UBC Theses and Dissertations

UBC Theses Logo

UBC Theses and Dissertations

High-dynamic range projection using a steerable MEMS mirror array Hoskinson, Reynald 2009-01-08

You don't seem to have a PDF reader installed, try download the pdf

Item Metadata

Download

Media
24-ubc_2010_spring_hoskinson_reynald.pdf [ 13.48MB ]
Metadata
JSON: 24-1.0068959.json
JSON-LD: 24-1.0068959-ld.json
RDF/XML (Pretty): 24-1.0068959-rdf.xml
RDF/JSON: 24-1.0068959-rdf.json
Turtle: 24-1.0068959-turtle.txt
N-Triples: 24-1.0068959-rdf-ntriples.txt
Original Record: 24-1.0068959-source.json
Full Text
24-1.0068959-fulltext.txt
Citation
24-1.0068959.ris

Full Text

High-dynamic range projection using a steerableMEMS mirror arraybyReynald HoskinsonB.A., McGill University, 1996M.Sc., The University of British Columbia, 2001A THESIS SUBMITTED IN PARTIAL FULFILMENT OFTHE REQUIREMENTS FOR THE DEGREE OFDoctor of PhilosophyinThe Faculty of Graduate Studies(Electrical and Computer Engineering)The University Of British Columbia(Vancouver)December 2009c© Reynald Hoskinson 2009iiAbstractThis thesis describes a novel way to improve the contrast and peak brightnessof conventional projectors by directing the light from the lamp away from thedark parts of the image towards the light parts before it reaches the projec-tor’s primary image modulator. A Microelectromechanical Systems (MEMS)micromirror array is inserted into the optical path between the lamp and theimage forming element. Each element of the array can be tip/tilted, divert-ing light to the areas that need it most, at the expense of the darker parts ofthe image. In effect, this method will produce a low resolution approximationof the image on the image-forming element. The micromirror array will allowthe projector to adapt its light source to the image being projected in orderto maximize light efficiency and throughput. By directing the light away fromthe dark parts earlier in the display chain, the amount of light that needs tobe blocked will be reduced, thus decreasing the black level of the final image.Moreover, the ability to dynamically allocate more light to the bright parts ofthe image will allow for peak brightnesses higher than the average maximumbrightness of display. Although the text primarily refers to DLP-type (digitallight projection) projectors, this technology will benefit all currently availableprojector types.Employing such an mechanism within a projector’s display chain requirescontributions to a number of different fields related to displays. We studied thetypical light path within a projector to determine how best to add micromirrorsinto the display chain. The tradeoffs between the distance on the screen that alight spot from a mirror (mobile light, or ML) could be moved, and its spatialextent were established. For a given micromirror tilt angle, the range of an MLcan be increased at the expense of a larger blur kernel. Micromirrors suitablefor this application were designed, simulated and fabricated. A novel way ofoptimizing the tradeoffs between tilt angle, mirror size, and mirror resonancefrequency by splitting the mirrors into smaller functional subsections was em-ployed. We developed several algorithms that determine favourable placementof the mobile lights from each of the micromirrors in the array, in order to bestimprove the image. From simulations, the projector average brightness could beincreased by a factor of 1.2 if micromirrors were available that could be tiltedto ±3.5◦ with the addition of this technology, without changing the projectorlamp. If the requirement for perfect image reconstruction is relaxed, the im-provement factor increases to 2.25. A prototype was system was developed thatallows for fast control of mirror elements, and the positive effect of employingadaptive light distribution in this manner was demonstrated.iiiContentsAbstract . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . iiContents . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . iiiList of Tables . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . viList of Figures . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . viiNomenclature . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xAcknowledgements . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xii1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11.1 Background . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11.2 A concept for an improved projection display . . . . . . . . . . . 31.3 Other adaptive display mechanisms for projectors . . . . . . . . . 41.3.1 Dynamic aperture . . . . . . . . . . . . . . . . . . . . . . 41.3.2 High dynamic range projectors and displays . . . . . . . . 41.4 Greyscale levels . . . . . . . . . . . . . . . . . . . . . . . . . . . . 61.5 Contributions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 71.6 Structure of the thesis . . . . . . . . . . . . . . . . . . . . . . . . 82 Optical system . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 92.1 Conventional projection display systems . . . . . . . . . . . . . . 92.2 ´Etendue . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 92.3 Projector optics . . . . . . . . . . . . . . . . . . . . . . . . . . . . 102.3.1 Lamp . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 102.3.2 Light collection . . . . . . . . . . . . . . . . . . . . . . . . 122.3.3 Integrator . . . . . . . . . . . . . . . . . . . . . . . . . . . 132.3.4 Light valve . . . . . . . . . . . . . . . . . . . . . . . . . . 142.4 Design of the optical system of the HDR projector . . . . . . . . 162.4.1 Circle of confusion . . . . . . . . . . . . . . . . . . . . . . 172.5 Luminance clipping due to AMA tilt . . . . . . . . . . . . . . . . 212.5.1 Clipping quantified . . . . . . . . . . . . . . . . . . . . . . 232.5.2 ´Etendue versus contrast . . . . . . . . . . . . . . . . . . . 272.5.3 Distribution of light from AMA mirror . . . . . . . . . . . 28Contents iv3 Micro-electromechanical mirrors . . . . . . . . . . . . . . . . . . . 303.1 Mirror positioning using electrostatic actuation . . . . . . . . . . 313.2 Micromirror design considerations . . . . . . . . . . . . . . . . . 363.3 Previous micromirror designs . . . . . . . . . . . . . . . . . . . . 383.4 Mirror designs for large-angle deflection . . . . . . . . . . . . . . 403.5 PolyMUMPs micromirrors . . . . . . . . . . . . . . . . . . . . . . 413.5.1 PolyMUMPs micromirror design . . . . . . . . . . . . . . 413.5.2 PolyMUMPs mirror characterization . . . . . . . . . . . . 443.6 Micragem mirror arrays . . . . . . . . . . . . . . . . . . . . . . . 483.6.1 Micragem mirror designs . . . . . . . . . . . . . . . . . . . 483.6.2 Simulations of Micragem mirrors . . . . . . . . . . . . . . 513.7 Proposed new Micragem design . . . . . . . . . . . . . . . . . . . 563.8 Future mirror designs . . . . . . . . . . . . . . . . . . . . . . . . 604 Light allocation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 614.1 Mobile light sources . . . . . . . . . . . . . . . . . . . . . . . . . 614.2 Gamma . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 624.3 Mirror allocation . . . . . . . . . . . . . . . . . . . . . . . . . . . 634.4 Gaussian pyramid approach . . . . . . . . . . . . . . . . . . . . . 654.5 Iterative adjustment . . . . . . . . . . . . . . . . . . . . . . . . . 664.6 Median cut approach . . . . . . . . . . . . . . . . . . . . . . . . . 684.6.1 Adjusting for limited ML range . . . . . . . . . . . . . . . 704.6.2 Median cut with iterative adjustment . . . . . . . . . . . 704.7 Simulation results . . . . . . . . . . . . . . . . . . . . . . . . . . 744.7.1 Gaussian pyramid results . . . . . . . . . . . . . . . . . . 794.7.2 Median cut results . . . . . . . . . . . . . . . . . . . . . . 814.7.3 Observations . . . . . . . . . . . . . . . . . . . . . . . . . 844.7.4 Implications for prototype . . . . . . . . . . . . . . . . . . 854.7.5 Sub-frame positioning . . . . . . . . . . . . . . . . . . . . 864.7.6 Allowing under-illuminated pixels . . . . . . . . . . . . . . 874.8 Estimated system performance . . . . . . . . . . . . . . . . . . . 894.8.1 Brightness improvement factor . . . . . . . . . . . . . . . 894.8.2 Contrast improvement . . . . . . . . . . . . . . . . . . . . 904.9 AMA projector image fidelity . . . . . . . . . . . . . . . . . . . . 904.10 Visual difference prediction . . . . . . . . . . . . . . . . . . . . . 924.10.1 VDP tests on simulation results . . . . . . . . . . . . . . . 934.10.2 Quantifying the effect of some under-illuminated pixels . 945 Physical implementation . . . . . . . . . . . . . . . . . . . . . . . . 985.1 Prototype overview . . . . . . . . . . . . . . . . . . . . . . . . . . 985.1.1 Optical system . . . . . . . . . . . . . . . . . . . . . . . . 995.1.2 AMA driver . . . . . . . . . . . . . . . . . . . . . . . . . . 1005.1.3 Synchronization . . . . . . . . . . . . . . . . . . . . . . . 1005.2 Prototype results . . . . . . . . . . . . . . . . . . . . . . . . . . . 102Contents v6 Conclusions and future work . . . . . . . . . . . . . . . . . . . . . 1096.1 Summary and conclusions . . . . . . . . . . . . . . . . . . . . . . 1096.2 Summary of Contributions . . . . . . . . . . . . . . . . . . . . . . 1106.3 Future work . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 110Bibliography . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 113viList of Tables3.1 Published characteristics for 2-DOF micromirrors . . . . . . . . . 404.1 Disparities tested, their corresponding blur kernel sizes, and thefull-width at half maximum (FWHM) of the blur kernel. . . . . . 754.2 Image characteristics . . . . . . . . . . . . . . . . . . . . . . . . . 764.3 Mechanical tilt angle (degrees) needed for select levels of blur anddisparity . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 784.4 Results of allowing a percentage of pixels to be under-illuminated. 96viiList of Figures1.1 Schematics of conventional and AMA projectors . . . . . . . . . 32.1 One ray bundle traversing a projection system . . . . . . . . . . 102.2 Illustration of variables in ´etendue . . . . . . . . . . . . . . . . . 112.3 Simplified optical function Of DMD device . . . . . . . . . . . . . 152.4 Lightpath of projector with an AMA, showing one ray bundle. . 172.5 Illustration of circle of confusion . . . . . . . . . . . . . . . . . . 192.6 How blurred light from an AMA mirror reaches the DMD . . . . 212.7 DMD light cone showing clipping due to AMA . . . . . . . . . . 222.8 A ray of height r (solid) and angle θ is diverted by the AMA by α 232.9 Collection efficiency of a UHP lamp as a function of system ´etendue 242.10 Change in angular distribution of light at DMD due to AMA tilt 252.11 Estimated losses for AMA tilts due to the DMD aperture. . . . . 262.12 Increasing inherent contrast via change in on-state light angle . . 273.1 Parallel-plate actuation . . . . . . . . . . . . . . . . . . . . . . . 313.2 Micromirror actuation through parallel-plate electrostatics. . . . 323.3 Electrostatic field applied to a mirror with torsional springs . . . 333.4 The dimensions of a micromirror spring . . . . . . . . . . . . . . 333.5 Schematic of in-plane comb drive . . . . . . . . . . . . . . . . . . 353.6 Concept of increased deflection angle through composite mirrordesign . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 413.7 PolyMUMPs micromirror, cross-section . . . . . . . . . . . . . . 423.8 Electrodes underneath composite mirror . . . . . . . . . . . . . . 433.9 PolyMUMPs micromirror, top view of model . . . . . . . . . . . 433.10 Scanning-electronic microscope image of PolyMUMPs mirrors . . 443.11 Photograph of a PolyMUMPs composite micromirror . . . . . . . 453.12 Static mirror deflection of PolyMUMPs mirror . . . . . . . . . . 463.13 Displacement amplitude vs. frequency for PolyMUMPs mirror . 473.14 One MUMPs mirror as measured by a white-light interferometer.The radius of curvature was measured to be 2.8mm. . . . . . . . 473.15 4 micromirrors from an early design using the Micragem process 483.16 Several micromirrors within the gimbal frame system . . . . . . . 493.17 Array of composite mirrors in Micragem row design . . . . . . . 503.18 Two rows of electrodes of the Micragem micromirror design . . . 503.19 Model of micromirror produced in ANSYS. . . . . . . . . . . . . 513.20 Implications of gimbal bending in the Micragem design . . . . . . 52List of Figures viii3.21 Simulated and measured individual mirror tilt around X axis . . 543.22 A scanning-electron microscope picture of micromirror springs . . 553.23 Image from WYKO white light interferometer of one compositemirror tilting. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 553.24 Illustration of mirrors electrically connected together by fixed rods 573.25 Electromechanical simulation of micromirror, Electrode A at 180V 573.26 Simulation of micromirror with electrode B at 180V. . . . . . . . 583.27 Simulation of micromirror with electrodes A and B at 200V. . . . 583.28 Simulation of micromirror: electrode A at 170V, and electrode Bat 190V . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 594.1 The light distribution from an non-tilted AMA, illustrating overfill 644.2 Gaussian pyramids for image and ML . . . . . . . . . . . . . . . 664.3 Basic steps in iterative adjustment algorithm . . . . . . . . . . . 674.4 An image divided to 28 regions using the median cut algorithm . 694.5 Diagram of placements of MLs using the median cut algorithm . 714.6 False-colour estimation of light distribution given by the ML lo-cations in Figure 4.5. . . . . . . . . . . . . . . . . . . . . . . . . . 724.7 Adjusted ML locations from 4.5 after an additional optimizationstep . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 734.8 Estimated blur for one ML of a 5×5 array . . . . . . . . . . . . 754.9 Original sample image 1: Mt. Robson. . . . . . . . . . . . . . . . 764.10 Original sample image 2: Rocky beach. . . . . . . . . . . . . . . . 774.11 Sample image 3: ANSI checkerboard. . . . . . . . . . . . . . . . . 784.12 Gaussian pyramid results for the Mt. Robson image . . . . . . . 794.13 Gaussian pyramid results for the Rocky Beach image. . . . . . . 804.14 Gaussian pyramid results for the ANSI checkerboard image. . . . 804.15 Allocation results: median cut algorithm, Mt. Robson image . . 814.16 Allocation results: modified median cut algorithm, Mt. Robsonimage . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 824.17 Allocation results: modified median cut algorithm, Rocky beachimage . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 824.18 Allocation results: modified median cut algorithm, ANSI checker-board image . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 834.19 Effect of allowing some pixels to be under-illuminated on im-provement factor for the Mt. Robson image. . . . . . . . . . . . . 884.20 VDP results from testing the effect of quantization, limited con-trast and allowing no under-illuminated pixels. . . . . . . . . . . 954.21 VDP results from testing the effect of quantization, limited con-trast and allowing 2% under-illuminated pixels. . . . . . . . . . . 965.1 Schematic of prototype, showing the major components. . . . . . 985.2 Photograph of prototype . . . . . . . . . . . . . . . . . . . . . . . 995.3 Schematic of the control signal flow in the prototype. . . . . . . . 1015.4 Light intensity from a Mitsubishi PK20 projector over time . . . 1015.5 Output of the AMA prototype with different separation settings 103List of Figures ix5.6 One ML at approximately -1mm disparity . . . . . . . . . . . . . 1045.7 One ML at approximately 8.5mm disparity . . . . . . . . . . . . 1055.8 14mm disparity, relative change. Units of percent difference. . . . 1065.9 30mm disparity, relative change. Units of percent difference. . . . 1075.10 Relative change for four different disparity settings, showing mul-tiple mirrors actuated. . . . . . . . . . . . . . . . . . . . . . . . . 108xNomenclaturef/# Ratio of focal length to diameter (f-stop)Im Original ImageIp Image of light distribution at DMDAMA Analog Micromirror arrayCE Collection efficiencyCMP Chemical-mechanical polishingCRT Cathode ray tubeDLP Digital Light ProjectionDMD Digital Micromirror DeviceDOF Depth of fieldDRIE Deep reactive ion etchingFEM Finite element modellingfL foot-lambertFWHM Full width at half maximumGP Gaussian pyramidHDR High Dynamic RangeHID High Intensity DischargeLCD Liquid Crystal DisplayLED Light Emitting DiodeMEMS Microelectromechanical SystemsML Mobile light sourcePolyMUMPs Polysilicon Multi-User MEMS ProcessPSF Point-spread functionList of Figures xiPWM Pulse-width modulationSCSi Single Crystal SiliconSOI Silicon-on-InsulatorTMA Thin-film Micromirror ArrayUHP Ultra-high pressure lampVDP Visual difference predictorxiiAcknowledgementsI would like to thank the many people who have guided me through my uni-versity career, as well as the friends and family who made it such an enrichingexperience.My supervisor Dr. Boris Stoeber has been an extraordinary guide into thewonderful world of MEMS over the past four years, and an always available andpatient mentor. Dr. Sid Fels has provided invaluable perspective throughoutthe sometimes windy road of my time at UBC. Helge Seetzen first alerted meto the possibility of overturning the conventional display paradigms and hisconfidence and ambition was infectious.My parents, Michael and Gisele, and brothers and sister Emile, Paul, andAngela for all your constant encouragement.Thanks to all those who have helped out on this project including Iman,Jeremy, Gerwin, Daniel, and Stefan. I’d like to acknowledge all my labmatesover the years including Ian, Ramin, Dave, Mike, Allison, and Paul.And finally, thank you, Caitlin Akai for your quiet and not-so-quiet supportduring my studies.1Chapter 1IntroductionElectronic displays convert signal information representing graphical elementsinto a viewable image. They can be categorized based on how the image isviewed. The three main categories are direct view, virtual, and projection. Ina virtual display, the image is in focus only on the retina, while direct viewdisplays include the surface upon which the image is displayed as part of thedevice. A projection display forms the displayed image on an auxiliary surface.The images are typically meant to be viewed by groups of people at the sametime.Projectors are carefully engineered to channel as much of the light emanatingfrom the lamp onto the image-forming element, and to relay the formed imagethrough the projection optics to the screen. How efficient the projector is atdoing this has major repercussions. Brightness is the primary characteristicdetermining projector price and quality, and projector efficiency is one key indetermining the final brightness of the projected image. Simply increasing thebrightness of the lamp to make the image brighter is not always an option. Thelamp is typically the most expensive piece of the projector, even more so thanthe image forming element. Since the light that is not directed to the screenends up as heat, a brighter lamp carries with it the need for bulkier and noisierfans and lamp electronics. No matter how carefully the lamp reflector and relayoptics are engineered, a fundamental limitation on efficiency is encountered: theimage is formed by letting light through to the screen for the bright parts of theimage, and blocking the light for the dark regions. The image-forming elementis evaluated by how well it passes light, and also how well it blocks it, becauseit is its ability to block light that finally creates the image seen on the screen.1.1 BackgroundThe two dominant types of projection displays available today are based oneither Digital Light Projection (DLP), or Liquid Crystal Displays (LCD). Toconstruct images, both employ different types of light valves, elements thatselectively block light. A lamp provides uniform illumination to the light valve,and the liquid crystals in the LCD, for example, selectively reduce illuminationof a pixel on the screen in order to form the dark parts of the image. TheDigitalMicromirrorDevice(DMD),insideaDLPprojectorfunctionsinasimilarmanner. A DMD is an array of micromirrors, one for each pixel, each one ofwhich can be tilted in one direction so that incident light reflects towards theChapter 1. Introduction 2projection lens and then out onto the screen, or another direction so the lightis reflected to a heat sink and that spot on the screen remains dark.The light source in a conventional projector supplies a uniform brightnessdistribution on the light valve, limiting the maximum brightness for a displayedimage. For most images, however, only a fraction of the total area is illuminatedat peak brightness. A conventional projector simply blocks the light which isnot necessary for a scene, thereby wasting this fraction of light, while it couldbe used to further illuminate the bright parts of the image.Furthermore, the currently available light valves are “leaky” and cannotblock all the light for black image areas (Dewald et al. 2004). The dynamicrange of a projector is the ratio between the brightest and darkest levels it candisplay. The dynamic range, or contrast of a display device can be defined asthe ratio c = Ib/Id between the brightest Ib and darkest Id pixel generated bya projector. Merely increasing the illumination through the projector lamp ina conventional projector does not necessarily increase the dynamic range, asboth the darkest and brightest level rise by the same relative amount. Currentprojector technology could benefit from an improvement in both dynamic rangeand peak brightness of the projected image.Overall projector contrast is a combination of light valve contrast and theoptical system. Pinning down current projector contrast norms is a difficulttask because manufacturers often claim contrast ratios substantially higher thanwhat is typically measurable in real world usage. Claims of ratios of up to2000:1 are common for commodity projectors, while flat panel displays claimratios much higher, above 40 000:1. Since no standardized, specific test hasbeen accepted by the industry, these claims are often widely inflated. For thelight valve itself, (Dewald et al. 2004) charts the evolution of the DMD contrast.The first DLP introduced into production in 1996 had 220:1 contrast, while thecurrent ‘Dark Metal 3’ iteration has a contrast of 1000:1. Since 2004, contrastimprovement effort has focused on the optical system (Dewald et al. 2004; TexasInstruments 2005b; Brennesholtz 1996).The contrast and dynamic range of projectors is not sufficient to displaymany of the real-world scenes, which can have up to eight orders of magnitudeof luminance range (Reinhard et al. 2005). Scenes such as sunsets, fireworks,and daylight must have their luminances tone-mapped to the limited range ofcurrent projectors. The peak brightness of a projector also limits the typeof environment in which it can be used to its full potential; the brighter theroom illumination, the more lower-end detail will be lost, lending a ’washedout’ image.The emerging popularity of 3D movies even further strain a projector’s abil-ity to display an image. Typically the efficiency of a theatre projector dropsto 14% of what it is for 2D content (Brennesholtz 2009). Current digital cin-ema projectors “strain” to achieve 4.5 foot-lamberts (fl) of screen luminance,especially on larger screens, rather than the 14 fl recommended by the Societyof Motion Picture Television Engineers in the standard ANSI/SMPTE 196M(SMPTE 2003). Even at 4.5 fl, lamps must be run at maximum power, lead-ing to high electric bills and short lamp life. These limitations are felt by theChapter 1. Introduction 3Figure 1.1: Fig. 1(a) Schematic of a conventional DLP projector, and (b)schematic of an enhanced DLP projector with second MEMS mirror array(AMA)studios as well. Because the displayed luminances are so different, they affectthe perception of colours (Brennesholtz 2009), and so studios must perform twoseparate expensive colour-correction processes, one for 2D and one for 3D.Portable projectors are also appearing on the marketplace, making batterylife, which is mostly dependent on the energy the lamp uses, an issue. Theseapplications specifically, and projectors in general, would benefit from a methodto use the light from the projector lamp more efficiently.1.2 A concept for an improved projectiondisplayThis thesis details a new concept that addresses the contrast shortcomings ofcurrently available projectors by adding a low-resolution intermediate mirrorarray to provide a non-homogeneous light source. This intermediate mirrordevice is capable of directing the uniform light from the projector lamp incidenton its surface to different areas on the light valve, in effect projecting a low-resolution version of the original image onto the light valve as shown in Figure1.1. Adding this intermediate mirror device will improve the dynamic range intwo ways: by directing the light to the bright parts of the image, the achievablepeak brightness will be increased. Simultaneously, the amount of light thatneeds to be blocked in the dark regions of the image will be reduced, thusdecreasing the brightness of the black level.This intermediate device can be realized with a low-resolution analog mi-cromirror array (AMA), made using microelectromechanical system (MEMS)technology. The tip and tilt angle (two degrees of freedom) of the micromirrorsChapter 1. Introduction 4in the array can be set continuously in order to direct light to an arbitrarylocation on the light valve such as the DMD.1.3 Other adaptive display mechanisms forprojectorsIn this section we describe other methods for dynamically changing illuminationincident on the light valve. While these other techniques can successfully reducethe dark level of the projector under certain conditions, none increase brightness.1.3.1 Dynamic apertureThe addition of a dynamic iris which is adjusted per frame of video is a muchmore limited way of adapting the projector’s light source to the content (Iisakaet al. 2003) (Toyooka et al. 2005). The dynamic iris is a physical aperturenear the lamp that can change size with each video frame, limiting the totalillumination that reaches the screen. For dark scenes, the iris closes, limitingthe light reaching the light valve, and thus increasing the contrast betweenimages by decreasing the dark level of dark scenes while increasing illuminationfor brighter scenes. The image data is adjusted in real time to reflect the irisposition. The system has to suppress unexpected changes in brightness, sincethese changes are interpreted by the human visual system as flicker.While this method can decrease the black level of certain images, it is only aglobal adjustment, not allowing improving the contrast in a scene with severallocalized bright or dark areas. Also, while this approach decreases the blacklevel for certain select images, it can’t increase the maximum brightness of ascene like the AMA can.1.3.2 High dynamic range projectors and displaysThe concept of using two light modulators in series put forward in (Seetzenet al. 2004) has primarily been applied to flat-panel displays, but has also beenapplied to several projector designs to improve contrast. With the dynamicrange c1 : 1 of the first display, and the dynamic range c2 : 1 of the seconddisplay, the theoretical contrast of the combined system is (c1c2) : 1. Using twolight modulators also results in a multiplicative increase in the number of dif-ferent brightness levels. Capitalizing on the limits inherent in the human visualsystem’s contrast sensitivity over different spatial scales, one of the modulatorscan be of much lower resolution than the final image.Seetzen et al. have developed an algorithm that generates commands foreach of two sequential light modulators from a given image (Seetzen et al. 2004;Trentacoste et al. 2007). The image can either be a of conventional 24-bit for-mat, or high-dynamic range (HDR), which can represent a much higher rangeof luminances. The authors demonstrated the reconstruction of HDR imagesusing two flat-panel display prototypes. In the first, a projector served as aChapter 1. Introduction 5backlight and low-resolution modulator, illuminating an LCD panel that servedas the high-resolution modulator. In their later DR-37 HDR display, an arrayof light emitting diodes (LEDs) are used as heterogeneous low-resolution back-ground illumination of a conventional LCD screen, where the LCD provided thehigh resolution image correction (Seetzen et al. 2004). In the algorithm, theoriginal HDR image is first downsampled to the resolution of the LED array,then the actual brightness distribution provided by the LEDs is determined bytaking into account the point-spread function (physical brightness distribution)of an LED through the display optics. The high-resolution LCD correctionis then generated to correct for any perceivable differences between the LEDillumination and the target image.Directly adapting these methods to projectors presents several problems.Conventional projectors use high-intensity discharge (HID) lamps as describedin Section 2.3.1, which are not suitable to be placed in a tightly-packed array.Also, projector lamps typically operate efficiently only at one output intensity,so are not themselves suitable as modulators. Instead, the previous approachesdescribed below place two subtractive modulators in series after a single projec-tor lamp whose output does not change. While this approach can successfullyreduce the dark level of the projector, it also inevitably reduces the overallbrightness of the projector, because neither modulator can transmit light with-out loss.(Pavlovych and Stuerzlinger 2005) added a second light modulator by fo-cusing the light from a regular DLP projector onto an LCD panel, and thenprojecting the image of the LCD panel onto a display screen. Overall contrastof this projection system is improved due to the reduction in dark level by thesecond LCD modulator, but at the cost of a significant reduction in overallbrightness, as LCD panels typically transmit only 10% or less of unpolarizedincident light (Seetzen et al. 2004). Also, the extra dynamic range availablein the dark regions will only be perceivable in a very dark room; the darklevel increases regardless as ambient room illumination increases. Even in amoderately-lit room, the effect of the lower dark levels will thus be lost, result-ing in a net decrease in overall perceivable contrast compared to an unmodifiedprojector, because of the darker peak brightness.(Damberg et al. 2007) similarly employ two LCD panels in series. Theirprototype includes the second LCD inside the projector, and uses six panels intotal – two for each colour channel. The authors report an order of magnitudeincrease in contrast relative to the original projector, but as in (Pavlovych andStuerzlinger 2005), this comes at the expense of the overall brightness of theimage due to the light losses from the second LCD panel.(Kusakabe et al. 2009) have developed a high-resolution dual-modulationprojectorwiththreeparallelmodulatorsforchrominance, andonehigh-resolutionmodulator for luminance, a concept also suggested by Damberg et al. They usea LCoS device with 8192 × 4320 pixels for the luminance modulator, and three4096 × 2160 LCoS panels for chrominance modulation. They claim a dynamicrange of 1.1 million to 1, and 10-bit greyscale. Chrominance can be displayed ina much lower resolution than luminance because humans are much less sensitiveChapter 1. Introduction 6to fine detail in chrominance. The ANSI contrast ratio using a 4 × 4 checker-board pattern was measured at the screen to be 250:1, which they attribute tothe flare of the relay and projection lenses. As is the case with the other high-contrast projectors in the literature, the increase in contrast was solely due tothe reduction in black levels, an improvement that is only relevant in extremelylow brightness environments.An AMA-projector would be particularly suited to displaying HDR images inan efficient manner. (Trentacoste et al. 2007) cite excessive power consumptionas one of the reasons that Seetzen et al. moved from using a projector as a lightsource to an array of LEDs. The lamp of the projector had to emit enough lightto illuminate the entire LCD panel at its highest possible brightness, whetherit was needed or not. For high-dynamic range images that include regions withvery high brightnesses, this would require a very bright lamp, especially after theefficiency of the LCD panel (16%) was taken into account. Trentecoste notes,however, that a random selection of 100 HDR image had less than 10% of theimage content in the high luminance range, and the average luminance wasmuch lower than the highest. For the images tested, the HDR projector displayoutput a factor of between 12.5 and 4.75 too much light, depending on the image.Clearly, a projector that could adapt its spatial brightness distribution to thecontent of the image would be very advantageous for these types of images.1.4 Greyscale levelsBesides peak brightness and contrast, a key attribute of any display is howmany discrete steps of luminance in between the brightest and darkest settingsare achievable. Conventional displays usually offer 8-bit control (256 steps) overluminance for each of the three color channels red, green and blue. While thisis an adequate number of steps for conventional image formats, the much largerrange of luminance of HDR displays also requires more brightness levels. Visualpsychologists such as (Barten 1992) have charted the number of steps necessaryto cover a given range of luminance, given the limits of the human visual system.From Barten, 962 distinct steps are sufficient for a display that can reach from0.05cd/m2 to 2700cd/m2. With two modulators as proposed in (Seetzen et al.2003), the number of steps possible in each modulator is multiplied to get thetotal number of luminance steps for the combined display. Two linear 8-bitdevices in series theoretically provide 2562 distinct steps, which easily exceedsthe requirement calculated by Barten.An AMA-enhanced DLP projector could also provide additional grey levelsfrom two different ways. First, multiple mobile light sources from different AMAmirrors can be directly overlapped. If only one micromirror area on the DMDis illuminated, all n AMA mirrors could divert their light to that spot, resultingin n possible grey levels from the AMA. The DMD could further modulatethis light, so the maximum number of grey levels is 256n. Secondly, the lightfrom each AMA mirror will have an approximately Gaussian distribution on theDMD. The resulting continuously-varying intensities from the AMA mirrors willChapter 1. Introduction 7also provide more brightness levels than would be possible using just the DMD.There are several reasons, however, why providing extra greyscale controlusing the AMA is not practical. First, the intermediate brightness levels men-tioned above are not fully controllable because of the limited resolution of theAMA. With an AMA for each pixel in the primarily modulator, more greyscalewould be assured. For AMAs with few mirrors, however, there is not enoughdegrees of freedom to ensure accurate display of more greyscale levels across theentire image. The AMA we are proposing would have just 100 or fewer mirrors,due to the cost and complexity of controlling them, while the secondary modu-lators used in Seetzen et al. contain hundreds of individually-controllable LEDsor areas from a projector.Another reason that added levels of greyscale are not needed is that theaddition of an AMA element will not make the display radically brighter as isthe case with Seetzen et al. With a baseline of 1 as the brightness a non-AMAprojector would be able to show a given image, let c be the improvement factor,or how much brighter we can show the image using the same projector with anAMA added. c will depend on the content of the image, including the sum totaland distribution of pixel intensities. If all intensities are within a small areain the image, the improvement factor could be large. An image where all thepixel intensities are near maximum and evenly distributed throughout, on theother hand, would not be able to to be brightened much using the AMA. Foran average image, we anticipate an improvement factor of 2. This would notadd enough brightness to require extra greyscale steps besides the 256 offeredby the primary modulator. Because of these reasons, we will not attempt toproduce more levels of greyscale than are available in the primary modulator.1.5 ContributionsThis thesis represents several novel contributions to the field of Electrical andComputer Engineering.• A method to dynamically reallocate the light from a projector lamp fromdark regions to bright regions on an image-dependent basis in order toincrease projector peak brightness, contrast, and efficiency. This is alsodescribed in our paper (Hoskinson and Stoeber 2008).• A theoretical framework for examining the tradeoffs between optical pa-rameters that affect AMA system performance.• The design and fabrication of an analog micromirror array suitable foran AMA projector, with composite mirrors that optimize the tradeoffsbetween mirror tilt angle, size, and dynamic behaviour (Hoskinson et al.2007a; Hoskinson et al. 2007b).• Algorithms and software implementations that allocate the mobile lightsdepending on the image, taking into account the physical limitations ofthe mirrors and projector.Chapter 1. Introduction 8• A prototype implementation demonstrating this method, showing thatregions can become brighter as well as darker.1.6 Structure of the thesisIn Chapter 2, we discuss the components that make up a projector, and theoptics that need to be considered when inserting the AMA into the projector.In Chapter 3, MEMS micromirrors are discussed, and the mirrors designedand fabricated for this application are detailed. We propose algorithms thatallocate the light from the AMA mirrors depending on image features in Chapter4. In Chapter 5, the prototype is described, and results showing the proofof principle of the AMA projector are discussed. Chapter 6 concludes, anddiscusses potential future directions.9Chapter 2Optical systemThis chapter includes a detailed investigation of how light is propagated througha projector. Very generally, projection displays use an optical imaging systemto magnify a small picture created by modulating the light from an illumina-tion system with a two-dimensional light valve. The light valve can modulateincident light independently for each pixel in the image.2.1 Conventional projection display systemsLooking at the light path in its entirety, all projectors can be broken down intoa number of functional subsections. Figure 2.1 show the subsections of singlechip DLP projectors. For purposes of clarity only the light reaching one pixelof the DMD is shown; in reality there is a ray bundle that reaches each pixelof the DMD. First in the light path, a reflector collects the light from a smallarc lamp, (usually metal halide or halogen), and directs it into the illuminationoptics. In a single-chip DLP projector, the lamp reflector minimizes the spotsize of the light at the colour wheel. After the colour wheel is the integrator,which spatially redistributes the image of the arc from a highly-peaked to amore uniform distribution with an aspect ratio that matches that of the lightvalve. This affects the final distribution of the light on the screen. For DLPprojectors, the integrator is usually a rod, made of hollow mirrored tunnels.From the integrator rod, the light travels through relay/folding optics, whichform an image of the integrator rod face on the DMD. The image of the DMDis then transmitted to the screen using a projection lens system.2.2 ´EtendueBefore a detailed look at the main components of a DMD projector, we firstdefine ´etendue, as it informs much of the discussion of the performance limits inprojector design. ´Etendue has been described as “the optical engineer’s versionof the second law of thermodynamics”(Brennesholtz and Stupp 2008). It is thegeometric capability of an optical system to transmit light. It can also refer tothe optical beam itself, as a product of its divergence and cross-sectional areaperpendicular to the propagation direction (Brennesholtz 1996).´Etendue can be defined asE =integraldisplay integraldisplaycosθdAdΩ, (2.1)Chapter 2. Optical system 10Lamp/ref_lector colourwheelintegrator fold mirrorrelay lensDMDprojectionlensFigure 2.1: One ray bundle traversing a projection systemwhere E is integrated over the area of interest. The angle θ is between thecentroid of the solid angle element dΩ and the normal to the surface elementdA, asshowninFigure2.2. Notethatthereisnoterminthisequationrelatingtooptical intensity: ´etendue is solely a geometric property. The units for ´etendueare mm2 steradian.When a beam is modified by a well-corrected optical element, ´etendue ispreserved. For example, when a lens focuses a beam to a spot, the area of thebeam is reduced but the convergence angle of the beam increases, so ´etendue ispreserved. The ´etendue of a ray bundle of light can never decrease; in an areathat involves scattering, it will increase.2.3 Projector opticsThis section explains the major components of a projector, with special atten-tion paid to the properties that constrain addition of the AMA system. Wealso concentrate on how optical design choices limit projector brightness andefficiency.2.3.1 LampThe first element in the projector’s optical path is the lamp, which today istypically an arc lamp, also referred to as an HID lamp (high-intensity discharge).A particular type of HID lamp, the UHP (ultra-high pressure) lamp by Phillipshas come to dominate all but the very high-end projector market (Brennesholtzand Stupp 2008). It is usually sold as an integrated lamp and reflector withChapter 2. Optical system 11Figure 2.2: The angle θ between the area of interest A,and the centroid of thesolid angle element dΩ in ´etendue.a power supply designed to maximize the lamp life. Lamps typically last from400-2000 hours. The arc length is typically 1.0 to 1.3 mm, which, along withthe reflector, determines the ´etendue of the resulting optical beam. The smallerthe arc length, the smaller the ´etendue, as long as the reflector can efficientlycollect the luminance.While HID lamps remain the most popular projection lighting source, LEDsand lasers have also been used. LEDs are used in small, portable projectors thatcan achieve up to 50 lumens, a projection market area that has seen much recentdevelopment (Conner 2006; Van Giel et al. 2007; Pan et al. 2008; Kanayamaet al. 2006). LEDs have a much lower lumen output per unit of ´etendue thanHID lamps, and so have not been used to date in many larger projectors.Lasers have also been used as a light source for projectors. See Brennesholtz(Brennesholtz 2007) for a concise description of laser projection technologies.Because of the small divergence angles of laser beams, they have very small´etendue values. A laser source also offers the possibility for much higher (105)brightnesses than HID lamps (Brennesholtz 2007). Lasers can also have a muchnarrower emission spectrum, allowing for more saturated colours and highercolour efficiency than HID lamps.Lasers have been incorporated as projector light sources in three main ways:1. As a raster scanner, with a laser for each primary colour, combined to onespot showing one pixel at a time. The spot moves over the image quicklyenough so that each image is integrated by a human observer. The laserintensity is modulated at video rate.2. Linear arrays such as the grating light valve (Bloom 1997), which producethe image through diffractive elements, one line at a time. A scanningmirror moves the single line through the image over time.Chapter 2. Optical system 123. As a conventional light source, with a DMD or other light valve technol-ogy. An example is the projection television the Mitsubishi LaservueTM(Mitsubishi ).Despite being proposed for displays since 1960 (Brennesholtz 2007), lasershave predominantly been used only in very small ‘pico’ projectors such as fromthe company Microvision (Yalcinkaya et al. 2006), and large, institutional sys-tems such as that from Evans & Sutherland (Sutherland 2009). The primaryreason they have so far been kept out of commodity displays is cost per lumen:laser sources bright enough for non-pico commodity projectors continue to becommercially cost-prohibitive. While there have been attempts at producinginexpensive laser sources for displays such as from the company Novalux (Nivenand Mooradian 2006), that company has since run into financial problems andhas been sold. Another barrier is safety: there needs to be a solution for high-brightness front projection systems to protect from eye damage. The horizonfor common laser projection continues to appear as far (or as close) as in 1960.Multiple lampsUsing multiple lamps to create a higher-brightness image is an option both fortraditional projectors and those with an AMA. The light from the lamps aretypically combined via an arrangement of lenses, prisms and reflectors. Anotherarrangement in LED projectors such as the Mitsubishi PK10 is to have multipleLEDs of different colours combined in series using dichroic mirrors.Multiple HID lamps are currently used in some high-end projectors wherebrightness requirements are paramount over cost. Using multiple light sourcesincreases the ´etendue of resulting beam compared to a single source, because theeffective surface area of the combined lamp and the resulting beam’s divergenceangle is much higher. For this reason, multiplying the number of lamps doesnot multiply the resulting luminance of the projector by the same factor; onepublished value for two lamps gave an improvement factor 1.5 (Mang et al.2008).Regardless of the method used to combine the light from separate lamps,the AMA could be used with the resulting beam in the same way as with thesingle lamp situation described in this thesis.2.3.2 Light collectionAs much of the light from the lamp is collected and channeled forwards towardsthe rest of the display in as small and collimated a beam as possible using areflector. Because of aberrations, the ´etendue of the beam after the collectionoptics is much larger than the original ´etendue of the lamp. The main strategiesfor light collection fall into two categories: imaging and non-imaging optics.In imaging optics, which is used in almost all non-LED projectors, animage of the arc is produced somewhere on the optical path. Two main subcat-egories of imaging optics for projection are critical illumination, in which theChapter 2. Optical system 13image of the lamp is at the light valve, and K¨ohler illumination, in which theimage of the lamp is at the entrance pupil of the projection lens. K¨ohler illumi-nation has the advantage of being inherently uniform at the light valve, but isless efficient, even taking into account the separate homogenizing step neededin critical illumination. Critical illumination is thus the standard strategy forlight collection in data projectors.In non-imaging optics, no image of the arc is produced. The emphasisis instead on the optical beam and its aspect ratio, area, and divergence. Anoverview of techniques used in this branch of optics can be found in (Winstonet al. 2005). There has been research into collection systems for HID-lamp pro-jectors with non-imaging optics such as (Jacobson et al. 1998). Utilizing a non-imaging collector allowed the authors to concentrate on optimizing collectionefficiency tradeoffs, in particular the tradeoff between the collection efficiencyof the reflector and the resulting ´etendue. Efficiency in this case refers to howmuch of the light from the lamp is collected and sent forward to the rest of theprojector. In general, the greater efficiency, the larger the ´etendue. All collec-tion systems in use in HID lamp projectors today are imaging, however, becauseoptimizing this tradeoff with non-imaging optics as they propose requires a dif-ferent lamp design and reflector optics, practical implementation considerationsthat have kept their ideas out of consumer projectors thus far.Non-imaging collection optics, however, are widely used with LED projec-tors, so perhaps there is room for some of this work to re-enter the arc-lampworld. As we shall see below, the most conceptually difficult part of incorporat-ing the AMA into a projector is that we are adding a non-imaging optic deviceinto a imaging-optic system.2.3.3 IntegratorAll modern projectors include some sort of integrator, such as an integratingrod or a lenslet integrator in order to homogenize the light output from thelamp/reflector, and change the circular cross-section of the lamp output beamto the rectangular aspect ratio of the light valve.A light tunnel, which is used for DMD projectors, is a rectangular elementwith reflective interior surfaces. It can either be air filled and dependent onmirrors for reflection, or solid, utilizing total-internal reflection. The tunnel hasthe same aspect ratio as the light valve it is meant to illuminate. Light fromthe lamp/reflector module is focused on the entrance to the light tunnel, andeach ray of light is reflected a number of times until it reaches the exit. Witheach reflection, each ray becomes less and less correlated with its input position,so that with an adequately-sized tunnel, the light at the output plane will bespatially uniform. The magnitude of the divergence angle (the angle betweenthe optical axis and the ray) of each ray does not change. An image of exitpupil of the integrator is placed with a lens on the DMD at the appropriatemagnification using a relay lens.Employing an integrator incurs some light losses (Brennesholtz and Stupp2008). An integrator rod introduces from 4-6 extra optical surfaces, each re-Chapter 2. Optical system 14sulting in a loss of beam intensity. At 0.75% loss per surface, this represents aloss of 4.5%. Another source of loss is from overfill: the light body produced bythe integrator is larger than the light valve in order for the edges of the picturesto be illuminated evenly. Typically, the overfill is 5-10% per linear dimension.Overfill is needed because the intensity of the light leaving the integrator doesnot have an entirely flat distribution. It falls off on the edges. To obtain ho-mogeneous illumination of the light valve, it is placed so that it occupies onlythe central, relatively homogeneous cross-section of the beam. Another reasonfor overfill is in case there is any system misalignment between the illuminationpath and the light valve; without overfill some pixels around the edge of thelight valve would be permanently under-illuminated.2.3.4 Light valveThe light valve in a DLP projector is a DMD, developed by Texas InstrumentsInc., and manufactured by them worldwide (Hornbeck 1983) (Hornbeck 1996).DLP projectors usually have either one or 3 DMD chips as light valves. In one-DMD systems, colour is achieved by means of a rotating colour wheel. Threeimages are shown per frame, one for each colour. Each mirror in the DMDis approximately 16.3µm square, with a 17µm pitch, and can be positioned intwo angular orientations, ±12◦ (10◦ in older versions, some newest models have±14◦). Zero degrees is not addressable. Because the mirror size is fixed, thesize of the array is larger for higher resolutions. For instance, a 1280 × 1024pixel array is 21.8 mm × 17.4 mm.Figure 2.3 shows a simplified version of how the individual DMD mirrorsreflect light either to a heat sink or out to the projection lens. When themirror in this diagram is set to the on state of −12 degrees, light is reflected tothe projection lens (which is centred at zero degrees in the diagram’s referenceframe), while a +12 degree mirror tilt sends light to the ‘light dump’ (centredat 48 degrees), where it is emitted as heat. Figure 2.3 also shows the mainsource of stray light that effects contrast: flat-state light. Although the DMD isonly bistable, flat-state light is still present from the integrated energy falling inthe area between on and off states. This light comes partly from the transitionbetween the on and off states of the mirrors, but since the mirrors can switchat a rate of more than 1kHz, this does not form a significant amount comparedto other sources of stray light. A larger proportion comes from the reflectionsfrom the mirror vias, which attach the mirror surface to the hinge mechanismunderneath it, and the gaps between the mirrors (Pinho 2003). The flat statelight also includes fixed contributions from the DMD package, such as windowreflectance and border metal (Texas Instruments 2005b).The DMD itself is just an array of mirrors, which reflect light coming fromany direction, without an inherent ´etendue. To fulfill its function as a lightmodulator, the incoming/outgoing light has to be limited to an angle less thanthat of its full tilt angle of 24 degrees (-12 to +12). That way, the light goingto the heat sink can be fully separated from that going to the projection lenspupil. The ´etendue of the DMD is therefore defined as a product of the totalChapter 2. Optical system 15LampProjectionlensFlat statelightOf_f-statelightOn (-12º)Of_f (+12º)-36º-12º12º36º60ºDMDmirrorFigure 2.3: Simplified optical function Of DMD device, illustrating the con-straints on input angle from the lamp to the DMDarea of the DMD and the angle of light it can accept and still fulfill its function.The ´etendue of a DMD can be calculated as follows. We begin with thegeneral case of a flat surface normal to the optical axis, and a uniform divergenceangle θ1/2. Equation 2.1 can then be integrated in closed form asE = n2AΩ = n2Apisin2θ1/2, (2.2)where n is the refractive index of the medium (air in this case), A the emissionarea, θ1/2 the half angle of the emission cone, and Ω the projected solid angle(Brennesholtz 1996).The numerical aperture,NA = nsinθ1/2 (2.3)is related to the f/# (pronounced f-stop, or f-number) of the projection lens,the ratio of its focal length to diameter Df/# = fD. (2.4)When the NA is small,NA = 12f/#. (2.5)Chapter 2. Optical system 16In this case, the half angle is limited by the aperture of the projection lens, sothe ´etendue can also be written asE = piA4(f/#)2. (2.6)Because we know that the acceptance angle of the projection lens pupil fora DMD projector is 24 degrees, we can work out the f/# of the projection lensas 2.4. In projectors, the NA of the projection pupil has to be small enoughto prevent overlapping flat and on-state light. The relay optics use appropriatemagnification to match the NA at the integrator with the NA of the projectionoptics. From the relay optics, the light hits the DMD, often first being reflectedthrough a total-internal-reflection (TIR) prism. The light then travels either toa heat sink or the optical projection system, which magnifies the image to thescreen while maintaining throughput and uniformity.In a projector, smaller ´etendue is better for the light source, while bigger isbetter for all of the other optical components such as relay lenses, fold mirrors,and light valve. If the ´etendue of an optical element is larger than the ´etendueof the light beam, it is capable of using the entire beam. Usually, the light valveis the element with the smallest ´etendue, due to its higher cost relative to theother optical components. An optical system that crops some of the light isreferred “´etendue limited”(Brennesholtz 1996).2.4 Design of the optical system of the HDRprojectorThe AMA must be inserted between the lamp and the DMD in order to changethe illumination distribution on the DMD. Here we call the distribution of lightfrom one mirror of the AMA at the DMD a mobile light source (ML). We wouldlike to achieve the optimal spot size of each ML, while maintaining adequatelight coverage of the DMD overall. The image of the DMD with this variableillumination will then be projected to the screen by the projection lens. Aswell as minimizing the AMA spot size, we would like to maximize the spotdisplacement for a given mirror tilt angle. However, as we shall see, these twogoals are in opposition.A conventional projector has one or more lenses between the output of theintegrator and the DMD. The lenses serve to relay the image of the integratoronto the DMD. We have the option of either removing these lenses entirely whenadding the AMA, or replacing them with ones that fulfill the considerationslisted above. Without a lens at all, the light will blur too quickly, given thatadequate physical separation is needed to separate the incoming and outgoingbeams from the AMA. We explain the rate of blur below. Figure 2.4 shows thetraversal of one bundle of rays as it traverses through a projector with an AMA.Chapter 2. Optical system 17Lamp/ref_lector colourwheelintegrator fold mirrorrelay lensDMDprojectionlensLamp/ref_lector colourwheelintegrator relay lensDMDprojectionlensAMAImage to screen Blurred illimination at DMDFigure 2.4: Lightpath of projector with an AMA, showing one ray bundle.2.4.1 Circle of confusionIn photography, the range of scene depths that appear in focus in an image iscalled the depth of field (DOF). In this section, we adapt some of the basiccalculations of depth of field to estimate what the shape of the beam from onemirror of the AMA will be when it reaches the DMD. Readers who would likea more in-depth overview of depth-of-field could consult (Jacobson et al. 2000;Hecht 2002; Nagahara et al. 2008). This is a first-order, paraxial analysis ofthe AMA projector system intended only to illustrate the basic relationshipsbetween the parameters of AMA size, ML size, and ML range.A lens between the AMA and the DMD should optimally relay the light atthe proper magnification onto the DMD. To determine the optical lens focallength and placement, we start with the simple lens formula (Hecht 2002)1f =1u +1v, (2.7)which gives a relationship between distances of object, image, and lens focallength, where f is the focal length of the lens, u the distance between the objectand the lens, and v the distance between the lens and the image, as shown inFigure 2.5a. If we put the AMA at the object plane u and the DMD at theimage plane v, we will get a perfectly in-focus image of the AMA on the screen,minimizing the spot size of the AMA mirror. However, since every point on theChapter 2. Optical system 18object plane is mapped to a corresponding point on the image plane, tilting themirrors will not move the light from one region to another when the AMA is infocus on the DMD. If anything, the light from a tilted mirror of the AMA willbe blocked by the aperture of this relaying lens and not make it to the DMD atall.To achieve the desired effect of redirecting light from one region to another,the AMA is placed at a distance dprime from the object plane of the lens, as shown inFigure 2.5b. By the time the light from a point on the AMA reaches a distanceu from the lens, it describes a circle c1, which in turn is imaged onto the DMDplane, forming the circle c2. This has the effect of blurring the image of theAMA at the DMD plane.Chapter 2. Optical system 19Figure 2.5: Illustration of circle of confusion. A point on the AMA spreads toa region c1, that in turn is imaged onto the DMD as the circle c2. In c), themirror is tilted by θ, causing the light cone to be shifted by 2θ. The dotted linein c) represents the shifted principle ray of the cone.Chapter 2. Optical system 20Using simple plane geometry, we can show that c1 is proportional to the lens(aperture) diameter and the separation between AMA and focal plane, and isindependent of the lens focal length. It can be calculated asc1 = Duprime|(u−uprime)| = Duprimedprime. (2.8)The diameter c2 of the circle of confusion on the image side of the lens is thensimplyc2 = |m|c1, (2.9)where m is the magnification of the lens system, obtained by the formulam = −vu. (2.10)The sign convention is that for real inverted images, m is negative, and forvirtual upright images, m is positive.The final calculation for the diameter of the circle of confusion becomesc2 = |Dmdprimeuprime|. (2.11)D is also the lens aperture, limiting the angle of incoming light from the AMA.To lower the rate of increase of blur diameter as the disparity increases, wecould reduce the aperture of the lens. However, reducing the aperture too muchwill negatively affect the system efficiency of the projector. D should be of asize that makes its numerical aperture nearly equal to that of the DMD. Thetradeoffs between illumination aperture and system efficiency are described inmore detail in Section 2.5.When an AMA mirror is tilted as shown in Figure 2.5c, the incident lightis redirected for a distance dprime before it reaches the object plane. Points on theplane at u will be imaged onto corresponding points on the plane at v, so thetilt angle of the AMA only displaces light up until dprime. The displacement dt atthe DMD plane can thus be calculated asdt = mdprimetan(2θ). (2.12)The angle of incoming light is limited by the projector’s ´etendue, and as wedescribed in Section 2.2, we can expect a beam divergence half angle of 12degrees. From Equations 2.8 and 2.12 it is evident that as we increase theseparation dprime to increase the displacement of the ML on the DMD, the circle ofconfusion also grows, so we are blurring the light from the AMA mirror.The magnification affects both the displacement and the blur. Increasingthe magnification m of the AMA on the DMD would increase dprime, but would alsoincrease the blur c2 by the same factor. Likewise, if the AMA was much biggerthan the DMD, m < 1 would decrease the blur, but also decrease dprime, and thusreduce the range of the ML on the DMD. Magnification is therefore not a usefulway of manipulating blur in this application. Figure 2.6 summarizes how theblurred image of the AMA is imaged onto the DMD.Chapter 2. Optical system 21Figure 2.6: Illustration of how blurred light from an AMA mirror reaches theDMD. The AMA tilts θ, and the resulting light is redirected by 2θ. Over dprimedistance, this moves the ML a distance dt.2.5 Luminance clipping due to AMA tiltWhen its mirrors are tilted, the AMA increases the ´etendue of the beam as awhole because the divergence angle of the beam increases. The implication isthat not all light diverted from one section of the DMD to another will reach thescreen. Some of it will be blocked by the DMD entrance pupil because althoughit would be directed to the right location spatially, some angular componentswill be outside of the acceptance cone of the DMD, and therefore will be clipped,not contributing to the added brightness of the projector. This is illustrated inFigure 2.7.To determine the extent of any clipping, we must determine what effecta tilted AMA mirror will have on a ray as it travels to the DMD. As a firstapproximation, we trace an affected ray’s path through a single thin-lens systemusing the matrix method described in (Halbach 1964). By representing a ray bya vector of its height r from the optical axis and the angle θ in radians the raysmakes with the optical axis, we can trace the ray through an optical system bymultiplying the vector with two-by-two matrices that represent discrete eventssuch as translation, or propagation through a lens surface. A simplified situationis represented in Figure 2.8, where a ray originally travelling in the direction θis deflected by a tilted AMA mirror by α, then travels through a lens with focallength f bracketleftbiggrprimeθprimebracketrightbigg=bracketleftbigg1 d20 1bracketrightbiggbracketleftbigg 1 0−1/f 1bracketrightbiggbracketleftbigg1 d10 1bracketrightbiggbracketleftbiggrθbracketrightbigg. (2.13)Equation 2.13 can be simplified tobracketleftbiggrprimeθprimebracketrightbigg=bracketleftbiggr−rd2/f +θ(d1 +d2 −d1d2/f)−r/f +θ−θd1/fbracketrightbigg. (2.14)With this formulation, the angular difference αprime resulting from diverting theinitial ray by α using the AMA is α − (αd1)/f. We can use the NewtonianChapter 2. Optical system 22Figure 2.7: a) The regular behaviour of a DMD mirror showing the incominglight cone, the cone in the on position reaching the projection lens, and theunwanted flat and off-state light. In b) some light has been diverted using theAMA, but because the cone is now not entirely directed towards the projectionpupil, not all the light makes it to the projection lens.expression for magnification (Hecht 2002)m = −f/xo, (2.15)wherexo = d1 −f (2.16)to arrive at the expression for the change in angle of a diverted ray asαprime = αm. (2.17)Equation 2.17 shows that if we begin with a large AMA, so that the magnifica-tion onto the DMD is less than 1, the angular change at the DMD is increased,leading to more clipping at the DMD aperture. A large magnification, on theother hand, adversely affects the circle of confusion as calculated in Equation2.11. The desired tilt angle of the AMA therefore becomes a compromise be-tween the flexibility of large displacements and the necessity to limit clipping.To some degree, clipping is mitigated by the non-uniform angular distribu-tion of the light source in a projector. Moving away from the central ray of theillumination cone, the light intensity decreases. Cutting off the edges of thiscone thus does not have as big an effect as if the same area was cut off in themiddle of the cone. Projector lamp research papers such as (Derra et al. 2005)Chapter 2. Optical system 23Figure 2.8: A ray of height r (solid) and angle θ is diverted by the AMA by αinclude an ´etendue vs. collected lumens curve, showing how efficient the pro-jection system will be at transferring the light from lamp to screen given otherlimiting system parameters. Heuristic descriptions of the lamp architecture canallow one to calculate the collection efficiency CE for a given ´etendue E andarc gap d. For the popular UHP (ultra-high pressure) lamp,CE = arctan( E3.8d2 +0.9d+0.8). (2.18)The curve resulting from this equation for the typical 1mm UHP arc lengthis shown in Figure 2.9. The ´etendue (20.17mm2str) of a 1024×768 pixel DMD,with dimensions of 14.08mm×10.5mm and a f/# of 2.4 (half-angle of 12◦) isalso included for reference as a dashed line. The estimated system collectionefficiency from a lamp with this ´etendue is 75%. Limiting that DMD’s f/#to 3.0, which is a half-angle of 9.5◦, would reduce the ´etendue to 12.65, butthe collection efficiency only 8%. This would allow the AMA mirrors, assuminga magnification of 1, to tilt the light in any direction up to 2.5◦ with littlelight loss. It would also substantially increase the normal contrast ratio of theprojector.2.5.1 Clipping quantifiedTo quantify the effect of a non-uniform angular light source on AMA clipping,we start by expressing ´etendue as defined in Equation 2.2 in terms of the coneaperture 2θ by holding the DMD area A constant, and the index of refractionn = 1:E(θ) = n2Apisin2θ. (2.19)Chapter 2. Optical system 240 20 40 60 80 1000102030405060708090Etendue (mm2 str)Collection efficiency (%)Figure 2.9: Collection efficiency of a 1mm arc length UHP lamp as a functionof system ´etendue. The dotted line represents the ´etendue of a 1024×768 pixelDMD.The lamp arc length d is also fixed, so Equation 2.18 can also now be expressedin terms of θ.CE(θ) = arctan( E(θ)3.8d2 +0.9d+0.8). (2.20)Here the collection efficiency CE(θ) represents the percentage light collectedwithin the cone. The light collected en in a concentric ring n isen = CE(θn)−CE(θn−1) (2.21)For a lambertian light source, no rings are needed; the entire aperture will haveuniform intensity. All others will have a different weight e associated with eachring.At any point on the DMD, the light making it through the projection aper-ture from a tilted AMA mirror has an angular distribution that intersects thatof the light originally meant for that space on the DMD, as shown in Figure2.10. To find the amount of light that makes it through the aperture, we sumthe areas of intersection of each weighted ring en and the aperture. The area Aof intersection between two circles of radius r and R separated by a distance disA(r,R,d) = r2 cos− 1d2 +r2 −R22dr +R2cos−1d2 +R2 −r22dR−1/2radicalbig(−d+r +R)(d+r−R)(d−r +R)(d+r +R)(2.22)Chapter 2. Optical system 25Figure 2.10: As the tilt angle d of the AMA increases, the angular distributionof the light at the DMD also changes. Only the light within the DMD aperturereaches the screen.(Weisstein 2007). The area of intersection Arn in a particular ring n is An −An−1. After accounting for overlapping areas, and weighting each area by er,we can determine the expected amount of clipping for a given tilt angle andlight distribution. The estimated loss as a percentage of total light incident onthe tilted AMA mirror is given in Figure 2.11.For convenience, Figure 2.11 shows losses as a function of the mechanical tiltangle of the AMA mirror. If we look at the actual change in angle of the lightα, it is twice that of the mechanical tilt, but the result is the same. However, αis modified by the magnification of the system as shown in Equation 2.17 beforeit reaches the DMD, so in a real system the x-axis will be scaled by systemmagnification.As expected, the loss reaches 100% as the AMA tilt reaches the magnitudeof the DMD tilt angle. At the lower degrees of tilt, the magnitude of loss isrelatively low, with a 10% loss at 2.5◦, a 20% loss at 3.5◦, and a 34% loss at4.5◦. Care will have to be taken to allocate mirrors with these losses in mind inorder to maximize the possible overall improvement using an AMA projector.They are taken into account when estimating the brightness improvement ofthe AMA in Chapter 4. On the other hand, in the cases where blocking light isdesirable, changing the angular distribution of the light incident on the DMDwould be a most effective way of blocking it.Chapter 2. Optical system 26Figure 2.11: Estimated losses for AMA tilts due to the DMD aperture.One major caveat is that these losses only need to occur if the light is divertedby the AMA towards the “off-state” light of the DMD. There is no reason for anaperture in any other direction (Janssen and Shimizu 1995). Dewald (Dewaldet al. 2004) obtains improved results for both contrast and brightness when theaperture is shaped to best fit the on-state light from the DMD and block the off-state and flat-state light. A ‘cat eye’ aperture shape elongated in the directionorthogonaltotheDMDtiltanglewasfoundtogivethebestcompromisebetweenlight loss and contrast increase for their system. A projector aperture speciallyfit to accommodate the increased aperture of the AMA would mitigate most ofthese clipping losses.Chapter 2. Optical system 272.5.2 ´Etendue versus contrastSo far we have described only the situation where the incoming light cone fullycovers the available ´etendue of the DMD, as in Figure 2.7a. In a real system,however, this is a worst-case scenario. Completely filling the available apertureleads to significant contrast reduction as scattering from the DMD extends theflat-state light into the on-state area, and thus becomes unwanted light thatmakes it through the projection pupil to the screen (Dewald et al. 2004).To avoid this, the on-state light can be shifted in relation to the projectionpupil by increasing the illumination angle, as shown in Figure 2.12. This directsthe scattered light farther from the pupil, thus increasing contrast. However, thea fraction of the on-state light is also cut off, asymmetrically reducing ´etendue,and thus reducing the total brightness of the display.Figure2.12: Byincreasingtheangleoftheon-statelight, theunwantedscatteredlight can be further directed away from the projection pupil, thus increasingcontrast. Some of the on-state light also is cut off, leading to a lower overallprojector brightness.The contrast ratio of the projector is significantly improved at the expenseof some overall brightness loss. In an application report (Texas Instruments2005b), Texas Instruments recommends this approach, showing that increasingthe illumination angle by 2◦ for a 10◦ DMD, so that the numerical aperture isreduced by 10%, increases the contrast of a projector by 14%, while availablelumens are decreased by only 4%. The losses are directly analogous to theclipping that occurs due to AMA tilt, discussed in Section 2.5.1. Our estimateloss from the clipping of 2◦ given in Figure 2.11 is a more pessimistic at 7%,showing that there is room for improvement in our model.Because they assume full aperture coverage, the clipping losses quantified inFigure 2.11 would be a worst-case scenario. In an actual system, the tradeoffChapter 2. Optical system 28between contrast and brightness can be optimized in interesting ways due tothe benefits to both contrast and peak brightness of an AMA projector. Sincethe AMA enhances contrast by re-directing unwanted light, there is no needto artificially constrain the illumination aperture beyond the native ´etendue ofthe DMD, allowing for higher native brightness. The asymmetric case detailedin Figure 2.12 could be used to enhance native contrast, allowing the AMA toselectively redirect this extra blocked light into the projection pupil.In summary, the key tradeoffs in pairing the AMA with light source andlight valve is between the blur kernel on one side, and the system efficiency andML range on the other. A smaller blur kernel corresponds to a combinationof smaller range and/or reduced system efficiency. The tilt angle of the AMAshould also be minimized to reduce extra system losses. In Chapter 4, weexamine how these factors affect the possible improvement achievable by theAMA in order to determine how to balance these factors.2.5.3 Distribution of light from AMA mirrorThe distribution of light energy with the circle of confusion is called the opticalpoint-spread function (PSF). The characteristics of the PSF are of interest toa number of different fields. Here we adapt work in the paper (Nagahara et al.2008), which details a method to extend the depth of field of a camera by addinga microactuator that translates the detector of the camera along the optical axis,then uses deconvolution to obtain an image with an extended depth of field. Thedeconvolution step they do requires a estimate of the PSF size and shape, whichwe adapt here to our circumstances.An idealized model for characterizing the PSF is the pillbox function is givenin (Nagahara et al. 2008) asp(r,c) = 4pic2Π(rc), (2.23)where c is the diameter of the circle of confusion, r is the distance of an imagepoint from the center of the blur circle, and Π(x) is the rectangle function, withthe value 1 if |x| < 1/2 and 0 otherwise. (Nagahara et al. 2008) approximatethe effect of aberrations by instead using a Gaussian functionp(r,b) = 2pi(gc)2e−2r2(gc)2 , (2.24)where g is a constant. The parameter g between 0 and 1 has a large effect overthe shape of the Gaussian, with smaller values giving a more narrow peak. Thisparameter should be determined through an optical simulation for a given lightsource, or empirically through optical measurements. It will be a function of thelamp and reflector characteristics, which determine the distribution of intensityas a function of deviation from the chief ray. We define the chief ray to be thecentral ray from the bundle that emanates from each point on the object. Forthe light sources we are working with, the chief ray will also be the ray withChapter 2. Optical system 29the highest intensity from its bundle. Collection systems that point more of thelight straight will have higher peaked Gaussian, while sources that have more ofa lambertian distribution, with a gradual dropoff of intensity as the deviationangle increases, will have a broader Gaussian profile of the PSF.Once we have come up with an estimate of the PSF of one spot on the AMA,we can estimate the distribution of light from one mirror in the array on theDMD, referred to here as the mobile light (ML) by convolving the area of themirror with the PSF. We can then use the ML to get an estimate of the lightdistribution from the entire AMA as it appears on the DMD.Let g(x,y) be the ML, the luminance distribution on the DMD from onemicromirror, and f(x,y) be the distribution if the ML was in focus (althoughstill with the displacement due to mirror tilt), and p(x,y) be the point spreadfunction estimated according to Equation 2.24. Theng = p∗f, (2.25)where ∗ is the convolution operator. Since convolution in the spatial domain isequivalent to multiplication in the Fourier domain, we can write Equation 2.25asG = PF, (2.26)where G(ω,ν),F(ω,ν), and P(ω,ν) are the Fourier transforms of g, f, and prespectively (Subbarao 1987). So to obtain the final light distribution from theAMA, we calculate the displacement and magnification of each AMA mirror,place each mirror’s displaced and magnified spot into an image, and then con-volve (or multiply, in the Fourier domain) the entire image by the PSF. This isthe approach taken in Chapter 4.If the DMD and AMA are parallel, and disregarding any local lens aberra-tions or apodization of the light incident to the AMA, the MLs will all be equal.If they are not parallel, the distribution of each ML will need to be calculatedseparately, because each will have a different PSF, due to the differing distancesbetween points on the AMA and points on the DMD. For the purposes of thisthesis, we assume the AMA and DMD to be parallel.30Chapter 3Micro-electromechanicalmirrorsMicroelectromechanical systems (MEMS) are small devices that are fabricatedwith many of the same processes as integrated circuits. Often starting with asilicon wafer, materials are deposited, patterned and etched in a sequence ofsteps, producing a complex three-dimensional structure (Senturia 2001). Unlikean integrated circuit, MEMS devices usually incorporate mechanical structures,some of which may be free to move. Because they use IC batch-processingtechniques, MEMS are usually relatively inexpensive to produce in bulk becausemany are fabricated in parallel.Micromirrors have had an important role throughout the history of MEMS;they were one of the first applications of MEMS technology in general (Pe-terson 1982). The primary application domains for MEMS in optics, orderedby approximate precedence, have been projection displays (Hornbeck 1997),components for optical fiber communications (sources, switches, cross-connects,routers, etc.) (Bishop et al. 2002), and optical sensing and imaging (Kim et al.2004). The optical efficiency of micromirrors can be very high, limited only bythe non-reflective gaps between the individual mirrors in an array. The ratioof optical area to overall mirror area is referred to as fill-factor. A micromirrorarray with a fill-factor of less than one will reflect less light in total than a plainmirror with equal surface quality. Losses from the fill-factor of the AMA willtake away from the brightness gains made by optical redirection. For this rea-son, structures the mirror needs to function must be built underneath or intothe mirror itself, so that adjacent mirrors can be placed as close as possible.Mircromirror arrays have been demonstrated with a fill factor of 99% (Junget al. 2006).Because of their small size, accuracy of control, and speed, micromirrors areideal for positioning the projector light. Below is a description of the principlesbehind actuating micromirrors, design issues which drove the development pro-cess of a custom MEMS micromirror array for implementation in a projector,and a description of the fabricated micromirrors.Chapter 3. Micro-electromechanical mirrors 31Figure 3.1: Parallel-plate actuation3.1 Mirror positioning using electrostaticactuationThe primary mode of actuation of micromirrors has traditionally been elec-trostatic, due to its scalability and low power consumption. Other actuationmethods include magnetic (Judy and Muller 1997) and thermal (Tuantranontet al. 2000) actuation, which are more difficult to confine, or require more powerrespectively compared to electrostatic actuation. When a voltage is applied be-tween two separated surfaces, opposite charges build up in the two surfaces.These opposite charges attract, creating an attractive electrostatic force (Sen-turia 2001). Typically the mobile surface is attached to a spring system thatprovides a restoring force when the mobile surface (the mirror) approaches thefixed surface.We can categorize the different types of micromirrors by the arrangementof springs used to separate the mirror from the substrate, and the topology ofthe electrostatic surfaces themselves. In this thesis we focus on three cases ofelectrostatic actuation: parallel-plates, torsional motion, and comb-drives, allof which have been used in micromirrors.Parallel-plate actuationIn the simplest type, parallel-plate actuation, one fixed surface provides oneelectrode, and the other, mobile surface is the mirror itself, as well as the secondelectrode, as seen in Figure 3.1. The force pulls the mirror straight down towardthe fixed electrode. This ‘piston’ movement can be used to change the phaseof incident light (Cowan et al. 1998) for adaptive optics applications. Theelectrostatic force can be expressed asFp = epsilon10AV22g2 (3.1)Chapter 3. Micro-electromechanical mirrors 32 ⊥ v+ Mirror (grounded) Hinges and springs Electrodes ϕ Figure 3.2: Micromirror actuation through parallel-plate electrostatics.where epsilon10 is the permittivity of free space (epsilon10 = 8.85pF/m for air), A is the areaof the parallel plate, V is the voltage, and g is the gap between the mirror andthe electrode (Senturia 2001). The mirror is attached to one or more springswhich resist the electrostatic attraction. If the gap is defined asg = g0 −zwhere g0 is the initial gap, and z the displacement, then the gap size isg = g0 − epsilon10AV22kpg2 (3.2)where kp is the parallel-plate spring constant. As the gap gets smaller, theelectrostatic force increases according to Equation 3.1. The critical voltage atwhich the electrostatic force becomes larger than the restoring mechanical forceis commonly referred to as the pull-in voltage. For voltage-controlled parallel-plate actuators, this corresponds to exactly g = 2/3g0, or a displacement ofone-third of the way towards the electrode. For voltages below the pull-involtage, the system can be operated at a stable point, allowing for continuousdisplacement control. For voltages above the pull-in, the mirror quickly snapsto the attracting electrode, making the last two thirds of the distance to theelectrode unusable for variable displacement.Torsional motion by electrostatic actuationFigure 3.2 shows how torsional springs suspending a one-degree of freedom mir-ror provide an axis of rotation, around which the mirror can tilt to one side orthe other, depending on which electrode is activated. In parallel-plate actua-tion as described above the electrostatic field is constant over the entire mirrorsurface. For a tilting micromirror, the gap and hence the magnitude of theelectrostatic torque changes over the mirror surface, as shown in Figure 3.3.The electrostatic torque of a tilted micromirror can be thought of as com-posed of an infinite number of infinitesimally-small parallel-plate capacitors,each with a force calculated from Equation 3.1. The torque can thus be writtenasτe = xintegraldisplay epsilon10V 22(g0 −z(x))2Ldx, (3.3)Chapter 3. Micro-electromechanical mirrors 33Figure 3.3: Electrostatic field applied to a mirror with torsional springstlwlFigure 3.4: The dimensions of a micromirror springwhere x and dx represent the position and width of the infinitesimal capacitorused for integration, g0 is the original gap between mirror and electrode, andz(x), the position-dependent displacement of the tilted mirror from its originallocation, equal to xtan(ϕ), according to Figure 3.3. L is the effective electrodelength (the length of the electrode under the mirror), into the plane of Figure3.3. See (Zhang et al. 2001) for an approximation to the numerical solution ofτe.The electrostatic torque τe causes the micromirror to rotate, which in turncauses an opposing mechanical spring torque in the beams suspending the mirrorabove the substrate. The torque Mt can be expressed in terms of the torsionalspring constant kt;Mt = ktϕ, (3.4)ϕ being the mirror angle. In the case of MEMS mirrors, the shape of the springscan often be approximated by a beam with a rectangular cross-section, with onedimension, for instance the thickness t much larger than the other dimension(width w), t >> w. In which case kt can be calculated analytically askt = 2Gwt33lparenleftbigg1− 192tpi5wtanh(piw2t )parenrightbigg,t >> w (3.5)where G is the shear modulus of the beam material, and l the length of theChapter 3. Micro-electromechanical mirrors 34torsion beam (Young and Budynas 2001), as shown in Figure 3.4.For the case of a square cross-section where t ≈ w, this can be simplified(Hibbeler 2008) tokt = t4G7.1l. (3.6)The shear modulusG = E2(1+v) (3.7)is related to the Young’s modulus of elasticity, E, by the Poisson’s ratio, v. Equi-librium is reached when the electrostatic torque equals the opposing mechanicalrestoring momentτe = Mt. (3.8)Equilibrium allows the mirror to stay tilted at a certain angle, as long as thevoltage is applied.Similar to the piston-type configuration, the range of angles that the mirrorcan reach at equilibrium is not necessarily the full range of motion available tothe mirror before itcomes inphysical contact withthe substrate layer. While themechanical torsion spring force varies linearly with deflection, the electrostaticforce is a non-linear function of deflection, and it varies with the square of thevoltageτe ∼ v2. (3.9)This means that as the applied voltage is increased, there is a point where theelectrostatic force overcomes the mechanical force of the torsion springs. Theo-retically, the pull-in angle or equivalently, snap-down point can be determinedby looking at the tilt angle as a function of applied voltage, solving Equation3.8 for V. The full derivation is not given here; see (Zhang et al. 2001), forexample, for details.Unlike in the parallel-plate case, where the pull-in is always 2/3g0, the pull-in point of this torsional system depends on the placement of the electrodes. If ais total width of the micromirror, then a1, shown in Figure 3.3 can be defined asαa, and a2 as βa. If the mirror angle when the mirror is tilted all the way to theelectrode is ϕmax, then the normalized angle can be defined as ϕ0 = ϕ/ϕmax.The solution can be calculated analytically (Zhang et al. 2001) when α = 0 asβϕ0 = 0.4404 (α = 0,0 ≤ β,θ ≤ 1). (3.10)This equation shows that the pull-in angle depends greatly on where the elec-trodes are placed. α has negligible impact on this calculation because it corre-sponds to the electrode area at the very centre of the mirror, the area wherethe electrostatic torque has the smallest effect. A small value of β will mean alarge tilt angle. This corresponds geometrically to the case where the electrodesare offset inwards from the edge of the mirror. If β ≤ 0.4404, ϕ0 = 1, and themirror will be able to be tilted over the entire course of its range until it contactsthe substrate.Chapter 3. Micro-electromechanical mirrors 35Figure 3.5: Schematic of in-plane comb driveHowever, this offset removes the electrode area that creates the largesttorque, increasing the voltage needed to drive the mirror. A value of β = 1will extend the electrodes to the edge of the mirror, and make the pull-in 0.4404of the maximum angle. As will be discussed below, we are limited to mirrorsizes small enough that we cannot easily sacrifice electrode area for tilt angle,because of limits to the maximum voltage we can apply to the mirror. We canmake the springs softer by lengthening them to reduce the required voltage, butthat adversely affects the fill-factor if the springs are fabricated in the reflectivelayer. We therefore limit ourselves to the case where β = 1.Comb drivesAnother type of electrostatic actuation for micromirrors uses comb drives, whichare interdigitated capacitors. Deflection of comb-like structures parallel to theplane of the substrate were originally introduced in 1989 in (Tang et al. 1989).Comb drives are typically arranged with one movable set of fingers, the rotor,and one stationary set, the stator. An electrostatic force between both sets ofcomb fingers engages the mobile fingers further between the stationary fingers,while the lateral distance between the fingers remains constant. Neglectingfringe fields, the electrostatic force inthe directionof movementcanbe expressedasFc = nepsilon10hd V 2, (3.11)where V is the applied voltage between stator and rotor, epsilon10 is the dielectricconstant, n the number of fingers, h the height of the comb fingers, and d thegap between fingers (Legtenberg et al. 1996), as shown in Figure 3.5.To apply comb drives to micromirrors, out-of-plane motion is required toproduce the tilt. Several strategies have been used to deflect the micromirrorsin an out-of-plane axis. Lateral actuation can be used for tilting micromirrorsby using a leverage mechanism as in (Milanovic et al. 2001; Kiang et al. 1998).Another strategy is to introduce a vertical offset between the moving fingers(rotor) and the fixed fingers (stator) for out-of-plane rotation, referred to asChapter 3. Micro-electromechanical mirrors 36a vertical comb-drive actuator (Hah et al. 2004b). Because of the increasedsurface area compared to only two flat comb fingers of the same length, combdrives create more force for the same amount of voltage. If X is the maximumdisplacement for a given drive voltage, the ratio between the force of a combdrive, and the force of an equivalently-sized parallel-plate actuator Fp isFcFp = 2g20y20 =9X22w2 , (3.12)where w is the minimum line width determined by the fabrication technologylimit (Motamedi 2005). Especially when w is small or a large X is desired,vertical comb drive actuators can produce more force for a given voltage thanparallel-plate actuation.Theoretically, if aligned perfectly comb drives do not suffer from pull-in.Practically, comb drives always have some misalignment, which causes lateralinstability, or side pull-in, which constrains motion in the same way as verti-cal pull-in (Borovic et al. 2006). A critical aspect of comb-drive design is thespacing between adjacent comb teeth (Krishnamoorthy et al. 2003) because thegenerated force is inversely proportional to the gap. The smaller the gap, how-ever, the more dramatic the effect of misalignment, which leads to instabilities.Soft suspensions and large forces, designed to achieve large traveling range, canexacerbate this problem.Self-alignment can mitigate this effect for vertical comb-drives: up to 98% ofthe theoretical maximum travel before pull-in has been reported (Krishnamoor-thy et al. 2003). Comb drives tend to increase the complexity of a micromirrordesign because they typically require many layers in order to hide the combdrives under the mirror surface to maintain adequate fill factor. Examples ofmicromirrors that use comb drives include (Jung et al. 2006) which has a highreported fill-factor of 99%, but very complex fabrication with multiple DeepReactive Ion Etching (DRIE) steps, and flip-chip bonding. Another exampleis (Tsai et al. 2008), which does not employ gimbals, but instead uses a cross-bar spring structure. The relevant characteristics for these mirrors are given inTable 3.1.3.2 Micromirror design considerationsThe design of the individual micromirrors which constitute the array must trade-off multiple competing considerations, such as mirror surface quality, or howoptically flat and reflective the mirror is, the maximum tilt angle achievable,the actuation speed, and cost. The weighting of these parameters for a par-ticular micromirror design depends on the intended application. For example,applications that require just one mirror, such as laser scanning, offer high tiltangles for single mirrors (Tsang and Parameswaran 2005). To achieve this, ex-tensive use is made of the chip area around the mirror. This makes such designsunsuitable for applications where multiple closely-packed mirrors are required,such as adaptive optics.Chapter 3. Micro-electromechanical mirrors 37When an array of micromirrors is considered, other design factors not presentin single mirror designs become important. For arrays of mirrors, design trade-offs focus on tilt angle and fill-factor, as defined in the introduction to Chapter3.Another practical constraint is the nature of the fabrication steps used tocreate the mirror design. Custom-designed processes offer the highest flexibility,but require expensive equipment in-house, and substantial development time.Using standardized multi-user MEMS processes allows for quick and inexpensiveprototyping compared to custom microfabrication processes. However, eachprocess has its own limitations, which impose additional design constraints.Micromirror arrays that have been fabricated using standard processes includemirrors that tilt like a seesaw (Hornbeck 1983), and linear arrangements ofmirrors that can tip or tilt, meaning tilt in two orthogonal directions (Tsaiet al. 2004b).To characterize the behaviour of the micromirror in motion, the micromirrorcan be modeled as a second-order mechanical system (Rao 2003; Conant 2002).The main aspects that pertain to micromirrors are briefly related here. For arotational system,τ = kϕϕ+b ˙ϕ+I ¨ϕ, (3.13)where ϕ is the mechanical mirror angle, τ the applied torque, kϕ the torsionalstiffness, b the damping constant, and Iϕ the polar moment of inertia aroundthe axis of rotation. In the ideal case, this system is linear, so in responseto a non-harmonic excitation, we can calculate and superimpose the responsesto each frequency component of the excitation waveform. At steady-state, thedynamic response to a sinusoidal torque of magnitude τ0 at frequency ωd isϕ = ϕ0 sin(ωdt+φ) (3.14)The resonant frequency ω is the frequency at which the amplitude of responseis greatest, and is given by (Conant 2002)ω =radicalBiggkϕIϕ −b22I2ϕ. (3.15)Single scanning-type micromirrors that continuously tilt over a range areusually driven at resonance. For applications such as projectors, we want staticcontrol of micromirrors, where a mirror tilt angle is specified, addressed, andheld until the next tilt angle is desired. The dynamic properties of the mirror arestill important, however, because they govern the time it takes between whenthe signal is given and the mirror settles at its desired location. The higher theresonance frequency of the mirror, the faster it can be positioned.We can approximate the shape of the mirror as a solid block in order tosimplify the calculation of its moment of inertia Iϕ. A block of length l1, widthw1 and thickness t1, rotating around the length axis can be approximated asIϕ = 112m(w21 +t21). (3.16)Chapter 3. Micro-electromechanical mirrors 383.3 Previous micromirror designsThe most successful optical MEMS component to date is the Digital Micromir-ror Device (DMD), used in Digital Light Processing (DLP) projectors (Kesselet al. 1998), invented in 1987 and used for projectors since 1996. In most con-figurations, there is a one-to-one correspondence between pixels on the displayand micromirrors on the DMD. The mirrors on a DMD are made of aluminum,and are approximately 16 µm square. Electrostatic actuation is used to rotatethe mirror from -12◦ to +12◦. These extremes are the only two addressable an-gles, so greyscale is achieved through pulse-width-modulation (PWM) – shiftingbetween the on and off states at high rates, the ratio of which determines thebrightness level.The DMD chip cannot be used as an AMA in this application because ithas no intermediate positions between its two discrete states. However, bothbefore and after the DMD was invented there has been a substantial amount ofresearch into variable-angle micromirrors for use in optical switching and otherapplications (Bishop et al. 2002; Tsai et al. 2004b; Dutta et al. 2000; Dokmeciet al. 2004; Taylor et al. 2004).Variable-angle mirrors have many other applications besides projectors. Forinstance, inthetelecommunicationindustry, transparentswitchingsystemsareacandidate to replace expensive high speed signal regeneration opto-electronics.All-optical switching requires micromirror arrays to steer optical beams fromone input port to any output port with little propagation loss. The passivenature of these systems permits routing of optical signals independent of theirwavelength, modulation, and polarization. One of the earliest serious venturesusing scanning mirrors in the networking industry is the now-discontinued Lu-cent LambdaRouter (Bishop et al. 2002), also described in (Aksyuk et al. 2003).Such an array is not practical as an AMA because of the small fill-factor of thedesign (estimated at 53%), based on the need to only reflect multiple narrowfibre-optic beams.Limiting previous work to mirror designs with a high fill-factor significantlyreduces the field, underscoring the difficulty in designing a mirror with inter-esting mechanical properties that can still be tightly packed. Many currentnovel mirror designs feature exterior actuators and springs that dwarf the mir-ror itself, such as (Wang et al. 2003). While these types of mirrors illustrateinteresting actuation and control ideas, they are impractical for use in an arrayof micromirrors.Some mirror arrays have a high fill-factor in one direction only, such as(Taylor et al. 2004; Tsai et al. 2004a; Hah et al. 2004a). The mirrors in theseconfigurations can be stacked tightly in one dimension, but extended compo-nents to the sides of the mirrors prevent them being stacked tightly in twodimensions.The Thin-film Micromirror Array (TMA) developed by Daewoo, intended tobe a competitor to the DMD, is one design that achieves a fill-factor of over 90%(Hwang et al. 1998). Thin-film piezoelectric actuators are used to obtain linearcontrol over each mirror in only one tilt direction. The angle of tilt determinesChapter 3. Micro-electromechanical mirrors 39the grey level, making PWM as is done with the DMD unnecessary. The TMAdesign seems to have been shelved, and is not commercially available.(Tuantranont et al. 1999) attempt to increase the fill-factor of a micromirrorarray by using a lenslet array to collect the light from the inactive portions of themicromirrors. The authors report that these mirrors significantly improve thefar-fielddiffractionpattern. However, whilethelensletarraycanincreasethefill-factor, it does have other adverse effects on the overall system performance, suchas introducing a limited depth of focus, chromatic aberration and distortion ofthe image. It also affects the complexity of the system; lenslets must be alignedto the mirrors, which increases the cost of assembling and adjusting the system.Many mirror designs in the literature use common MEMS fabrication pro-cesses available from foundries. The Polysilicon Multi-User MEMS Process(PolyMUMPs) is probably the most-used and cited foundry process in the mi-cromirror literature. It is a surface micromachining process that uses two struc-tural polysilicon layers and two sacrificial phosphosilicate glass layers, as de-tailed in Section 3.5.1. One important property of the MUMPs process that isparticularly important for mirror design is that it is a conformal process; fea-tures of underlying layers can be clearly seen though the topmost layers. Thoseparticularly concerned with optical properties must take this effect, also knownas “print-through” into account when considering where to place the electrodesthat will drive them.(Comtois et al. 1995) designed a hexagonal micromirror for the MUMPs pro-cess, with mirrors 50 µm across, and arranged with 75 µm centre-to-centre spac-ing between nearest micromirrors. The mirrors had a piston mode for adaptiveoptics applications such as wavefront correction, with one big electrode under-neath that pulls the entire mirror down. A later paper (Comtois et al. 1999)lists many of the considerations a MEMS designer wishing to maximize fill-factorhas to deal with, some of them specific to MUMPs but most generalizable to allsurface-micromachining processes.Sandia Lab’s SUMMiT-V is another surface micromachining multi-user pro-cess. Besides having two more polysilicon layers than MUMPs (5 instead of3), the SUMMiT-V also employs chemical-mechanical polishing (CMP), whichallows for, among other things, flatter surface features because the polishingreduces the effect of “print through.” (Cowan et al. 1998) provide a comparisonof fill-factor achieved on various mirror designs using the MUMPs and SandiaSUMMiT-V process. The paper found that the topography of the mirrors is atleast as important to the overall reflectance properties of a micromirror arrayas the fill-factor. A 2D micromirror array with a relatively high fill-factor builtusing the SUMMiT-V process is detailed in (Tsai et al. 2004b). The authorsclaim relatively large continuous scan angles (±4◦ and ±3.4◦ degrees for thetwo orthogonal degrees of freedom), with a fill-factor of 96%. To achieve this,the torsional springs are placed under the mirrors. The electrodes are terracedupwards towards the middle of the mirror, ostensibly to reduce the actuationvoltage. The terraced electrodes are made from the bottom four polysiliconlayers; the top layer is reserved for the mirror. Maximum scan angle is achievedat over 90V, which is relatively high compared to many other designs.Chapter 3. Micro-electromechanical mirrors 40A variation on (Tsai et al. 2004b) by the same authors uses a ring of combdrive sections (Tsai et al. 2008), along with a cross-bar arrangement of torsionalsprings to achieve 2DOF actuation without gimbals. The springs and combdrives are all hidden below the mirror to maintain fill-factor. They achieve tiltangles of ±5.4◦ at 42 V, and ±2.3◦ at 61 V for rotations about the x and y axis,respectively.(Dagel et al. 2006) describes a hexagonal tip/tilt/piston mirror array with anarray fill-factor of 95%. The micromirrors tilt using a novel leverage mechanismunderneath the mirror. The base of the lever is attracted to a electrode onthe substrate, pulling the portion of the lever on the other side of a torsionalspring upwards. Bumper features on the substrate keep the leverage mechanismoperating as the torsional springs bend at high voltages by acting as a pivot.Many tip/tilt mirror systems use gimbals to suspend the mirrors, such as(Bishop et al. 2002; Lin et al. 2001; Wen et al. 2004). Usually, a frame surroundsthe mirror and is attached to it by two torsional springs, forming an axis ofrotation. The frame itself is then attached to the surrounding material by twosprings in orthogonal directions, allowing the mirror to tip and tilt on torsionsprings.Although the basic structure of the 2-DOF mirror using gimbals is simple,there has been a fair amount of literature on the optimal design of these mirrors,such as the placement of the electrodes underneath the mirror/gimbal structure.Most gimbal designs in the literature use four electrodes, some of which havepairs of electrodes separated, with two placed under the mirror and another twounder the frame. Placing the electrodes under the frame means that the framehas to be relatively thick, which negatively affects fill-factor.Table 3.1: Published characteristics for 2-DOF micromirrorsDesign Tilt X Tilt Y process fill-factor mirror size(Tsai et al. 2008) ±5.4◦ ±2.3◦ SUMMiT V 96% (est) 96µm(Jung et al. 2006) ±1.8◦ 0.2◦ custom 99% 360µm(Dagel et al. 2006) ±2.5◦ ±2.5◦ SUMMiT V 98% (est) 500µm(Aksyuk et al. 2003) ±4.4 ±4.4 custom 53% (est) 600µmTable 3.1 shows selected micromirrors reported in the literature. Only mirrorconfigurations that could be put in a 2D array are listed.3.4 Mirror designs for large-angle deflectionDue to the complexity of MEMS micromirror fabrication and the limited fa-cilities available at UBC, we limited ourselves to designs that could be fabri-cated using publicly-available established multi-user MEMS processes. We chosetwo processes available through the CMC Microsystems (CMC 2009), a Cana-dian organization that provides microsystems resources to researchers at manyCanadian academic institutions. As outlined below, the two fabrication pro-cesses used in this project were the surface micromachining process PolyMUMPsChapter 3. Micro-electromechanical mirrors 41(Koester et al. 2003), offered by the company MEMSCAP, and Micragem (CMC2005), a Silicon-on-Insulator based process offered by the company Micralynethrough CMC Microsystems until 2008.Both of these fabrication processes have a limited maximum gap betweenthe mirror layer and the substrate. Along with the mirror size, this gap limitsthe maximum tilt angle of the mirror. Using multiple smaller mirrors in placeof larger mirrors allows for greater angles, as shown in Figure 3.6.Figure 3.6: Concept of increased deflection angle through composite mirrordesignA contribution in this thesis is therefore to subdivide each mirror in thearray into several smaller mirrors, and address all of the smaller mirrors withthe same leads. The subdivided mirrors become composite mirrors, each treatedas one mirror in the larger array. This increases the scan angle compared to asingle mirror with the same surface area, while maintaining the gap size which isgiven by the process. It also increases the resonance frequency when comparedto the single mirror case; see Equation 3.15.3.5 PolyMUMPs micromirrors3.5.1 PolyMUMPs micromirror designFabrication of the first prototype set of composite micromirror arrays was doneusing the Polysilicon Multi-user MEMS process PolyMUMPs (Koester et al.2003). PolyMUMPs is a surface micromachining process where successive thinlayers of structural and sacrificial material are deposited, lithographically pat-terned, and etched to form a 3D structure. Polysilicon is used as a structuralmaterial, and thin films of phosphosilicate glass as sacrificial layers. There are3 polysilicon layers in total, the first of which (Poly0), is not releasable fromthe silicon wafer on which it is deposited. A thin, non-structural metal layer isdeposited last for surface reflectance, and to provide high conductivity.For this micromirror design, which is also described in (Hoskinson et al.2007a) all of the electrodes and leads were formed from the Poly0 layer, whichallows for 3µm features. The releasable Poly1 layer between Poly0 and Poly2was not used in order to increase the gap between the mirror (Poly2) and theelectrodes (Poly0). The Poly2 (1.5µm thick) and Metal layers (0.5µm thick)Chapter 3. Micro-electromechanical mirrors 42are used for the mirror surface. 2.75µm, the combined height of the two oxidelayers between Poly0 and Poly2 defines the gap between the mirror and the 3polysilicon electrodes. An illustration of the cross-section of one mirror is shownin Figure 3.7, and the electrodes are depicted in Figure 3.8. Three electrodeswere used as that is the minimum number that still provides two degrees offreedom of tilt angle, and also minimizes the number of electrical connectionsneeded for each mirror.Figure 3.7: PolyMUMPs micromirror, cross-section. Both oxide layers are re-moved in the release step, allowing the micromirror to tilt.To ensure the best possibility that designs made with the MUMPs processare successfully fabricated, there are a number of design rules, given in (Koesteret al. 2003), which set minimum features sizes and minimum overlap betweenfeatures on different layers. These rules have profound consequences on mirrordesign choices. For instance, the design rules imply that vertical structures haveto be very stiff, so flexible spring elements have to be fabricated in horizontallayers. In this design, a central post is used to support the mirror. In order todecrease spring stiffness, the springs are cut into the mirror surface, emanatingfrom the centre anchor. A model of the top of the mirror is shown in Figure3.9.The mirror surface is electrically grounded through a connection to the sus-pending posts, facilitating electrostatic actuation. In the micromirror arraydesign, 62 hexagonal mirrors, each 100µm across their maximum width, consti-tute one composite mirror.Chapter 3. Micro-electromechanical mirrors 43Figure 3.8: An illustration of the electrodes (blue) under a part of one sectionof a composite mirror. One mirror’s outline is shown in black. All electrodesthat cause the mirrors to tilt in the same direction are connected together. Thecentral portion under the mirror connects the mirror surface to ground throughthe central post.Figure 3.9: PolyMUMPs micromirror, top view of modelChapter 3. Micro-electromechanical mirrors 443.5.2 PolyMUMPs mirror characterizationFigure 3.10 shows a photograph through a microscope of several mirrors fabri-cated with the PolyMUMPs process. Because the polysilicon is conformal (eachsuccessive layer follows the vertical contours of those that have been appliedbefore it), the electrodes and leads providing voltage are visible. Etch holes inthe mirror are also visible. Etch holes are required when using the PolyMUMPsprocess for proper release of large polysilicon structures such as these mirrors.Figure 3.11 shows a one composite mirror and the surrounding bond pads.Figure 3.10: Scanning-electronic microscope image of PolyMUMPs mirrors.Some mirrors have been removed to show underlying electrode structure.Figure 3.12 shows the static deflection angle as a function of voltage frommeasurements using a white-light interferometer. The maximum angle observedbefore pull-in was 0.75◦. Pull-in was observed at 87.3 volts. This deflection an-gle compares favourably to other tip/tilt micromirrors fabricated via the Poly-MUMPs process such as the 12 mrad (.00021◦) achieved in (Lin et al. 2001) fora 130µm square mirror.Resonance frequencies for the mirrors were measured using a Laser DopplerVibrometer in two ways. The first involved a frequency sweep in which thevelocity amplitude of the mirror’s response to a sinusoidal 20V peak-to-peakvoltage input, in addition to DC bias voltage of 40V, was measured at discretefrequency points. Dividing the velocity amplitude measured at each point byits input frequency gives the displacement amplitude. The second method wasto excite the mirror with a step input, and again measure the resulting veloc-ity. The Fast-Fourier Transform (FFT) of this response gives the displacementspectrum. Figure 3.13 shows that the results from both measurements matchwell, and both show the resonance frequency at approximately 95 kHz. Similarresults have been achieved for different bias voltages.Chapter 3. Micro-electromechanical mirrors 45Figure 3.11: Photograph of composite mirror using the PolyMUMPs process.The bond pads used to electrically connect the wire bonds to the mirrors arevisible in the top and right sides of the picture.Figure 3.14 shows the surface characteristics of one fabricated mirror frommeasurements taken with a white-light interferometer. This data shows the“print through” effect from the underlying lines and electrodes. The surfacecurvature due to residual stress from the metallization is also evident. Theradius of curvature of this mirror was measured at 2.8mm by plotting a cross-sectional slice of the interferometer data.While the print-through is unavoidable with the PolyMUMPs process, thesurface curvature could have been largely avoided. The final metallization stepof the MUMPs process has been shown to be the primary cause of surfacecurvature. We chose to use the optional metal layer on these mirrors despitethat because (Cowan et al. 1998) have shown that it nearly doubles the opticalefficiency of the mirror compared to the case without any metallization.With metallization, the curvature can be reduced by combining the Poly1and Poly2 layers to provide a 3.5µm mirror layer rather than the 1.5µm Poly2layer alone. The increased thickness of the two combined polysilicon layerswould have substantially stiffened the mirror against curvature. However, thisalsowouldhavereducedtheelectrostaticactuationgapsizeto2µmfrom2.75µm,which would in turn reduce the maximum tilt angles of the mirrors. We there-fore decided to only use the Poly2 layer to gain the added tilt angle, at the costof increased mirror curvature.In retrospect, the design for this mirror array was perhaps too ambitious fora first iteration. To provide the three separate voltages to the mirror electrodesfor the 62 mirrors within one composite mirror, conductor bridges had to bedesigned across other conductors. With 25 composite mirrors, there were 1550Chapter 3. Micro-electromechanical mirrors 46Figure 3.12: Static mirror deflection of PolyMUMPs mirror for a constant volt-age supply to one electrode.mirrors in total on the array, and a lot of potential for errors by the designer,leading to false connections. As a consequence, not all composite mirrors couldbe controlled independently, and some sections of the composite mirrors did notmove with the other sections.Chapter 3. Micro-electromechanical mirrors 47Figure 3.13: Two measures of displacement amplitude vs. frequency for Poly-MUMPs mirror (Hoskinson et al. 2007a). Using a 40V bias voltage signal, a20V peak-to-peak AC voltage amplitude was used for the sweep, and 20V wasalso used for the step excitation.Figure 3.14: One MUMPs mirror as measured by a white-light interferometer.The radius of curvature was measured to be 2.8mm.Chapter 3. Micro-electromechanical mirrors 483.6 Micragem mirror arrays3.6.1 Micragem mirror designsIn the Micragem process, 10 or 12µm deep cavities in glass carry structuredmetallization used for electrodes, leads and bond pads. The metal can be placedin the cavities or on the glass surface. A Silicon-on-Insulator (SOI) wafer is thenanodically bonded to the glass, and etched so that only a 10µm thick SingleCrystal Silicon (SCSi) layer is left. Another metal layer is then deposited andpatterned on the SCSi surface, after which the SCSi is patterned through deepreactive ion etching (DRIE).Figure 3.15: Picture of 4 micromirrors, forming one pixel of an early designmicromirror array made in using the Micragem processThe constraints of the Micragem process design rules dictated many of thechoices made in our design process. There is only one mechanical layer, so allof the hinges and frames supporting the mirror have to be incorporated intothe same level as the mirror surface. All of these necessary support structuresdirectly subtract from the reflective mirror surface, and therefore directly impactthe overall fill-factor in the array.The micromirrors we have designed using the Micragem process employ athin gimbal system, which suspends square mirrors over the 12µm cavity. Aswith the MUMPs design, multiple mirrors are controlled simultaneously to formone AMA composite mirror by linking their electrodes. We chose to take ad-Chapter 3. Micro-electromechanical mirrors 49Figure 3.16: Several micromirrors with a portion of the thin gimbal frame sys-tem. Each frame suspends six square mirrors over a 12µm cavity. Each mirrorcan be tilted around the two indicated orthogonal axes x and y. Six rows of sixmirrors (36 total) are controlled together to form a composite mirror.vantage of this during the design of the micromirrors themselves. In our design,each row of mirrors within the same composite mirror shares the same gimbalframe, which improves fill-factor and promotes homogeneity between submir-ror deflections. An early design, shown in Figure 3.15, has two mirrors per row,while later designs had 5 or 6, a portion of which is shown in Figure 3.16. Multi-ple mirrors within one frame minimizes the ratio between frame area and mirrorarea. Instead of the loss of reflective area from four frame sides and two framehinges per mirror, the loss from the frame area and outer hinges are amortizedover a larger number of mirrors. The two gimbal springs and two of the foursides are only needed once per group rather than once per mirror. Figure 3.17shows a photograph of several of the composite mirrors in the array.The other main consideration affected by the design rules was the differencein minimum feature sizes between the SCSi layer used for the mirror surfaceand the metal layer used for the electrodes. In the SCSi layer, the minimumfeature size (both length and width) and dark size (cut) is 2µm. For the metallayer used for the electrodes, both the minimum feature and dark size is 10µm.This had a substantial impact on our array design, because wires 10µm thickand 10µm apart had to connect each electrode and its corresponding bond padon the outside of the array. The maximum vertical gap distance of the process(12µm) meant that the mirrors had to be relatively small to achieve enough tiltChapter 3. Micro-electromechanical mirrors 50Figure 3.17: Array of composite mirrors in Micragem row designangle to move the light by a sufficient amount. There was also a practical limitas to the number (84) of bond pads available in a convenient package for theprototype array.Placing a row of mirrors within a gimbal frame opened up design possibilitiesfor the electrode arrangement. Electrodes could be routed in daisy-chain formwithin the rows. The minimum number of electrodes needed to achieve two-degree of freedom actuation is 3. If the electrodes are structured in a way thatallows them to be connected in rows, the electrode paths can be minimized.With one wire between electrodes, it is also possible to connect rows togetherin a compact manner, as shown in Figure 3.18. This results in a square orrectangular array of mirrors in a composite mirror. Each row of mirrors is inone frame. Each frame also constitutes a row in the electrode layer.Figure 3.18: Two rows of electrodes, showing row-wise connection, and how tworows are connected to each other. The outlines of the mirror features are alsoshown.Chapter 3. Micro-electromechanical mirrors 513.6.2 Simulations of Micragem mirrorsCoupled electrostatic-mechanical simulations were undertaken to predict andanalyze the behaviour of the micromirrors using the commercial finite-elementmodelling (FEM) software ANSYSTM(ANSYS 2009). The EMTGEN macro inANSYS was used to generate TRANS126 elements between the nodes on thebottom surface of the micromirror and the electrodes. TRANS126 elements areuseful for simulating electrostatic-mechanical coupling between a MEMS deviceand a plane, and are valid if the gap between the mirror and the electrode issmall compared to the dimensions of the mirror, as is the case with this design.Figure 3.19 shows the result of a simulation of one row of micromirrors thatwe have designed, under a potential of 120V applied to the middle electrodeC underneath each mirror. The simulation predicted a maximum negative andpositive deflection of −3.1µm and 1.65µm, respectively.Figure 3.19: Model of micromirror produced in ANSYS.Figure 3.19 shows that the mirror frame itself bends towards the electrodesas electrostatic force is applied. The longer the frames are for a given width,the more they are able to bend. When the frame bends, the mirrors closestto the middle of the frame become closer to the electrodes underneath. Theytherefore are subject to higher electrostatic forces than those towards the edges.The mirrors in the middle of the frame therefore tilt more for a given tilt voltagethan those on the edges.This effect is not symmetric, because half of the mirrors in the row areworking with the bend and half working against it. The effect is to tilt themirrors slightly towards the centre of the frame. When a voltage is applied, theflat mirrors in Figure 3.20 a) bend at different rates depending on their distancefrom the electrode and what side of the centre of the row they are on, as shownin Figure 3.20 b).Chapter 3. Micro-electromechanical mirrors 52Figure 3.20: (a) The gimbal frame holding a row of mirrors can bend. (b)This causes an applied electrostatic field to be unequally converted into force,depending on the initial orientation of the mirror due to frame bendingThe bending of the frame could be mitigated by making the frames wider,or by putting fewer mirrors within one frame, and thus making the framesshorter. A more compliant spring on the frames also be of help to rotate therow around the y-axis. Frame bending does introduce some interesting sideeffects. When the mirror tilts about the x axis, the effect on the reflected lightbeam is an asymmetrical compression. This effect is controllable in the sensethat an additional bias can be applied to all 3 electrodes to exaggerate thecompression. The asymmetrical nature of the compression is not controllable,because the total electrode size is fixed for each individual mirror.The ANSYS simulation results for x-axis rotation as well as experimentalresults are shown in Figure 3.21. The experimental results were obtained usinga white-light interferometer.When we compare Figure 3.21 a and b, we see that the simulated mirrorsactuate at smaller voltages than the measured values. This could be due toan overestimation in the simulation of the amount of over-etch. The Micragemdesign rules (CMC 2005) specify a minimum feature size for the SOI layer as2µm, and a undercut of±1µm, which can significantly affect the spring constantof the micromirror’s torsional springs. Depending on the rate of variation (whichis not specified in the design rules), this can have a significant impact on theperformance of the composite mirrors, which depend on having every mirrorwithinthe grouphavingverysimilarperformance. Figure 3.22showstwospringsimaged with a scanning electron microscope, and shows that even within oneChapter 3. Micro-electromechanical mirrors 53torsional spring, the widths of the individual serpentine coils can change. Itis evident the that the springs were assumed to have smaller widths than theydid in actuality, and were thus more stiff than those in the simulation. Thus inthe simulation, less voltage was needed to produce a given deflection than wasmeasured with the fabricated mirrors.Figure 3.23 shows a fabricated mirror under an applied voltage as recon-structed by the data from a WYKO white-light interferometer. It is evidentthat the individual mirrors of this array show a slightly different tilt, particu-larly Mirror 5.Chapter 3. Micro-electromechanical mirrors 54Figure 3.21: Simulated (a) and measured (b) individual mirror tilt around Xaxis. The mirror numbers correspond to the labels in Figure 3.19.Chapter 3. Micro-electromechanical mirrors 55Figure 3.22: A scanning-electron microscope picture of micromirror springs,showing the differences in spring width.Figure 3.23: Image from WYKO white light interferometer of one compositemirror tilting.Chapter 3. Micro-electromechanical mirrors 563.7 Proposed new Micragem designA future mirror design better suited for this application could use the centrespring concept of the PolyMUMPs mirrors with the flat surfaces achievable withthe Micragem process. To illustrate this, we simulated a new mirror designusing the Micragem process. Again we chose to use three electrodes to providethree degrees of freedom. Minimizing the number of electrodes is especiallyimportant for the Micragemg process because of the relatively large minimumfeature size of 10 µm in the electrode layer, compared to 2µm in the SCSilayer. Equally important, a hexagon can be regularly tiled, fit together withoutany spaces. Similarly to Figure 3.18, an unlimited number of each of the threetypes of electrodes can be connected together to form composite mirrors withoutthe need for bridges. The middle electrodes for each of the composite mirrorsare connected in a row, and rows are connected via loop between rows. Theother two electrodes are also connected into rows, and these rows are connectedtogether at the ends.Figure 3.25 shows the mirror shape and the shape of the three equal-sizedelectrodes underneath. The mirror is attached by springs to a central 10×10µmpost that suspends the mirror above the etched glass layer and electrodes. Thesprings are designed to provide nearly equal spring force in every direction. Eachexterior side of the hexagon is 125 µm. From any corner to its opposite corner,the distance is 250 µm. The minimum distance from centre to edge is 125√3,or 216.5µm.This design has an estimated fill factor of 92%. The calculated tilt anglesfor the four figures are as follows: Figure 3.25: 2.09◦ for electrode A at 180V,Figure 3.26: 1.91◦ for electrode B at 180V, Figure 3.27: 2.08◦ for electrodes Aand B at 200V, and Figure 3.28: 1.64◦ for electrode A at 170V, and electrodeB at 190V. These are highest tilt angles achieved before the mirrors reachedpull-in during the simulation.Unlike the previous Micragem design, the mirrors do not need mechanicalsupport from the rest of the SCSi surface, but they still do need a commonelectrical connection to provide a voltage signal that opposes that of the elec-trodes underneath. Because of the smaller feature size of the SCSi surface, wechose to include this connection as fixed rods in the mirror layer. One such arod is shown in Figure 3.26. It is connected to the central post, so it remainsstationary and is under no force as the mirror is actuated. As shown in thesimulations, the small cut in the mirror surface this introduces does not causethe mirror to bend in an uneven manner. In a larger array, these rods could beconnected to small connecting spaces between mirrors, as illustrated in Figure3.24.Unfortunately, the Micragem process used to fabricate these mirrors is nolonger being offered as a public service by neither Micralyne nor CMC, so theprocess must be replicated elsewhere before these mirrors can be fabricated.Chapter 3. Micro-electromechanical mirrors 57Figure 3.24: Illustration of mirrors electrically connected together by fixed rodsanchored to the central posts of the mirrors.Figure 3.25: Electromechanical simulation of micromirror with electrode A ac-tuated with 180V. The three electrodes underneath are also indicated.Chapter 3. Micro-electromechanical mirrors 58Figure 3.26: Simulation of micromirror with electrode B at 180V.Figure 3.27: Simulation of micromirror with electrodes A and B at 200V.Chapter 3. Micro-electromechanical mirrors 59Figure 3.28: Simulation of micromirror: electrode A at 170V, and electrode Bat 190VChapter 3. Micro-electromechanical mirrors 603.8 Future mirror designsThe requirement that the mirrors must be able to be fabricated using an ex-isting multi-user process severely limited the design space. In the PolyMUMPsdesigns, the mirror surfaces had a large radius of curvature and had their sur-faces marred by the “print through” effect from the underlying springs. Thelimited depth of the PolyMUMPs process of 2.75µm maximum gap height widthalso limited the achievable tilt angle of the mirrors. For a tilt angle of ±4◦, themirror would have to be less than 30µm in length, and would thus require veryhigh voltages to actuate.The mirrors fabricated with the Micragem process had optically flat surfaces,but the constraints on the minimum feature size of the electrode layer had largerepercussions on the fill-factor of the array. While the row design mitigatedthis issue somewhat, it introduced a new problem of frame bending, creatingnon-uniform tilt angles. This makes the relationship between angle and voltagehighly non-linear, and very complex to account for in the allocation algorithm.The shape of the ML also depends on the tilt angle as the frame bends, addingfurther complexity. While it may be possible to use the non-uniform tilt advan-tageously to shape the light to the characteristics of the image, the shape of thelight would depend on the voltage given to each electrode under the mirror.A future mirror design better suited for this application could use the cen-tre spring concept of the PolyMUMPs mirrors with the flat surfaces achievablewith the Micragem process. An attempt at this was made, but due to issueswith fabrication the micromirror array was inoperative. Unfortunately, the Mi-cragem process used to fabricate these mirrors is no longer being offered as apublic service by neither Micralyne nor CMC. A new process, possibly Sandia’sSUMMiTV (Sandia National Laboratories 2007), will have to be used for thenext iteration of mirrors.If the Sandia process is used, a design similar to that used in (Tsai et al.2008) would provide more than adequate tilt range for the AMA device. The fill-factor of these mirrors is also high at 96%, which is crucial for this application.It could also be tiled into composite mirrors depending on the desired size ofthe array. The design would have to be adapted for this application, as despiteits theoretically high fill-factor, it has not been demonstrated in a large array,and would also need to be adapted for use in composite mirrors.61Chapter 4Light allocationThis chapter describes how the mirrors are allocated for a given image, how thefinal image is formed, and provides an evaluation of how much extra brightnesscan be coaxed from a projector through the addition of an AMA.In Chapter 2, we examined the implications of adding and AMA device intoa projector’s optical path in terms of optics: the shape and distribution of thelight reflecting from each AMA onto the DMD. In this chapter, we estimate thepotential improvement to an image displayed on the projector with the additionof the AMA. To do that, we need to convert the optical specification of the MLand its range into one expressed in terms of image pixels. Section 2.5.3 gave adescription of the convolution kernel that can be used to estimate the shape ofthe ML for a given simple physical arrangement of AMA and DMD with lenses.This is used in Section 4.3 to calculate the ML shape in pixel coordinates.In this chapter, low-dynamic range (LDR) images with formats that aredevice-dependent are primarily used as examples. These image formats specifyrelative luminance changes on the device used to reproduce the image, ratherthan the absolute, scene-referred luminances of the original scene. They are alsogamma-encoded so that the relative luminance changes represented by the pixelvalues are not linear.4.1 Mobile light sourcesA key aspect of improving dynamic range by using two modulators as describedin (Seetzen et al. 2004) is that one of them can be of much lower resolution thanthe final displayed image. The light distribution of the low-resolution modulatoris then corrected by the high-resolution modulator to achieve the desired image.The light sources in the low-resolution modulator (LRM) are in fixed locations,irrespective of where the bright parts of the image are. If the locations of theLRM light sources and the bright parts of the image coincide, the full capabilityof this concept can be exploited. If they do not, the closest neighboring lightsources must be used. Because the intensities of the light sources fall off overdistance, they will have to be set to higher intensities than in the matchedcase. Also, more light has to be blocked, because the neighbours will now beputting out more light than the needs of their locations. As an example, takethe LED modulator detailed in (Seetzen et al. 2004). A bright spot in the imagemay occur in between two adjacent LEDs. In that case, the neighboring LEDsneed to provide higher light intensities than the local maximum of the image.Chapter 4. Light allocation 62The excess light from these LEDs would then have to be blocked by the high-resolution LCD. This would result in a lower contrast and higher power usagethan would be possible if the LEDs could be positioned exactly where they areneeded for each image.In an AMA approach, the intensity of light of each mobile light source (ML)can not be changed, but the location on the high-resolution light valve is vari-able. This means that every ML can be targeted to exactly where it is needed.The combination of all of the virtual light sources must be sufficient to cor-rectly display the image. The allocation scheme should also take advantage ofthe flexibility afforded by these MLs to the peak brightness of the image.Similar to the approach by (Seetzen et al. 2004), the signals sent to the low-resolution modulator are calculated first, then the light distribution from thefirst modulator on the second modulator is simulated, and finally, the correctedimage sent to the high-resolution second light modulator is calculated. Lightfrom an ML can be described by convolution of a rectangular area of pixels witha point spread function on the DMD, as detailed in Chapter 2.4.2 GammaSome discussion on how typical images are specified is needed so that we canmatch the representation of the MLs to the representation of luminance valuesin an LDR image. Pixels in most standard image formats such as HD video(Poynton 2003) are encoded with a non-linear gamma function. Gamma is therate of change of the relative luminance of image elements of a reproduction asa function of the relative luminance of the same image elements of the originalimage (Fairchild 1995).Video transmissions were first encoded with a gamma to compensate for thenon-linear characteristics of cathode-ray tube (CRT) displays. CRT electronguns typically have a current/drive voltage relationship described by a powerlawI = kV γd, (4.1)where I is the current from the electron gun, proportional to the light intensity,k is a constant, V is the drive voltage, and γd is the gamma of the electron gun(Poynton 2003).To achieve a linear lumen output, transmissions such as those specified bythe National Television System Committee (NTSC) and later the sRGB formatfor digital images (Stokes et al. 1996) were encoded with the inverse of γd.γd = 2.2 has been used as a benchmark for the behaviour of a typical CRT, sothe signal gamma, γs, was specified as 1/2.2. However, actual direct-view CRTshave a γd in the range from 2.31 to as high as 3.1, so γsγd = 1.27, instead of the‘ideal’ value of 1 for linear imaging. Tests show consumers prefer a nonlineargamma (Roufs et al. 1994; Barten 1996). This may be due to the influencethe surrounding luminance has on a viewer’s perceived contrast of an image(Fairchild 1995).Chapter 4. Light allocation 63The exponent γd is different for different monitors. Displays with other rela-tionships between input signals and luminance such as LCDs typically includecircuitry to do inverse gamma correction as a pre-processing step. (Kessel et al.1998) describes how the DMD is a linear luminance modulator, and so its drivermust include inverse gamma pre-processing.The method described in Chapter 2 to estimate the distribution of light onthe DMD from the AMA is in normalized, relative, linear units. An inversegamma step is therefore performed on the incoming image so that it is in anapproximately linear, relative luminance scale. Because the second modula-tor, such as a DMD, usually assumes a standard gamma, the corrected high-resolution signal sent to the DMD should be gamma re-encoded in anticipationof another inverse gamma step happening automatically in the DMD driverhardware. The AMA manipulates light in a linear manner, and so has a gammaof 1. If a non-linear gamma in the output such as 1.27 is preferred, this wouldbe taken into account in the original image before it is decomposed into the firstand second modulator components.4.3 Mirror allocationFinding the optimal light distribution for the AMA corresponds to choosingthe locations for each of the n MLs from the array of n analog micromirrors.The procedure for exploring the solution space is as follows. The approximationstarts with a representation of the light incident on the DMD, coming from theentire AMA, as a block of pixels in DMD-space. The size of the AMA lightrepresentation is the pixel resolution of the DMD with an additional overfill ofapproximately 20% per side. This amount of overfill is typical in projectiondisplays (Brennesholtz and Stupp 2008), in order to maintain an approximatelyuniform brightness distribution over the DMD, and as a guard against anymisalignment in the optics.For an 800×600 pixel DMD, we initially represent the light incident on theDMD from the AMA as a white image of size 1000 × 800 pixels, centered onthe DMD. The image is subdivided into n rectangles, each of which represents 1ML. The position of the centre of each rectangle in image (DMD) coordinates isrecorded. Each rectangle is then blurred using the convolution kernel specifiedin Equation 2.24, to get the shape and relative intensity distribution of each ML.The convolution kernel used to blur the light is arrived at using the physicalparameters of the optical system, as detailed in Chapter 2.Initially, the mirrors of the AMA are in a non-actuated, flat state so allMLs are equally distributed on the DMD. The summation of the MLs in thisinitial distribution should give a relatively uniform distribution on the DMD sothat any image can be displayed in a conventional manner. As detailed below,different levels of blur were tested to see their effect on the overall performanceof an AMA projector. The relatively-large overfill of 200 pixels per side wasarrived at because it was the minimum size that gave an approximately uni-form distribution over the 800 × 600 resolution of the projector, as shown inChapter 4. Light allocation 64Figure 4.1 for each of the levels of blur tested. With any smaller an overfill,the intensities at the edges of the area incident on the DMD were much lessthan those in the middle. The amount of overfill chosen was a balancing act forthis simulation because the amount of blur itself affects the uniformity of theoverall distribution. With a 200 pixel per-side overfill, the uniformity on theDMD ranged from 1% deviation at small levels of blur, to 15% at higher levelsof blur.Figure 4.1: The light distribution from an non-tilted AMA, illustrating overfill.The rectangle in the middle represents Ip, the light incident on the DMD.Once the blur of each of the MLs is established, the algorithms detailed belowsearch for the position of each of the n MLs that provide the best improvementto the displayed image. It is assumed that there is adequate granularity in themirror control to position the MLs at any pixel in the image. If numpixels isthe number of pixels in the DMD, there are numpixelsn total possibilities ofmirror configurations, which is an astronomical number of positions to evaluatefor even an 800×600 DMD and a 3 ×3 array of AMA mirrors using a bruteforce method where every solution is evaluated. Additionally, the analog natureof the AMA mirrors potentially allows for the placement of MLs between pixels,which would mean even more possibilities.To guide the allocation algorithm, it needs to be established what constitutesan improvement. In HDR pictures where the scene is defined in terms of actualluminances, the improvement is the ability to show images correctly that havehigher peak brightnesses than the original projector was able to generate. WithLDR images defined in terms of relative luminance, improvement can be definedas the ability to boost the entire range of pixel brightness values; in effect to bea projector with a higher original lumen output.We propose two different approaches to mirror allocation: Gaussian pyra-midswithiterativeredistribution, andmediancut. BothhavebeenimplementedChapter 4. Light allocation 65in Matlab and tested; their implementation and performance are detailed below.Given that neither of these solutions is guaranteed to give the best possibleimprovement, an advantage of testing two algorithms is to be able to verify theeffect of input parameters (range, blur level, number of mirrors) on the resultingimprovement. If patterns evident from one simulation’s results are echoed inthe other, we can be more confident that they reflect a near-optimal solution ateach parameter.4.4 Gaussian pyramid approachTo locate luminance regions in the image to be matched with a ML, the originalimage and an image of the ML can be represented as Gaussian pyramids, firstdescribed in (Burt and Adelson 1983). We adapted a Gaussian pyramid imple-mentation for Matlab described in (Simoncelli 2007) to form a pyramid in whichthe original image is iteratively low-pass-filtered with a binomial filter to reduceits high-frequency content, while the image is simultaneously downsampled toreduce each of its dimensions by a factor of two. The image of the ML is alsoreduced in the same manner, as shown in Figure 4.2. At lower resolutions, eachpixel is a weighted sum of those around it at higher resolutions, so exploringthe solution space at a coarse image representation allows easier recognition ofthe global luminance distribution in the image. Once the best ML placementis found at one level, it is refined at successively higher-resolution levels of thepyramid.The algorithm starts with a target image to be projected, Im, and a simu-lated distribution of light on the DMD, designated as Ip. Ip can be calculatedfrom the light distribution A(ri,ci) of the individual MLs asIp(r,c) =nsummationdisplayi=1A(ai −r,bi −c), (4.2)where each pixel (in row r and column c ) is estimated from the sum of the con-tributions from all MLs of the AMA micromirrors, where (ai,bi) is the locationof the center of the ML from mirror i. The maximum of Ip when no mirrors aretilted is 1, representing the original lumen output of the projector. The pixelsin the target image Im are also from 0−1. An improvement kk = min(Ip/Im) (4.3)can then be defined as the minimum each pixel in the target image can be scaledand remain larger than the corresponding pixel in Ip, for every pixel in Im. Theimages Im (original image) and Ip (image from sum of MLs) are constructed for5 levels of the Gaussian pyramid.Chapter 4. Light allocation 66Figure 4.2: Gaussian pyramids are constructed for the image (left) and theML. Both the ML and the image are downsampled so that each level has halfthe resolution of the previous.4.5 Iterative adjustmentAfter the initial placement of mirrors, a “greedy” algorithm we will call iter-ative adjustment is used to optimize the placement of the PSFs by iterativeredistribution. This algorithm uses the locally optimum choice at each iterationstage, assuming that in most cases this will lead close to the global optimum.The algorithm described below has not been optimized for real-time application,although using pyramids is a highly efficient way of performing large Gaussianblurs, and are used in other real-time applications such as (Rempel et al. 2007).This algorithm is also not guaranteed to find the optimal placement of mirrors,because it does not search the solution space exhaustively for the optimum.However, it is sufficient for a proof of concept to show the benefits of the AMAprojector.The main steps in the algorithm are outlined in Figure 4.3. It is evaluatedfor each level, starting at the coarsest. First, the relative dimmest pixel, whichis the pixel p in Ip that has the lowest intensity relative to the target image isfound, p = min(Ip/Im). To maximize k, we must reposition MLs to providemore light to p.The closest ML (Ai) in distance to p is found, and then the point q closestChapter 4. Light allocation 67Figure 4.3: Basic steps in iterative adjustment algorithmto p within the range of Ai is calculated. This ensures that the only solutionswithin the range of the ML are evaluated. The midpoint between the centre ofAi and q is also computed.Ai is then removed from the distribution Ip, and Ip/Im is evaluated withAi inserted at three points successively: its original location, the midpoint, andpoint q. If the image obtained when Ai added is not better at q or the midpoint,Ai is left in its original location, and the next-closest ML, Ai+1 is evaluated inthe same manner. Better in this context means min(Ip/Im) becomes larger. Ifall of the MLs are iterated through without an improvement, the algorithm hasreached a local minima, and the adjustment at this stage stops.Finally, the placement of the MLs will provide a heterogeneous illuminationIp for the second modulator, in this case the DMD. To produce a projectedresulting image close to the original LDR or HDR image, but with boost in con-trast and brightness compared to a representation using a uniform illuminationof the DMD, the high-resolution correction Idmd for the DMD must be found.The final result for the projected image will be a per-pixel optical multiplica-tion of the two modulators: kIm(i,j) = Idmd(i,j)Ip(i,j), where k representsthe improvement in brightness. Idmd is thus simply equal to kIm(i,j)/Ip(i,j),quantized to the 256 possible greyscale states of the DMD.The maximum possible improvement factor k will depend on the image’s lu-minance distribution. If every single displayed pixel is required to have sufficientluminance, k = min(Ip/Im). At a higher k, some pixels cannot be reproducedat the luminance specified by kIm. For a completely white image, k would be 1,Chapter 4. Light allocation 68the minimum. This would still give the same brightness as a non-AMA projec-tor, minus any extra losses introduced by the AMA. The improvement factor kmay also be fixed beforehand. For instance, k could be specified to ensure thata stream of video has a constant relative brightness over time.There may also be situations where the maximum brightness available is notneeded – for instance, when rendering very dark images. In this case, the AMAcould direct some light away from the DMD, discarding it before it reaches theDMD and affects the dark level.As currently implemented, iterative adjustment is fairly computationallyintensive, because the ML is represented as an image that at high levels of blur,is almost as large as the target image itself. At full resolution, moving andchecking one ML requires as many additions, subtractions and comparisons asthe number of pixels in the image, in this case 800×600, proportionally more forhigher-resolution images. Over the course of the iterative adjustment processas many locations for the MLs are tested, this becomes the dominant source ofcomputational cost.4.6 Median cut approachWhile performing successive iterative adjustment steps at each level of the Gaus-sian pyramid is an allocation method that shows the potential for improvementto projector peak brightness, it would be difficult to implement in a real-timesetting, which would be necessary for on-board projector conversion given con-ventional video signals.An alternative to allocating the MLs via the process of analyzing a Gaussianpyramid is to divide the original image into equal energy zones, and allocate oneML for each zone. The median cut algorithm described in (Debevec 2008), avariation from the original described in (Heckbert 1982), is an efficient methodof subdividing an image into zones of approximately equal energy.To divide the image into 2n regions, the algorithm progresses as follows:first, the image is added to the region list as a single region. Each region inthis list is subdivided along its longest dimension so that the light energy in theimage becomes a constant-time operation of adding and subtracting rectanglesafter the construction of the table.Although neither Debevec nor Heckbert describe it, it is possible to dividethe image up into an arbitrary number of regions, not necessarily a power oftwo. The cut that subdivides a region into two is placed so that the two portionsare unequal as the situation warrants.The algorithm is thus slightly changed to the following:1. Add the image into the region list as one region, along with the numberof desired divisions d.2. If d > 2, subdivide into floor(d/2) (i.e. round d/2 down) and ceil(d/2)(round up) regions, and add these regions to the list, keeping track of thenew d for each. In order to maintain the constraint that all regions atChapter 4. Light allocation 69the final level be of approximately equal energy, choose the cutting line sothat it divides the region into portions of energy floor(d/2) and ceil(d/2)as closely as possible, along the largest dimension of the parent region.Repeat until d < 3.3. If d = 2, subdivide region into 2 along the longest dimension.A useful property of the original algorithm is that the cut is always made onthe long side of the rectangle, which minimizes the aspect ratio of the two divi-sions. To approximately maintain this property when dealing with an arbitrarydesired final number of regions, the region is divided into only 2 or 3 section periteration. The cut is always done at a place that reflects this ratio, in order tokeep the energy of all regions approximately equal.Figure 4.4: An image divided to 28 regions using the median cut algorithm,with the centroids in each region represented as dots.Figure 4.4 shows an image cut into 28 regions. The centroids of each regionare marked with squares. An ML is placed with its centre at the centroid ofeach region. The main advantage of dividing the image in this manner is thatthe image can be divided quickly into as many regions as there are MLs. Withthe Gaussian pyramid scheme, there are still many more pixels than there areMLs at the coarsest level of the pyramid, so the initial allocation is much lessrepresentative of the actual intensity distribution of the pixels.Chapter 4. Light allocation 70One potential drawback of using this scheme for ML allocation is that thesize of each region is a function only of its summed light energy, without regardto the size of the ML. Very small regions might only fit a small fraction of thetotal light energy of an ML, and very large regions might be larger than a singleML. Also, an equal aspect ratio of a region is not guaranteed, so some regionsmight be of much different shape than an ML.4.6.1 Adjusting for limited ML rangeAnother drawback is that the MLs could have a limited range of movement fromtheir original positions due to a limited tilt angle of the micromirrors, whichis not taken into account in the above algorithm. In the Euclidean bipartiteminimum matching problem (Agarwal and Varadarajan 2004), we are given anequal number of points from two sets, and would like to match a point fromone set with a distinct point from the other set, so that the sum of distancesbetween the paired points is minimized. We can use an algorithm that solvesthis problem to match each centroid from the median cut step to a location ofa ML in its rest (non-tilted) state, and thus minimize the sum total distancebetween the pairs in the two groups. This will minimize the sum total angle thatthe mirrors must tilt to achieve the points specified in the median cut solutionset. The Euclidean bipartite matching problem can be done in polynomial timeusing Kuhn’s Hungarian approach (Kuhn 1955).The bipartite minimum matching problem does not limit any one pair tobe less than a given amount, only that the sum total of all pairs is minimized.This means that there might be some pairs that exceed the range of motion ofan ML for a given maximum mirror tilt angle. If any of the distances betweenthe non-tilted ML and the centroid are larger than the range, they are placedat the furthest point along the line that connects the two points that they canreach. Figure 4.5 shows the results of using the median cut algorithm on theimage in Figure 4.4. Figure 4.6 shows the estimated light distribution given bythe locations in Figure 4.5.Because the MLs are only added after their locations are determined, thecomputational complexity of this algorithm is much less than that of the Gaus-sian pyramid with iterative adjustment. Many of the operations for buildingthe median cut data structure could be performed in parallel. It is anticipatedthat a variant of this algorithm could be implemented on board the projectorto interpret image data and direct the AMA at video rates.4.6.2 Median cut with iterative adjustmentIf speed is not an overriding concern, the solution obtained by using the me-dian cut algorithm can be optimized by performing an iterative adjustment stepas described above in Section 4.5. After the initial median cut solution is de-termined, the pixel with the minimum improvement is found, the closest MLmoved towards it, and the image is evaluated at points along the path. Figure4.7 shows the same example as in Figure 4.5 after such an adjustment. SomeChapter 4. Light allocation 71Figure 4.5: Diagram of placements of MLs for the image in Figure 4.4 using themedian cut algorithm, with a mirror range of 100 pixels. The starting (smallred dots) and ending (arrowheads) locations of each ML are shown, as well aswhere they would have gone had they had unlimited range (green dotted line).The border of the DMD is shown as a rectangle. Some of the MLs start frombeyond the border because of the required overfill.of the locations are no longer at a point directly on the path from their originallocation to where the median cut algorithm would have placed them. In thisparticular example, the improvement factor k was 1.52 for the original mediancut placement, and a range 100 pixels. After optimization, k rose to 1.85 withthe same range.Chapter 4. Light allocation 72Figure 4.6: False-colour estimation of light distribution given by the ML loca-tions in Figure 4.5. The scale bar indicates improvement factor from a baselineof 1.Chapter 4. Light allocation 73Figure 4.7: Adjusted ML locations from 4.5 after an additional optimizationstep. The orientation of the green dotted lines show that some of the MLs havebeen diverted. They are no longer all on the same path from starting position(red) to ending position (arrow terminus) to ideal position according to themedian cut algorithm (end of dotted line).Chapter 4. Light allocation 744.7 Simulation resultsThe algorithms described above were implemented in Matlab. With them wecalculate anestimatedimprovementfactor, k, the amountwe are able to increasethe peak brightness of the scene, while maintaining adequate illumination forthe entire image. We neglect any additional losses introduced by the AMA,such as from fill-factor.The final achievable improvement k can be used to evaluate the choices forML size, range and number given as initial conditions to the simulation. To de-termine the best combination of these parameters to guide further development,the variables examined are:• Range: The distance, in pixels, along which a mobile light source canmove. This depends on the optical system and the maximum tilt angleachievable by the AMA micromirrors.• Blur: The PSF obtained in Chapter 2 determines the spatial localizationof a mobile light source. As discussed previously, the more the disparity,the larger the blur, and also the larger the range of the mobile light source.• Number of mirrors: The number of mirrors within the AMA. Withmore mirrors, more details in the image can be covered with separateemphasis. It is therefore estimated that a larger number of mirrors willprovide more improvement. However, more mirrors also imply a greaterphysical cost and complexity of the mirror array and its drivers.The range of each mirror and the amount of blur are related, and can beapproximated using the methods detailed in Chapter 2. With measurementsor assumptions of the blur and achievable tilt angle of an AMA mirror, theallocation system can be used to simulate the projector’s effectiveness. Thesimulation can then be done completely on the DMD plane, so that we canevaluate the effectiveness of the allocation algorithm, and examine how thephysical parameters impact the degree to which the AMA can improve projectorperformance.Thefirststeptousetheallocationalgorithmasanevaluativetoolbeginswithan AMA configuration with no mirrors tilted. Five levels of blur, characterizedby the size of the convolution kernel, were tested. These levels were determinedby the disparity distances in Table 4.1, as explained in Chapter 2. Figure 4.8shows each levels of blur for one ML from a 5×5 array with an original size of200×160 pixels. As the blur increases, the intensity becomes progressively morediffuse and less localized. The peak intensity also decreases. For each of thefive levels of blur, five AMA configurations were tested. The smallest numberof mirrors was a 3×3 array, followed by 4×7, 5×5, 8×8, and 10×10. Weincluded the 4×7 array because that was the dimensions of the AMA mirrorsused in the prototype. In all configurations, the die size of the AMA was heldconstant, so the number of mirrors determines the relative size of the ML, witharrays having more mirrors also having proportionately smaller ML sizes.Chapter 4. Light allocation 75Table 4.1: Disparities tested, their corresponding blur kernel sizes, and thefull-width at half maximum (FWHM) of the blur kernel.Disparity (mm) Kernel size (pixels) Kernel FWHM (pixels)0 1 110 171 12020 279 19740 410 29060 486 34390 554 391Figure 4.8: Estimated blur for one ML of a 5 × 5 array, with progressivelyincreasing disparity in millimeters between focal plane and DMD.Finally, with each combination of blur and number of AMA mirrors, differentranges of movement (in pixels) of the MLs were tested: 50, 100, 300, 500, and1000. 1000 pixels was included to allow an ML in one corner of a 800x600image to be moved to the opposite corner. Since the centre of some of the MLsmay originate outside of the DMD entirely, this is not quite enough to allowfor unfettered movement, but for the images tested this never represented aconstraint.In the physical system, the range of movement and the blur are related: thesmaller the blur, the smaller the range of movement. For these simulations,we chose a larger range of values than what is physically possible given thelimitations of the current prototype, in order to fully map the tradeoffs betweendifferent parameters.Test images were chosen to give different views into the behaviour of thesystem. The first, “Mt. Robson,” shown in Figure 4.9, has dark section at thebottom and section of high intensity in the sky and on the peak of the mountain.It is anticipated that light will be moved away from the bottom to those regionsnear the top of the image with the highest intensity. The second test image,“Rock Beach” shown in Figure 4.10 also has a bright sky region, but most of thedarker regions are interspersed with brighter regions, which will make it moredifficult to move light from one region to another without affecting fine detail.The third, “ANSI checkerboard” in Figure 4.11 is an artificial test pattern thatis often used to evaluate the contrast of a projector. It has eight regions offull-intensity white interspersed with eight regions of full-intensity black. It isChapter 4. Light allocation 76Table 4.2: Image characteristicsPixel attributes Mt. Robson Rocky beach ANSImean 44.45 45.35 128st. dev 43.04 67.20 128general dark section at bottom intensity full on/offbright section top interspersed sectionsanticipated that an AMA projector will not be able to improve this image to alarge degree due to the sharp demarcation of bright and dark areas.Figure 4.9: Original sample image 1: Mt. Robson.Before applying the AMA algorithm, it is useful to list some first-order imagestatistics to get an estimate as to potentially how much each image could beimproved. After an inverse gamma has been applied to the Mt. Robson imageto obtain linear luminance changes, the mean pixel value becomes 44.45 out of255 (17%), with a standard deviation of 43.04.For the Rocky beach image (Figure 4.10), the mean is 46.34 (18%), with astandard deviation of 67.2, a minimum pixel value of 0, and maximum of 255.For the ANSI checkerboard image in Figure 4.11, there is only two pixel values:0 and 255. They are evenly distributed, giving a mean pixel value of 128 out of255 (50%), and a standard deviation also of 128. These values are summarizedin Table 4.2.Chapter 4. Light allocation 77Figure 4.10: Original sample image 2: Rocky beach.The goal of these simulations is to determine whether an AMA projectorwill still be tenable given the optical constraints given in Chapter 2 and themechanical constraints given in Chapter 3. In terms of ease of engineering, alarger blur is easier to use than a smaller blur; a smaller range is easier to obtainthan a larger range, and less mirrors would be preferable to more mirrors in thearray.All of these parameters can be modified to a certain extent. Both range andblur are constrained by optical limitations. The workable range can be extendedand blur reduced by limiting the ´etendue of the projector lamp, but only withinlimits, since this will also reduce the total lumens and therefore the efficiencyof the system. The number of mirrors is primarily constrained by the cost ofthe drivers, but also has an effect on the fill-factor of the AMA. More AMAmirrors also require more computation time, the degree of which depends onthe algorithm chosen.The following simulations are not meant to provide an exhaustive survey ofall the possible parameter values that might effect an AMA projector’s perfor-mance, but rather probes parameter values that seem technically feasible. Theyshow that the improvement is possible even with limited ML range and rela-tively large blur kernels. They also provide a framework to use for the designof a more polished prototype. They will provide a guide to the ramificationsof micromirror design choices and projector optics. At that point, more tar-geted simulations using smaller ranges of blur, range and mirror numbers willChapter 4. Light allocation 78Figure 4.11: Sample image 3: ANSI checkerboard.Table 4.3: Mechanical tilt angle (degrees) needed for select levels of blur anddisparity, assuming a magnification of 1.blur disparity (mm)range (pixels) 10 20 40 60 9050 2.0 1.0 0.5 0.3 0.2100 4.0 2.0 1.0 0.7 0.5300 11.4 5.9 3.0 2.0 1.3500 17.5 9.7 5.0 3.3 2.21000 27.2 17.5 9.7 6.8 4.4be needed to further optimize the system.Table 4.3 shows the implications of tested choices of blur and range on therequired tilt angle of the AMA mirrors. The numbers were calculated usingEquation 2.12, assuming a magnification of 1. Combinations that include lowlevels of blur with high range require the most angle. Limiting the range hasseveral implications. While 300 pixels is less than half the distance across an800x600 image, the tip/tilt mirrors near the centre of the image could move theML 300 pixels in any direction, for a maximum of 600 pixel range.While a large range of movement is expected to produce the best results inthe simulations, the larger tilt angles needed to produce this movement havean adverse effect on the ´etendue of the resulting beam. Diverting light withthe AMA from one region to another changes the angle of the incoming beam.The projector lens pupil accepts a beam with an angle of 24◦. Therefore, someof the more extreme possibilities such as 1000 pixel range and 10 mm blurdisparity, which requires 27◦ tilt, resulting in a change in beam angle of 54◦,Chapter 4. Light allocation 79are not physically practical, but are included here for the sake of completenessin identifying trends.4.7.1 Gaussian pyramid resultsFigures 4.12, 4.13, and 4.14 show the improvement obtained for each of the testparameter combinations using the Gaussian pyramid with iterative adjustmentapproach.Figure 4.12: Gaussian pyramid results for the Mt. Robson image. The improve-ment factor k possible with various mirror number combinations, range of themirrors in pixels, and blur disparities are shown as entries on the bar graph.Chapter 4. Light allocation 80Figure 4.13: Gaussian pyramid results for the Rocky Beach image.Figure 4.14: Gaussian pyramid results for the ANSI checkerboard image.Chapter 4. Light allocation 814.7.2 Median cut resultsThe median cut algorithm did not produce results as effective as those producedwith the Gaussian pyramid approach. Because the results for the median cutalgorithm alone were so small, we include only the results for the Mt. Robsonimage in Figure 4.15. The median cut with iterative adjustment is included inFigures 4.16, 4.17, and 4.18. Notice that in some of the conditions, notablyblurs from 10 and 20mm disparity, there was no improvement, corresponding toa k of 1.Figure 4.15: Results from allocating the MLs using the median cut algorithmfor the Mt. Robson image.Chapter 4. Light allocation 82Figure 4.16: Results from allocating the MLs using the modified median cutalgorithm for the Mt. Robson image.Figure 4.17: Results from allocating the MLs using the modified median cutalgorithm for the Rocky Beach image.Chapter 4. Light allocation 83Figure 4.18: Results from allocating the MLs using the modified median cutalgorithm for the ANSI checkerboard image.Chapter 4. Light allocation 844.7.3 ObservationsThe results above show that the Gaussian pyramid algorithm consistently re-turns the best solution for all three images tested. The average k over allconditions for the pyramid algorithm was 1.56, while the median cut alone was1.29. The median cut with an additional iterative adjustment step was 1.49,although that adds the most computationally expensive part of the Gaussianpyramid approach, and thus does not provide much of the speed advantage thatthe median cut approach alone does. The relative differences between algorithmsare consistent over the three images tested.Because the score is set by the pixel with the least light relative to its need,outliers can have a large effect on the final outcome. In the results for themodified median cut algorithm shown in Figure 4.16, the scores for an 8x8array for 10 blur, ranges 500 and 1000 show no improvement, even though arange of 300 has an improvement of 3.5. Because the algorithms do not do anexhaustive search, the solution that was found for 300 range is not found for500 or 1000 in this case.RangeIgnoring the physical implications for the moment, in terms of possible improve-ment factor, the more range, the better, because solutions obtained for lesserranges will always be achievable by higher ranges (although because of the non-exhaustive search, it isn’t guaranteed that the proposed algorithms will findthem). While the iterative algorithm described in Section 4.4 incorporates withthe maximum range at every iteration, the median cut algorithm applies thelimits of the maximum range after a solution is found, so it is assumed that thesmaller ranges will affect Median Cut results the most.For all algorithms tested, limiting the mirror movement to 50 and 100 pixelsseverely impacts the possible improvement, irrespective of blur level or mirrornumbers. Since range depends on disparity and mirror tilt angle, smaller rangeswould be easier to engineer. They would also cause the least amount of anancillary light loss due to the change of incident angle on the DMD. For thisreason, the range of 300 pixels is chosen as the optimal of those we have tested.This range should be re-evaluated with every new physical prototype, whenthe assumptions detailed above can be validated and adjusted according to theprototype’s measured parameters.BlurIt was anticipated that the lower the blur, the more opportunity there wouldbe for improvement. Less blur means a smaller ML size, so the intensity couldbe better targeted to where it is needed. The results show that with a largernumber of mirrors, the best results are achieved with high range and low blur.As the number of mirrors is decreased, the differences between k scores fordiffering levels of blur become less pronounced.Chapter 4. Light allocation 85From the results of the simulation, there was a clear difference between thepyramid algorithm and the median cut depending on blur level. For the pyra-mid algorithm, less blur gave better results averaged over all other conditions.For the median cut with iterative adjustment, the trend was less pronounced.Individual inspection of the results revealed that a relatively small number ofunder-illuminated pixels brought down the results, rendering the change in blurless of an important factor in determining k. No blur at all was also tested, butno improvement was ever achieved, so it was omitted from the graphs.Number of mirrorsFrom first principles, it seems evident that the more separately-controllablemirrors are in the array, the better the opportunity for improvement, becausethey offer more degrees of freedom to tailor the light distribution on the DMDto the desired image. More mirrors can separately cover more features in theimage. There are more opportunities to stack MLs. Although the size alsodepends on blur, more mirrors also means a smaller ML. Qualitatively, the bargraphs show clear upwards trends as the number of mirrors increase.Quantitatively, the number of mirrors revealed a difference in the two mainalgorithms, Gaussian pyramid and median cut. Averaged over all other con-ditions, the pyramid algorithm did show a clear upward trend as the numberof mirrors increased. The median cut algorithm, however, did not show muchimprovement as the number of mirrors increased.Image typeThe results show that there is a large variation between improvement factorsdepending on the image. The overall average improvement factors for the Mt.Robson, Rocky beach, and ANSI images are 1.81, 1.45, and 1.09 respectively.Table 4.2 shows some possible reasons for this. ANSI has very high frequencycomponents that are not fit well by the blurred MLs, as exemplified by itshigh standard deviation of pixel intensities. Mt. Robson had a lower standarddeviation, but still large differences in pixel intensity over the image, with adark section at the bottom and light at top, typical of many nature scenes.Rocky beach, although with a similar mean pixel intensity, does not have thesame global distribution of light and dark areas to work with.4.7.4 Implications for prototypeNow that we have described the implications for blur and range, we can narrowdown the choices to those that are physically realizable. While higher angleshave been shown in the literature, ±3.5◦ tilt around both the X and Y axis is areasonable limit for this discussion. As Table 4.3 shows, this is a realistic com-promise that is close to that achieved in the literature with multi-user processes,and below what can be achieved with a customized fabrication process.Chapter 4. Light allocation 86From the results of the simulations, the combinations of 60 blur and 500range, or 40 blur and 300 range are closest to this maximum target. The simula-tions show that the improvement factor found for these conditions are relativelysimilar. Over all image/algorithm combinations, a blur of 60mm and range of500 corresponds results in an average of 1.58 improvement factor, while 40/300gives 1.61. When further broken down by algorithm, 40/300 consistently scoreshigher than 60/500. Using a worst-case analysis to quantify the clipping due tosome of the light tilted by the AMA missing the DMD aperture, 3.5◦ results ina 20% clipping loss, described in Section 2.5, reducing the improvement factorto 1.26.These results show that significant improvement is possible for a range ofphysically-realistic parameters. While these results show that the Gaussianpyramid and median cut approach both have promise, more tests are neededwhen the exact physical parameters of the projector and beam incident on theDMD are known. Some of the assumptions required to run the tests couldhave had some impact on the results. In particular, the shape of the Gaussianused in the blur kernel given in Equation 2.24 is influenced by the parameterg, which will have to be re-evaluated as the projector optics change. We chosea value of 0.6 for g in these simulations, as it provides an average, non-peakeddistribution. When a prototype where the physical parameters can be properlyobtained, these tests should be performed again to verify the performance ofthe algorithms.In general, the results show that lower blur, more mirrors, and higher range,all contribute to better possible improvement. In part this depends on thealgorithm chosen; the more efficient median cut algorithm works best whenthe blur is a bit higher, and with a medium range of pixels. For the subset ofpossibilities that are currently physically realizable, lower blur is more importantthat higher range.4.7.5 Sub-frame positioningThe above results assume that the ML is stationary for the duration the image isdisplayed. A frame of video is normally shown at 30-60Hz, so the micromirrorsmust be able to be positioned much more quickly than that so they can bestationary for the majority of the frame, then repositioned between frames. Asshown in Chapter 3, the MEMS micromirrors we propose can be positioned at10 kHz, an order of magnitude faster than what is required for this purpose.We could potentially exploit this additional speed by repositioning the mi-cromirrors within a frame of video. This would allow us, for instance, to coveran area that only requires half of the intensity that an ML offers in half thetime, then use the other half of the ML’s intensity somewhere else.Simulations were performed to determine the potential advantage of thissub-frame positioning. To position the MLs four times within one frame, theintensity of the ML was divided by four, and the number of MLs multiplied byfour. Foursub-MLswereassignedtoeverylocationintheimagethattheoriginalMLs were assigned to, and the same algorithms as above were performed. TheChapter 4. Light allocation 87four sub-MLs could move independently, but were still limited by the same rangeconstraints as before. It was found that for the conditions tested above, thereis no advantage to having sub-frame positioning.4.7.6 Allowing under-illuminated pixelsThe above results show that the choice of algorithm had the most significantimpact on the improvement factor. A potential reason for this is that all pixelsmust be illuminated properly. This means that the improvement factor is set bythe pixel with the minimum ratio of light given to light required (Ip/Im). Allit takes is one outlying pixel to bring down the improvement factor. Out of thethree versions of the algorithms tested, the Gaussian pyramid approach givesthe most opportunity to minimize the effect of the outliers during its extensiveiterative adjustment steps, perhaps explaining some of the difference in resultsachieved relative to the median cut approach.If a particular pixel is an outlier, it may be that the rest of the image couldhave been shown at a much higher brightness if that pixel were ignored. It maybe that several of the pixels will not be noticed by the viewer if they are under-illuminated. We tested how much brighter the image could get if we allowed acertain percentage of pixels to be dimmer (relatively) than the rest of the image.Figure 4.19 shows the performance of the three algorithms tested for the Mt.Robson image, with a blur from 60 mm disparity, an AMA with 7x4 mirrors,and a range of 300 pixels. Although the Gaussian pyramid (GP) algorithm per-forms best for the case where all pixels are illuminated correctly, it is eventuallyoutperformed by the two median cut (MC) versions when some pixels are al-lowed to be under-illuminated. For the case in which all pixels receive enoughlight, the pyramid representation scores highest with a 2.44 improvement factor,while median cut with iterative adjustment (MCIA) scores 2.32, and just usingMC placement scores 1.9.But at just 200 under-illuminated pixels (which, for an 800x600 image, is0.004%), GP is overtaken by both median cut versions. GP scores 2.71, whileMC alone scores 2.8, and MCIA scores 3.07. That is a 26% increase in brightnessover the highest score obtained for perfect reconstruction. The increase raisesto 36% if 500 pixels are allowed to be under-illuminated. The other two imagestested are consistent with this trend.Figure 4.19 shows that the median cut algorithms are sensitive to outliers,and much more overall brightness increases can be achieved if the requirementsare loosened. Also, visual evidence of the under-illuminated pixels might bedifficult to detect due to the inevitable presence of other sorts of display errors.The model of the ML used in the algorithm for a real projector will most likelybe an approximation that differs slightly from what is physically present dueto manufacturing variability. We estimate how under-illuminating pixels effectsperceivable image quality in Section 4.9.Even if allowing under-illuminated pixels is not desirable under normal con-ditions, these tests show that there is a lot of flexibility to tailor the performanceto different conditions. For instance, when displaying a video sequence on anChapter 4. Light allocation 88Figure 4.19: Effect of allowing some pixels to be under-illuminated on improve-ment factor for the Mt. Robson image, for up to a) 50 underilluminated pixels,and b) 500 pixels.Chapter 4. Light allocation 89AMA projector, the potential improvement factor will vary over time as theimage changes, depending on the sum total intensity in the image and its distri-bution. If it changes too quickly or the magnitude of the change is too abrupt,the viewer will notice the artificial change in relative brightnesses of the dif-ferent features in the image. Figure 4.19 shows that fixing the improvementfactor for this video sequence doesn’t necessarily mean fixing it at the minimumvalue out of all images in the sequence. By allowing some pixels to be under-illuminated, the improvement factor can be set at a higher level that matcheswhat is achievable for an average image in the sequence, not the minimum.4.8 Estimated system performance4.8.1 Brightness improvement factorAdding the AMA will incur some light losses that partially offset the gainsdescribed above. We can estimate the efficiency of a production version of theAMA at reflecting incident light by examining the statistics from the DMD.After taking into account the transmittance of the packaging window of 97%,the active area fill-factor fD of the DMD of 88%, its mirror specular reflectivityrD of 89.4%, and diffraction efficiency, the DMD reflects eD = 68% of incidentlight (Texas Instruments 2005a).Although the DMD is a highly-engineered device, the AMA could achievehigher efficiencies because of its different structure and use. The fill-factor ofthe AMA could be increased, because the AMA requires many less mirrors thanthe DMD, and thus could have less total space between them. A fill-factor fAof 95% could be achievable given a different fabrication process, as it is slightlylower than the 96% achieved in (Tsai et al. 2008) for micromirrors that offerperformance that would be adequate for this application.Another factor that would mitigate the light efficiency losses due to theaddition of the AMA is that the AMA will replace an existing fold mirror in theprojector. We can expect the fold mirror to have a similar specular reflectivitycoefficient as the AMA, so rd can be removed from the calculation of the AMAefficiency.As described in Section 2.5, another source of loss is due to a portion ofthe diverted light missing the aperture. For a ±3.5◦ tilt AMA, the estimatedlosses due to clipping lc is approximately 20% worst-case, if all the AMA mirrorswere tilted to their maximum, and no other mitigation plans such as apertureshaping were included. For this calculation, we assume the mirrors are tiltedto an average angle of 2.5◦, giving a expected loss from clipping as 10% fromFigure 2.11. The expected efficiency of the AMA can thus be calculated relativeto the DMD reflectivity aseA = eD × (1−lc)fAfDrD(4.4)which results in an estimated efficiency of the AMA as eA = 74%. Although thisis a significant loss of light, we still anticipate that the AMA-enabled projectorChapter 4. Light allocation 90wouldhaveanetgainofpeakbrightnessgiventheresultsfromSections4.7.1and4.7.2. With 40mm blur disparity and 300 pixel range, the average improvementfactor was 1.61, giving a net improvement factor of 1.20 after taking into accountthese losses.If we allow some pixels to remain under-illuminated as explained in Section4.7.6, the case becomes even stronger. With 200 under-illuminated pixels, theaverage score for the median cut algorithm with iterative adjustment was 3.07.After taking into account losses, this still leaves a 2.25 improvement.4.8.2 Contrast improvementAn AMA projector will also improve projector contrast, both on the low-endwith a reduced black level, and on the high-end with increased brightness. TheAMA is primarily a redistributive system, so the contrast improvement is thesquare of the brightness improvement, since the black level would be reduced thesame amount. If we use the brightness improvement factor of 1.2 for an imagewhere no under-illuminated pixels allowed, the overall contrast improvementfactor would be 1.4. For a system that allows under-illuminated pixels, whilethe brightness improvement is 2.25, the contrast improvement would be 5, avery substantial change.Ultimately, the magnitude of improvement will depend on critical designdecisions during the projector design phase, decisions that substantially affectany estimate of increased contrast due to the AMA. The major initial decisionwhich will depend on the projector’s intended use is the choice of illuminationnumerical aperture. Typically the DMD illumination angle is increased slightlyto increase projector contrast, at the expense of some brightness. This need notbe the case given the AMA’s effect on contrast, which would increase the overallimprovement factor but decrease the contrast improvement factor.4.9 AMA projector image fidelityAfter the introduction of the increase in performance that can be achieved whensome pixels are not adequately illuminated, it is natural to wonder about theeffect on visual fidelity the addition of the AMA would have on the projectedimage. In this section we quantify any undesired visual artifacts that are intro-duced when an AMA is added to the system.No display available today can perfectly reproduce an arbitrary image. Con-ventional displays have limited contrast, so there will be pixels that have moreor less light than they should. Other problems are display-technology specific.In projectors, there is distortion from the projection lens, and vignetting, thespatial fall-off of brightness from center to fringe. These imperfections can be ag-gravated by high-gain screens causing their own brightness variations. Evidenceof how frequent projector imperfections arise can be found in the research donein the area of tiling multiple projectors, where imperfections must be correctedbefore an adequate combined image is formed. Tiling the images from multipleChapter 4. Light allocation 91projectors allows for the display of very high-resolution images over a large area,but inter and intra-projector differences can cause distracting artifacts. A goodsurvey of projector imperfections in the context of tiling is (Majumder et al.2008).Conversely, external factors influence the appearance of the projected imageeven when it is perfectly displayed. The level of ambient illumination in theroom determines the minimum dark level of the image. In an environment withsignificant ambient illumination, the steps between the first few darkest pixelvalues could be imperceptible, because they correspond to changes in luminancethat are too small to perceive. Since the perception of luminance is roughlylogarithmic, these same imperceptible intensity levels could be perceptible in adarker room.In this hypothetical case, an image displayed with an AMA projector witha larger improvement factor (and thus a higher overall brightness) with somepixels under-illuminated might still have more perceivable detail than the sameimage displayed at a lower brightness with all pixels correctly illuminated, be-cause the missed pixels are not perceivable at that level of ambient illumination.Colour, which will not be directly addressed in this work, is also a majorissue in display accuracy. For instance, (Heckaman 2006) show that adding awhite channel to the colour wheel may increase brightness, but negatively affectsthe colour saturation of the projected image.The particular errors quantified in this work are those directly attributableto the addition of the AMA. Specifically, we will examine the additional sourceof error that comes from having a non-homogeneous backlight of lower resolutionthan the DLP itself. The errors can be categorized into five main areas:1. Quantization: There are only 256 greyscale values in a DLP. Therewill thus be errors due to quantization as the DMD cannot compensateaccurately enough for the continuous, non-homogeneous illumination fromthe AMA.2. Over-illumination. The DMD doesn’t have complete control over con-trast, so it will leak light, causing some pixels to be over-illuminated.Other components in the optical path also contribute to leakage from oneregion to its neighbours, such as scattering from the projector lens. Thesefactors contribute to an amount of per-pixel error. Normally this is ex-pressed as a constant loss of contrast over the whole image, because of theconstant illumination source. With a projector that includes an AMA,the light that must be blocked by the DMD varies spatially, causing theamount of light leakage to also vary spatially.To estimate over-illumination, it is necessary to have an estimate of theamount of leakage in the projector for a given amount of luminance. Thiscan be estimated from the contrast ratio of the device, and the peakbrightness. If the peak brightness of the projector is x, and contrast ratiois 1 : y, then the estimated brightness of a pixel set to its darkest level (0)in the DMD is x/y. If the pixel is set to a brightness of z, the estimatedChapter 4. Light allocation 92result is z+x/y. While this is not a perfect estimate due to the variabilityof lens scattering and possible non-linear effects, it provides a base level tocompare the difference in an AMA-projector to a conventional projector.3. Under-illumination. The improvement factor, k, sets the improvementfactor of the AMA projector for a given image. If k is set too high, the im-age cannot be reproduced perfectly because the total luminance availableis constrained by the projection lamp. The result could be a loss of tex-ture in the displayed image. Choosing k thus becomes a tradeoff betweenmaximizing total image brightness and maintaining image accuracy.To maximize overall brightness, it is tempting to sacrifice some accuracyby allowing some pixels to be under-illuminated. We can ensure thisdoes not happen by specifying to the allocation algorithm that all pixelsmust have sufficient light, but how many under-illuminated pixels couldbe present without adversely affecting the viewing experience, and underwhat conditions?4. Simulation imperfections. In the analysis above, it is assumed thatthe distribution of light incident on the DMD is perfectly known. Inreality, due to real-time constraints there will be approximations madethat cause imperfections in the model of the shape of each individual ML,and possibly in its location due to slight micromirror differences acrossthe array that affect their tilt angles. Although these differences can becompensated for by using more complex models and feedback mechanisms,there will always be some error due to this mismatch.5. Real time constraints. In the allocations algorithms proposed above,the final result is optimized at the resolution level of the final image. Ina real-time application, the bandwidth of the hardware might not be fastenough to support this. Simplified approximations and/or lower resolu-tions might be necessary to obtain adequate performance, but would alsoresult in other sources of errors.4.10 Visual difference predictionTo quantify the effect of these inaccuracies, the simulated optical combinationof the AMA image on the DMD (IP) and the DMD image are compared againstthe original image. To avoid extensive psychophysical user testing, we use an au-tomated visual difference predictor (VDP) (Ramanarayanan et al. 2007), whichhas been extended for application to high dynamic range images as well as con-ventional images (Mantiuk et al. 2005). The VDP algorithm provides a metricto distinguish the subset of differences between two images that a standard hu-man observer would be able to detect. The algorithm filters the images throughvarious stages that mimic the light scattering in the cornea, lens, and retina,and our non-linear response to luminance and contrast. The limitations of thehuman visual system mean many physical differences between images are notChapter 4. Light allocation 93detectable by a human observer. See (Mantiuk et al. 2005; Trentacoste 2006) forthorough introductions to the methodology used in visual-difference estimates.Visual-difference prediction has been used before to evaluate novel displays,notably in (Trentacoste 2006) for Brightside/Dolby’s high-dynamic range dis-play, the DR-37. Like Trentacoste, we use the implementation HDR-VDP de-scribed in (Mantiuk et al. 2005). Our objective is to establish visual equivalencebetween images made with an AMA-equipped projector, compared to a regularprojector that has the same peak brightness as the improved image. For in-stance, if the peak brightness of the AMA projector when none of the mirrorsare tilted is l, and the AMA projector improves the projected image to 2l bytilting the AMA mirrors, then this simulated image is compared via VDP toan image perfectly displayed at 2l. We compare images of equal luminance be-cause luminance can affect the perception of features within the image, and wewanted to keep the differences to those directly attributable to introduction ofthe AMA.The VDP algorithm computes for each pixel in the image the probability ofthe viewer detecting distortion. However, this does not mean that the resultsreturned are accurate on a per-pixel basis. A major step in the VDP pipeline isto process the images with a series of filters sensitive to spatial frequency andorientation that approximate the spatial processing done in the visual cortex.The filtering operations are performed in the frequency domain, so the loca-tion of a difference is dependent on its spatial frequency. For computationalefficiency, images are not filtered at every spatial frequency or orientation, re-sulting in banded areas of detection. In reality these bands are smoother, andif more frequencies were tested, the features would be more representative ofthe shape of the difference (Trentacoste 2006). As long as we keep in mind tointerpret the results in terms of a probabilistic area of difference, the VDP is auseful tool for evaluating whether two images would look the same to a typicalviewer.4.10.1 VDP tests on simulation resultsBecause a real-time performance implementation is outside the scope of thisthesis, and the prototype is not in a state where simulation imperfections canbe properly evaluated, we presently omit analysis of these sources of error. Wefirst test via simulation whether the artifacts for quantization and the over-illumination issues inherent with an AMA-projector are normally detectable.We tested using a 7x4 array of mirrors, using a range of 300 pixels, and adisparityof60mm. WithDMDpixelsbeingapproximately14µmsquare, wecanuse simple trigonometry to calculate that the micromirror mechanical tilt angleneeded to traverse 300 pixels with this disparity, assuming no magnification, is±2◦, which is within the range of published micromirrors.Quantization is achieved as follows. In our simulation, both the image of anML and Ip, the distribution of light on the DMD from all MLs are stored asdouble-precision floating-point arrays. Because the DMD values are calculatedChapter 4. Light allocation 94as Idmd = kIm/Ip, they are initially floating-point values as well, but are thenquantized to 256 values, to Iprimedmd. The resulting image is then calculated asImq = Iprimedmd ∗Ip. (4.5)In this equation and those that immediately follow, it is implicit that the cal-culation happens on a per-pixel basis, e.g. with each pixel in the DMD imagebeing multiplied by the corresponding pixel in Ip.The limited contrast of the projector is simulated by adding a portion of Ipto the resulting image.Imc = Imq +Ip/c, (4.6)where c refers to the contrast ratio of the DMD. For instance, if we take aconservative value of 1 : 400 contrast ratio for the DMD, c is 400. Imc iscompared against kIm, where k is the improvement factor.The physical parameters of both the target (simulated image) and maskimage (perfect image) need to be specified for the VDP algorithm. For thesetests, the image was specified to be displayed at a size of 0.81 m by 0.45 m ata distance of 1.2 m.The peak brightness of the image also must be specified. As described inSection 1.1, in the standard ANSI/SMPTE 196M the Society of Motion PictureTelevision Engineers recommends that the screen luminance be set at 14 foot-lamberts (SMPTE 2003). A foot-lambert (fL) is approximately 3.426cd/m2, so14 fL is 48cd/m2.Such relatively low peak luminances are sufficient for conventional film im-ages shown in a theatre setting where the ambient light is minimized. For thehome or business with much higher ambient light levels, higher brightness ca-pabilities are needed. The recommended luminance for a television accordingto the SMPTE is 50 fL, or 170cd/m2. Modern LCD and plasma televisions areapproximately 300 cd/m2, whereas the DR-37 from Seetzen et al. can reach upto 8500 cd/m2.To evaluate an AMA projector using the VDP algorithm, we chose an inter-mediate level of 200cd/m2 to use for peak brightness. The median cut algorithmwith iterative adjustment on the Mt. Robson image obtained a 2.22 improve-ment for ‘perfect’ reconstruction. Simulation of the quantization and contrastwere added to the simulation results and tested with VDP. At this peak bright-ness, the VDP algorithm predicted that there was a probability of detectionP > 95% for 0.0002% of the pixels, as shown in Figure 4.20. For an 800×600image, this works out to 96 pixels. The affected pixels are hardly visible at thebottom right-hand corner of the image.4.10.2 Quantifying the effect of some under-illuminatedpixelsGiven that quantization and contrast issues did not significantly affect the imageat this peak brightness, there is potential to significantly increase the overallbrightness of the image if the constraints for perfect reproduction are relaxed.Chapter 4. Light allocation 95Figure 4.20: VDP results from testing the effect of quantization, limited contrastand allowing no under-illuminated pixels.Table 4.4 shows the results of increasing the improvement factor to variouslevels, depend on the percentage of pixels that become under-illuminated. For0.01% pixel error, which would allow for an improvement factor of 2.77, VDPcalculated that only 0.0002% of pixels might be noticed. That is the same asthe VDP result when all pixels are sufficiently illuminated, and only contrastand quantization artifacts are taken into account.It is worth noting that we are not modifying Ip with these tests, just theimage sent to the DMD. Also, it is not necessarily the brightest pixels thatare under-illuminated; they could be anywhere in the image. In this particularimage they start out in the bottom corner from where light has been redirectedto other brighter parts of the image.The results in Table 4.4 show that there is a significant potential for furtherincreasing the peak brightness if the constraints for perfect results are relaxed.More than 5x improvement can be obtained with 5% of pixels under-illuminated.The VDP statistics suggest it is possible to obtain a lot more improvementwithout sacrificing visual quality. When the image is inflated too much, however,the peaks of the image no longer are as bright as they were originally, andbecome easily-recognizable difference features. Subjectively, there was a bigjump between 1% and 2% of pixels underexposed; the extra 1% seems to comemostly from the central peak highlight, as shown in Figure 4.21.Visual-difference prediction allows us to make educated estimates as to theChapter 4. Light allocation 96Percentage pixels improvement fraction of pixelsunder-illuminated P > 95%0.0 2.22 0.0002%0.01 2.77 0.0002%0.1 3.32 0.0065%1.0 3.89 0.0923%2.0 4.29 0.37%5.0 5.07 1.13%Table 4.4: Results of allowing a percentage of pixels to be under-illuminated.Figure 4.21: VDP results from testing the effect of quantization, limited contrastand allowing 2% under-illuminated pixels.effect of adding an AMA to the illumination path of the projector. We haveshown that significant brightness increases can be achieved with no loss in de-tectable picture quality. There is also room for further increases if some losses inpixel quality are acceptable. In the future, with detailed knowledge of the ambi-ent light level and projector characteristics such as its gamma and native peakbrightness, there may be opportunities to distinguish between details that areand are not perceptible, and intelligently allocate the resources from the AMAaccordingly. Image details that would not be perceptible can be identified beforeallocation, allowing for higher, more targeted brightness increases. Perception-Chapter 4. Light allocation 97driven resource allocation using an AMA could help projectors become moreefficient and responsive to different environments, rather than putting the onussolely on the viewer to manually make changes to the room and the projectorsettings to obtain an acceptable picture.98Chapter 5Physical implementationFigure 5.1: Schematic of prototype, showing the major components.5.1 Prototype overviewWe have built a prototype to demonstrate dual light modulation using an AMAand DMD. The components of prototype are mounted on an optical table toallow for precise adjustments. Figure 5.1 shows a block diagram with the pro-totype components.Chapter 5. Physical implementation 995.1.1 Optical systemInitially, we planned on adding the AMA to an existing projector, a KnollSystems HD102 DMD projector (identical to the Infocus model 4805), withoutmodifying any of the other optical components. This proved impractical due tothe structure of the cavity that encloses the end of the integrating rod, foldingmirror and the DMD, where the AMA would have had to be placed. The cavitywas too small to place the AMA and still have adequate room for a relay lensbetween the AMA and DMD.Instead, we used this projector as a light source, cutting a hole to allow thelight from the lamp to exit the projector before it reaches its DMD. The light iscollected with a 60mm lens and directed to the AMA. A 45mm lens relays thelight from the AMA to a second projector, a Mitsubishi PK20 projector that isrelatively easy to open up to allow access to the DMD. The light incident to theDMD is reflected normally through a prism to the PK20 projection lens onto ascreen. Figure 5.2 shows a photograph of the prototype.Figure 5.2: Photograph of prototype, including projector light source, AMA,relay lens, and DMD.Both of the projectors use sequential colour. In the Knoll projector, a 6-section colour wheel of repeating red, green and blue filters is used, while thePK20 uses red, green and blue LEDs. The light from the LEDs is not usedfor this prototype. We use a video splitter to send the same VGA signal toboth projectors, so there is frame-level synchronization. Since the VGA signaldoes not dictate the orders of colours in a colour-sequential display, the coloursChapter 5. Physical implementation 100within the frame are not synchronized, and so are not correct. For this reasonwe use only greyscale images with the prototype.In the future, the colour wheel from the Knoll projector can be taken outentirely to obtain an approximately 3x increase in brightness for a greyscaleimage. Alternatively, a different light source that can be synchronized with thePK20 LEDs should be investigated to allow for colour images.5.1.2 AMA driverCMC Microsystems offers packaging options for the Micragem AMA, the mostsuitable for this application being an 84-pin pin-grid array (PGA) package.The fabricated AMA chip is mounted in the package, and connected to thepins of the package with up to 84 wire bonds. In our design, each compositemirror requires three independent analog voltages for the three electrodes thatcontrol the mirrors. With 84 pins, the maximum number of mirrors that can becontrolled is thus 28. The prototype chip has 7×4 mirrors. Since 1 pin mustbe reserved to electrically ground to the mirror surface, one of the mirrors onlyhas two electrodes activated.As shown in Figure 5.2, the PGA package is electrically connected to asmall printed circuit board though a socket that allows for easy chip insertionand removal. The printed circuit board (PCB) was designed to take 3 standard40-pin ribbon cables and route their signals to the 84 pins of the socket andthen the PGA. The three ribbon cables are in turn connected to three Ana-log Devices AD5535 digital-to-analog converter (DAC) evaluation boards whichprovide the high-voltage signals, as shown in Figure 5.3. The converters have14-bit resolution between 0 and 200V. Each DAC chip provides 32 channels ofanalog output, so there are 96 possible controls in total, meaning that everyelectrode in the AMA can be controlled using this apparatus. To control thethree DACs, We selected the Altera DE2 floating-point gate FPGA system asa convenient testbed for the prototype. Software was written for the DE2’sCyclone II FPGA to supply the Serial Peripheral Interface (SPI) required asinput by the DAC. As implemented, the FPGA can provide a clock signal tothe DAC at the AD5535’s highest rate possible, 30 MHz. At this clock signalrate, the channel update rate is 1.2 MHz. All 32 channels can thus be updatedat a rate of 37 kHz, which is more than sufficient given the resonance frequencyof the micromirrors is approximately 10 kHz.5.1.3 SynchronizationThe Altera DE2 development board includes a VGA output that can be con-trolled by the FPGA. We used this to demonstrate the capability of synchro-nizing the AMA with a DMD. A VGA signal to the DMD is synchronized with20 nanosecond accuracy to the AMA mirror control. This level of synchroniza-tion provides the opportunity for sub-video-frame synchronization between theAMA and the DMD. This would allow for moving the AMA mirrors to multiplepositions within one frame of video, as detailed in Section 4.7.5.Chapter 5. Physical implementation 101Figure 5.3: Schematic of the control signal flow in the prototype.To examine how the frame of video could be divided into sub-frames suit-able for the AMA, we measured the light output from the PK20 projector beforeits disassembly and inclusion in the AMA projector prototype. A photodiodeconnected to an oscilloscope measured the change in light intensity over time.Figure 5.4 shows two signals obtained from the projector showing its behaviourto two different input signals. We found that the each of the colours was dis-played 4 times per frame. The frame was 1/60s long. Each colour had a differentperiod within the frame: 1.78ms for green, 1.06ms for blue, and 1.38ms for red.The difference in periods for colours are possibly due to the relative strengthof the red, green and blue LEDs, and that the human visual system is mostsensitive to green in terms of brightness, and least to blue.Figure 5.4: Light intensity from a Mitsubishi PK20 projector over time. In theupper signal, the horizontal axis is time, and the vertical axis is intensity. Thelower signal shows the VGA sync signal. In a), the projector was given an imagewith full red and blue, while b) shows a signal with all three colours in varyingproportions.The FGPA writes to each DAC at every subfield change so that a differentstate of the AMA mirrors could be specified for each colour field. Due to databandwidth issues, only static images are implemented for sub-frame changes atthis time. Because the colour is not synchronized between the two projectors inthe prototype, the image display functionality of the FPGA code is not used inChapter 5. Physical implementation 102the current version of the prototype.5.2 Prototype resultsDue to the problems with the micromirror fabrication, no fully-functional AMAchips were available for integration into the prototype. The goals for this versionof the prototype were therefore modest. We wanted to show that we could makeregions of the image brighter or darker, while still keeping the DMD imageundistorted. This would validate the main claims of the work. Net gains inbrightness compared to an unmodified projector were not pursued because wewere using a different light source, and we significantly change the light path inorder to give ourselves enough freedom to experiment, which resulted in somelight losses. At this point our goal was not to compete against professionally-optimized projectors, but simply to show the potential of this method.An image was displayed with the AMA projector by directing a VGA signalfrom a computer to both to the light source (Knoll projector) and the DMD(PK20 projector). The AMA was controlled by a set of predetermined D/Avoltages which can be cycled through using buttons on the Altera DE2. Thesevoltages actuate the AMA mirrors, which in turn change the final intensitydistribution of the projected image. The objective was to measure the relativeluminance changes in the image as the AMA mirrors were actuated, in orderto validate the concept that the AMA can affect projector contrast and peakbrightness.A Nikon D70s SLR camera was used to capture images of the projectedlight as the settings for disparity and AMA voltages were manipulated. Thelight reflecting from the AMA to the DMD and finally out of the projectionlens was the sole source of illumination for the images. The Nikon camera hasits own gamma response, which make the pixel values reported by the cam-era non-linear measurements of luminance, even when a RAW format is used.Without knowing the actual camera response, no accurate measure of relativeintensity changes can be made. To obtain approximately linear measurementsof luminance, we constructed HDR images from multiple images taken with dif-ferent exposures using the software Photosphere by Greg Ward (Ward 2009).Photosphere uses an algorithm similar to that described in Mitsunga and Nayar(Mitsunaga and Nayar 1999) to estimate the radiometric response function ofa camera from images taken at different exposures. Once an estimate of theresponse function is made, the estimated (linear) radiometric distribution inthe scene can be calculated. Inanici and Glavin (Inanici and Galvin 2004) haveshown that this technique is a useful tool for capturing luminance values within10% of actual over a wide range of luminances, which is adequate for this ini-tial study. The Nikon camera has an exposure bracketing function that takesmultiple pictures of the same scene at differing exposure settings. The resultingimages were input into Photosphere to construct the HDR images.Figure 5.5 illustrates the effect of changing the disparity on the level of blur.As the blur increases, details of the AMA disappear, and the resulting MLChapter 5. Physical implementation 103Figure 5.5: Images showing the output of the AMA prototype with differentseparation settings. The DMD is showing a black grid of 50 pixels squares on awhite background. One AMA mirror in the centre right has been actuated.becomes more and more disperse even as the displacement of the ML increases.The ML can not be tracked between different disparities by the grid lines alone,because although care was taken to move the DMD only along the optical axis,the alignment of the DMD in the projector made this difficult to do in practiceover a wide range of disparities. For this reason, we can only make qualitativemeasurements between disparities.The grid lines are from the DMD, and are in 50 pixel increments. We canthus measure the size of the image AMA in units of DMD pixels by countingthe grid lines. Physically, the AMA array is 9 mm ×5 mm, while the DMD is11.17 mm ×8.38 mm. We can see from the image in Figure 5.5 that the AMA at-1mm disparity is approximately 650×450 pixels, which gives a magnificationlevel of 1 in the horizontal axis and 1.25 in the vertical axis. The image of theAMA gets larger as the disparity increases because of the increasing size of theblur kernel.In Figures 5.6, 5.7, 5.8, and 5.9, we examine the region affected by the oneAMA mirror tilted in Figure 5.5. To more clearly show the areas of relativechange that we are concerned about, the intensity of these images is in units ofpercentage difference, calculated asPd = 100It −InIn, (5.1)where It is the image with actuated AMA mirrors, and In is the image of non-actuated mirrors, and Pd the resulting percentage difference. The calculation isdone on a per-pixel basis.The mirror shown was actuated with 200V. From Figure 3.21, we see thetilt angle is between 0.4◦ and 2.2◦ at this voltage, depending on which mirrorChapter 5. Physical implementation 104Figure 5.6: One ML at approximately -1mm disparity (the DMD is is on therelay-lens side of the focal plane). Units of percent difference.in the composite arrangement is measured, due to their differing orientationswhen actuated as described in Section 3.6.2. For this reason we anticipate theshape of the ML will be spatially shortened in one dimension compared to theshape of the dark region that is left when the ML moves.Figure 5.6 shows the ML nearly in focus, with a slight bias in light to the topleft. At such low disparities, the ML hardly moves at all. The mirror is tiltingthe light towards the top left, showing that the separation is actually slightlynegative. If the DMD had been exactly on the focal plane, the ML would nothave been displaced at all, except from the bending artifacts that occur whenthese mirrors are actuated. The size of the ML was measured by measuring thegrid in terms of image pixels, measuring the size of the ML in image pixels, andthen solving for the size of the ML in terms of DMD pixels, with the knowledgethat the grid is in 50 DMD pixel increments. With this process, we measuredthe size of the ML in this case as approximately 75 pixels per side.Figure 5.7 shows how the ML has become blurred as the disparity increasesto 8.5mm. The size of the ML has grown to 100 pixels per side, and has startedto change shape due to mirror bending. The ML has been displaced 60 pixels inChapter 5. Physical implementation 105Figure 5.7: The same region with the mirror at the same tilt angle, at 8.5mmdisparity. Units of percent difference.a diagonal direction towards the bottom left. The actual ML is less wide thanthe region it is leaving because of the bending artifacts in the mirror. The angleof tilt is not uniform over the composite mirror, causing the ML to distort as itis tilted.At 14mm disparity, the size of the ML is now 120 pixels on its longest side,as shown in Figure 5.8. The ML has been displaced approximately 90 pixels.Figure 5.9 shows the ML at 30mm of disparity. Although hardly visible inFigure 5.5, showing a closeup of the region in false colour with units of relativedifference from the non-tilted case shows that the ML has shifted 160 pixels.The ML is further elongated, and now is spread over an area of 160 pixels perside.Also evident from Figures 5.6 5.7, 5.8, and 5.9 is that the peak intensitychange attenuates with the blur. At very low levels of blur such as 8.5mm,there is a 75% improvement in peak brightness, and a 25% decrease in peakdarkness. At 30mm of disparity, the peak has been reduced to a 40% increase,and 25% decrease. The non-symmetrical nature of these increases are likelyChapter 5. Physical implementation 106Figure 5.8: 14mm disparity, relative change. Units of percent difference.caused by the lens-effect of the bending AMA mirrors, as the ML becomesclearly smaller than the space it leaves as the disparity increases.Using Equation 2.12, we can use the measurements above to compare theestimated tilt angle of the micromirrors to the measurements shown in Figure3.21. A DMD pixel is 14µm square, and we have shown above that the mag-nification is approximately 1. Using the pixel measurements above, the 8.5mmdisparity works out to 2.7◦, 14mm disparity to 2.6◦, and the 30mm disparity to2.1◦. These discrepancies can be attributed to measure error of the ML range,the slight magnification of the vertical dimension of the AMA, the bending ofthe AMA mirrors themselves, and the differences in spring values between AMAmirrors in the array. However, all are within 0.5◦ of the value measured withthe white-light interferometer for 200V actuation.Figure 5.10 shows multiple mirrors actuated, creating a non-homgenous lightfield across most of the image. As the ML range increases with the blur disparity,some of the MLs blend into each other to form larger regions both brighter anddarker than the original luminance.Overall, these measurements show that the approach of using an AMA issuccessful at redistributing light from one region of the projected image to an-other. We also demonstrated that the AMA does not geometrically distort theimage from the DMD in any way. The effect that disparity has on both rangeChapter 5. Physical implementation 107Figure 5.9: 30mm disparity, relative change. Units of percent difference.and blur was shown, and validates the approach to simulations taken in Chapter4. While difficulties with micromirror fabrication severely impact the attainabletilt angle, it was adequate to show that the AMA can make areas of the imageboth brighter and darker as the mirrors are actuated.Chapter 5. Physical implementation 108Figure 5.10: Relative change for four different disparity settings, showing mul-tiple mirrors actuated.109Chapter 6Conclusions and futurework6.1 Summary and conclusionsThis thesis has presented a method to significantly improve the brightness andcontrast capability of a projector through the addition of an analog micromirrorarray. By channeling light to where it is needed and away from where it is not,an AMA projector makes better use of its light source, which is one of the mostexpensive components of today’s projectors. After the addition of an AMA, thelight reaching the primary image modulator, such as a DMD, is not consideredto be uniform. Instead, the distribution from the AMA mirrors is simulated,and the compensation for the non-homogeneity is applied to the original imagebefore it is sent to the DMD. The result is an image of higher contrast and peakbrightness than would otherwise be possible with the same projection lamp.Dispensing with the assumption of homogeneity creates other opportunitiesfor increases in efficiency. To obtain a uniform distribution, the light from thelamp extends past the borders of the DMD so that apodization is minimized.This overfill light is normally wasted. In an AMA projector, however, the overfillcan be used to further illuminate the image because homogeneity is no longerassumed.The optical issues that arise when adapting a projector with an AMA wereanalyzed. Because of the geometrical nature of the light from a non-laser pro-jector lamp, light from each mirror of the AMA is blurred before reaching theDMD. A methodology for simulating the effects of blur, and its relation to mir-ror range, and number of mirrors was implemented. It was found that a largerrange of movement for the mobile light from an AMA mirror (ML) implies alarger blur. This can be mitigated up to a point by increasing the maximummechanical tilt angle of the mirrors.Two-degree of freedom analog micromirror arrays were designed, fabricatedand analyzed for this purpose using multi-user MEMS fabrication processes. Anovel way of optimizing the tradeoffs between tilt angle, mirror size, and mirrorresonance frequency by splitting the mirrors into smaller functional subsectionsthat move synchronously was employed.We developed several algorithms that determine favourable placement of themobile lights from each of the micromirrors in the array, in order to best improvethe image. Results from these implementations show that with a mirror arrayChapter 6. Conclusions and future work 110of 28 mirrors, the average brightness could be increased by a factor of 1.2 andthe contrast by 1.4 if micromirrors were available that could be tilted to ±3.5◦with the addition of this technology, without changing the projector lamp. Theimprovement factor rises to 2.25 and contrast factor by 5 if a minimal numberof pixels are allowed to be under-illuminated. Additional micromirrors in thearray would also increase this improvement.Finally, we constructed a physical prototype adapted from a commodityprojector. Mirror control electronics were designed to provide a test-bed for theAMA projector prototype. We verified with the prototype that the AMA canincrease the peak brightness and contrast of the projector.6.2 Summary of ContributionsIn summary, this thesis has examined a novel method to dynamically reallocatethe light from a projector lamp from dark regions to bright regions on an image-dependent basis in order to increase projector peak brightness, contrast, andefficiency. This is also described in our paper (Hoskinson and Stoeber 2008).This approach required novel work in a number of areas:• A theoretical framework for examining the tradeoffs between optical pa-rameters that affect AMA system performance.• The design and fabrication of an analog micromirror array suitable foran AMA projector, with composite mirrors that optimize the tradeoffsbetween mirror tilt angle, size, and dynamic behaviour (Hoskinson et al.2007a; Hoskinson et al. 2007b).• Algorithms and software implementations that allocate the mobile lightsdepending on the image, taking into account the physical limitations ofthe mirrors and projector.• A prototype implementation demonstrating this method, showing thatregions can become brighter as well as darker.6.3 Future workThe addition of the AMA into a projector involves several added costs. Furtherresearch and development will have to be done to minimize them. For instance,the AMA could be made with a process that facilitates the incorporation of high-voltage CMOS that can be connected via through-holes to the mirror electrodes.This would remove the need for separate D/A converters and wire-bonding, andallow for more efficient use of electrode space, since routing leads would no longerbe needed. This would also allow for many more individually-controlled mirrors.This would involve a substantial research effort to design a new fabricationprocess that best combine the high-voltages CMOS with the MEMS materialssuch as single-crystal silicon.Chapter 6. Conclusions and future work 111Shaping the DMD aperture to let in as much on-state light given the abilityof the AMA to divert unwanted light could mitigate losses that occur in conven-tional non-AMA projectors from contrast control. Normally, some of the usableillumination is clipped before reaching the DMD to prevent stray unwanted lightfrom affecting contrast, but with the added ability to control where light reachesthe DMD, and at what angle, more initial stray light can be tolerated withoutaffecting overall contrast. Shaping the DMD aperture would also help minimizeAMA clipping loss.This work analyzed only static images. The sample images tested showedthat the improvement factor can vary substantially from image to image. Asystem must be implemented to ensure that this does not cause perceptibleartifacts as the global brightness changes over time for video. The adaptive iris(Toyooka et al. 2005) included in some projectors faces the same problem, andhas been mitigated to some degree.Since the distribution of light on the DMD is already non-homogeneous,the integrating rod normally present in a projector could be omitted, furtherreducing the cost of the device. The AMA could be thought of as an “activeintegrator”; in an integrator, light that is originally near the centre of the distri-bution is moved to the exteriors. The AMA instead moves this light to whereverit is needed in the image, not necessarily the centre. The non-homogeneous na-ture of the incident light could be taken into account during the normal image-processing step of the projector. This would also increase system efficiencyversus an equivalent non-AMA projector. In this case, because the light inci-dent on the AMA would be non-homogeneous, a separate model of every MLwould be needed.The large gap in improvement factor between perfect reconstruction and thatwithevenonly200under-illuminatedpixels(1.3vs. 2.5)showsthatthereisworkto do in optimizing the allocation algorithms to avoid outliers. Further researchcould also be done to determine whether outliers could be less perceptible thana similar error on a regular projector. It may be that the greater dynamic rangeof an AMA-projected image inherently masks errors to some degree.The visual difference prediction employed in Section 4.9 was only for valida-tion purposes, but it may be possible to employ a small subsection as part of theallocation algorithm in a predictive capacity. If the pixels could be weighted by a‘perceptibility’ metric, then the AMA projector could make intelligent decisionsas to the potential impact of under-illuminated regions during the allocationalgorithm. Light could even be diverted away from regions of low perceptualsignificance towards regions of high perceptual significance for a greater overallimprovement than if every image pixel had an identical weighting.Another avenue for future work is with the non-homogeneous correctionmechanism needed to produce accurate images in an AMA projector. It couldbe extended to cover ambient light conditions with the addition of a camera.Various methods for compensating for room illumination have been suggested inthe literature, such as (Nayar et al. 2003; Wang et al. 2005; Ashdown et al. 2006;Park et al. 2008). All involve capturing the state of the surface projected uponusing a camera, and compensating for any non-uniformities by changing theChapter 6. Conclusions and future work 112image projected. Each approach comes at the cost of dynamic range, since someof the image bits normally used for greyscale values are now used for radiometriccompensation. The output of a projector can saturate if the compensationexceeds the projector’s dynamic range, resulting in perceptible artifacts in thecompensated image.A similar technique for radiometric compensation could be used with acamera-enabled AMA projector that could alleviate dynamic range limitations.Images from a camera-equipped AMA projector could be fed into the image pro-cessing algorithm, and non-homogeneous screen brightnesses accounted for bythe image processing algorithm when determining the control sequences for theAMA and DMD. The added dynamic range introduced with the AMA wouldalleviate the problems of previous approaches.This work has demonstrated that an AMA-enhanced projector can makeprojectors intelligent allocators of their light sources. This ability opens upseveral interesting research directions, as the projector could conceivably takeinto account the ambient lighting in the room, the psychophysical limitations ofthe viewers, and the content being presented to display the best possible imageover a wide range of conditions.113BibliographyAgarwal, P., and Varadarajan, K. 2004. A near-linear constant-factorapproximation for euclidean bipartite matching? In Proceedings of the twen-tieth annual symposium on Computational geometry, ACM New York, NY,USA, 247–252.Aksyuk, V., Basavanhally, N. R., Bishop, D. J., Bolle, C. A., Chang,C. C., Frahm, R., Gasparyan, A., Gates, J. V., George, R., Giles,C. R., Kim, J., Kolodner, P. R., Lee, T. M., Neilson, D. T., Nijander,C., Nuzman, C. J., Paczowski, M., Papzian, A. R., Pardo, F., Ramsey,D. A., Ryf, R., Scotti, R. E., Shea, H., and Simon, M. E. 2003. 238x238micromechanical optical cross connect. IEEE Photonics Technology Letters 15,4, 587–589.ANSYS, 2009. ANSYS. http://www.ansys.com. Accessed 03/24/09.Ashdown, M., Okabe, T., Sato, I., and Sato, Y. 2006. Robust Content-Dependent Photometric Projector Compensation. Proceedings of the Confer-ence on Computer Vision and Pattern Recognition Workshop (CVPRW’06),Projector-Camera Systems (PROCAMS), 6.Barten, P. G. J. 1992. Physical model for the contrast sensitivity of thehuman eye. Proceedings of SPIE 1666, 57–72.Barten, P. 1996. Effect of Gamma on Subjective Image Quality. SID Intl.Symp. Digest of Technical Papers.Bishop, D. J., Giles, C. R., and Austin, G. P. 2002. The Lucent Lamb-daRouter: MEMS technology of the future here today. IEEE CommunicationsMagazine 40, 3, 75–79.Bloom, D., 1997. The grating light valve: revolutionizing display technology.Borovic, B., Lewis, F., Liu, A., Kolesar, E., and Popa, D. 2006.The lateral instability problem in electrostatic comb drive actuators: modelingand feedback control. Journal of Micromechanics and Microengineering 16, 7,1233–1241.Brennesholtz, M. S., and Stupp, E. H. 2008. Projection Displays, 2nd ed.Wiley.Bibliography 114Brennesholtz, M. S. 1996. Light collection efficiency for light valve projec-tion systems. In Projection Displays II Proceedings of SPIE, vol. 2650, 71–79.Brennesholtz, M. 2007. Invited Paper: The evolution of projection dis-plays: From mechanical scanners to microdisplays. Journal of the Society forInformation Display 15, 759.Brennesholtz, M. S., 2009. 3D: Brighter is Better,http://displaydaily.com/2009/05/27/3d-brighter-is-better/ Last accessed05/06/2009.Burt, P., and Adelson, E. 1983. A multiresolution spline with applicationto image mosaics. ACM Transactions on Graphics 2, 4, 217–236.CMC. 2005. Introduction to Micragem: A silicon-on-insulator based micro-machining process. Tech. rep., Micralyne Inc. Version 4.0.CMC, 2009. Canadian microelectronics corporation, www.cmc.ca.Comtois, J. H., Bright, V. M., Gustafson, S. C., and Michalicek,M. A. 1995. Implementation of hexagonal micromirror arrays as phase-mostlyspatial light modulators. Proceedings of SPIE 2641, 76–87.Comtois, J., Michalicek, A., Cowan, W., and Butler, J. 1999. Surface-micromachined polysilicon moems for adaptive optics. Sensors and Actuators78, 54–62.Conant, R. 2002. Micromachined Mirrors. PhD thesis, University of Cali-fornia, Berkeley.Conner, A. 2006. LEDs for Microdisplay Illumination. SID Internationalsymposium digest of technical papers 37, 2, 1819.Cowan, W. D., Bright, V. M., Lee, M. K., Comtois, J. H., andMichalicek, M. A. 1998. Design and testing of polysilicon surface-micromachined piston micromirror arrays. Proceedings of SPIE 3292, 60–70.Dagel, D., Cowan, W., Spahn, O., Grossetete, G., Grine, A., Shaw,M., Resnick, P., and Jokiel Jr, B. 2006. Large-stroke MEMS deformablemirrors for adaptive optics. Journal of Microelectromechanical Systems 15, 3,572–583.Damberg, G., Seetzen, H., Ward, G., Heidrich, W., and Whitehead,L. 2007. High Dynamic Range Projection Systems. Proceedings of the 2006Society for Information Display Annual Symposium.Debevec, P. 2008. A median cut algorithm for light probe sampling. InInternational Conference on Computer Graphics and Interactive Techniques,ACM New York, NY, USA.Bibliography 115Derra, G., Moench, H., Fischer, E., Giese, H., Hechtfischer, U.,Heusler, G., Koerber, A., Niemann, U., Noertemann, F., Pekarski,P., et al. 2005. UHP lamp systems for projection applications. JOURNALOF PHYSICS-LONDON-D APPLIED PHYSICS 38, 17, 2995.Dewald, D., Segler, D., and Penn, S. 2004. Advances in contrast en-hancement for DLP projection displays. Journal of the Society for InformationDisplay 11, 177–181.Dokmeci, M., Pareek, A., Bakshi, S., Waelti, M., Fung, C., Heng,K. H., and Mastrangelo, C. 2004. Two-axissingle-crystalsiliconmicromir-ror arrays. Journal of Microelectromechanical Systems 13, 6, 1006–1017.Dutta, S. B., Ewin, A. J., Jhabvala, M. D., Kotecki, C. A., Kuhn,J. L., and Mott, D. B. 2000. Development of individually addressablemicromirror arrays for space applications. In Proceedings of SPIE – Volume4178, 365–371.Fairchild, M. 1995. Considering the surround in device-independent colorimaging. Color Res. Appl 20, 6, 352–363.Hah, D., Huang, S. T.-Y., Tsai, J.-C., Toshiyoshi, H., and Wu, M. C.2004. Low-voltage, large-scan angle MEMS analog micromirror arrays withhidden vertical comb-drive actuators. Journal of Microelectromechanical Sys-tems 13, 2, 279–289.Hah, D., Patterson, P. R., Nguyen, H. D., Toshiyoshi, H., and Wu,M. C. 2004. Theory and experiments of angular vertical comb-drive actua-tors for scanning micromirrors. IEEE Jorunal of Selected Topics in QuantumElectronics 10, 3, 505–513.Halbach, K. 1964. Matrix representation of Gaussian optics. AmericanJournal of Physics 32, 90.Hecht, E. 2002. Optics, 4th ed. Addison Wesley, San Francisco, CA, USA.Heckaman, R. 2006. Effect of DLP projector white channel on perceptualgamut. Journal of the SID, 755.Heckbert, P. 1982. Color image quantization for frame buffer display. ACMSIGGRAPH Computer Graphics 16, 3, 297–307.Hibbeler, R. 2008. Mechanics of Materials, 7th ed. Pearson Prentice Hall,New Jersey.Hornbeck, L. J. 1983. 128 x 128 deformable mirror device. IEEE Transac-tions on Electron Devices ED-30, 539–545.Hornbeck, L. J., 1996. Active yoke hidden hinge digital micromirror device.Bibliography 116Hornbeck, L. 1997. Digital Light Processing TM for High-Brightness, High-Resolution Applications. Electronic Imaging EI ‘97 Projection Displays III,IS&T and SPIE, San Jose CA 3013, 27–40.Hoskinson, R., and Stoeber, B. 2008. High-dynamic range image projec-tion using an auxiliary mems mirror array. Optics Express 16, 7361–7368.Hoskinson, R., Mansoor, I., and Stoeber, B. 2007. High mechani-cal bandwidth polysilicon micromirrors for large area / high deflection anglemicromirror arrays. In Canadian Conference on Electrical and Computer En-gineering.Hoskinson, R., Mansoor, I., and Stoeber, B. 2007. Submirror pixels forlarge scan angle micromirror arrays. In Canadian Workshop on MEMS andMicrofluidics.Hwang, K.-H., Song, Y.-J., and Kim, S.-G. 1998. Thin-film micromirrorarray for high brightness projection displays. Japanese Journal of AppliedPhysics 37, 12B, 7074–7077.Iisaka, H., Toyooka, T., Yoshida, S., and Nagata, M. 2003. NovelProjection System Based on an Adaptive Dynamic Range Control Concept. InProcoeedings Int Disp Workshops, vol. 10, 1553–1556.Inanici, M., and Galvin, J. 2004. Evaluation of High Dynamic Range Pho-tography as a Luminance Mapping Technique. Tech. rep., Lawrence BerkeleyNational Laboratory, Lighting Research Group, LBNL-Report.Jacobson, B., Gengelbach, R., Stewart, C., and Rutan, D. 1998.Metal Halide Lighting Systems and Optics for High Efficiency Compact LCDProjectors. In Proc. SPIE, vol. 3296, 38–45.Jacobson, R., Ray, S., Attridge, G. G., and Axford, N. 2000. TheManual of photography: Photographic and Digital Imaging, 9th ed. Focal Press.Janssen, P., and Shimizu, J., 1995. High contrast illumination system forvideo projector. Patent, Aug. 15. US Patent 5,442,414.Judy, J., and Muller, R. 1997. Magnetically actuated, addressable mi-crostructures. Microelectromechanical Systems, Journal of 6, 3, 249–256.Jung, I., Krishnamoorthy, U., and Solgaard, O. 2006. High fill-factortwo-axis gimbaled tip-tilt-piston micromirror array actuated by self-Alignedvertical electrostatic combdrives. Microelectromechanical Systems, Journal of15, 3, 563–571.Kanayama, H., Maeda, M., Miwa, T., Ikeda, T., Murata, H., andChihara, K. 2006. Ultra Small Projector with High Efficiency IlluminationSystem. In International Conference on Consumer Electronics (ICCE’06).Bibliography 117Kessel, P. V., Hornbeck, L., Meier, R., and Douglass, M. 1998. AMEMS-based projection display. In Proceedings of the IEEE, vol. 86, 1687–1704.Kiang, M., Solgaard, O., Lau, K., and Muller, R. 1998. Electrostaticcombdrive-actuated micromirrors for laser-beamscanning and positioning. Mi-croelectromechanical Systems, Journal of 7, 1, 27–37.Kim, S., Barbastathis, G., and Tuller, H. 2004. MEMS for optical func-tionality: Special issue on electroceramics in micro-electro-mechanical systems.Journal of Electroceramics 12, 1-2, 133–144.Koester, D., Cowen, A., Mahadevan, R., Stonefield, M., andHardy, B. 2003. PolyMUMPs design handbook. Tech. rep., MEMSCAP.Revision 10.0.Krishnamoorthy, U., Lee, D., and Solgaard, O. 2003. Self-alignedvertical electrostatic combdrives for micromirror actuation. Microelectrome-chanical Systems, Journal of 12, 4, 458–464.Kuhn, H. 1955. The Hungarian method for the assignment and transportationproblems. Naval Research Logistics Quarterly 2, 83–97.Kusakabe, Y., Kanazawa, M., Nojiri, Y., Furuya, M., andYoshimura, M. 2009. A high-dynamic-range and high-resolution projectorwith dual modulation. In Proceedings of SPIE, vol. 7241.Legtenberg, R., Groeneveld, A., and Elwenspoek, M. 1996. Comb-drive actuators for large displacements. J. Micromech. Microeng 6, 3, 320–329.Lin, J. E., Michael, F. S. J., and Kirk, A. G. 2001. Investigationof improved designs for rotational micromirrors using multi-user MEMS pro-cesses. MEMS Design, Fabrication, Characterization, and Packaging SPIE4407, 202–213.Majumder, A., Bhasker, E., and Juang, R. 2008. Advances towardshigh-resolution pack-and-go displays: A survey. Journal of the Society forInformation Display 16, 481.Mang, O., Chou, C., and Yuan, O. 2008. Optimized design and analysisof dual-lamp illumination modules compatible with existing projector systems(Journal Paper). Optical Engineering 47, 07.Mantiuk, R., Daly, S., Myszkowski, K., and Seidel, H.-P. 2005. Pre-dicting visible differences in high dynamic range images - model and its calibra-tion. In Human Vision and Electronic Imaging X, IS&T/SPIE’s 17th AnnualSymposium on Electronic Imaging (2005), B. E. Rogowitz, T. N. Pappas, andS. J. Daly, Eds., vol. 5666, 204–214.Bibliography 118Milanovic, V., Last, M., and Pister, K. 2001. Torsional micromirrorswith lateral actuators. Transducers 2001.Mitsubishi. Laservue.Mitsunaga, T., and Nayar, S. 1999. Radiometric self calibration. In Com-puter Vision and Pattern Recognition, 1999. IEEE Computer Society Confer-ence on., vol. 1.Motamedi, M., Ed. 2005. MOEMS: micro-opto-electro-mechanical systems.SPIE Press.Nagahara, H., Kuthirummal, S., Zhou, C., and Nayar, S. 2008. Flexi-ble Depth of Field Photography. In European Conference on Computer Vision(ECCV).Nayar, S., Peri, H., Grossberg, M., and Belhumeur, P. 2003. A Pro-jection System with Radiometric Compensation for Screen Imperfections. FirstIEEE International Workshop on Projector-Camera Systems (PROCAMS-2003).Niven, G., and Mooradian, A. 2006. Trends in laser light sources forprojection display. In 13th International Display Workshops (IDW ’06).Pan, J., Wang, C., Lan, H., Sun, W., and Chang, J. 2008. Highefficiency pocket size projector with a compact projection lens and LED basedlight source system. Applied optics 47, 19, 3406–3414.Park, H., Lee, M., Seo, B., Park, J., Jeong, M., Park, T., Lee, Y.,and Kim, S. 2008. Simultaneous Geometric and Radiometric Adaptationto Dynamic Surfaces With a Mobile Projector-Camera System. Circuits andSystems for Video Technology, IEEE Transactions on 18, 1, 110–115.Pavlovych, A., and Stuerzlinger, W. 2005. A High-Dynamic RangeProjection System. Progress in biomedical optics and imaging 6, 39.Peterson, K. E. 1982. Silicon as a mechanical material. Proceedings of theIEEE 70, 5, 420–457.Pinho, G. P. 2003. Optics of digital cinema. Projection Displays IX (Pro-ceedings of SPIE/IS&T Volume 5002), 123–131.Poynton, C. 2003. Digital Video and HDTV: Algorithms and Interfaces.Morgan Kaufmann.Ramanarayanan, G., Ferwerda, J., Walter, B., and Bala, K. 2007.Visual equivalence: towards a new standard for image fidelity. ACM Transac-tions on Graphics (TOG) 26, 3.Rao, S. S. 2003. Mechanical Vibrations, 4th ed. Prentice Hall.Bibliography 119Reinhard, E., Ward, G., Pattanaik, S., and Debevec, P. 2005. HighDynamic Range Imaging: Acquisition, Display, and Image-Based Lighting(The Morgan Kaufmann Series in Computer Graphics). Morgan KaufmannPublishers Inc., San Francisco, CA, USA.Rempel, A., Trentacoste, M., Seetzen, H., Young, H., Heidrich,W., Whitehead, L., and Ward, G. 2007. Ldr2Hdr: on-the-fly reverse tonemapping of legacy video and photographs. ACM Transactions on Graphics(TOG) 26, 3.Roufs, J., Koselka, V., and van Tongeren, A. 1994. Global brightnesscontrast and the effect on perceptual image quality. In Proceedings of SPIE,vol. 2179.Sandia National Laboratories, 2007. Sandia Ultra-planar, Multi-level MEMS Technology 5 (SUMMiT V), http://www.mems.sandia.gov/tech-info/summit-v.html.Seetzen, H., Whitehead, L., and Ward, G. 2003. A high dynamic rangedisplay using low and high resolution modulators. Society for InformationDisplay (SID) Digest of Technical Papers, 1450–1453.Seetzen, H., Heidrich, W., Stuerzlinger, W., Ward, G., Whitehead,L., Trentacoste, M., Ghosh, A., and Vorozcovs, A. 2004. High dy-namic range display systems. ACM Transactions on Graphics (Siggraph 2004)23, 3, 760–768.Senturia, S. D. 2001. Microsystem design. Kluwer Academic Publishers,Norwell, MA, USA.Simoncelli, E. P., 2007. matlabPyrTools,http://www.cns.nyu.edu/ lcv/software.html.SMPTE. 2003. 196M Indoor Theater and Review Room Projection – ScreenLuminance and Viewing Conditions.Stokes, M., Anderson, M., Chandrasekar, S., and Motta, R. 1996. AStandard Default Color Space for the Internet–sRGB. Microsoft and Hewlett-Packard Joint Report, Version 1.Subbarao, M. 1987. Direct recovery of depth-map I: Differential methods. InProc. IEEE Computer Society Workshop on Computer Vision, Miami Beach.Sutherland, E. ., 2009. Untitled, http://www.es.com/Accessed18/04/2009.Tang, W., Nguyen, T., and Howe, R. 1989. Laterally driven polysiliconresonant microstructures. Micro Electro Mechanical Systems, 1989, Proceed-ings,’An Investigation of Micro Structures, Sensors, Actuators, Machines andRobots’. IEEE, 53–59.Bibliography 120Taylor, W. P., Brazzle, J. D., Osenar, A. B., Corcoran, C. J.,Jafri, I. H., Keating, D., Kirkos, G., Lockwood, M., Pareek, A.,and Bernstein, J. J. 2004. A high fill factor linear mirror array for awavelength selective switch. Journal of Micromechanics and Microengineering14, 1, 147–152.Texas Instruments. 2005. DMD 0.7 XGA 12 DDR DMD Discovery.Texas Instruments. 2005. Single-panel DLP projection system optics. Tech.rep. Application Report – Discovery DLPA002.Toyooka, T., Yoshida, S., and Iisaka, H. 2005. Illumination control sys-tem for adaptive dynamic range control. Journal of the Society for InformationDisplay 13, 105.Trentacoste, M., Heidrich, W., Whitehead, L., Seetzen, H., andWard, G. 2007. Photometric image processing for high dynamic range dis-plays. Journal of Visual Communication and Image Representation 18, 5,439–451.Trentacoste, M. 2006. Photometric image processing for high dynamicrange displays (MSc thesis). The Faculty of Graduate Studies (Computer Sci-ence), The University Of British Columbia, Vancouver.Tsai, J.-C., Fan, L., Hah, D., , and Wu, M. 2004. A high fill-factor, largescan-angle, two-axis analog micromirror array driven by leverage mechanism.In International Conference on Optical MEMS and Their Applications.Tsai, J.-C., Huang, S., and Wu, M. 2004. High fill-factor two-axis analogmicromirror array for 1xN2 wavelength-selective switch. In IEEE InternationalConference on Micro Electro Mechanical Systems, 101–104.Tsai, J., Chiou, S., Hsieh, T., Sun, C., Hah, D., and Wu, M. 2008.Two-axis MEMS scanners with radial vertical combdrive actuators—design,theoretical analysis, and fabrication. Journal of Optics A: Pure and AppliedOptics 10, 044006.Tsang, S.-H., and Parameswaran, M. 2005. Self-locking vertical operationsingle crystal silicon micromirrors using silicon-on-insulator technology. InElectrical and Computer Engineering, 2005. Canadian Conference on, 429–432.Tuantranont, A., Bright, V. M., Zhang, W., and Lee, Y. C. 1999.Packaging of lenslet array on micromirrors. Proc. SPIE 3631, 156–164.Tuantranont, A., Bright, V., Liew, L., Zhang, W., and Lee, Y. 2000.Smart phase-only micromirror array fabricated by standard CMOSprocess. InMicro Electro Mechanical Systems, 2000. MEMS 2000. The Thirteenth AnnualInternational Conference on, 455–460.Bibliography 121Van Giel, B., Meuret, Y., Bogaert, L., Murat, H., De Smet, H., andThienpont, H. 2007. Efficient and Compact Illumination in LED ProjectionDisplays. In SID International symposium digets of technical papers, vol. 38,Society for information display.Wang, K., B¨ohringer, K. F., and Sinclair, M. 2003. Low voltage andpull-in state adjustable dual servo scanning mirror. In The 5th Pacific RimConference on Lasers and Electro-Optics.Wang, D., Sato, I., Okabe, T., and Sato, Y. 2005. Radiometric Com-pensation in a Projector-Camera System Based on the Properties of HumanVision System. Proceedings of IEEE International Workshop on Projector-Camera Systems 2005 2, 8.Ward, G., 2009. Photosphere, http://www.anyhere.com/.Weisstein, E. W., 2007. Circle-circle intersection. From MathWorld–A Wolfram Web Resource. http://mathworld.wolfram.com/Circle-CircleIntersection.html.Wen, J., Hoa, X. D., Kirk, A. G., and Lowther, D. A. 2004. Analysis ofthe performance of a MEMS micromirror. In IEEE Transactions on Magnetics,vol. 40, 1410–1413.Winston, R., Minano, J., and Benitez, P. 2005. Nonimaging Optics.Academic Press.Yalcinkaya, A., Urey, H., Brown, D., Montague, T., and Sprague,R. 2006. Two-axis electromagnetic microscanner for high resolution displays.J. Microelectromech. Syst 15, 786–94.Young, W. C., and Budynas, R. 2001. Roark’s formula for stress andstrain. McGraw-Hill, September.Zhang, X., Chau, F., Quan, C., Lam, Y., and Liu, A. 2001. A study ofthe static characteristics of a torsional micromirror. Sensors & Actuators: A.Physical 90, 1-2, 73–81.

Cite

Citation Scheme:

        

Citations by CSL (citeproc-js)

Usage Statistics

Share

Embed

Customize your widget with the following options, then copy and paste the code below into the HTML of your page to embed this item in your website.
                        
                            <div id="ubcOpenCollectionsWidgetDisplay">
                            <script id="ubcOpenCollectionsWidget"
                            src="{[{embed.src}]}"
                            data-item="{[{embed.item}]}"
                            data-collection="{[{embed.collection}]}"
                            data-metadata="{[{embed.showMetadata}]}"
                            data-width="{[{embed.width}]}"
                            async >
                            </script>
                            </div>
                        
                    
IIIF logo Our image viewer uses the IIIF 2.0 standard. To load this item in other compatible viewers, use this url:
http://iiif.library.ubc.ca/presentation/dsp.24.1-0068959/manifest

Comment

Related Items