Open Collections

UBC Theses and Dissertations

UBC Theses Logo

UBC Theses and Dissertations

Inverse tone mapping of standard dynamic range content for high dynamic range applications Mohammadi, Pedram 2020

Your browser doesn't seem to have a PDF viewer, please download the PDF to view this item.

Notice for Google Chrome users:
If you are having trouble viewing or searching the PDF with Google Chrome, please download it here instead.

Item Metadata

Download

Media
24-ubc_2020_may_mohammadi_pedram.pdf [ 5.94MB ]
Metadata
JSON: 24-1.0388672.json
JSON-LD: 24-1.0388672-ld.json
RDF/XML (Pretty): 24-1.0388672-rdf.xml
RDF/JSON: 24-1.0388672-rdf.json
Turtle: 24-1.0388672-turtle.txt
N-Triples: 24-1.0388672-rdf-ntriples.txt
Original Record: 24-1.0388672-source.json
Full Text
24-1.0388672-fulltext.txt
Citation
24-1.0388672.ris

Full Text

Inverse Tone Mapping of Standard Dynamic Range Content for High Dynamic Range Applications by  Pedram Mohammadi  B.A.Sc., Ferdowsi University of Mashhad, Iran M.A.Sc., Ferdowsi University of Mashhad, Iran  A THESIS SUBMITTED IN PARTIAL FULFILLMENT OF THE REQUIREMENTS FOR THE DEGREE OF  Doctor of Philosophy in THE FACULTY OF GRADUATE AND POSTDOCTORAL STUDIES (Electrical and Computer Engineering)  THE UNIVERSITY OF BRITISH COLUMBIA (Vancouver)  February 2020  © Pedram Mohammadi, 2020 ii  The following individuals certify that they have read, and recommend to the Faculty of Graduate and Postdoctoral Studies for acceptance, the dissertation entitled: Inverse Tone Mapping of Standard Dynamic Range Content for High Dynamic Range Applications  submitted by  Pedram Mohammadi in partial fulfillment of the requirements for the degree of Doctor of Philosophy in Electrical and Computer Engineering  Examining Committee: Panos Nasiopoulos Supervisor  John D. Madden Supervisory Committee Member  Edmond Cretu University Examiner Ian Mitchell University Examiner   iii  Abstract High Dynamic Range (HDR) technology has revolutionized the field of digital media, affecting different aspects such as capturing, compression, transmission and display. By modeling the behavior of Human Visual System (HVS) when perceiving brightness and color, HDR technology offers a life-like viewing experience that is far superior to what Standard Dynamic Range (SDR) technology could achieve. While HDR technology has a disruptive impact in different fields, it also opens new revenue sources for SDR content owners and broadcasters that will continue producing real-time events in SDR format for the near future. For the latter case, SDR content need to be efficiently converted to HDR format, taking advantage of the superior visual quality of HDR displays. Over the years, several attempts aimed at converting SDR content to HDR format, a process known as inverse Tone Mapping (iTM). The design of inverse Tone Mapping Operators (iTMOs) is considered a difficult task, as it tries to expand the brightness and color information to ranges not originally captured by SDR cameras.  In this thesis, we propose novel iTMOs that can effectively deal with all types of SDR content from dark, to normal and bright scenes, producing high visual quality HDR content. Our proposed methods work in the perceptual domain, which allows us to take advantage of the sensitivity of the human eye to brightness changes in different areas of the scene during the mapping process. To preserve the overall artistic impression, we developed methods that divide the SDR frame into dark, normal (average brightness), and bright regions, allowing us to keep intact dark and bright areas, without darkening or brightening up the frame. We also address the issue of the color shift in SDR to HDR mapping by proposing a perception-based color adjustment method that preserves the hue of colors with insignificant iv  changes in brightness, producing HDR colors that are faithful to their SDR counterparts. Subjective and objective evaluations have shown that our proposed iTMOs outperform the state-of-the-art methods in terms of overall visual quality of the generated HDR video and generating HDR colors that closely follow their SDR counterparts. v  Lay Summary In this thesis, we propose novel iTMOs that are capable of dealing with all types of SDR frames from dark, to normal and bright frames generating high visual quality HDR videos. Our proposed methods work in the perceptual domain to model the sensitivity of the human eye to brightness changes in different regions of the scene. To maintain the overall visual impression of the input SDR frame, our methods divide the input SDR frame into three regions in terms of their brightness namely dark, normal, and bright. Furthermore, we achieve a good balance between the overall contrast and brightness of the HDR frame by optimizing our mapping process. We also propose a perception-based color adjustment method that maintains the color accuracy between input SDR and generated HDR frames and preserves the hue of colors as we move from SDR to HDR while resulting in negligible luminance change. vi  Preface  This thesis presents research conducted by Pedram Mohammadi under the guidance of Dr. Panos Nasiopoulos. A list of publications resulting from the work presented in this thesis is provided below. The main body of Chapter 2 is taken from the publication in [P1]. The content of Chapter 3 appears in the conference paper [P2] and the submitted journal paper of [P3]. The content of Chapter 4 appears in the conference paper [P4] and the submitted journal paper [P5]. The work presented in [P1] – [P5] was performed by Pedram Mohammadi, including literature review, designing and implementing the proposed methods, performing all evaluations, analyzing the results and writing the manuscripts. Dr. Panos Nasiopoulos, Dr. Mahsa T. Pourazad, and Dr. Maryam Azimi provided guidance and editorial input. The subjective studies in this work were covered under the UBC Ethics Board (H12-00308). This thesis was written by Pedram Mohammadi, with editing assistance from Dr. Panos Nasiopoulos. List of Publications Based on Work Presented in This Thesis [P1] P. Mohammadi, M. T. Pourazad and P. Nasiopoulos, "An Entropy-based Inverse Tone Mapping Operator for High Dynamic Range Applications," 2018 9th IFIP International Conference on New Technologies, Mobility and Security (NTMS), Paris, 2018, pp. 1-5. [P2] P. Mohammadi, M. T. Pourazad and P. Nasiopoulos, "A High Contrast Video Inverse Tone Mapping Operator for High Dynamic Range Applications," 2019 IEEE International Symposium on Circuits and Systems (ISCAS), Sapporo, Japan, 2019, pp. 1-5. vii  [P3] P. Mohammadi, M. T. Pourazad and P. Nasiopoulos, "A Perception-based Inverse Tone Mapping Operator for High Dynamic Range Video Applications," Manuscript submitted, September 2019. [P4] P. Mohammadi, M. Azimi and M. T. Pourazad, "An Entropy-Based Inverse Tone Mapping Operator with Improved Color Accuracy for High Dynamic Range Applications," 2019 IEEE International Conference on Consumer Electronics (ICCE), Las Vegas, NV, USA, 2019, pp. 1-2. [P5] P. Mohammadi, M. T. Pourazad and P. Nasiopoulos, "A Fully-Automatic Content Adaptive Inverse Tone Mapping Operator with Improved Color Accuracy," Manuscript submitted, September 2019. viii  Table of Contents Abstract ......................................................................................................................................... iii Lay Summary .................................................................................................................................v Preface ........................................................................................................................................... vi Table of Contents ....................................................................................................................... viii List of Tables ................................................................................................................................ xi List of Figures ............................................................................................................................. xiv List of Abbreviations ................................................................................................................. xxi Acknowledgements ................................................................................................................... xxii Dedication ................................................................................................................................. xxiii Chapter 1: Introduction and Overview .......................................................................................1 1.1 Overview of state-of-the-art iTMOs ............................................................................... 3 1.1.1 Akyuz et al.’s Operator ............................................................................................... 4 1.1.2 Meylan et al.’s Operator ............................................................................................. 6 1.1.3 Banterle et al.’s Operator ............................................................................................ 7 1.1.4 Huo et al.’s Operator ................................................................................................... 9 1.1.5 Rempel et al.’s Operator ........................................................................................... 11 1.1.6 Kovaleski et al.’s Operator ....................................................................................... 12 1.1.7 Bist et al.’s Operator ................................................................................................. 13 1.1.8 Luzardo et al.’s Operator .......................................................................................... 15 1.1.9 Endo et al.’s Operator ............................................................................................... 17 1.1.10 Kim et al.’s Operator............................................................................................. 18 1.2 Thesis Outline ............................................................................................................... 19 ix  Chapter 2: An Entropy-based inverse Tone Mapping Operator ............................................22 2.1 Proposed iTMO ............................................................................................................. 22 2.1.1 Converting SDR Light Values to Perceptual Values ................................................ 22 2.1.2 Segmenting the SDR PQ Values to Three Brightness Regions ................................ 26 2.1.3 Mapping SDR PQ Values to HDR Domain .............................................................. 31 2.2 Experimental Evaluations ............................................................................................. 35 2.3 Conclusions ................................................................................................................... 37 Chapter 3: A Perception-based inverse Tone Mapping Operator for High Dynamic Range Video Applications .......................................................................................................................38 3.1 Proposed iTMO ............................................................................................................. 38 3.1.1 Converting SDR Light Values to Perceptual (PQ) Domain, Computing the Histogram and Segmentation to Three Brightness Regions ................................................. 39 3.1.2 Mapping SDR PQ Values to HDR Domain .............................................................. 43 3.2 Experimental Evaluations ............................................................................................. 53 3.3 Conclusions ................................................................................................................... 69 Chapter 4: A Fully-Automatic Content Adaptive Inverse Tone Mapping Operator with Improved Color Accuracy ...........................................................................................................70 4.1 Proposed iTMO ............................................................................................................. 71 4.1.1 Converting SDR Light Values to PQ Domain and Computing Histogram .............. 72 4.1.2 Segmenting the SDR PQ Values to Three Brightness Regions ................................ 72 4.1.3 Mapping SDR PQ Values to HDR Domain .............................................................. 78 4.1.4 Proposed Color Adjustment Method ........................................................................ 81 4.2 Experimental Evaluations ............................................................................................. 87 x  4.3 Conclusions ................................................................................................................. 101 Chapter 5: Conclusions and Future Work ..............................................................................102 5.1 Summary of the Contributions .................................................................................... 102 5.2 Significance and Potential Applications of the Research ........................................... 104 5.3 Future Work ................................................................................................................ 106 Bibliography ...............................................................................................................................107     xi  List of Tables Table 3.1 Visual quality scores by HDR VDP-2.2 for dark frames in Figure 3.10 for three different values of w1 and w2 ....................................................................................................................... 54 Table 3.2 Visual quality scores by HDR VDP-2.2 for normal frames in Figure 3.10 for three different values of w1 and w2 ........................................................................................................ 54 Table 3.3 Visual quality scores by HDR VDP-2.2 for bright frames in Figure 3.10 for three different values of w1 and w2 ........................................................................................................ 54 Table 3.4 Visual quality scores by PU-SSIM for dark frames in Figure 3.10 for three different values of w1 and w2 ....................................................................................................................... 54 Table 3.5 Visual quality scores by PU-SSIM for normal frames in Figure 3.10 for three different values of w1 and w2 ....................................................................................................................... 55 Table 3.6 Visual quality scores by PU-SSIM for bright frames in Figure 3.10 for three different values of w1 and w2 ....................................................................................................................... 55 Table 3.7 Mean and standard deviation of MOS scores for our contrast, brightness and hybrid approaches using the dataset in Figure 3.13 ................................................................................. 59 Table 3.8 Visual quality scores by HDR VDP-2.2 for dark frames in Figure 3.10 using hybrid, brightness and contrast approaches ............................................................................................... 61 Table 3.9 Visual quality scores by HDR VDP-2.2 for normal frames in Figure 3.10 using hybrid, brightness and contrast approaches ............................................................................................... 61 Table 3.10 Visual quality scores by HDR VDP-2.2 for bright frames in Figure 3.10 using hybrid, brightness and contrast approaches ............................................................................................... 61 Table 3.11 Visual quality scores by PU-SSIM for dark frames in Figure 3.10 using hybrid, brightness and contrast approaches ............................................................................................... 62 xii  Table 3.12 Visual quality scores by PU-SSIM for normal frames in Figure 3.10 using hybrid, brightness and contrast approaches ............................................................................................... 62 Table 3.13 Visual quality scores by PU-SSIM for bright frames in Figure 3.10 using hybrid, brightness and contrast approaches ............................................................................................... 62 Table 3.14 Mean and standard deviation of MOS scores for the six iTMOs based on the dataset in Figure 3.16 .................................................................................................................................... 64 Table 3.15 Visual quality scores by HDR VDP-2.2 for dark frames in Figure 3.19 for six different iTMOs ........................................................................................................................................... 67 Table 3.16 Visual quality scores by HDR VDP-2.2 for normal frames in Figure 3.19 for six different iTMOs ............................................................................................................................ 67 Table 3.17 Visual quality scores by HDR VDP-2.2 for bright frames in Figure 3.19 for six different iTMOs ........................................................................................................................................... 67 Table 3.18 Visual quality scores by PU-SSIM for dark frames in Figure 3.19 for six different iTMOs ........................................................................................................................................... 68 Table 3.19 Visual quality scores by PU-SSIM for normal frames in Figure 3.19 for six different iTMOs ........................................................................................................................................... 68 Table 3.20 Visual quality scores by PU-SSIM for bright frames in Figure 3.19 for six different iTMOs ........................................................................................................................................... 68 Table 3.21 Execution times of six iTMOs (in seconds) for a 4K SDR frame .............................. 68 Table 4.1 Mean, SD, and 95% CI of the average ΔL for different values of the saturation factor in our proposed color adjustment method ......................................................................................... 87 Table 4.2 Mean, SD, and 95% CI of the average ΔL using our proposed iTMO with three different color adjustment methods ............................................................................................................. 88 xiii  Table 4.3 Mean, SD, and 95% CI of the average ΔL using Kim et al.’s iTMO with three different color adjustment methods ............................................................................................................. 89 Table 4.4 Mean and SD of MOS scores for the six iTMOs based on the dataset in Figure 4.17 . 93 Table 4.5 Visual quality scores by HDR VDP-2.2 for dark frames in Figure 4.20 for six different iTMOs ........................................................................................................................................... 96 Table 4.6 Visual quality scores by HDR VDP-2.2 for normal frames in Figure 4.20 for six different iTMOs ........................................................................................................................................... 96 Table 4.7 Visual quality scores by HDR VDP-2.2 for bright frames in Figure 4.20 for six different iTMOs ........................................................................................................................................... 96 Table 4.8 Visual quality scores by PU-SSIM for dark frames in Figure 4.20 for six different iTMOs....................................................................................................................................................... 97 Table 4.9 Visual quality scores by PU-SSIM for normal frames in Figure 4.20 for six different iTMOs ........................................................................................................................................... 97 Table 4.10 Visual quality scores by PU-SSIM for bright frames in Figure 4.20 for six different iTMOs ........................................................................................................................................... 97 Table 4.11 Mean and SD of MOS scores for the six iTMOs based on the dataset in Figure 4.21..................................................................................................................................................... 100  xiv  List of Figures Figure 1.1 (a) Overall brightness range seen by human eye through adaptation. (b) Brightness range seen by human eye at a certain adaptation level. (c) Brightness range supported by SDR technology. (d) Brightness range supported by HDR technology. ................................................. 1 Figure 1.2 Mapping curves used by Akyuz et al.’s operator for three different γ values and k = 1000 nits. ......................................................................................................................................... 5 Figure 1.3 (a) An example of a dark SDR frame with (b) its HDR frame generated by Akyuz et al.’s iTMO with γ = 1, and k = 1000 nits. ....................................................................................... 5 Figure 1.4 Mapping curve used by Meylan et al.’s operator for Dispmax = 1000 nits, ρ = 660 nits, and ω = 30 nits. ............................................................................................................................... 7 Figure 1.5 (a) An example of a dark SDR frame along with (b) its HDR frame generated by Meylan et al.’s iTMO with Dispmax = 1000 nits and ρ = 660 nits. ............................................................... 7 Figure 1.6 Mapping curves used by Banterle et al.’s operator for Dispmax = 1000 nits, and three different values for Lwhite. ................................................................................................................ 8 Figure 1.7 (a) An example of an SDR frame along with (b) its HDR frame generated by Banterle et al.’s iTMO with Dispmax = 1000 nits and Lwhite = 10................................................................... 9 Figure 1.8 Mapping curves used by Huo et al.’s operator for σ = 1000 nits, Ls = 0, and n = 0.86........................................................................................................................................................ 10 Figure 1.9 (a) An example of an SDR frame along with (b) its HDR frame generated by Huo et al.’s iTMO with σ = 1000 nits. ...................................................................................................... 11 Figure 1.10 Mapping curve used by Rempel et al.’s operator. ..................................................... 11 Figure 1.11 (a) An example of a bright SDR frame along with (b) its HDR frame generated by Rempel et al.’s iTMO, and (c) HDR frame generated by Kovaleski et al.’s iTMO. .................... 12 xv  Figure 1.12 Mapping curves used by Bist et al.’s operator for three different γ values. .............. 14 Figure 1.13 (a) An example of a dark SDR frame along with (b) its HDR frame generated by Bist et al.’s iTMO. ................................................................................................................................ 14 Figure 1.14 Mapping curve used by Luzardo et al.’s operator for Dispmax = 1000 nits, a = 1.25, d = 4, mSDR = 21.47 nits, mHDR = 27.60 nits, b = - 4.2758, and c = 5.2758. ..................................... 16 Figure 1.15 (a) An example of a dark SDR frame along with (b) its HDR frame generated by Luzardo et al.’s iTMO with Dispmax = 1000 nits, a = 1.25, d = 4, and mSDR = 21.47 nits. ........... 17 Figure 1.16 (a) An SDR frame along with (b) its HDR frame generated by Endo et al.’s iTMO. 17 Figure 1.17 Mapping curve used by Kim et al.’s operator for a = 5, α = 0.4, and β = 0.6. .......... 18 Figure 1.18 (a) An example of a dark SDR frame along with (b) its HDR frame generated by Kim et al.’s iTMO with for a = 5, α = 0.4, and β = 0.6. ....................................................................... 19 Figure 2.1 Difference in how our eyes perceive brightness information and how cameras capture that information. ............................................................................................................................ 23 Figure 2.2 Luminance values encoded by the PQ and gamma functions. .................................... 24 Figure 2.3 (a) An SDR frame along with (b) the histogram of its luminance channel in PQ domain........................................................................................................................................................ 25 Figure 2.4 An example of histogram of SDR PQ values. Our proposed segmentation method aims at finding X1, and X2 to divide the PQ values into three brightness regions. ................................ 27 Figure 2.5 (a) Plots of E1, E2, and E1 + E2. X1 is where E1 + E2 is maximum. (b) Plots of E3, E4, and E3 + E4. X2 is where E3 + E4 is maximum. (c) Result of segmentation using our proposed segmentation method. Black, gray, and white areas represent dark, normal, and bright regions in the SDR frame, respectively. ........................................................................................................ 29 xvi  Figure 2.6 (a) – (c) An example of dark, normal, and bright SDR frames, respectively. (d) – (f) Segmentation results using our proposed method for each frame, respectively.  Black, gray, and white portions demonstrate dark, normal, and bright regions in the SDR frame, respectively. ... 30 Figure 2.7 Changes in X1 and X2 using our proposed segmentation method. (a) and (b) show the changes in X1 and X2, respectively. ............................................................................................... 31 Figure 2.8 An example of mapping curve used in our proposed iTMO. ...................................... 32 Figure 2.9 An example of the two curves associated with slope limits for dark and bright regions. These two curves demonstrate the limits for which we can choose our mapping curves. ........... 33 Figure 2.10 Preference scores of the subjects in mean opinion score (MOS) when comparing the visual quality generated by our iTMO over that of Meylan et al.’s, and Banterle et al.’s operators........................................................................................................................................................ 36 Figure 3.1 Workflow of our proposed iTMO. The red dotted line encloses components of our approach. ....................................................................................................................................... 39 Figure 3.2 (a) An SDR frame along with (b) the histogram of its luminance channel in PQ domain........................................................................................................................................................ 40 Figure 3.3 (a) PQ curve for mapping SDR light values to PQ domain. (b) Rate of change (derivative) of the PQ curve. (c) and (d) Zoomed areas corresponding to dark and bright regions, respectively. .................................................................................................................................. 42 Figure 3.4 (a) - (c) An example of dark, normal and bright SDR frames, respectively. (d) - (f) represent the result of our segmentation method for each frame, respectively. Black, gray, and white portions demonstrate dark, normal, and bright regions in the input SDR frame, respectively........................................................................................................................................................ 43 xvii  Figure 3.5 An example of mapping curve used in our iTMO. X1 is the boundary between dark and normal regions, and X2 is the boundary between normal and bright regions. .............................. 44 Figure 3.6 An example of an overall dark SDR frame with (b) its corresponding HDR frame with our contrast approach. ................................................................................................................... 47 Figure 3.7 An example of an overall bright SDR frame with (b) its corresponding HDR frame with our contrast approach. ................................................................................................................... 47 Figure 3.8 (a) An example of an overall dark SDR frame with (b) its corresponding HDR frame generated using our contrast approach. (c) Corresponding HDR frame generated using our brightness approach. ..................................................................................................................... 51 Figure 3.9 (a) An example of an overall bright SDR frame with (b) its corresponding HDR frame generated using our contrast approach. (c) Corresponding HDR frame generated using our brightness approach. ..................................................................................................................... 52 Figure 3.10 Nine original tone mapped HDR frames randomly chosen from the dataset in [58].53 Figure 3.11 (a) An example of an overall dark SDR frame with (b) its corresponding HDR frame with our contrast approach. (c) Corresponding HDR frame with our brightness approach, and (d) corresponding HDR frame with our hybrid approach with w1 = w2 = 0.5. ................................... 56 Figure 3.12 (a) An example of an overall bright SDR frame with (b) its corresponding HDR frame with our contrast approach. (c) Corresponding HDR frame with our brightness approach, and (d) corresponding HDR frame with our hybrid approach with w1 = w2 = 0.5. ................................... 57 Figure 3.13 One frame of each of the nine 4K SDR videos used in our subjective test. .............. 58 Figure 3.14 Preference scores of the subjects in mean opinion score (MOS) when comparing the visual quality of the generated HDR videos using our hybrid approach with w1 = w2 = 0.5 over that of brightness and contrast approaches separately. ........................................................................ 59 xviii  Figure 3.15 Post-hoc analysis using Tukey’s method for mean values of MOS scores and α=0.05. Methods within the same group generate statistically insignificant HDR videos. ....................... 60 Figure 3.16 One frame of each of the nine 4K SDR videos used in our subjective test. .............. 63 Figure 3.17 Mean opinion scores (MOS) indicating the visual quality of the HDR videos generated by our iTMO against Meylan et al.’s, Rempel et al.’s, Kovaleski et al.’s, Bist et al.’s, and Kim et al.’s operator ................................................................................................................................. 64 Figure 3.18 Post-hoc analysis using Tukey’s method for mean values of MOS scores and α=0.05. Methods within the same group generate statistically insignificant HDR videos. ....................... 65 Figure 3.19 Nine original tone mapped HDR frames randomly chosen from the dataset in [58]. 66 Figure 4.1 The block diagram of our proposed iTMO. The main components of our method are enclosed in the red dotted line. The generated HDR frame in this diagram has maximum brightness of 1000 nits. However, in general, our method is capable of generating HDR videos with arbitrary maximum brightness. .................................................................................................................... 71 Figure 4.2 (a) An SDR frame along with (b) the histogram of its luminance channel in PQ domain........................................................................................................................................................ 73 Figure 4.3 An example of histogram of SDR PQ values. Our proposed segmentation method aims at finding X1, and X2 to divide the SDR PQ values to three brightness regions. .......................... 75 Figure 4.4 Plots of E1, E2, and E1+E2. X1 is where E1+E2 is maximum. (b) Plots of E3, E4, and E3+E4. X2 is where E3+E4 is maximum. (c) Original SDR frame in Figure 4.2 (a). (d) Segmentation result using our proposed method. Black, gray, and white areas represent dark, normal, and bright regions in the SDR frame, respectively. ....................................................................................... 76 Figure 4.5 (a) - (c) An example of dark, normal and bright SDR frames, respectively. (d) - (f) represent the result of our proposed segmentation method for each frame, respectively. ............ 78 xix  Figure 4.6 (a) An SDR frame. (b) Our mapping curve associated with it. X1, and X2 are the thresholds found using our segmentation method and Y1, and Y2 are their corresponding PQ values in HDR domain. ............................................................................................................................ 80 Figure 4.7 Performance of our proposed iTMO for (a) dark, (b) normal, and (c) bright SDR frames. (d), (e), and (f) show the tone mapped HDR frames generated using our proposed iTMO, respectively. .................................................................................................................................. 81 Figure 4.8 Changes in hue (h) and Lightness (L) with respect to Chroma (C) for six basic colors as we change the saturation factor s from zero to two for Schlick et al.’s method. (a) and (b) demonstrate the changes in h and L with respect to C for six basic colors, respectively. The points where s = 1 are marked with black squares .................................................................................. 82 Figure 4.9 Changes in hue (h) and Lightness (L) with respect to Chroma (C) for six basic colors as we change the saturation factor s from zero to two for Mantiuk et al.’s method. (a) and (b) demonstrate the changes in h and L with respect to C for six basic colors, respectively. The points where s = 1 are marked with black squares. ................................................................................. 83 Figure 4.10 Changes in hue (h) and Lightness (L) with respect to Chroma (C) for six basic colors as we change the saturation factor s from zero to two for our proposed color adjustment method. (a) and (b) demonstrate the changes in h and L with respect to C for six basic colors, respectively. The points where s = 1 are marked with black squares. ............................................................... 85 Figure 4.11 A subset of our 400 SDR frames dataset randomly taken from [49]. ....................... 86 Figure 4.12 Average of ΔL along with its 95% CI over our dataset of 400 SDR frames for three different values of s. ...................................................................................................................... 87 Figure 4.13 Average of ΔL along with its 95% CI over our dataset of 400 SDR frames using our proposed iTMO for three different color adjustment methods. .................................................... 88 xx  Figure 4.14 Average of ΔL along with its 95% CI over our dataset of 400 SDR frames using Kim et al.’s iTMO for three different color adjustment methods. ........................................................ 89 Figure 4.15 An example of our mapping curve. X1, and X2 are the boundaries for dark, and bright regions, respectively while Y1, and Y2 are their corresponding PQ values in HDR domain, respectively. .................................................................................................................................. 89 Figure 4.16 (a) and (b) Absolute difference for Y1 and Y2 generated by our proposed iTMO using the segmentation method in Chapter 2, respectively, compared to (c) and (d) the proposed segmentation method in this Chapter. ........................................................................................... 90 Figure 4.17 One frame of each of the nine 4K SDR videos used in our subjective test. .............. 91 Figure 4.18 Preference scores of the subjects in mean opinion score (MOS) when comparing the visual quality of the HDR videos generated by our iTMO over that of Endo et al.’s, Luzardo et al.’s, Bist et al.’s, iTMO in Chapter 3, and Kim et al.’s iTMO. ................................................... 93 Figure 4.19 Post-hoc analysis using Tukey’s method for mean values of MOS scores and α=0.05. Methods within the same group generate statistically insignificant HDR videos. ....................... 94 Figure 4.20 Nine original tone mapped HDR frames randomly chosen from the dataset in [58]. 95 Figure 4.21 Ten SDR frames randomly chosen from [48] for our subjective test........................ 98 Figure 4.22 Preference scores of the subjects in mean opinion score (MOS) when comparing the color accuracy of output HDR frame generated by our proposed iTMO over that of Endo et al.’s, Luzardo et al.’s, Bist et al.’s, iTMO in Chapter 3, and Kim et al.’s operator. .............................. 99 Figure 4.23 Post-hoc analysis using Tukey’s method for mean values of MOS scores and α=0.05. Methods within the same group generate statistically insignificant HDR videos. ..................... 100  xxi  List of Abbreviations ANOVA  Analysis of Variance CI                                                        Confidence Interval CIE                                                      Commission Internationale de l'Eclairage CNN                                                    Convolutional Neural Network HD                                                       High Definition HDR                                                    High Dynamic Range HVS                                                    Human Visual System iTM                                                     inverse Tone Mapping iTMO                                                  inverse Tone Mapping Operator ITU                                                     International Telecommunication Union JND                                                    Just Noticeable Difference LF                                                       Light Field MOS Mean Opinion Score PQ                                                      Perceptual Quantizer PU  Perceptually Uniform SD                                                      Standard Deviation SDR                                                   Standard Dynamic Range SSIM                                                  Structural Similarity Index TMO                                                  Tone Mapping Operator VDP Visual Difference Predictor xxii  Acknowledgements This thesis would never have been done without the help and support from numerous people. I would like to take this opportunity to express my gratitude to the following individuals. Primarily, I would like to give my sincerest gratitude to my supervisor Dr. Panos Nasiopoulos for his continuous guidance and support throughout my Ph.D. I thank Panos for his encouragement, patience, enthusiasm, and broad knowledge in multimedia. I also thank Panos for the freedom and independence he gave me during my PhD studies. He has always been a great mentor, a role model, and a friend. I would like to extend my sincere thanks to Dr. Mahsa T. Pourazad for her support and guidance during my PhD studies. I thank Mahsa for her patience, encouragement, and great inspiring suggestions. I was lucky to have the opportunities to collaborate with her throughout my PhD studies.  I would also like to thank my Ph.D. committee members, Dr. Victor Leung, Dr. Jane Wang, and Dr. John Madden for investing time and energy to provide guidance and help throughout my graduate studies. Their encouragement and insightful comments have been extremely valuable to me. I also thank my other colleagues at the UBC Digital Multimedia Lab, Maryam Azimi, Stelios Ploumis, Ahmad Khaldieh, Hamid Reza Tohidypour, Fujun Xie, Anahita Shojaei, Ilya Ganelin, Nusrat Mehajabin, Doris Xiang, and Joseph Khoury.  I thank you all for creating a vibrant and supportive research environment, for the stimulating discussions, for your friendship, and for the fun times we have had together. xxiii  Dedication  To my Mom, Mahboobeh, and my Dad, Bahman, whom without their unconditional love and support none of this would be happening.1  Chapter 1: Introduction and Overview The human eye, through adaptation, is capable of perceiving a wide range of brightness values from starlight to sunlight [1]. At a certain adaptation level, our eyes are capable of perceiving a brightness range of 5 orders of magnitude (i.e., the difference in powers of ten between highest and lowest brightness values) measured in units of candelas per square meter (cd/m2) or nits (See Figure 1.1) [1]. However, due to its limitations, Standard Dynamic Range (SDR) technology can capture and display only a small portion (about two to three orders of brightness magnitude) of what our eyes can perceive. High Dynamic Range (HDR) technology aims at overcoming the limitations of its SDR counterpart by diligently modeling the capabilities  Figure 1.1 (a) Overall brightness range seen by human eye through adaptation. (b) Brightness range seen by human eye at a certain adaptation level. (c) Brightness range supported by SDR technology. (d) Brightness range supported by HDR technology.  2  of Human Visual System (HVS) in perceiving brightness. Doing so, HDR technology can capture and display a brightness range of up to 5 orders of magnitude. While this emerging video technology has a disruptive impact on the capturing and display industries, it also opens new opportunities for SDR content owners and broadcasters such as Netflix and Hollywood studios that will continue producing real-time events in SDR format for the near future. For the latter case, SDR content will have to be efficiently converted to HDR format, taking advantage of the advanced visual quality of HDR displays to offer to viewers a much better quality of experience than what the original SDR content would ever do. Thus, it is of high importance to be able to generate HDR content by converting existing legacy SDR content to the new format, taking advantage of the higher visual quality that it offers. Over the years, several attempts have focused on converting existing SDR content to HDR format, a process better known as inverse Tone Mapping (iTM) [2]–[27]. The design of inverse Tone Mapping Operators (iTMOs) has proven to be a challenging task as one tries to recreate the information that is lost during SDR capturing process [28], [29]. Moreover, the fact that most of the existing iTMOs only focus on specific types of SDR content either considering normal (average brightness) SDR content, or focusing on dark, and/or bright ones does not help in providing a general solution. One of the challenges in designing an efficient iTMO arises from the fact that as we try to expand the SDR brightness levels, we inevitably increase the chance of introducing and/or increasing noise in dark regions and banding in bright regions of the input SDR frame. Thus, keeping these visual artifacts at a non-perceivable level is an important consideration when designing iTMOs. Another important challenge in designing iTMOs arises from the fact that as we try to expand the SDR brightness levels, we introduce changes in colors between input SDR and generated HDR frames [30]–[33]. Consequently, an efficient color adjustment method capable 3  of maintaining the color accuracy between input SDR and generated HDR frame is another important consideration when designing iTMOs.  In this thesis, we propose novel iTMOs capable of dealing with all types of SDR content, from dark, to normal and bright scenes. Our proposed iTMOs keep the visual artifacts at a non-perceivable level and maintain the color accuracy between input SDR and generated HDR frame. In Chapter 2, we propose an entropy-based iTMO to convert SDR content to HDR format. In Chapter 3, we propose a hybrid iTMO for converting SDR content to HDR format capable of achieving the best possible trade-off between overall brightness and contrast of the generated HDR video. In Chapter 4, we propose a content adaptive iTMO that addresses the shortcomings of the methods in Chapters 2 and 3. In addition, we address the issue of color shifts when converting SDR content to HDR format and propose a novel perception-based color adjustment method that is capable of maintaining the color accuracy between input SDR and generated HDR frame.  The following sections in this chapter provide an in-depth look into state-of-the-art iTMOs (Section 1.1) and provide a summary of the scientific contributions of this thesis (Section 1.2). 1.1 Overview of state-of-the-art iTMOs  Over the years, several efforts focused on converting SDR content to HDR format. In general, an iTMO takes as input the luminance channel of the SDR frame and generates the HDR equivalent. Note that luminance is measured in units of candelas per square meter (cd/m2) or else known as nits and can be computed based on International Telecommunication Union (ITU) BT.709 standardization [34] using the following equation: 0.2126 0.7152 0.0722SDR SDR SDR SDRL R G B                                                                                               (1.1) where LSDR is the luminance channel of the SDR frame, and RSDR, GSDR, and BSDR are the red, green, and blue channels of the SDR frame, respectively. In general, SDR luminance values are in the 4  range [0.1 – 100] nits. Note that the corresponding calculation of luminance for BT.2020 standard can be found in [35]. Throughout this thesis, we refer to brightness and luminance interchangeably. In the following sub-sections, we provide a comprehensive overview of some of the most prominent iTMOs. 1.1.1 Akyuz et al.’s Operator One of the simplest iTMOs to implement is the method proposed by Akyuz et al. [2] which uses the following linear mapping curve: ,min,max ,minSDR SDRHDRSDR SDRL LL kL L                                                                                                  (1.2) where LHDR, and LSDR are the HDR and SDR luminance channels, respectively, LSDR,min, and LSDR,max are the minimum and maximum SDR luminance values, k is the maximum brightness of the target HDR display, and γ is a parameter responsible for the shape of the mapping curve that is chosen by the user. Figure 1.2 shows the mapping curves used by this iTMO for three different γ values, and k = 1000 nits. These curves map the SDR luminance values from [0.1 – 100] nits to [0.01 – 1000] nits in the HDR domain. After performing a series of subjective tests, the authors concluded that a simple linear mapping curve (γ = 1) is sufficient to have good quality HDR content. This iTMO does a good job in converting normal and bright SDR frames to HDR format. However, it falls short when it comes to dark SDR frames and generates HDR frames that appear overly bright with visible visual artifacts. Figure 1.3 (a) shows a dark SDR frame randomly chosen from [36] while Figure 1.3 (b) shows the HDR frame generated by Akyuz et al.’s operator with γ 5  = 1, and k = 1000 nits. As it can be seen, since mapping is done using a simple line, the generated HDR frame is not of the highest visual quality and appears overly bright with visible visual artifacts in dark regions of the frame. Throughout this thesis, generated HDR frames with different iTMOs are tone mapped using Reinhard et al.’s [37] Tone Mapping Operator (TMO) for displaying purposes. Throughout this thesis, since there are no public databases containing SDR and HDR pairs of the same scene, we refrain from comparing generated HDR frames with ground truth.   Figure 1.2 Mapping curves used by Akyuz et al.’s operator for three different γ values and k = 1000 nits.                                                 (a)                                                                                  (b)  Figure 1.3 (a) An example of a dark SDR frame; (b) its HDR frame generated by Akyuz et al.’s iTMO with γ = 1, and k = 1000 nits.   6  1.1.2 Meylan et al.’s Operator Meylan et al. [3], [14] proposed an iTMO with the goal of recovering the highlights in SDR frames. The main idea is to detect dark, and bright regions of the SDR frame and expand them using a piece-wise linear mapping curve with one breaking point. The mapping curve in this iTMO is presented by the following equation:  11 2SDR SDRHDRSDRs L                     if    LLs s L       otherwise                                                                                  (1.3) where LHDR, and LSDR are the HDR and SDR luminance channels respectively, ω is the boundary between dark, and bright regions, s1, and s2 are the slopes of the mapping curve associated with dark and bright regions, respectively. These slopes can be calculated as follows: 1s                                                                                                                                                                                     (1.4) max2100Disps                                                                                                                                                               (1.5) where Dispmax is the maximum brightness of the target HDR display and ρ is the portion of the maximum brightness of the HDR display that will be allocated to dark regions. To determine a value for ρ that works well for all types of SDR frames, the authors conducted a series of comprehensive subjective tests and concluded that ρ = 0.66 × Dispmax is a good estimate for generating high quality HDR content.  Figure 1.4 shows the mapping curve used in this iTMO for an HDR display with maximum brightness of 1000 nits (Dispmax = 1000 nits), ρ = 660 nits, and ω = 30 nits. This curve maps the SDR luminance values from [0.1 – 100] nits to [0.01 – 1000] nits in the HDR domain. Figure 1.5 (a) shows a dark SDR frame while Figure 1.5 (b) shows the HDR frame generated by Meylan et al.’s operator with Dispmax = 1000 nits and ρ = 660 nits. We observe that although this mapping 7  does a good job in recovering the highlights, in the case of dark SDR frames the generated HDR frame is often noisy and appears overly bright compared to SDR input, resulting in reduced overall contrast.  1.1.3 Banterle et al.’s Operator Banterle et al. proposed an iTMO to address the issue of loss of contrast by Akyuz et al.’s and Meylan et al.’s iTMOs [21]–[24]. The main idea behind this method is to choose a previously  Figure 1.4 Mapping curve used by Meylan et al.’s operator for Dispmax = 1000 nits, ρ = 660 nits, and ω = 30 nits.                                                (a)                                                                                  (b)  Figure 1.5 (a) An example of a dark SDR frame along with (b) its HDR frame generated by Meylan et al.’s iTMO with Dispmax = 1000 nits and ρ = 660 nits.   8  proposed tone mapping curve and calculate its inverse curve. Authors chose Reinhard et al.’s tone mapping curve [37] due to its simplicity. The mapping curve used in this iTMO is presented by the following equation:  2max 21 41 12HDR white SDR SDR SDRwhiteL Disp L L L LL                                                                   (1.6) where LHDR, and LSDR are the HDR and SDR luminance channels, respectively, Dispmax is the maximum brightness of the target HDR display, and  Lwhite is a parameter responsible for the shape of the mapping curve and is chosen by the user. Based on comprehensive subjective testing, the authors concluded that if Lwhite ≈ Dispmax, the generated HDR frames would demonstrate good contrast with no banding in bright regions of the frame. After expanding the SDR luminance channel, the bright regions of the frame are identified using the median cut algorithm [38] and are further expanded to increase the contrast in these regions. Figure 1.6 shows the mapping curves used in this iTMO for an HDR display with maximum brightness of 1000 nits (Dispmax = 1000  Figure 1.6 Mapping curves used by Banterle et al.’s operator for Dispmax = 1000 nits, and three different values of Lwhite.  9  nits), and three different values of Lwhite. These curves map the SDR luminance values from [0.1 – 100] nits to [0.01 – 1000] nits in HDR domain. One major disadvantage of this iTMO is that due to the shape of the mapping curve, the dark areas of the SDR frame appear darker in the generated HDR frame, causing loss of visible details in those areas. Figure 1.7 (a) shows an example of an SDR frame and Figure 1.7 (b) depicts its generated HDR frame using Banterle et al.’s iTMO with Dispmax = 1000 nits and Lwhite = 10. We observe that the details in the dark areas of the frame such as the trees in the background are lost in the generated HDR frame. 1.1.4 Huo et al.’s Operator To take advantage of the capabilities of HVS in perceiving brightness, Huo et al. [25], [26] proposed an iTMO that aims at modeling the eye retina response when viewing an SDR frame. After computing the retina response based on [39], the pixel-based luminance adaptation level of the SDR frame is calculated. Finally, the SDR frame and pixel-based luminance adaptation level are applied to the inverse of the eye retina response and the HDR frame is generated. The mapping curve used in this iTMO is computed using the following equation:                                               (a)                                                                                  (b)  Figure 1.7 (a) An example of an SDR frame along with (b) its HDR frame generated by Banterle et al.’s iTMO with Dispmax = 1000 nits and Lwhite = 10. 10   1,maxnn nSDRHDR sSDRLL LL                                                                                                                                      (1.7) where LHDR, and LSDR are the HDR and SDR luminance channels, respectively, LSDR,max is the maximum SDR luminance value, Ls is the surrounding luminance at each pixel of the SDR frame, σ is the adaptation level, and n is a sensitivity control parameter [39]. Based on comprehensive subjective tests, authors concluded that n = 0.86 results in good quality HDR frames. Figure 1.8 shows the mapping curve used in this iTMO for an HDR display with maximum brightness of 1000 nits (σ = 1000 nits), surrounding luminance values of zero (Ls = 0), and n = 0.86. This curve maps the SDR luminance values from [0.1 – 100] nits to [0.01 – 1000] nits in HDR domain. Figure 1.9 (a) shows a dark SDR frame while Figure 1.9 (b) presents the HDR frame generated using Huo et al.’s operator with σ = 1000 nits. As it can be seen, this iTMO underperforms when it comes to dark SDR frames and results in HDR frames that appear darker with loss of visible details (see the brick walls).  Figure 1.8 Mapping curves used by Huo et al.’s operator for σ = 1000 nits, Ls = 0, and n = 0.86.  11  1.1.5 Rempel et al.’s Operator To address the need for real-time iTMOs, Rempel et al. [27] proposed a real-time, fully-automatic (no human intervention) iTMO capable of generating good quality HDR frames. In this iTMO a simple linear curve is chosen as the mapping curve. Afterwards, the bright regions of the SDR frame are identified using a simple fixed threshold and are further expanded to generate the HDR frame. Figure 1.10 depicts the mapping curve used in this iTMO. This curve maps the SDR                                               (a)                                                                                  (b)  Figure 1.9 (a) An example of an SDR frame along with (b) its HDR frame generated by Huo et al.’s iTMO with σ = 1000 nits.   Figure 1.10 Mapping curve used by Rempel et al.’s operator.  12  luminance values from [0.1 – 100] nits to [0.01 – 1000] nits in the HDR domain. Although normal regions are well represented in the HDR domain, this iTMO falls short in recovering details in dark and bright regions of the SDR frame. Figure 1.11 (a) shows a bright SDR frame while Figure 1.11 (b) depicts the generated HDR frame using Rempel et al.’s iTMO. As it can be seen, details in the dark regions are lost in the HDR frame. In addition, there are visible visual artifacts in the bright regions of the frame, such as the ice cubes on the right side of the HDR frame. 1.1.6  Kovaleski et al.’s Operator Kovaleski et al. [4], [5] proposed an iTMO with the specific task of improving the performance of Rempel et al.’s operator in dark and bright regions. Kovaleski et al. hypothesized that the generated artifacts in Rempel et al.’s results were due to the simple fixed thresholding for   (a) (b)  (c)  Figure 1.11 (a) An example of a bright SDR frame along with (b) its HDR frame generated by Rempel et al.’s iTMO, and (c) HDR frame generated by Kovaleski et al.’s iTMO.   13  detecting bright regions. To improve the detection of this threshold, the authors proposed to implement cross-bilateral filtering [40], [41] to detect dark and bright regions of the frame and avoid visual artifacts in those areas. Figure 1.11 (c) shows the generated HDR frame using Kovaleski et al.’s operator. It can be observed that Kovaleski et al.’s iTMO does a better job in recovering the details in dark areas and removing banding in bright regions. However, the details and contrast in the bright regions are reduced. 1.1.7  Bist et al.’s Operator Bist et al. [6], [7] proposed an iTMO with the aim of preserving the artistic intent of the original SDR content. Preserving artistic intent in this case means that if the input SDR frame is a dark, normal or bright scene, then the generated HDR frame needs to convey the same overall impression. This iTMO is based on the assumption that the input SDR frame is generated using professional stylized workflow and the target HDR display has maximum brightness of 1000 nits. The mapping curve used in this iTMO is calculated as follows:  1000HDR SDRL L                                                                                                                                                            (1.8) where LHDR, and LSDR are the HDR and SDR luminance channels respectively, and γ is an exponent found through comprehensive subjective tests. Based on subjective tests, authors concluded that γ could be modeled as a function of the overall brightness of a frame. To calculate γ, first the SDR frame is converted from RGB to L*a*b* color space [42]. This is done because the L*a*b* color space does a better job in approximating human vision. Afterwards, the median value of the L* channel (Lmedian) is computed and clipped to the range [5 – 95] nits to avoid outlier values. Finally, γ is calculated as: 101001 logmedianL     .                                                                                                                                                  (1.9) 14   Figure 1.12 shows the mapping curves used by Bist et al.’s iTMO for three different γ values and a target HDR display with maximum brightness of 1000 nits. This curve maps the SDR luminance values from [0.1 -100] nits to [0.01 – 1000] nits in HDR domain. Although this iTMO does a good job in converting normal SDR frames to HDR format, it underperforms in the case of dark and bright scenes, with the generated HDR frame appearing overly dark and with reduced details in dark regions. Figure 1.13 (a) shows a dark SDR frame (γ=2.2) while Figure 1.13 (b)  Figure 1.12 Mapping curves used by Bist et al.’s operator for three different γ values.                                                 (a)                                                                                  (b)  Figure 1.13 (a) An example of a dark SDR frame along with (b) its HDR frame generated by Bist et al.’s iTMO.   15  presents the generated HDR frame using Bist et al.’s operator. We observe that the HDR frame appears darker than its SDR counterpart with loss of visible details in dark regions of the frame. 1.1.8 Luzardo et al.’s Operator To further improve upon the performance of Bist et al.’s iTMO, Luzardo et al. proposed an iTMO capable of generating HDR content with maximum brightness above 1000 nits while preserving the artistic intent of the SDR content [8]. In this case, the non-linear mapping curve used is described as follows: aSDRHDR adSDRLLL b c                                                                                                                                                          (1.10) where LHDR, and LSDR are the HDR and SDR luminance channels respectively. Based on extensive subjective tests, the authors concluded that when a = 1.25 and d = 4, the generated HDR frames have good contrast and no visible visual artifacts. Parameters b and c are calculated based on the following equations:  maxmax1aSDR HDRadHDR SDRm Disp mbm m Disp                                                                                                                                  (1.11)  maxmax1ad aSDR HDR SDRadHDR SDRm m m Dispcm m Disp                                                                                                                             (1.12) where Dispmax is the maximum brightness of the target HDR display and mSDR and mHDR represent the middle gray values in SDR and HDR luminance channels, respectively. As the SDR luminance channel is already provided, mSDR = 21.47 nits. To calculate mHDR, the authors used the multi-linear regression approach and proposed the following equation: , ,0.017 0.097 0.008 0.028HDR g SDR SDR ov SDRm L C P                                                                            (1.13) 16  where Lg,SDR is the geometric average of the SDR luminance channel [8], CSDR is the contrast of the SDR frame [8], and Pov,SDR is the percentage of over-exposed pixels in the SDR frame. Over-exposed pixels are defined as the ones with luminance values greater than or equal 99 nits. Figure 1.14 depicts the mapping curve used in this iTMO for Dispmax = 1000 nits, a = 1.25, d = 4, mSDR = 21.47 nits, mHDR = 27.60 nits, b = - 4.2758, and c = 5.2758. Although this iTMO preserves the artistic intent of normal and bright frames, when it comes to dark SDR frames, generated HDR frame appears darker as compared to its SDR input resulting in loss of visible details. Figure 1.15 (a) shows the dark SDR frame in Figure 1.13 (a) while Figure 1.15 (b) shows the generated HDR frame using Luzardo et al.’s operator with Dispmax = 1000 nits, a = 1.25, d = 4, and mSDR = 21.47 nits. As it can be seen from Figure 1.15 (b), Luzardo et al.’s iTMO does a better job in recovering the details in dark areas as compared to Bist et al.’s iTMO. However, the generated HDR frame still appears somewhat brighter compared to its SDR counterpart and loses visible details in dark regions of the frame.  Figure 1.14 Mapping curve used by Luzardo et al.’s operator for Dispmax = 1000 nits, a = 1.25, d = 4, mSDR = 21.47 nits, mHDR = 27.60 nits, b = - 4.2758, and c = 5.2758.  17  1.1.9 Endo et al.’s Operator Recently, deep learning-based iTMOs have attracted a lot of attention with these methods being able to generate good quality HDR content [9]–[13], [15], [16]. One of the first iTMOs in this category is the work proposed by Endo et al. [9]. The authors employ supervised learning based on deep Convolutional Neural Networks (CNNs) to generate different exposures of the input SDR frame. Afterwards, the multiple exposures are combined together using Debevec et al.’s method [43] to generate the final HDR content. Although this method is capable of generating good quality HDR content, it falls short in recovering the details of the bright regions of the frame. Another drawback of this approach is its high computational complexity. Figure 1.16 (a) shows an                                               (a)                                                                                  (b)  Figure 1.15 (a) An example of a dark SDR frame along with (b) its HDR frame generated by Luzardo et al.’s iTMO with Dispmax = 1000 nits, a = 1.25, d = 4, and mSDR = 21.47 nits.                                                (a)                                                                                  (b)  Figure 1.16 (a) An SDR frame along with (b) its HDR frame generated by Endo et al.’s iTMO.   18  SDR frame while Figure 1.16 (b) presents the generated HDR frame by Endo et al.’s operator. As it can be seen from Figure 1.16 (b), although the HDR frame is of high visual quality, details of the bright areas (e.g., moon) are lost in the HDR frame.  1.1.10 Kim et al.’s Operator Kim et al. [10] proposed a learning-based iTMO where the input SDR frame is first divided into its base and detail layers. The relationship between details layers of the input SDR and the generated HDR frames is estimated through a training phase using a set of SDR and HDR frame pairs [10], while at the same time the base layer of the input SDR frame is expanded using a non-linear mapping curve presented by the following equation: 11SDRaLHDR SDR aeL Le                                                                                                    (1.14) where LHDR, and LSDR are the HDR and SDR luminance channels respectively, and a, α, and β are set to 5, 0.4, and 0.6, respectively. Figure 1.17 depicts the mapping curve used in this iTMO for a = 5, α = 0.4, and β = 0.6. Although this approach does a good job in recovering details of dark  Figure 1.17 Mapping curve used by Kim et al.’s operator for a = 5, α = 0.4, and β = 0.6.   19  regions in the frame, when it comes to dark SDR frames, HDR frame will appear brighter compared to its SDR counterpart and contains visible visual artifacts in different regions of the generated HDR frame. Figure 1.18 (a) shows a dark SDR frame while Figure 1.18 (b) presents the generate HDR frame by Kim et al.’s operator for a = 5, α = 0.4, and β = 0.6. As it can be seen from Figure 1.18 (b), visible details of the dark areas are well represented in the HDR frame, with some visible visual artifacts in the same areas (see the sky). In addition, the generated HDR frame looks overly bright and loses details in the bright regions. 1.2 Thesis Outline In this thesis, we propose novel iTMOs capable of generating high visual quality HDR videos while keeping the visual artifacts at non-perceivable levels. Our proposed iTMOs are able to deal with all types of SDR content from dark, to normal and bright scenes. Through imitating the behavior of HVS when perceiving brightness, our iTMOs are capable of dividing the SDR frame into three brightness regions and then mapping those regions in an efficient manner to a higher dynamic range. Our final iTMO, based on our subjective and objective evaluations,                                               (a)                                                                                  (b)  Figure 1.18 (a) An example of a dark SDR frame along with (b) its HDR frame generated by Kim et al.’s iTMO with for a = 5, α = 0.4, and β = 0.6.   20  achieves the best possible trade-off between overall contrast and brightness of the HDR frame through maximizing a weighted sum of our proposed brightness and contrast difference functions. In Chapter 2, we propose an entropy-based iTMO for converting SDR content to HDR format. To divide the SDR frame into three regions in terms of their brightness we propose an entropy-based segmentation. The mapping curve in this work is a piece-wise linear curve with two breaking points and the slopes for each segment of the curve are chosen manually based on comprehensive subjective tests. The segmentation approach used in this chapter, although efficient for images, is too sensitive to small variations in pixel distribution between consecutive frames, which causes flickering in video applications. Another challenge is that the proposed iTMO in this chapter is not content adaptive, as the slopes are chosen manually and are fixed for all types of SDR frames. In Chapter 3, we propose a hybrid iTMO capable of producing high visual quality HDR videos. The mapping curve used in this work is same as the one in Chapter 2. The slopes for each segment of the curve are chosen in a content adaptive manner through our hybrid approach of maximizing our proposed contrast and brightness difference functions. By doing so, our proposed iTMO in this chapter achieves the best possible trade-off between overall contrast and brightness of the generated HDR frame. To make the segmentation process less sensitive to changes in pixel distribution among frames we proposed fixed boundaries for dark, normal, and bright regions. Although this iTMO creates high quality HDR videos, it is not completely content adaptive, as the boundaries for three brightness regions are fixed for all types of SDR frames. In Chapter 4, we propose a novel fully content adaptive iTMO that works in perceptual domain and can deal with all types of SDR content from dark, to normal and bright scenes, and is capable of producing high visual quality HDR videos. Our proposed iTMO in this chapter 21  preserves the overall impression of the original SDR content by dividing it to three brightness regions using our proposed segmentation method that is based on the known theory of maximum entropy thresholding [44]. Unlike our proposed segmentation approach in Chapter 2, which results in flickering due to its high sensitivity to pixel distribution variations, this segmentation approach employs a cumulative approach that is less sensitive to these changes and avoids flickering. Although the slopes of the mapping curves are calculated the same way as in Chapter 3, the segmented regions differ as they are determined by our new segmentation method.  Furthermore, we address the issue of color shifts when converting SDR frames to HDR format and propose a perception-based color adjustment method that, unlike common color adjustment approaches, results in negligible luminance change and prevents hue shifts.  22  Chapter 2: An Entropy-based inverse Tone Mapping Operator   To address the need for a general solution that can deal with all types of SDR content, in this chapter we propose an entropy-based iTMO that is capable of dealing with all types of SDR frames from dark, to normal and bright ones. Our proposed iTMO, unlike others, works in the perceptual domain to consider the sensitivity of the human eye to brightness changes in different regions (i.e., dark, normal, and bright) of the scene. This helps our iTMO to treat each region differently based on how our eyes perceive the different levels of brightness. This not only allows us to preserve the overall brightness of each region but also enables our iTMO to deal with all types of SDR frames (e.g., dark, normal, and bright frames), something that most iTMOs fail to address. Our subjective evaluations show that, on average, our method outperforms other state-of-the-art methods in terms of the overall visual quality of the generated HDR frame.  The rest of this chapter is organized as follows: Section 2.1 provides the details of our proposed iTMO. Section 2.2 discusses the results of our experimental evaluations and Section 2.3 concludes this chapter. 2.1 Proposed iTMO 2.1.1 Converting SDR Light Values to Perceptual Values  Our main goal is to design an iTMO capable of dealing with all types of SDR content, ranging from dark to normal and bright scenes. We base our approach on the fact that our visual system is more sensitive to brightness changes in dark regions rather than bright and normal ones. In order to take advantage of this HVS characteristic, we need to convert the captured luminance values to values that match our visual system. As it can be seen in Figure 2.1, cameras capture luminance information in linear fashion, not following the characteristics of our visual system. In fact, humans interpret the same amount of luminance difference in a different (more like a 23  logarithmic) way, depending on whether it is daylight or moonlight [1]. In general, HVS is more sensitive to light changes that happen in moonlight than daylight. In SDR technology, gamma correction (encoding) [45], also known as the BT.1886 standard, is used to convert camera captured values to values perceived by the HVS. However, gamma encoding was designed to address SDR values with maximum luminance of 100 nits, and thus cannot be used for HDR, which covers a much larger luminance range. For this reason, a new perceptual conversion function has been designed for HDR, known as the Perceptual Quantizer (PQ) function [46]. The PQ function is presented by the following equation: 2111 231mmlightPQ mlightc c LLc L                                                                                                                                                               (2.1) where Llight is the input light values in the range [0.005 – 10000] nits, while LPQ is the corresponding PQ values in the range [0 – 1]. The values for c1, c2, c3, m1, and m2 are equal to 0.8359, 18.8515, 18.6875, 0.1593, and 78.8437 respectively.   Figure 2.1 Difference in how our eyes perceive brightness information and how cameras capture that information.   24  Figure 2.2, shows the gamma and PQ curves for light values in the range [0.005 – 10000] nits. The difference in the beginning parts of the two curves is due to the fact that PQ assigns more range for mapping of the dark regions of the frame. We observe that the PQ function assigns half of the output values to luminance values ranging from 0.005 to 100 nits, while keeping the rest for luminance values ranging from 100 to 10,000 nit [47]. On the other hand, gamma function uses all of its output values for luminance values ranging from 0.1 to 100. The PQ function is designed to optimize the distribution of light values with respect to the HVS properties, transforming physically linear values into perceptually linear ones. Perceptually linear means that any variation of intensity at any brightness level is seen the same way by the human eye. The latter forms the fundamental base for our implementation as we want to identify what the human eye can see and map that information to a higher dynamic range. To this end, after computing the SDR luminance channel using equation (2.2), we convert the SDR luminance values to the PQ domain, so the mapping process takes advantage of the human visual system characteristics. 0.2126 0.7152 0.0722SDR SDR SDR SDRL R G B                                                                                               (2.2)  Figure 2.2 Luminance values encoded by the PQ and gamma functions. 25  where LSDR is the luminance channel of the input SDR frame and RSDR, GSDR, and BSDR are the red, green, and blue channels of the input SDR frame.   Our next step is to compute the histogram of the resulting SDR PQ values, which groups them into uniform size bins (i.e., with equal number of brightness values in each bin). The height of each bin indicates the number of pixels in the frame corresponding to these PQ values. Figure 2.3 (a) shows an example of a normal SDR frame along with the histogram of its luminance  (a)  (b) Figure 2.3 (a) A normal SDR frame along with (b) the histogram of its luminance channel in PQ domain.  26  channel in the PQ domain in Figure 2.3 (b). Note that SDR luminance values in this example are from 0.1 to 100 nits, which in the PQ domain correspond to 0.0623 to 0.5081 range. 2.1.2 Segmenting the SDR PQ Values to Three Brightness Regions  As we mentioned before, our eyes’ sensitivity to changes in brightness varies in dark, normal and bright regions. One of the main objectives in converting SDR content to HDR format is maintaining the artistic intent and the overall visual impression. To this end, we need to treat dark, normal, and bright regions of the content differently, as the eyes’ sensitivity to color changes based on brightness [48]. Our proposed segmentation method is based on the theory of maximum entropy thresholding [44]. In general, entropy is known as a measure of the uncertainty of a random variable [49]. The lower the probability of occurrence of an event, the more uncertain (higher entropy) we are about it happening and vice versa. Entropy of an image, E, is defined as follows [49]:  2logi iiE p p                                                                                                                                                         (2.3) where pi is the number of pixels in the i-th bin of the histogram of the frame over the total number of pixels in the frame. Figure 2.4 depicts the histogram of SDR PQ values for the SDR frame in Figure 2.3 (a). Our goal is to find the thresholds X1, and X2 that divide the SDR PQ values to three brightness regions. To this end, the first step is to identify Xmin and Xmax that are the first and last non-zero elements of the histogram, respectively. To have equal search regions for X1, and X2, we also find Xmid which is the middle point of the histogram. For the histogram in Figure 2.4 we have Xmin = 0.0623, Xmax = 0.5081, and Xmid = 0.2843. Afterwards, at each PQ point in the interval [Xmin – Xmid] we calculate two values: i) first E1, which denotes how uncertain we are that a PQ point in the interval belongs to dark regions of the SDR frame, as follows: 27   1 2( ) ( )( ) log( ) ( )minXmin midtotal totali XN i N iE X    ,   X X XN i N i                                                                   (2.4) where N(i) is the number of pixels corresponding to i-th PQ point in the histogram and Ntotal(i) is the total number of pixels starting from Xmin until the i-th PQ point in the histogram, and ii) E2, that denotes how uncertain we are that a PQ point in the interval belongs to normal regions of the SDR frame, as follows:  2 2( ) ( )( ) log( ) ( )midXmin midtotal totali XN i N iE X    ,   X X XN i N i                                                                     (2.5) To find X1 we compute the sum of E1 and E2. At each PQ point, E1 + E2 denotes how uncertain we are that a PQ point belongs to dark or normal regions. The boundary between dark and normal regions, X1, is the point where E1 + E2 is maximum. In other words, X1 is the point where we are most uncertain that it belongs to dark or normal regions.   Figure 2.4 An example of histogram of SDR PQ values. Our proposed segmentation method aims at finding X1, and X2 to divide the PQ values into three brightness regions.   28  We go through the same procedure for finding X2 only in the [Xmid – Xmax] interval. At each PQ point in this interval, we compute two values: i) E3, that determines how uncertain we are that a PQ point in the interval belongs to normal regions as follows:  3 2( ) ( )( ) log( ) ( )midXmid maxtotal totali XN i N iE X    ,   X X XN i N i                                                                 (2.6) where N(i) is the number of pixels corresponding to i-th PQ point in the histogram and Ntotal(i) is the total number of pixels starting from Xmid until the i-th PQ point in the histogram, and ii) E4, that determines how uncertain we are that a PQ point belongs to bright regions, as follows:  4 2( ) ( )( ) log( ) ( )maxXmid maxtotal totali XN i N iE X    ,   X X XN i N i                                                                    (2.7) The boundary between normal and bright regions, X2, is the point where E3 + E4 is maximum. In other words, X2 is the point where we are most uncertain that it belongs to normal or bright regions.  Figure 2.5 (a) shows the plot for E1, E2, and the summation of the two for the histogram in Figure 2.4. As we mentioned before, E1 denotes how uncertain we are that a PQ point belongs to dark regions. Therefore, as it can be seen from Figure 2.5 (a) as we go towards Xmid (normal regions) E1 increases. In addition, E2 denotes how uncertain we are that a PQ point belongs to normal regions. Therefore, as we go towards Xmin (dark regions), E2 increases. As we discussed before and can also be seen in Figure 2.5 (a), X1 is the point where E1 + E2 is maximum.  Figure 2.5 (b) shows the plot for E3, E4, and the summation of the two for the histogram in Figure 2.4. As we mentioned before, E3 denotes how uncertain we are that a PQ point belongs to normal regions. Therefore, as Figure 2.5 (b) shows, as we approach Xmax (bright regions) E3 increases. In addition, E4 denotes how uncertain we are that a PQ point belongs to bright regions. 29  Therefore, as we approach Xmid (normal regions), E4 increases. As we mentioned before and is depicted in Figure 2.5 (b), X2 is the point where E3 + E4 is maximum. Figure 2.5 (c) shows the result of our proposed segmentation method for the frame in Figure 2.3 (a). Black, gray, and white areas in the figure depict dark, normal, and bright regions in the frame respectively. As Figure 2.5 (c) shows, our proposed segmentation method works reasonably well in dividing the SDR frame to three brightness regions. To further examine the performance of our proposed segmentation method Figures 2.6 (a), (b), and (c) present three SDR frames                                               (a)                                                                                   (b)  (c) Figure 2.5 (a) Plots of E1, E2, and E1 + E2. X1 is where E1 + E2 is maximum. (b) Plots of E3, E4, and E3 + E4. X2 is where E3 + E4 is maximum. (c) Result of segmentation using our proposed segmentation method. Black, gray, and white areas represent dark, normal, and bright regions in the SDR frame, respectively.   30  representing dark, normal, and bright frames, respectively. In addition, Figures 2.6 (d), (e), and (f) represent the result of our proposed segmentation method for each of those frames, respectively. Black, gray, and white areas in the figure depict dark, normal, and bright regions in the frame, respectively. We can observe that our proposed segmentation method does a reasonably good job in dividing the SDR frame into three brightness regions. One issue concerning the proposed segmentation method is that it is too sensitive to small variations in pixel distribution that can result in flickering for video applications. This sensitivity arises from the fact that when calculating the entropy, we use the number of pixels in each bin of the histogram. Therefore, since in a video sequence the number of pixels vary from one frame to the next, this results in constant change in the values of X1, and X2 which can ultimately cause flickering in the generated HDR video. Figures 2.7 (a) and (b) show the changes in X1, and X2 using our proposed segmentation method for 25 consecutive frames with no scene change,                                 (a)                                                      (b)                                                        (c)     (d)                                                      (e)                                                         (f) Figure 2.6 (a) – (c) An example of dark, normal, and bright SDR frames, respectively. (d) – (f) Segmentation results using our proposed method for each frame, respectively.  Black, gray, and white portions demonstrate dark, normal, and bright regions in the input SDR frame, respectively. 31  respectively. As can be seen, our proposed segmentation method, due to its high sensitivity to changes in pixel distribution, results in large variations in the values of X1, and X2. We will address this issue in Chapter 4 and propose a more robust segmentation method that is less sensitive to changes in pixel distribution and prevents flickering. 2.1.3 Mapping SDR PQ Values to HDR Domain Many different mapping curves may be used to convert SDR PQ values to HDR domain. To choose the best possible mapping curve, we thoroughly tested different types of curves, compared the generated HDR frames (using SIM2 4000 nits HDR display) using each curve using comprehensive subjective tests and evaluated the overall visual quality of the generated frames, and chose the one that achieved the highest visual quality. These curves included piece-wise linear, exponential, and polynomial curves, to mention a few. The performance evaluations concluded that the piece-wise linear function yielded acceptable results while offering the simplest implementation, good control over range, and low computational cost. Since we divide the SDR                                              (a)                                                                                       (b) Figure 2.7 Changes in X1 and X2 using our proposed segmentation method. (a) and (b) show the changes in X1 and X2, respectively. 32  frame into three brightness regions, our mapping curve is a piece-wise linear curve with two breaking points. Figure 2.8 shows an example of a mapping curve used in our iTMO. This curve maps the SDR PQ values from [0.0623 – 0.5081] (corresponding to [0.1 – 100] nits) to [0.0151 – 0.9026] (corresponding to [0.005 – 4000] nits) in the HDR domain. s1, s2, and s3 in the figure are the slopes assigned to map dark, normal, and bright regions to HDR domain, respectively. An important issue in deciding the shape of our piece-wise linear mapping curve is the slope that we assign to dark and bright regions. The slopes should be assigned in a manner that produce high visual quality HDR frames while at the same time keeping the visual artifacts at non-perceivable levels. In our proposed method, to choose the appropriate set of slopes that work well for all types of SDR content we conducted an empirical analysis. In our test, we examined different slopes for mapping dark regions of the SDR frames in our database of 300 frames that were chosen randomly from [36] to represent a very large variety of content. By inspection of the generated HDR frames on a SIM2 HDR display [50], we concluded that there exist a set of slopes 1 ≤ s1 ≤ 3 for which the output contains no visual artifacts and offers more visible details in dark regions. In  Figure 2.8 An example of mapping curve used in our proposed iTMO.   33  other words, this range represents a good trade-off between visual quality and visual artifacts in dark regions of the generated HDR frame. We performed the same empirical analysis for a second time, but this time we examined different slopes for mapping the bright regions of the same SDR frames. By inspection of the generated HDR frame on a SIM2 HDR display [50], we concluded that there exist a set of slopes 1.5 ≤ s3 ≤ 3.9 for which the generated HDR frames contain less banding and more visible details in bright regions. In other words, this range offers a good trade-off between visual quality and visual artifacts in bright regions of the generated HDR frame. Based on our observations, we conclude that if we choose our slopes outside the ranges found for dark and bright scenes, the generated HDR frames will have some visible visual artifacts. Figure 2.9 demonstrates the slope limits for dark and bright regions of the SDR frame along with their corresponding mapping curves. These curves map the SDR PQ values in the range [0.0623 – 0.5081] ([0.1 – 100] nits) to [0.0151 – 0.9026] ([0.005 – 4000] nits). In order to have a good trade- Figure 2.9 An example of the two curves associated with slope limits for dark and bright regions. These two curves demonstrate the limits for which we can choose our mapping curves.  34  off between visual quality and visual artifacts in the generated HDR frame, we are limited to choosing a mapping curve that falls entirely between the red and blue curves. Any curve above the red or below the blue curve results in low visual quality HDR frames. The range of slope limits for dark and bright regions are (in degrees) 45○ ≤ θ1 ≤ 71○, and 56○ ≤ θ2 ≤ 75○, respectively. To have a single mapping curve that works well with all types of SDR content, we took the average of the maximum and minimum slopes that we could use in dark, and bright regions, ending up with a slope of s1 = 2 for dark areas and s3 = 2.7 for bright regions. One issue with the proposed segmentation method is that since our search region for X1 and X2 begins at the first and ends at the last non-zero elements of the histogram, we may have boundaries that do not correspond to dark and/or bright regions in the frame. Therefore, to get boundaries that are more accurate for each of the three brightness regions, we need to limit the search region for X1, and X2. It has been proven  that our eyes start to perceive colors for luminance values above 3 nits, and that the range 0.1 (0.0623 in PQ) to 3 (0.2133 in PQ) nits constitutes low but not quite dark lighting situations [51], [52]. Based on these observations, in our proposed segmentation method we limit the search region for X1 to [Xmin – 0.2133]. As we stated previously, we only chose our slope limits for dark and bright regions of the SDR frame, and we choose the slope for normal regions based on the other two slopes. However, since in a video sequence the values of X1 and X2 change from frame to frame, there is a chance of either having slopes that are less than one (which result in loss of details) or even negative slopes for normal regions of the SDR frame. Therefore, in order to prevent these two cases, the slope for normal regions must be greater than or equal to one (s2 ≥ 1). To this end, X1 and X2 must satisfy the following equation: 2 11.7 0.3589X X  .                                                                                                                                                      (2.8) 35  Therefore, after finding X1 in the range [Xmin – 0.2133] and X2 in the range [Xmid – Xmax] using our proposed segmentation method, we examine whether the value of X2 satisfies equation (2.8). If it does, we keep it. Otherwise, we set its values so that s2 = 1: 120.35891.7XX                                                                                                                                                              (2.9) 2.2 Experimental Evaluations  To evaluate the performance of our proposed iTMO we conducted a subjective test and compared the HDR frames generated by our proposed method against the ones generated using Meylan et al.’s and Banterle et al.’s operators in terms of the overall visual quality of the generated HDR frame. We used 23 SDR frames representing dark, normal, and bright scenes. Eighteen adults including 11 males and 7 females with ages between twenty-four and thirty-five years old participated in our test. All subjects were tested for color blindness and visual acuity using Ishihara and Snellen charts respectively. Prior to the test, a training session with three SDR frames representing dark, normal, and bright scenes was conducted to familiarize the subjects with the test procedure. During the test, the HDR frames generated by our proposed iTMO and the other two operators were shown on each side of the SIM2 HDR display [50], with a 14-pixels width black stipe between each pair. Subjects were asked to choose one of the two HDR frames (“A” or “B”) that appears more visually pleasing to them. In case the two HDR frames appear to have the same level of visual quality, subjects were instructed to report them as “equal”.   The subjective test results were analyzed using the well-established paired comparison methodology proposed in [53]. In addition, an outlier detection based on the method proposed in [53] was employed and two outliers were detected and their results were excluded from our analysis. 36  To validate the subjects’ choice consistency as well as taking any variations of brightness across the display into account, the position of the frames processed by our proposed iTMO was randomized from left to right side of the display and shown repeatedly to the subjects during the test.  Figure 2.10 shows the percentage of how many times, on average, subjects preferred the visual quality generated by our iTMO over that of Meylan et al.’s and Banterle et al.’s operators. As it can be observed, on average, our iTMO outperforms Meylan et al.’s operator by 87% and Banterle et al.’s operator by 68% of the time in terms of generating HDR frames that are more visually pleasing to the viewers. It is worth mentioning that we implemented both Banterle et al.’s, and Meylan et al.’s operators using the open source code available in [28] and set the maximum brightness of the generated HDR frame to 4000 nits. In addition, all the other parameters were chosen in a way that both of these operators yield the highest visual quality HDR frames.  Figure 2.10 Preference scores of the subjects in mean opinion score (MOS) when comparing the visual quality generated by our iTMO over that of Meylan et al.’s, and Banterle et al.’s operators.  37  2.3 Conclusions  In this chapter, we proposed an iTMO capable of generating high visual quality HDR frames. By working in the perceptual domain, our proposed iTMO is able to model the sensitivity of HVS to changes in brightness in different regions of the scene. Doing so enables our proposed iTMO to maintain the artistic intent, which among other things means to keep the dark areas dark and the bright areas bright, without darkening or brightening up the frame. To this end, we proposed an entropy-based segmentation method that divides the SDR frame into three brightness regions. To convert the SDR PQ values values to HDR domain we chose a piece-wise linear mapping curve with two breaking points. The slopes for each segment of our mapping curve were chosen based on comprehensive subjective testing and in a way that yield high visual quality HDR frames. Based on our subjective evaluations, our proposed iTMO outperforms other existing state-of-the-art methods in terms of generating high visual quality HDR frames.  As we mentioned before, since our segmentation method is too sensitive to variations in pixel distribution, it results in flickering for video applications. On the other hand, our proposed iTMO is not considered content adaptive since the slopes of our mapping curve are fixed for all types of SDR frames. In addition, since the slopes in our empirical analysis were chosen using a 4000 nits HDR display, the proposed iTMO in this chapter was only suitable for converting SDR frames to HDR format with maximum brightness of 4000 nits. To address these shortcomings, in the following chapter we introduce a general solution to the above issues and propose a content adaptive iTMO that prevents flickering and is capable of generating HDR videos for any target HDR range. 38  Chapter 3: A Perception-based inverse Tone Mapping Operator for High Dynamic Range Video Applications  In this Chapter, we propose a perception-based iTMO capable of producing high visual quality HDR videos. To overcome the issues of our proposed entropy-based segmentation in the previous chapter, we employ fixed boundaries for dark, normal, and bright regions of the SDR frames. By doing so, we eliminate the possibility of flickering for video applications. To choose the slopes of our piece-wise linear mapping curve we propose a hybrid optimization approach where we maximize a weighted sum of our proposed contrast and brightness difference functions. Finally, by taking into account the maximum brightness of the target HDR display, this iTMO is capable of generating HDR videos with arbitrary maximum brightness.  The rest of this chapter is organized as follows: Section 3.1 provides the details of our proposed iTMO. Section 3.2 presents the results of our experimental evaluations. Finally, Section 3.3 concludes this chapter. 3.1 Proposed iTMO  The components of our proposed iTMO are shown as part of the end-to-end workflow in Figure 3.1. The main components of our iTMO are enclosed in the red dotted line. These components include conversion of the SDR light values to perceptual (PQ) domain, computing the histogram of the resulting PQ values, dividing the frame into three brightness regions based on the eye sensitivity to brightness changes throughout the scene, calculating the slopes of the mapping curve by maximizing a weighted sum of contrast and brightness difference functions between input SDR and generated HDR frames, and finally mapping the SDR PQ values to HDR domain and 39  converting the HDR PQ values back to the light domain. The following sub-sections describe in detail the main components of our iTMO. 3.1.1 Converting SDR Light Values to Perceptual (PQ) Domain, Computing the Histogram and Segmentation to Three Brightness Regions  We base our design on the perceptual features of the HVS to exploit the sensitivity of our eye to different brightness levels. This helps us to efficiently apply our mapping process. To this end, our first task is to convert the SDR light values to perceptual values. This is an essential step as our eyes do not see light in a linear way, with sensitivity in perceiving brightness changes directly depending on the overall ambient brightness. More specifically, our eyes are more sensitive to brightness variations in dark regions than they are to normal and bright ones. The introduction of HDR led to the design of a Perceptual Quantizer (PQ) that takes advantage of HVS properties and optimizes the distribution of light values, converting physically linear values to perceptually linear ones [46]. Perceptual linearity indicates that our eyes perceive any change in light intensity at any brightness level equally. This is an essential part of our implementation, as  Figure 3.1 Workflow of our proposed iTMO. The red dotted line encloses components of our approach.  40  we aim to use this knowledge when mapping visual information from a lower dynamic range to a higher one.  Our next step is to compute the histogram of the resulting SDR PQ values, which groups them into uniform size bins (i.e., with equal number of brightness values in each bin). The height of each bin indicates the number of pixels in the frame with these PQ values. Figure 3.2 (a) shows an example of an SDR frame along with the histogram of its luminance channel in PQ domain in Figure 3.2 (b). Please note that the original SDR light values in this example are from 0.1 to 100 nits, which once transferred to the PQ domain range from 0.0623 to 0.5081.                                                                                          (a)  (b) Figure 3.2 (a) An SDR frame along with (b) the histogram of its luminance channel in PQ domain. 41   One of the main objectives in converting SDR content to HDR format is maintaining the artistic intent and the overall visual impression of the original SDR frame. To this end, we need to treat dark, normal, and bright regions of the content differently, as the eyes’ sensitivity to color changes when the brightness level varies. For instance, when the light levels drop to near total darkness, the response of the eye changes significantly according to the Scotopic response curve [48]. Therefore, to address this, our next task is to divide the SDR PQ values to dark, normal, and bright regions. Although PQ function takes into account the sensitivity of the human eye to brightness changes in different regions of a scene, it does not indicate where the boundaries of dark, normal, and bright regions are. Figure 3.3 (a) shows the PQ curve for mapping SDR light values in the range [0.1 – 100] nits to [0.0623 – 0.5081] in the PQ domain. As we mentioned before, our eyes are more sensitive to brightness changes in dark areas rather than normal and bright ones. This effect is clearly taken into account in Figure 3.3 (a) by the shape of the PQ curve. By inspecting Figure 3.3 (a) we can observe that dark areas are where the rate of change, as shown in Figure 3.3 (b), in the PQ curve is at its highest, while the bright areas are the ones where the rate of change is reaching its minimum value. Figures 3.3 (c) and (d) depict the zoomed areas corresponding to dark and bright regions, respectively. In bright regions, we observe that at approximately 0.5 nits the rate of change for dark regions is starting to settle to its minimum value, while at 30 nits it begins to settle to its minimum value for bright regions. Therefore, we conservatively define our dark, normal, and bright regions as the ones in the range [0.1 – 0.5] nits ([0.0623 – 0.1217] in PQ), (0.5 –30] nits ((0.1217 – 0.3962] in PQ), and (30 – 100] nits ((0.3962 – 0.5081] in PQ), respectively. Our proposed brightness segmentation follows closely the zone system proposed by Ansel Adams [54]. Figures 3.4 (a), (b), and (c) show three SDR frames, while 42  Figures 3.4 (d), (e), and (f) depict the results of our segmentation method for each SDR frames respectively where black, gray, and white regions represent dark, normal, and bright regions, respectively. As can be seen from the figures our fixed boundaries do a reasonably good job in dividing a frame into three brightness regions.  (a)  Figure 3.3 (a) PQ curve for mapping SDR light values to PQ domain. (b) Rate of change (derivative) of the PQ curve. (c) and (d) Zoomed areas corresponding to dark and bright regions, respectively. 43  3.1.2 Mapping SDR PQ Values to HDR Domain  Over the years, many different mapping curves have been used, varying from polynomial to exponential. As we discussed in Chapter 2, the piece-wise linear function is a great starting point for our implementation as it is computationally efficient, and is capable of generating satisfactory results. Considering that each SDR frame is segmented into three brightness regions perceived somewhat different by the human eye, our mapping curve is a piece-wise linear curve with two breaking points presented by the following equation: 1 , 1 ,, 2 , 2 ,3 , 3 ,0.12170.1217 0.39620.3962SDR PQ SDR PQHDR PQ SDR PQ SDR PQSDR PQ SDR PQs L a    if   LL s L a   if   Ls L a   if    L                                                                           (3.1)                               (a)                                                       (b)                                                        (c)                              (d)                                                       (e)                                                        (f) Figure 3.4 (a) - (c) An example of dark, normal and bright SDR frames, respectively. (d) - (f) Represent the result of our segmentation method for each frame, respectively. Black, gray, and white portions demonstrate dark, normal, and bright regions in the input SDR frame, respectively. 44  where LHDR,PQ, and LSDR,PQ are the HDR and SDR luminance channels in PQ domain respectively, s1, s2, and s3 are the slopes associated with each of our three brightness regions, a1, a2, and a3 are the y-intercepts associated with each of the three lines in our mapping curve.  Figure 3.5 shows an example of mapping curve used in our iTMO. X1 is the boundary between dark and normal regions while X2 indicates the boundary between normal and bright regions. This mapping curve maps the PQ values from the range [0.0623 – 0.5081] (or [0.1 – 100] nits) to [0.0151 – 0.9026] (or [0.005 – 4000] nits). It is worth mentioning that the maximum and minimum mapped values are dependent on the maximum and minimum brightness of the target HDR display. This dependency makes our method compatible with HDR displays supporting different brightness ranges.  A critical point in deciding the shape of our piece-wise linear mapping curve is the slope that we assign to dark, normal, and bright regions. The slopes should be assigned in a way that the final HDR frame looks superior to its SDR counterpart in terms of overall visual quality. One of  Figure 3.5 An example of mapping curve used in our iTMO. X1 is the boundary between dark and normal regions, and X2 is the boundary between normal and bright regions.  45  the main advantages of the HDR content over SDR is that it conveys a much higher contrast. Our initial main objective, thus, is to maximize the contrast in the resulting HDR frame. The contrast of the k-th region in HDR domain, CHDR,k , will be:  , , ,max, , ,min,, ,max, , ,min, , 1, 2, 3HDR k HDR PQ k HDR PQ kk SDR PQ k SDR PQ k k SDR kC L Ls L L s C    ,   k =                                                                         (3.2) where LHDR,PQ,min,k, LHDR,PQ,max,k, LSDR,PQ,min,k and LSDR,PQ,max,k are the minimum and maximum PQ values in the k-th region of the HDR and SDR domains respectively, CSDR,k is the contrast of the k-th region of the SDR frame, and sk is the slope assigned to k-th region of the SDR frame.  To maximize the contrast in each region, we treat this as a convex optimization problem, which allows us to find a global optimum solution in an efficient manner [55]. Let CSDR,1, and CHDR,1 be the contrast of the first region in the SDR and HDR domains, respectively. Therefore, our objective function would be:  2 2 2,1 ,1 1 ,121HDR SDR SDRC C s C                                                                                                                                  (3.3) where 22 is the second norm (2) to the power of two, and s1 is the slope assigned to first region. In order to make our maximization process content adaptive, we need to take into account the distribution of pixels in each of the three brightness regions of the SDR domain. For the first region, we take the distribution of pixel values into account by employing the expectation operator as follows:  2 2,1 ,1 1 ,1 12 ( 1)HDR SDR SDRE C C s C p                                                                                                                      (3.4) where p1 is the number of pixels in the first region over the total number pixels in the SDR frame, and E is the expectation operator. 46   For all of the three brightness regions, our final objective function, FC (s1,s2,s3), would be:          32 22 22 2 2 21 2 3 1 ,1 1 2 ,2 2 3 ,3 3 ,1, , 1 1 1 1C SDR SDR SDR k SDR k kkF s s s s C p s C p s C p s C p                                (3.5) where p1, p2, and p3 are the number of pixels in the first, second, and third regions over the total number of pixels in the SDR frame respectively and s1, s2, and s3 are the slopes assigned to the first, second, and third regions, respectively.  Our optimization problem is limited by two constraints. The first one is that the resulting HDR contrast in each region should be greater than or equal to the corresponding one in the SDR frame, meaning that the slope for each region must be greater than or equal to one. The second condition is that the sum of the three HDR regions should be less than or equal to the maximum dynamic range supported by the target HDR display. Given the above, our final maximization problem becomes:   1 2 3, ,CMaximize F s s s                                                                                                                                               (3.6) subject to 1   ,   1, 2, 3ks k                                                                                                                                                                  (3.7) 3,1k SDR k HDRks D D                                                                                                                                                                (3.8) where sk is the slope assigned to k-th region, DSDR,k is the difference between maximum and minimum PQ values in the k-th region of the SDR domain (actual bound), and DHDR is the difference between the maximum and minimum brightness values of the target HDR display in PQ domain. Solving this optimization problem for each frame yields the values of the three regional slopes for that frame, given that DSDR,k and DHDR are already known. 47   One issue with our approach of maximizing the contrast difference is that the resulting slopes may generate a huge difference between the SDR and HDR contrast, which in turn will yield unwanted visual artifacts such as noise in dark areas and banding in bright areas and in some extreme cases change the overall brightness of the frame.  Figures 3.6 and 3.7 show two examples of extreme cases for an overall dark and bright SDR frame, respectively. Figure 3.6 (a) depicts an overall dark SDR frame while Figure 3.6 (b) shows the corresponding HDR frame (tone mapped here using Reinhard’s tone mapping operator [37] for visualization purposes) generated by our contrast approach. Figure 3.7 (a) depicts an                                               (a)                                                                                  (b)  Figure 3.6 An example of an overall dark SDR frame with (b) its corresponding HDR frame with our contrast approach.                                               (a)                                                                                  (b)  Figure 3.7 An example of an overall bright SDR frame with (b) its corresponding HDR frame with our contrast approach. 48  overall bright SDR frame while Figure 3.7 (b) shows the corresponding HDR frame (tone mapped again for visualization purposes) generated by our contrast approach. As it can be seen from these two examples, although we achieve high contrast in each area of the input SDR frame, in these extreme cases the dark frame became disproportionally brighter and the bright frame disproportionally darker. This is caused by the fact that in our contrast difference maximization process we only consider minimum and maximum PQ values of each region in the SDR domain and try to increase their difference as much as we can in the HDR domain. As a result, the contrast becomes disproportionally bigger, which on one hand affects the overall brightness and on the other forces the PQ values to be farther apart which generates noise and banding. To overcome this issue, we also consider maximizing the difference of the individual PQ values between input SDR and generated HDR frames, first adding the objective functions for each SDR brightness regions and then maximizing the brightness difference of the entire frame.   To maximize the difference of the individual PQ values in each region, we treat this as a convex optimization problem, which allows us to find a global optimum solution in an efficient manner [55]. Let LSDR,PQ,1, and LHDR,PQ,1 be an SDR PQ value in the first region and its corresponding HDR PQ value, respectively. Therefore, based on equation (3.1), our initial objective function for these two points will be:   22, ,1 , ,1 1 , ,1 121HDR PQ SDR PQ SDR PQL L s L a                                                                                                        (3.9) where 22 is the second norm (2) to the power of two, s1 is the slope assigned to first region, and a1 is the y-intercept associated with the first line of our mapping curve. As equation (3.9) considers only one set of points in SDR and HDR domains, our objective function for all the PQ values in the first region would be as follows: 49            1 111 12 2, ,1 , ,1 , ,2 , ,22 22 2, , , , 1 , ,1 12221 , ,2 1 1 , , 12 2 21 , , 1 1 , , 1 11 1...11 ... 11 2 1HDR PQ SDR PQ HDR PQ SDR PQHDR PQ N SDR PQ N SDR PQSDR PQ SDR PQ NN NSDR PQ i SDR PQ ii iL L L LL L s L as L a s L as L a s L N a                                                                                                  (3.10) where LSDR,PQ,1, LSDR,PQ,2, …, LSDR,PQ,N1 and LHDR,PQ,1, LHDR,PQ,2, …, LHDR,PQ,N1 are SDR and HDR PQ values in the first region respectively, N1 is the total number of SDR PQ histogram bins in the first region.  To make our maximization process content adaptive, we need to take into account the distribution of pixel values in each of our three brightness regions in the SDR frame. This is achieved by taking the expectation operator of equation (3.10):       1 11 12 2 21 , , 1 1 , , 1 11 12 2 21 1 , , 1 1 1 , , 1 11 11 2 11 2 1N NSDR PQ i SDR PQ ii iN NSDR PQ i SDR PQ ii iE s L a s L N as p L a s p L N a                                                                                        (3.11) where E is the expectation operator, LSDR,PQ,i is the i-th PQ value in the first region of SDR domain, and p1 is the number of pixels in the first region divided by total number of pixels in the frame. It is important to point out that since N1a1 is a constant, the expectation operator has no effect on it. Expanding equation (3.11) over all three brightness regions, the final objective function, FB(s1,s2,s3), becomes:            1 11 2 1 21 11 2 1 22 2 21 2 3 1 1 , , 1 1 1 , , 1 11 12 2 22 2 , , 2 2 2 , , 2 21 1256 2562 23 3 , , 3 3 3 , , 3 31 1, , 1 2 11 2 11 2 1N NB SDR PQ i SDR PQ ii iN N N NSDR PQ i SDR PQ ii N i NSDR PQ i SDR PQ ii N N i N NF s s s s p L a s p L N as p L a s p L N as p L a s p L N a                          2                                              (3.12) 50  where p1, p2, p3 are the number of pixels in the first, second, and third brightness regions over the total number pixels in the SDR frame, respectively. Since we are dealing with 8-bit SDR frames that have 256 bins in their histogram, we have N1 + N2 + N3 = 28 = 256.  Our optimization problem is limited by two constraints. The first one is that the resulting HDR PQ values in each region should be greater than or equal to the corresponding ones in the SDR frame, meaning that the slope for each region must be greater than or equal to one. The second condition is that the sum of three HDR regions should be less than or equal to the maximum dynamic range supported by the target HDR display. Given the above, our final maximization problem becomes:   1 2 3, ,BMaximize F s s s                                                                                                                                             (3.13) subject to:  1   ,   1, 2, 3ks k                                                                                                                       (3.14) 3,1k SDR k HDRks D D                                                                                                                       (3.15) where sk is the slope assigned to k-th region, DSDR,k is the difference between maximum and minimum PQ values in the k-th region of the SDR domain, and DHDR is the difference between the maximum and minimum brightness of the target HDR display in PQ domain. Solving this optimization problem yields the values of the three regional slopes, given that DSDR,k and DHDR are already known.  Figures 3.8 and 3.9 show two examples of extreme cases of a dark and bright SDR frames, respectively. Figure 3.8 (a) depicts an overall dark SDR frame along with the HDR frame (tone mapped) generated by our contrast approach in Figure 3.8 (b) and the HDR frame generated by our brightness approach in Figure 3.8 (c). Figure 3.9 (a) depicts an overall bright SDR frame 51  along with the HDR frame (tone mapped) generated by our contrast approach in Figure 3.9 (b) and the HDR frame generated by our brightness approach in Figure 3.9 (c). As it can be seen from Figures 3.8 and 3.9, the brightness approach, that works more “locally” than the contrast approach, is able to maintain the overall brightness of each region in the generated HDR frame. Therefore, to take advantage of the high contrast offered by contrast approach and good overall brightness offered by brightness approach, we decided to combine them into a hybrid approach, maximizing a weighted sum of the contrast and brightness difference functions. In other words, our maximization problem becomes:     1 , 1 2 3 2 , 1 2 3, , , ,B n C nMaximize w F s s s w F s s s                                                                    (3.16)                                               (a)                                                                                  (b)  (c) Figure 3.8 (a) An example of an overall dark SDR frame with (b) its corresponding HDR frame generated using our contrast approach. (c) Corresponding HDR frame generated using our brightness approach. 52  subject to: 1   ,   1, 2, 3ks k                                                                                                                        (3.17) 3,1k SDR k HDRks D D                                                                                                                       (3.18) where FB,n(s1,s2,s3), and FC,n(s1,s2,s3) are the normalized brightness and contrast difference functions respectively and w1 and w2 are two weights with w1 + w2 = 1.  After forming our convex hybrid maximization problem, we solve it using first order Karush-Kuhn-Tucker (KKT) optimality condition of the corresponding Lagrangian [55] and obtain our slopes for each of the three brightness regions. The process for choosing the weight values w1 and w2 is explained in the following section.                                               (a)                                                                                  (b)  (c) Figure 3.9 (a) An example of an overall bright frame with (b) its corresponding HDR frame with our contrast approach. (c) Corresponding HDR frame with our brightness approach. 53  3.2 Experimental Evaluations To choose the best possible combination for w1 and w2 that yields highest visual quality HDR frames we performed objective quality assessment using HDR VDP-2.2 [56] and PU-SSIM [57] quality metrics. These two metrics are designed for evaluating the quality of a test HDR frame with respect to an original (ground truth) HDR frame. For the original HDR frame, we used the dataset in [58] and randomly chose nine HDR frames representing dark, normal, and bright scenes for performance evaluation under different brightness conditions. Figure 3.10 shows the tone mapped version of the nine HDR frames which include Hills, Bridge, Vegas, Store, Kitchen, Park, Bench, Peak, and Church. We tone mapped the nine HDR frames using Reinhard’s tone mapping operator and used the generated SDR frames as input for our hybrid iTMO. We generated different HDR frames using three different sets of values for w1 and w2: 1) w1=w2=0.5, 2) w1=0.3, w2=0.7 and 3) w1=0.7, w2=0.3. Finally, we evaluated the quality of each generated HDR frame with respect to the original HDR frame for each set of values using HDR VDP-2.2 and PU-SSIM. Tables 3.1, 3.2, and 3.3 show the results of HDR VDP-2.2 and Tables 3.4, 3.5, and 3.6 show the results of PU-                (a) Hills                    (b) Bridge                   (c) Vegas                         (d) Store                     (e) Kitchen                                                                      (f) Park          (g) Bench   (h) Peak   (i) Church Figure 3.10 Nine original tone mapped HDR frames randomly chosen from the dataset in [58]. 54  SSIM for different sets of values for w1 and w2 for dark (Vegas, Store, Church), normal (Kitchen, Park, Peak), and bright (Hills, Bridge, Bench) frames, respectively. It is important to point out that the higher the score of HDR VDR-2.2 and PU-SSIM, the higher the visual quality of the generated Table 3.1 Visual quality scores by HDR VDP-2.2 for dark frames in Figure 3.10 for three different values of w1 and w2 Frame w1=w2=0.5 w1=0.3, w2=0.7 w1=0.7, w2=0.3 Vegas 45.70955 45.59459 45.68054 Store 44.42627 44.4238 44.42818 Church 46.82211 46.72068 46.8166 Average 45.65264 45.57969 45.64177  Table 3.2 Visual quality scores by HDR VDP-2.2 for normal frames in Figure 3.10 for three different values of w1 and w2 Frame w1=w2=0.5 w1=0.3, w2=0.7 w1=0.7, w2=0.3 Kitchen 47.25607 47.20854 47.23436 Park 48.91358 48.90155 48.91529 Peak 46.67786 46.69233 46.67106 Average 47.61584 47.60081 47.6069  Table 3.3 Visual quality scores by HDR VDP-2.2 for bright frames in Figure 3.10 for three different values of w1 and w2 Frame w1=w2=0.5 w1=0.3, w2=0.7 w1=0.7, w2=0.3 Hills 48.62002 48.62003 48.60073 Bridge 46.78144 46.77396 46.76266 Bench 50.23429 50.21976 50.1995 Average 48.54525 48.53792 48.52097   Table 3.4 Visual quality scores by PU-SSIM for dark frames in Figure 3.10Table 3.5 Visual quality scores by HDR VDP-2.2 for dark frames in Figure 3.10 Frame w1=w2=0.5 w1=0.3, w2=0.7 w1=0.7, w2=0.3 Vegas 45.70955 45.59459 45.68054 Store 44.42627 44.4238 44.42818 Church 46.82211 46.72068 46.8166 55  HDR frame. Therefore, as it can be seen from the results of both metrics, on average, when w1=w2=0.5, our hybrid approach achieves the highest visual quality for all types of SDR frames. As a result, in our iTMO, we set the values of both w1 and w2 to 0.5.  Table 3.4 Visual quality scores by PU-SSIM for dark frames in Figure 3.10 for three different values of w1 and w2 Frame w1=w2=0.5 w1=0.3, w2=0.7 w1=0.7, w2=0.3 Vegas 0.959037 0.957373 0.956808 Store 0.966426 0.966354 0.966475 Church 0.970388 0.970428 0.969719 Average 0.965284 0.964718 0.964334  Table 3.5 Visual quality scores by PU-SSIM for normal frames in Figure 3.10 for three different values of w1 and w2 Frame w1=w2=0.5 w1=0.3, w2=0.7 w1=0.7, w2=0.3 Kitchen 0.972699 0.972298 0.971302 Park 0.969191 0.968345 0.967109 Peak 0.972083 0.971996 0.97188 Average 0.971324 0.97088 0.970097  Table 3.6 Visual quality scores by PU-SSIM for bright frames in Figure 3.10 for three different values of w1 and w2 Frame w1=w2=0.5 w1=0.3, w2=0.7 w1=0.7, w2=0.3 Hills 0.968888 0.968114 0.968038 Bridge 0.969359 0.965217 0.96527 Bench 0.988664 0.98168 0.981628 Average 0.975637 0.97167 0.971645    Table 3.16 Visual quality scores by PU-SSIM for dark frames in Figure 3.10 Frame w1=w2=0.5 w1=0.3, w2=0.7 w1=0.7, w2=0.3 Vegas 0.959037 0.957373 0.956808 Store 0.966426 0.966354 0.966475 Church 0.970388 0.970428 0.969719 56  Figures 3.11 and 3.12, depict two examples of extreme cases (dark and bright SDR frames). Figures 3.11 (a), (b), (c), and (d) depict an overall dark frame and the corresponding HDR frames (tone mapped) generated by our contrast approach, our brightness approach, and hybrid approach with w1 = w2 = 0.5, respectively. Figures 3.12 (a), (b), (c), and (d) depict an overall bright frame and the corresponding HDR frames (tone mapped) generated by our contrast approach, our brightness approach, and hybrid approach with w1 = w2 = 0.5, respectively. We observe that our hybrid approach produces results that have a good balance between overall brightness and contrast of the generated HDR frame.                                               (a)                                                                                  (b)                                               (c)                                                                                  (d) Figure 3.11 (a) An example of an overall dark SDR frame with (b) its corresponding HDR frame with our contrast approach. (c) Corresponding HDR frame with our brightness approach, and (d) corresponding HDR frame with our hybrid approach with w1 = w2 = 0.5.                                                (a)                                                                                  (b)                                               (c)                                                                                  (d) 57   To evaluate the performance of our hybrid iTMO against our brightness and contrast methods applied separately we also performed a subjective test. Nine 4K SDR video streams [59] (each 10 seconds long) including normal, dark, and bright content were converted to HDR format using our three approaches and displayed on two similar Sony BVM-X300 professional monitors [60]. Figure 3.13 depicts one frame of each of the nine 4K SDR videos, representative of different content and brightness conditions, used in our subjective test, which include Mountain, Santorini, Niagara, Night, Sunrise, Workplace, Building, Leaves, and Room. Thirteen male, and five female subjects with ages ranging from twenty-four to thirty-two took part in our subjective evaluations. Prior to the test, all subjects were screened for color blindness and visual acuity using the Ishihara                                               (a)                                                                                  (b)                                               (c)                                                                                  (d) Figure 3.12 (a) An example of an overall bright SDR frame with (b) its corresponding HDR frame with our contrast approach. (c) Corresponding HDR frame with our brightness approach, and (d) corresponding HDR frame with our hybrid approach with w1 = w2 = 0.5.                                                (a)                                                                                  (b)                                               (c)                                                                                  (d) 58  and Snellen charts, respectively. A training session allowed all subjects to become familiar with the test procedure. The subjective test results were analyzed using the methodology in [53]. Subjects were asked to choose if one of the two HDR videos (“A” or “B”) was visually more appealing than the other or if they looked the same (“equal”). To take into account any slight variations between the two displays, the position of the videos was randomized from left to right. Two outliers were detected using the method described in [53] and the data collected from them was omitted from our evaluation. Figure 3.14 shows the average of times the subjects preferred the visual quality produced by our hybrid iTMO against brightness and contrast approaches separately. As it can be seen, our hybrid method outperforms the brightness and contrast methods by 80% and 86% of the times respectively, in terms of being more visually pleasing to the observers. We further analyzed the MOS scores of our contrast, brightness, and hybrid approaches based on F-value and p-value of one-way analysis of variance (ANOVA) to determine whether there any statistical significance between any of the three separate approaches. To do this, we performed the Levene’s test [61] to guarantee the homogeneity of variances of the gathered data, and we concluded that one-way ANOVA can be applied. Table 3.7 presents the results of the one-way ANOVA test including the mean and standard deviation of the MOS scores for each approach.                       (a) Mountain           (b) Santorini             (c) Niagara            (d) Night                 (e) Sunrise                                      (f) Workplace          (g) Building              (h) Leaves                (i) Room  Figure 3.13 One frame of each of the nine 4K SDR videos used in our subjective test.                        (a) Mountain           (b) Santorini             (c) Niagara            (d) Night                 (e) Sunrise                                      (f) Workplace          (g) Building              (h) Leaves                (i) Room  Figure 3.93 One frame of each of the nine 4K SDR videos used in our subjective test.  Table 3.25 Mean and standard deviation of MOS scores for our contrast, brightness and hybrid approaches using the dataset in Figure 3.13                       (a) Mountain           (b) Santorini             (c) Niagara            (d) Night                 (e) Sunrise                                      (f) Workplace          (g) Building              (h) Leaves                (i) Room  Figure 3.10 One frame of each of the nine 4K SDR videos used in our subjective test.       59  The ANOVA test concluded that the null hypothesis (i.e., there is no statistically significant difference between the three approaches) probability (p-value) for the dataset in our subjective test is 0.02. Since the p-value is less than the significance level α=0.05 we reject the null hypothesis and conclude that the difference in MOS values obtained by our contrast, brightness, and hybrid approaches is statistically significant.  Figure 3.14 Preference scores of the subjects in mean opinion score (MOS) when comparing the visual quality of the generated HDR video using our hybrid approach with w1 = w2 = 0.5 over that of brightness and contrast approaches separately.   Figure 3.124 Preference scores of the subjects in mean opinion score (MOS) when comparing the visual quality of the generated HDR video using our hybrid approach with w1 = w2 = 0.5 over that of brightness and contrast approaches separately.   Table 3.7 Mean and standard deviation of MOS scores for our contrast, brightness and hybrid approaches using the dataset in Figure 3.13 iTMO Mean Standard deviation Contrast approach 1.0625 1.1236 Brightness approach 1.75 0.9309 Hybrid approach 14.0625 1.4818 F-value (p-value) 9.5520 (0.02725)   Table 3.26 Mean and standard deviation of MOS scores for our contrast, brightness and hybrid approaches using the dataset in Figure 3.13Table 3.27 Mean and standard deviation of MOS scores for our contrast, brightness and hybrid approaches using the dataset in Figure 3.13 iTMO Mean Standard deviation Contrast approach 1.0625 1.1236 Brightness approach 1.75 0.9309 Hybrid approach 14.0625 1.4818 F-value (p-value) 9.5520 (0.02725)   60   To evaluate the MOS-based performance difference between any two of our contrast, brightness and hybrid approaches, we performed Tukey’s post-hoc analysis for their MOS scores. Figure 3.15 shows the results of this test when α=0.05. Methods in the same group are the ones with statistically insignificant difference. As it can be seen, our brightness and contrast approaches fall within the same group while our hybrid approach falls in a separate one, meaning that our hybrid approach yields statistically distinct HDR videos as compared to our brightness and contrast methods in terms of subjective quality assessment. We also compared the performance of our hybrid iTMO against our contrast and brightness approaches using HDR VDP-2.2 and PU-SSIM objective quality metrics. To this end, we used the same nine original HDR frames in Figure 3.10 chosen randomly from the dataset in [58]. We tone mapped these HDR frames using Reinhard’s tone mapping operator and used the generated SDR frames as input for our contrast, brightness and hybrid approaches. Finally, we evaluated the quality of generated HDR frames with respect to the original HDR frames using HDR VDP-2.2 and PU-SSIM. Tables 3.8, 3.9, and 3.10 show the results of HDR VDP-2.2 for our three separate approaches for dark (Vegas, Store, Church), normal (Kitchen, Park, Peak), and bright (Hill, Bridge, Bench) frames respectively. In addition, Tables 3.11, 3.12, and 3.13 show the results of PU-SSIM for our three separate approaches for dark, normal, and bright frames, respectively. As it can be seen from the results of both metrics, on average, our hybrid approach achieves the highest visual quality for all types of SDR frames, reaffirming the result of our subjective test.  Figure 3.15 Post-hoc analysis using Tukey’s method for mean values of MOS scores and α=0.05. Methods within the same group generate statistically insignificant HDR output.   Table 3.31 Visual quality scores by HDR VDP-2.2 for dark frames in Figure 3.10 Figure 3.15 Post-hoc analysis using Tukey’s method for mean values of MOS scores and α=0.05. Methods within the same group generate statistically insignificant HDR output.   Table 3.32 Visual quality scores by HDR VDP-2.2 for dark frames in Figure 3.10 Figure 3.16 Post-hoc analysis using Tukey’s method for mean values of MOS scores and α=0.05. Methods within the same group generate statistically insignificant HDR output.   Table 3.33 Visual quality scores by HDR VDP-2.2 for dark frames in Figure 3.10 Figure 3.17 Post-hoc analysis using Tukey’s method for mean values of MOS scores and α=0.05. Methods within the same group generate statistically insignificant HDR output.  61  We evaluated the performance of our proposed hybrid iTMO by carrying out another subjective test, comparing its performance against those of Meylan et al.’s [14], Rempel et al.’s [27], Kovaleski et al.’s [5], Bist et al.’s [7] and Kim et al.’s [10] iTMOs in terms of the overall Table 3.8 Visual quality scores by HDR VDP-2.2 for dark frames in Figure 3.10 using hybrid, brightness and contrast approaches. Frame Hybrid approach Contrast approach Brightness approach Vegas 45.70955 43.67382 44.13756 Store 44.42627 41.28935 41.27666 Church 46.82211 45.11983 45.59701 Average 45.65264 43.361 43.67041  Table 3.9 Visual quality scores by HDR VDP-2.2 for normal frames in Figure 3.10 using hybrid, brightness and contrast approaches. Frame Hybrid approach Contrast approach Brightness approach Kitchen 47.25607 46.74428 47.11382 Park 48.91358 48.97533 48.91015 Peak 46.67786 46.81748 46.67316 Average 47.61584 47.51236 47.56571  Table 3.10 Visual quality scores by HDR VDP-2.2 for bright frames in Figure 3.10 using hybrid, brightness and contrast approaches. Frame Hybrid approach Contrast approach Brightness approach Hills 48.62002 48.57038 48.54986 Bridge 46.78144 46.74898 46.71023 Bench 50.23429 49.86225 49.79646 Average 48.54525 48.39387 48.35218   Table 3.34 Visual quality scores by PU-SSIM for dark frames in Figure 3.10Table 3.35 Visual quality scores by HDR VDP-2.2 for dark frames in Figure 3.10 Frame Hybrid approach Contrast approach Brightness approach Vegas 45.70955 43.67382 44.13756 Store 44.42627 41.28935 41.27666 Church 46.82211 45.11983 45.59701 62  visual quality of the generated HDR video. The parameters of each of the iTMOs were set to their default values as mentioned in each of their respective papers. For Meylan et al.’s, Rempel et al.’s, and Kovaleski et al.’s iTMOs we used the publicly available source code in [28]. We implemented Table 3.11 Visual quality scores by PU-SSIM for dark frames in Figure 3.10 using hybrid, brightness and contrast approaches. Frame Hybrid approach Contrast approach Brightness approach Vegas 0.959037 0.957037 0.95658 Store 0.966426 0.966426 0.966524 Church 0.970388 0.970388 0.969073 Average 0.965284 0.964617 0.964059  Table 3.12 Visual quality scores by PU-SSIM for normal frames in Figure 3.10 using hybrid, brightness and contrast approaches. Frame Hybrid approach Contrast approach Brightness approach Kitchen 0.972699 0.971699 0.970909 Park 0.969191 0.968191 0.968037 Peak 0.972083 0.972083 0.97168 Average 0.971324 0.970658 0.970209  Table 3.13 Visual quality scores by PU-SSIM for bright frames in Figure 3.10 using hybrid, brightness and contrast approaches. Frame Hybrid approach Contrast approach Brightness approach Hills 0.968888 0.968188 0.968106 Bridge 0.969359 0.965359 0.965189 Bench 0.988664 0.981664 0.981615 Average 0.975637 0.971737 0.971636    Table 3.46 Visual quality scores by PU-SSIM for dark frames in Figure 3.10 Frame Hybrid approach Contrast approach Brightness approach Vegas 0.959037 0.957037 0.95658 Store 0.966426 0.966426 0.966524 Church 0.970388 0.970388 0.969073 63  Bist et al.’s iTMO based on the description in its respective paper. Authors of [10] generously provided us with the implementation of their method. We used a new set of nine 4K SDR video streams (each 10 seconds long) [59] involving normal, dark, and bright content for this test. Figure 3.16 depicts one frame of each of the nine 4K SDR videos used in our subjective test, which include Bellagio, Cave, Fireworks, Flower, Garden, Lake, Desert, Reflection, and Surfing. As can be seen, our dataset includes dark, normal, and bright scenes to evaluate the performance of our iTMO against others under different brightness conditions. Participants in the test were twelve male and six female subjects with ages ranging from twenty-three to thirty-one. All subjects were screened for color blindness and visual acuity by Ishihara and Snellen charts respectively. Subjects who participated in this test were different from the ones who participated in our previous test. A training session allowed subjects to get familiar with the test procedure.  Once more, the two HDR videos were shown on two similar Sony BVM-X300 professional monitors, and the subjective test results were analyzed according to the method proposed in [53]. The order of all possible combinations of the video pairs was randomized, yielding a total of 45 video streams. Subjects were required to choose either one of the two HDR videos as more visually appealing or decide if they looked the same. The videos were randomly changed between the two displays to remove any                      (a) Bellagio              (b) Cave             (c) Fireworks            (d) Flower             (e) Garden                                       (f) Lake                   (g) Desert              (h) Reflection           (i) Surfing Figure 3.16 One frame of each of the nine 4K SDR videos used in our subjective test.                       (a) Bellagio              (b) Cave             (c) Fireworks            (d) Flower             (e) Garden                                       (f) Lake                (g) Desert              (h) Reflection           (i) Surfing Figure 3.18 One frame of each of the nine 4K SDR videos used in our subjective test.                       (a) Bellagio              (b) Cave             (c) Fireworks            (d) Flower             (e) Garden                                       (f) Lake                   (g) Desert              (h) Reflection           (i) Surfing Figure 3.16 On  frame of each of the nine 4K SDR videos used in o r subjective test.                       (a) Bellagio              (b) Cave             (c) Fireworks            (d) Flower             (e) Garden     64  effect caused by possible differences between the two professional monitors and their physical location. Two outliers were found using the methodology in [53] and their results were omitted from our analysis. Figure 3.17 shows the frequency of times subjects favored the visual quality produced by our iTMO against Meylan et al.’s, Rempel et al.’s, Kovaleski et al.’s, Bist et al.’s, and Kim et al.’s iTMOs. We observe that, on average, our iTMO outperforms Meylan et al.’s by 78%, Rempel et al.’s by 94%, Kovaleski et al.’s by 81%, Bist et al.’s by 91%, and Kim et al.’s by 69% of the times in terms of being more visually pleasing to the observers. We further analyzed the MOS scores of the six iTMOs in this subjective test in Table 3.14 based on F-value and p-value of one-way analysis of variance (ANOVA). To do this, we performed the Levene’s test to guarantee the homogeneity of variances of the gathered data, and we concluded that one-way ANOVA can be applied. The ANOVA test concluded that the null hypothesis probability (p-value) for the dataset  in our subjective test is 3.87×10-11, meaning that difference in MOS values by the six iTMOs is statistically significant at significance level α=0.05.  Figure 3.17 Mean opinion scores (MOS) indicating the visual quality of the output HDR video generated by our iTMO against Meylan et al.’s, Rempel et al.’s, Kovaleski et al.’s, Bist et al.’s, and Kim et al.’s    Table 3.55 Mean and standard deviation of MOS scores for the six iTMOs based on the dataset in Figure 3.16 Figure 3.20 Mean opi ion scores (MOS) indicating the visual quality of the output HDR video generated by our iTMO against Meylan et al.’s, Rempel et al.’s, Kovaleski et al.’s, Bist et al.’s, and Kim et al.’s     65  To evaluate the MOS-based performance difference between any two of the six iTMOs, we performed Tukey’s post-hoc analysis for their MOS scores. Figure 3.18 shows the results of this test when α=0.05. Methods in the same group are the ones with statistically insignificant difference. As it can be seen, Meylan et al.’s, Rempel et al.’s, Kovaleski et al.’s, Bist et al.’s, and  Kim et al.’s iTMOs fall within the same group while our proposed hybrid iTMO falls in a separate one, meaning our proposed iTMO yields statistically distinct HDR videos as compared to other five iTMOs in terms of subjective quality assessment. We also compared the performance of our proposed hybrid iTMO against Meylan et al.’s, Rempel et al.’s, Kovaleski et al.’s, Bist et al.’s, and Kim et al.’s iTMOs using HDR VDP-2.2 and PU-SSIM objective quality metrics. To this end, we used nine original HDR frames randomly Table 3.14 Mean and standard deviation of MOS scores for the six iTMOs based on the dataset in Figure 3.16 iTMO Mean Standard deviation Meylan et al.’s 1.75 1.4832 Rempel et al.’s 0.5 0.7302 Kovaleski et al.’s 1.5 1.2110 Bist et al.’s 0.75 0.8563 Kim et al.’s 2.625 1.1474 Proposed 35.375 3.7925 F-value (p-value) 15.7863 (3.87483×10-11)   Table 3.58 Mean and standard deviation of MOS scores for the six iTMOs based on the dataset in Figure 3.16 iTMO Mean Standard deviation Meylan et al.’s 1.75 1.4832 Rempel et al.’s 0.5 0.7302 Kovaleski et al.’s 1.5 1.2110 Bist et al.’s 0.75 0.8563 Kim et al.’s 2.625 1.1474 Proposed 35.375 3.7925 F-value (p-value) 15.7863 (3.87483×10-11)   Table 3.59 Mean and standard deviation of MOS scores for the six iTMOs based on the dataset in Figure 3.16 iTMO Mean Standard deviation Meylan et al.’s 1.75 1.4832 R mpel et al.’s 0.5 0.7302 Kovaleski et al.’s 1.5 1.2110 Bist et al.’s 0.75 0.8563 Kim et al.’s 2.625 1.1474 Proposed 35.375 3.7925 F-value (p-value) 15.7863 (3.87483×10-11)  Figure 3.18 Post-hoc analysis using Tukey’s method for mean values of MOS scores and α=0.05. Methods within the same group generate statistically insignificant HDR videos.     Figure 3.22 Post-hoc analysis using Tukey’s method for mean values of MOS scores and α=0.05. Methods within the same group generate statistically insignificant HDR output.    Table 3.57 Mean and standard deviation of MOS scores for the six iTMOs based on the dataset in Figure 3.16 Figure 3.18 Post-hoc analysis using Tukey’s method for mean values of MOS scores and α=0.05. Methods 66  chosen from the dataset in [58] representing dark, normal, and bright scenes to evaluate the performance of our iTMO against others under different brightness conditions. Figure 3.19 shows the tone mapped version of the nine HDR frames which include Waterfall, Snow, Arch, Lamp, Typewriter, Peppermill, Pier, Gorse, and Rock. We tone mapped these HDR frames using Reinhard’s tone mapping operator and used the generated SDR frames as input for the six iTMOs. Finally, we evaluated the quality of the generated HDR frames with respect to the original HDR frame using HDR VDP-2.2 and PU-SSIM. Tables 3.15, 3.16, and 3.17 show the results of HDR VDP-2.2 for the six iTMOs for dark (Lamp, Typewriter, Peppermill), normal (Pier, Gorse, Rock), and bright (Waterfall, Snow, Arch) frames respectively. In addition, Tables 3.18, 3.19, and 3.20 show the results of PU-SSIM for the six iTMOs for dark, normal, and bright frames respectively. As it can be seen from the results of both metrics, on average, our proposed hybrid iTMO achieves the highest visual quality for all types of SDR frames, reaffirming the result of our subjective test. We also compared the execution time of our proposed iTMO against other five iTMOs. We calculated the execution times based on unoptimized source codes for each iTMO. In addition, the execution times is calculated for a 4K SDR frame using MATLAB R2018a software on a computer with Intel(R) Core(TM) i7-5820K CPU @ 3.3GHz. Table 3.21 shows the results of the                           (a) Waterfall              (b) Snow                (c) Arch                (d) Lamp         (e) Typewriter                                         (f) Peppermill             (g) Pier                (h) Gorse             (i) Rock Figure 3.19 Nine original tone mapped HDR frames randomly chosen from the dataset in [58].  Table 3.61 Visual quality scores by HDR VDP-2.2 for dark frames in Figure 3.19                          (a) Waterfall              (b) Snow                (c) Arch                (d) Lamp         (e) Typewriter                                         (f) Peppermill             (g) Pier                (h) Gorse             (i) Rock Figure 3.19 Nine original tone mapped HDR frames randomly chosen from the dataset in [58]. 67  execution times for each iTMO in seconds. As it can be seen our proposed iTMO ranks second in terms of execution time which is reasonable considering the higher visual quality it provides. Table 3.15 Visual quality scores by HDR VDP-2.2 for dark frames in Figure 3.19 for six different iTMOs. Frame Proposed Kim Kovaleski Meylan Bist Rempel Lamp 53.0473 46.74879 46.963256 47.519966 49.58417 52.69622 Typewriter 48.10044 43.2599 44.020972 44.469251 45.87392 46.75414 Peppermill 46.37453 41.48034 42.870002 42.481676 43.26849 43.60474 Average 49.17409 43.82968 44.618077 44.823631 46.24219 47.68503  Table 3.16 Visual quality scores by HDR VDP-2.2 for normal frames in Figure 3.19 for six different iTMOs. Frame Proposed Kim Kovaleski Meylan Bist Rempel Pier 50.72516 43.02445 43.178084 42.929074 45.11719 46.03405 Gorse 46.3769 43.82617 45.403682 45.463806 44.55143 46.03896 Rock 46.54312 44.60946 46.654773 46.733931 44.51708 46.93629 Average 47.88173 43.82003 45.078846 45.04227 44.72857 46.33643  Table 3.17 Visual quality scores by HDR VDP-2.2 for bright frames in Figure 3.19 for six different iTMOs. Frame Proposed Kim Kovaleski Meylan Bist Rempel Waterfall 47.95183 45.67576 47.920803 47.880207 45.50425 47.99985 Snow 51.72745 46.24283 47.253952 47.31836 48.25494 49.8245 Arch 47.32708 44.84433 47.432354 47.294075 44.06702 47.08561 Average 49.00212 45.58764 47.535703 47.497548 45.94207 48.30332   Table 3.62 Visual quality scores by HDR VDP-2.2 for dark frames in Figure 3.19 Frame Proposed Kim Kovaleski Meylan Bist Rempel Lamp 53.0473 46.74879 46.963256 47.519966 49.58417 52.69622 Typewriter 48.10044 43.2599 44.020972 44.469251 45.87392 46.75414 Peppermill 46.37453 41.48034 42.870002 42.481676 43.26849 43.60474 Average 49.17409 43.82968 44.618077 44.823631 46.24219 47.68503  Table 3.63 Visual quality scores by HDR VDP-2.2 for normal frames in Figure 3.19 Frame Proposed Kim Kovaleski Meylan Bist Rempel 68   Table 3.18 Visual quality scores by PU-SSIM for dark frames in Figure 3.19 for six different iTMOs.  Frame Proposed Kim Kovaleski Meylan Bist Rempel Lamp 0.995766 0.982229 0.9740865 0.9764124 0.984618 0.996626 Typewriter 0.990925 0.970268 0.9592009 0.9623684 0.97299 0.992212 Peppermill 0.982823 0.962577 0.9482766 0.9419287 0.961448 0.975384 Average 0.989838 0.971692 0.9605213 0.9602365 0.973019 0.988074  Table 3.19 Visual quality scores by PU-SSIM for normal frames in Figure 3.19 for six different iTMOs. Frame Proposed Kim Kovaleski Meylan Bist Rempel Pier 0.97675 0.965017 0.960169 0.9601589 0.965688 0.969436 Gorse 0.972222 0.963704 0.9598112 0.9609173 0.967228 0.965017 Rock 0.977566 0.966256 0.9611561 0.961659 0.968312 0.967247 Average 0.975513 0.964992 0.9603788 0.9609118 0.967076 0.967233  Table 3.20 Visual quality scores by PU-SSIM for bright frames in Figure 3.19 for six different iTMOs. Frame Proposed Kim Kovaleski Meylan Bist Rempel Waterfall 0.979287 0.97418 0.9712239 0.9719816 0.973783 0.97463 Snow 0.976956 0.967769 0.962969 0.9631881 0.966271 0.969094 Arch 0.983075 0.975587 0.9684621 0.9704864 0.973628 0.974917 Average 0.979772 0.972512 0.9675517 0.9685521 0.971227 0.97288    Table 3.78 Execution times of six iTMOs (in seconds) for a 4K SDR frameTable 3.79 Visual quality scores by PU-SSIM for dark frames in Figure 3.19 Frame Proposed Kim Kovaleski Meylan Bist Rempel Lamp 0.995766 0.982229 0.9740865 0.9764124 0.984618 0.996626 Typewriter 0.990925 0.970268 0.9592009 0.9623684 0.97299 0.992212 Peppermill 0.982823 0.962577 0.9482766 0.9419287 0.961448 0.975384 Average 0.989838 0.971692 0.9605213 0.9602365 0.973019 0.988074  Table 3.21 Execution times of six iTMOs (in seconds) for a 4K SDR frame  Proposed Kim Kovaleski Meylan Bist Rempel Execution Time (sec.) 2.3973 49.6554 1.0892 2.7858 2.9084 48.0186    Table 3.72 Visual quality scores by HDR VDP-2.2 for dark frames in Figure 3.19Table 3.73 Execution times of six iTMOs (in seconds) for a 4K SDR frame  Proposed Kim Kovaleski Meylan Bist Rempel Execution Time (sec.) 2.3973 49.6554 1.0892 2.7858 2.9084 48.0186 69  3.3 Conclusions  In this chapter, we proposed a hybrid iTMO capable of generating high visual quality HDR videos. Based on our subjective and objective evaluations, our iTMO achieves the best possible trade-off between the overall brightness and contrast of the generated HDR video and is capable of generating HDR videos for any dynamic range supported by the target HDR display. To address the issue of flickering caused by our proposed iTMO in Chapter 2, we proposed fixed boundaries for dark, normal, and bright regions of the SDR frame. To achieve high contrast HDR videos, we aimed at maximizing our proposed contrast difference function between input SDR and generated HDR frames. Although this approach yields high contrast HDR videos, in extreme cases it affects the overall brightness of the frame with dark SDR frames appearing overly bright in HDR domain and bright SDR frames overly dark. To overcome this issue, we proposed a more local optimization approach and aimed at maximizing the difference of individual PQ values in SDR and HDR domain, a process that we called maximizing the brightness difference function between input SDR and generated HDR frames. Although our brightness approach preserves the overall brightness of the HDR frame, the generated results are not of high contrast. As a result, to take advantage of the capabilities of both contrast and brightness approaches, we proposed our hybrid approach and aimed at maximizing a weighted sum of the two functions. Our subjective and objective evaluations demonstrate that our proposed hybrid iTMO outperforms other state-of-the-art methods in terms of overall visual quality of the generated HDR video. One disadvantage of this iTMO is that the segmentation method is not content adaptive, as the boundaries for our three brightness regions are fixed for all types of SDR frames. In the following chapter, we will address this issue and propose a fully content adaptive segmentation method capable of producing flicker-free high visual quality HDR videos. 70  Chapter 4: A Fully-Automatic Content Adaptive Inverse Tone Mapping Operator with Improved Color Accuracy The proposed iTMO in Chapter 3 is not content adaptive as it employs fixed boundaries for determining dark, normal, and bright regions of the SDR frame. Furthermore, the proposed segmentation method in Chapter 2, although content adaptive, is too sensitive to small variations in pixel distribution and results in flickering for video applications. Therefore, a content adaptive segmentation method that is less sensitive to small variations in pixel distribution not only avoids flickering in the generated HDR video, but can also improve the visual quality of it. In this Chapter, we propose a fully content adaptive iTMO that divides the input SDR frame to three brightness regions based on a novel entropy-based segmentation approach. Our proposed segmentation is based on the theory of maximum entropy thresholding [44] and accurately divides the input SDR frame to three brightness regions (i.e., dark, normal, and bright) to maintain the overall visual impression of the scene, while avoiding flickering. Furthermore, we propose a perception-based color adjustment method that is capable of preserving the color accuracy between the input SDR and the generated HDR frame. By performing the color adjustment in the perceptual domain, the proposed method preserves the hue of the SDR colors and generates HDR colors that closely follow their SDR counterparts, irrespective of the iTMO used. Based on our subjective and objective evaluations, our proposed iTMO outperforms other state-of-the-art operators in terms of overall visual quality of the generated HDR video and generates HDR colors that closely follow their SDR counterpart. 71  The remainder of this chapter is organized as follows: Section 4.1 discusses the details of our proposed iTMO. Section 4.2 presents the results of our experimental evaluations, and Section 4.3 concludes this chapter. 4.1 Proposed iTMO The block diagram of our proposed iTMO is presented in Figure 4.1. The main components of our proposed iTMO are presented in the red-dotted line. First, the input SDR frame is linearized using inverse gamma encoding [45]. Then, the SDR luminance channel is computed and these normalized [0-1] values are converted to light values in the range [0.1 – 100] nits, which are subsequently converted to perceptually linear values. Afterwards we compute the distribution (histogram) of these values. This step is followed by segmenting these values into three brightness regions (i.e., dark, normal, and bright) using our proposed entropy-based segmentation. Next, we compute the slopes of our mapping curve using our previously proposed hybrid approach of maximizing a weighted sum of brightness and contrast difference between input SDR and output HDR frame in Chapter 3. These slopes help us perform the inverse tone mapping, converting the  Figure 4.1 The block diagram of our proposed iTMO. The main components of our method are enclosed in the red dotted line. The generated HDR frame in this diagram has maximum brightness of 1000 nits. However, in general, our method is capable of producing HDR videos with arbitrary maximum brightness.   72  SDR PQ values to HDR domain. To have HDR colors that closely match their SDR counterparts we employ our proposed perception-based color adjustment method. Finally, the HDR perceptual values are converted back to light values, generating the HDR frame. The following sub-sections offer an in-depth description of each of these steps. 4.1.1  Converting SDR Light Values to PQ Domain and Computing Histogram As we mentioned before, our visual system does not perceive the brightness information of a scene in a linear way like cameras do [1]. In fact, they are more sensitive to changes in brightness in dark regions rather than normal and bright ones. To take advantage of this capability, we decided to implement our iTMO in perceptual domain. Doing so enables us to detect regions of the SDR frame that are visually important to our eyes and efficiently map those regions to higher brightness levels. To convert the SDR light values to perceptual domain we employ Perceptual Quantizer (PQ) transform [46], which is designed to optimize the distribution of light values with respect to HVS characteristics. Therefore, as a first step in our iTMO, we convert the input SDR light values to PQ domain. Afterwards, we compute the histogram of these SDR PQ values. Figure 4.2 (a) shows an example of an SDR frame along with the histogram of its luminance channel in PQ domain in Figure 4.2 (b). The SDR light values in this example range from [0.1 – 100] nits which translates to [0.0623 – 0.5081] in PQ domain. 4.1.2 Segmenting the SDR PQ Values to Three Brightness Regions As we discussed earlier in this chapter, our eyes are more sensitive to brightness changes in dark regions than they are to normal and bright ones. Therefore, it is important for our proposed iTMO to be able to detect each of these three brightness regions. To do so, we segment the SDR 73  PQ values to three brightness regions namely dark, normal, and bright. Our proposed segmentation method is based on the theory of maximum entropy thresholding [44]. As we mentioned in Chapter 2, entropy is known as a measure of the uncertainty of a random variable [49]. The lower the probability of occurrence of random variable, the more uncertain (higher entropy) we are about it happening and vice versa. Entropy of an image, E, is defined as follows [49]:  (a)  (b) Figure 4.2 (a) An SDR frame along with (b) the histogram of its luminance channel in PQ domain.    (a)  74  2log ( )i iiE p p                                                                                                                                                                 (4.1) where pi is the number of pixels in the i-th bin of the histogram of the frame over total number of pixels in the frame. In addition, H, and W denote height and width of the SDR frame respectively.  Figure 4.3 depicts the histogram of SDR luminance values in the PQ domain for the SDR frame shown in Figure 4.2 (a). As a first step in our segmentation method, we define Xmin, and Xmax as the minimum and maximum PQ values in the SDR domain respectively. In addition, we define Xgray as the middle gray point (defined as middle code-word 128) in PQ domain, according to Ansel Adam’s famous zone system [54]. The values for Xmin, Xgray, and Xmax are dependent on the maximum brightness of the SDR content and are fixed for all the frames throughout the SDR video. For instance, for the case of [0.1 – 100] nits SDR content, we have Xmin = 0.0623, Xgray = 0.3651 (PQ equivalent of middle code-word 128), and Xmax = 0.5081. The position of Xmin, Xgray, and Xmax are marked with vertical red-dotted lines in Figure 4.3. Our goal is to find the thresholds X1, and X2 (shown by vertical blue-dotted lines in Figure 4.3) that divide the SDR PQ values to three brightness regions. To make our proposed segmentation method less sensitive to changes in pixel distribution among frames and to avoid flickering, we adopt a cumulative approach and work with the summation of pixels until each bin when calculating the entropy. To do so, at each PQ point in the interval [Xmin – Xgray] we calculate two entropy values. First, we compute E1, which denotes how uncertain we are that a PQ point in this interval belongs to the dark region of the SDR frame, as follows: 1 2( ) ( )( ) log( ) ( )minXmin graytotal gray total grayi XS i S iE X    , X X XN X N X                                                                             (4.2) 75  where S(i) is the sum of pixels until the i-th PQ point in the histogram. Also, Ntotal(Xgray) is the total number of pixels starting from Xmin until Xgray.  The entropy E1 for the histogram in Figure 4.3 is shown in Figure 4.4 (a) in red. We observe that, since E1 denotes how uncertain we are that a PQ point belongs to dark region, its value increases as we go towards Xgray (normal region). We also calculate E2, which denotes how uncertain we are that a PQ point in the interval belongs to normal region of the SDR frame, as follows: 2 2( ) ( )( ) log( ) ( )grayXmin graytotal gray total grayi XS i S iE X    , X X XN X N X                                                                              (4.3)  The entropy E2 for the histogram in Figure 4.3 is shown in Figure 4.4 (a) in blue. In this case, as we move towards Xmin (dark region), E2 increases as it denotes the uncertainty of belonging to the normal region. To find X1 we compute the sum of E1 and E2. At each PQ point, E1+E2 denotes  Figure 4.3 An example of histogram of SDR PQ values. Our proposed segmentation method aims at finding X1, and X2 to divide the SDR PQ values to three brightness regions.    Figure 4.5 An example of histogram of SDR PQ values. Our proposed segmentation method aims at finding X1, and X2 to divide the SDR PQ values to three brightness regions. Xmin, Xmax, and Xgray are minimum, maximum, and middle gray points in PQ domain.    76  how uncertain we are that a PQ value belongs to the dark or normal region. Threshold X1 that defines the boundary between dark and normal region (the point where we are most uncertain that it belongs to dark or normal region) is the point where E1+E2 is maximum (see the green curve in Figure 4.4 (a)). We go through the same process for finding X2 in the [Xgray – Xmax] interval, which defines the boundary between the normal and bright region. Once more, at each PQ point in this interval,                                              (a)                                                                                      (b)                                                     (c)                                                                                      (d) Figure 4.4 Plots of E1, E2, and E1+E2. X1 is where E1+E2 is maximum. (b) Plots of E3, E4, and E3+E4. X2 is where E3+E4 is maximum. (c) Original SDR frame in Figure 4.2 (a). (d) Segmentation result using our proposed method. Black, gray, and white areas represent dark, normal, and bright regions in the SDR frame, respectively.                                               (a)                                                                                      (b)         77  we compute two entropy values. First, we compute E3 that determines how uncertain we are that a PQ value in the interval belongs to the normal region as follows: 3 2( ) ( )( ) log( ) ( )grayXgray maxtotal max total maxi XS i S iE X    , X X XN X N X                                                                            (4.4) where S(i) is the sum of pixels until the i-th PQ point in the histogram. Also, Ntotal(Xmax) is the total number of pixels starting from Xgray until Xmax.  The entropy E3 for the histogram in Figure 4.3 is shown in Figure 4.4 (b) in red. We observe that since E3 denotes how uncertain we are that a PQ point belongs to normal region, its value increases as we approach Xmax (bright region). In addition to E3, we calculate E4, which determines how uncertain we are that a PQ value in the interval belongs to the bright region as follows: 4 2( ) ( )( ) log( ) ( )maxXgray maxtotal max total maxi XS i S iE X    , X X XN X N X          .                                                                    (4.5) The entropy E4 for the histogram in Figure 4.3 is presented in Figure 4.4 (b) in blue. In this case, as we approach Xgray (normal region), E4 increases as it denotes the uncertainty of belonging to bright region. To find X2 we calculate E3+E4 which denotes how uncertain we are that a PQ point belongs to normal or bright region. The boundary between the normal and bright region, X2, is the point where E3+E4 is maximum (the point where we are most uncertain that it belongs to normal or bright region). Difference in amplitude of entropy between Figures 4.4 (a) and (b) is due to difference in pixels distribution between [Xmin – Xgray] and [Xgray – Xmax] intervals.  It is important to mention that in case the pixel distribution in the [Xmin – Xgray] interval is zero, we have X1 = Xgray. The same conclusion is true for the [Xgray – Xmax] interval meaning that in case the pixel distribution is zero in this interval, we have X2 = Xgray. 78   Figure 4.4 (c) shows the original SDR frame in Figure 4.2(a) while Figure 4.4 (d) depicts the result of our proposed segmentation method. Black, gray, and white areas in Figure 4.4 (d) denote the dark, normal, and bright regions in the frame, respectively. Figure 4.5 further demonstrates the performance of our proposed segmentation method. Figures 4.5 (a), (b), and (c) show a dark, normal, and bright SDR frames, respectively, while Figures 4.5 (d), (e), and (f) show the segmentation results, where black, gray, and white areas represent dark, normal, and bright regions, respectively. As it can be observed, our proposed segmentation accurately divides the SDR frame to three brightness regions. 4.1.3 Mapping SDR PQ Values to HDR Domain As we mentioned in Chapter 2 and 3, to choose the best possible mapping curve for our iTMO, we ran a series of extensive subjective tests and examined different types of mapping curves. We concluded that piece-wise linear curve offers the simplest implementation, and low \                                      (a)                                             (b)                                                 (c)                                       (d)                                             (e)                                                 (f) Figure 4.5 (a) - (c) An example of dark, normal and bright SDR frames, respectively. (d) - (f) represent the result of our proposed segmentation method for each frame, respectively.  \                                      (a)                                             (b)                                                 (c)                                       (d)                                             (e)                                                 (f) Figure 4.9 (a) - (c) An example of dark, normal and bright SDR frames, respectively. (d) - (f) represent the result of our proposed segmentation method for each frame, respectively.  \   79  computational cost, while at the same time producing high visual quality HDR videos.  Since we divide the SDR PQ values into three brightness regions, our mapping curve is a piece-wise linear curve with two breaking points, which can be written as: 1 , 1 , 1, 2 , 2 1 , 23 , 3 2 ,SDR PQ min SDR PQHDR PQ SDR PQ SDR PQSDR PQ SDR PQ maxs L a     if   X L XL s L a    if   X <L Xs L a    if   X <L X                                                                                                           (4.6) where LSDR,PQ, and LHDR,PQ are SDR and HDR PQ values respectively. s1, s2, and s3 are the slopes assigned to each of our three brightness regions. a1, a2, and a3 are the y-intercepts associated with each segment of the mapping curve. Xmin, and Xmax are the minimum and maximum PQ values in the SDR domain respectively. X1 is the boundary between dark and normal regions and X2 is the boundary between normal and bright regions. Figure 4.6(a) shows the SDR frame in Figure 4.2 (a) while Figure 4.6 (b) shows our mapping curve associated with this frame. This curve maps the SDR PQ values in the range [0.0623 – 0.5081] (or [0.1 – 100] nits) to [0.0215 – 0.7518] (or [0.01 – 1000] nits) in HDR domain. It is important to point out that since the brightness range of HDR displays may vary, our proposed iTMO is capable of dealing with HDR displays with arbitrary brightness ranges.      To find the slope of each segment of our mapping curve, we use the hybrid approach introduced in Chapter 3, where we maximized a weighted sum of the brightness and contrast difference functions between input SDR and generated HDR frame. Figures 4.7 (a), (b), and (c) represent a dark, normal, and bright SDR frames, respectively, while Figures 4.7 (d), (e), and (f) depict the generated HDR frames using our proposed iTMO. All HDR frames are tone mapped 80  using Reinhard et al.’s tone mapping operator [37] for visualization purposes. As it can be  (a)  (b) Figure 4.6 (a) An SDR frame. (b) Our mapping curve associated with it. X1, and X2 are the thresholds found using our segmentation method and Y1, and Y2 are their corresponding PQ values in HDR domain.    (a)         81  observed, our proposed iTMO achieves a good balance between overall brightness and contrast of the frame and preserves the overall impression of the input SDR content. 4.1.4 Proposed Color Adjustment Method An important consideration in designing an efficient iTMO is maintaining the color accuracy between the input SDR and the generated HDR frames. This is a challenge, because as we try to expand the brightness levels from SDR to HDR, we may cause color shifts [30], [31]. Thus, an efficient color adjustment scheme is of high importance when designing iTMOs. One common approach for color adjustment in SDR to HDR mapping is proposed by Schlick et al. [62]. This method performs the color adjustment in the light domain and the RGB color space, aiming at preserving the hue of the input SDR colors, as follows: sSDRHDR HDRSDRCC LL                                                                                                                                                               (4.7)                               (a)                                                        (b)                                                        (c)                               (d)                                                        (e)                                                        (f) Figure 4.7 Performance of our proposed iTMO for (a) dark, (b) normal, and (c) bright SDR frames. (d), (e), and (f) show the tone mapped HDR frames generated using our proposed iTMO for each SDR frame, respectively.                                 (a)                                                        (b)                                                        (c)                               (d)                                                        (e)                                                        (f) Figure 4.11 Performance of our proposed iTMO for (a) dark, (b) normal, and (c) bright SDR frames. (d), (e), and (f) show the tone mapped HDR frames generated using our proposed iTMO, respectively.      82  where CHDR denotes one of the color channels (red, green, or blue) of the generated HDR frame,  CSDR represents the same color channel in the input SDR frame, LHDR and LSDR are the HDR, and SDR luminance channels, respectively and s is a saturation factor which is greater than zero.  We wish to examine the performance of equation (4.7) in terms of its effect on hue, chroma, and lightness of colors. To this end, we first convert the RGB values to XYZ and then the IPT color space as it has a better hue uniformity than Lab color space [63]. As we need to examine the changes in hue, chroma, and lightness, we convert IPT to LCh [64] color space as it has all the three variables we aim to examine. Figure 4.8 (a) shows the changes in hue (h) with respect to Chroma (C) as we change the saturation factor s in equation (4.7) for six basic colors. As it can be seen, changing s results in changes in the hue of colors, especially for red and blue. Figure 4.8 (b) presents the changes in Lightness (L) with respect to Chroma (C) as we change saturation factor s                                          (a)                                                                                (b) Figure 4.8 Changes in hue (h) and Lightness (L) with respect to Chroma (C) for six basic colors as we change the saturation factor s from zero to two for Schlick et al.’s method. (a) and (b) demonstrate the changes in h and L with respect to C for six basic colors, respectively. The points where s = 1 are marked with black squares                                           (a)                                                                                (b) Figure 4.13 Changes in hue (h) and Lightness (L) with respect to Chroma (C) for six basic colors as we change the saturation factor s from zero to two for Schlick et al.’s method. (a) and (b) demonstrate the changes in h and L with respect to C for six basic colors, respectively. The points where s = 1 are marked with black squares   83  in equation (4.7). We observe that the overall brightness of colors first decreases and then starts to increase as s increases.   Another common approach for color adjustment in SDR to HDR mapping is proposed by Mantiuk et al. [30] where the color adjustment is performed in the light domain and the RGB color space with the main aim of limiting the resulting luminance changes, as follows: 1  +1SDRHDR HDRSDRCC s LL                                                                                                                                                   (4.8) where CHDR denotes one of the color channels (red, green, or blue) of the generated HDR frame, CSDR represents the same color channel in the input SDR frame, LHDR and LSDR are the HDR, and SDR luminance channels, respectively and s is a saturation factor which is greater than zero.  Figure 4.9 (a) shows the changes in hue (h) for six basic colors with respect to Chroma (C) as the saturation factor s in equation (4.8) changes. As it can be seen, changing s results in changes                                          (a)                                                                                (b) Figure 4.9 Changes in hue (h) and Lightness (L) with respect to Chroma (C) for six basic colors as we change the saturation factor s from zero to two for Mantiuk et al.’s method. (a) and (b) demonstrate the changes in h and L with respect to C for six basic colors, respectively. The points where s = 1 are marked with black squares.                                             (a)                                                                                (b) Figure 4.15 Changes in hue (h) and Lightness (L) with respect to Chroma (C) for six basic colors as we change the saturation factor s from zero to two for Mantiuk et al.’s method. (a) and (b) demonstrate the changes in h and L with respect to C for six basic colors, respectively. The points where s = 1 are marked wit  black squares.     84  in the hue of colors, especially for red and blue. Figure 4.9 (b) presents the changes in Lightness (L) with respect to Chroma (C) as we change saturation factor s in equation (4.8). As it can be observed, the overall brightness of colors has a near-constant behavior with s.  To address the issue of hue shift caused by Mantiuk et al.’s and Schlick et al.’s operators and achieve better color accuracy between input SDR and output HDR frames, we propose to perform the color adjustment in the PQ domain as follows: ,, ,,HDR PQHDR PQ SDR PQSDR PQLC s CL                                                                                                                                                (4.9) where CHDR,PQ denotes one of the color channels (red, green, or blue) of the generated HDR frame in the PQ domain, respectively. CSDR,PQ represents the same color channel in the SDR frame in the PQ domain. LHDR,PQ, and LSDR,PQ are the HDR and SDR luminance channels in the PQ domain, respectively and s is a saturation factor which is greater than zero.  Figure 4.10 (a) shows the changes in hue (h) with respect to Chroma (C) as we change the saturation factor s in equation (4.9) for six basic colors. We can see that changing s does not affect the hue of colors and it stays constant regardless of the changes in s. In addition, Figure 4.10 (b) shows linear changes in Lightness (L) with respect to Chroma (C) for different values of s, meaning that the colors will appear brighter as we increase the saturation factor. In Figure 4.10, points where s = 1 are marked with black squares. Comparing Figures 4.8 (a), 4.9 (a), and 4.10 (a) demonstrates that our proposed color adjustment method preserves the hue of colors irrespective of the changes in s as opposed to Schlick et al.’s and Mantiuk et al.’s methods. Inspection of Figures 4.8 (b), 4.9 (b), and 4.10 (b) shows that Mantiuk et al.’s method does a better job in limiting changes in color Lightness (L) 85  compared to our proposed and Schlick et al.’s methods, with our proposed method showing a linear behavior. Our next step is to identify a suitable value for the saturation factor s that yields the smallest change in the HDR luminance channel for our color adjustment method. To this end, we conducted an experiment where we randomly selected 400 SDR frames from [36] and generated their corresponding HDR frames using our proposed iTMO coupled with the proposed color adjustment method, using three different saturation factors: 1) s = 0.8, 2) s = 1, and 3) s = 1.2,  resulting in 400×3 = 1200 pairs of SDR and HDR frames. Our dataset included a broad range of color hues and saturation levels representing human faces, indoor, and outdoor scenes. Figure 4.11 shows a small subset of our dataset that includes Fall, Leaves, Fruits, Vase, Umbrella, Holi, Rose, Parrot, Pencils, and Dance. For each pair, we compute the absolute difference between the luminance channel of the generated HDR frame before and after applying the color adjustment method (ΔL).                                          (a)                                                                                (b) Figure 4.10 Changes in hue (h) and Lightness (L) with respect to Chroma (C) for six basic colors as we change the saturation factor s from zero to two for our proposed color adjustment method. (a) and (b) demonstrate the changes in h and L with respect to C for six basic colors, respectively. The points where s = 1 are marked with black squares.                                           (a)                                                                                (b) Figure 4.17 Changes in hue (h) and Lightness (L) with respect to Chroma (C) for six basic colors as we change the saturation fac or s from zero to two for our proposed color adjustm nt method. (a) and (b) demonstrate the changes in h and L with respect to C for six basic colors, respectively. The points where s = 1 are marked with black squares.   86  An efficient color adjustment method should not affect the overall HDR luminance (ΔL ≈ 0). Table 4.1 presents the mean, standard deviation (SD) and the 95% confidence interval (CI) for the average values of ΔL for each pair of SDR and HDR frames and for three different saturation factors. We observe from Table 4.1 that when s = 1, our proposed color adjustment yields on average the lowest change in ΔL. Figure 4.12 is a graphical representation of results in Table 4.1. Given the above findings, we would like to examine the overall performance of our proposed color adjustment method for the extreme luminance changes generated by inverse tone mapping. To this end, we compared the performance of our proposed method against the methods of Schlick et al.’s and Mantiuk et al.’s for Kim et al.’s [10] and our iTMO in this chapter. For this test, we used the same dataset as our pervious test (Figure 4.11), which includes 400 SDR frames randomly chosen from [36]. Similar to our previous test, for each pair we compute the absolute difference between the luminance channel of the generated HDR frame before and after applying the color adjustment methods (ΔL). Table 4.2 presents the mean, SD and the 95% CI for the average values of ΔL for each SDR and HDR pair using our proposed iTMO with three different color adjustment methods. Similarly, Table 4.3 presents the results for Kim et al.’s iTMO. Figures 4.13 and 4.14 are the graphical representation of the results in Tables 4.2 and 4.3, respectively. We                      (a) Fall              (b) Leaves             (c) Fruits             (d) Vase         (e) Umbrella         (f) Holi                                                (g) Rose          (h) Parrot          (i) Pencils            (j) Dance Figure 4.11 A subset of our 400 SDR frames dataset randomly taken from [49].                       (a) Fall              (b) Leaves             (c) Fruits             (d) Vase         (e) Umbrella         (f) Holi                                                (g) Rose          (h) Parrot          (i) Pencils            (j) Dance Figure 4.19 A subset of our 400 SDR frames dataset randomly taken from [47].  Table 4.1 Mean, SD, and 95% CI of the average ΔL for different values of the saturation facotr in our proposed color adjustment method                      (a) Fall              (b) Leaves             (c) Fruits             (d) Vase         (e) Umbrella         (f) Holi                                                (g) Rose          (h) Parrot          (i) Pencils            (j) Dance Figure 4.11 A subset of our 400 SDR frames dataset randomly taken from [49].        87  observe that our proposed method and Mantiuk et al.’s color adjustment method yield negligible changes in HDR luminance irrespective of the iTMO used. In summary, our proposed color adjustment method yields negligible changes in luminance while, unlike other methods, preserves hue. 4.2 Experimental Evaluations In sub-section 4.1.2, we claimed that our proposed entropy-based segmentation method prevents flickering due to its cumulative approach in calculating Entropy. Here, we evaluate our iTMO’s performance regarding flickering. We chose 120 consecutive frames of an SDR video (24 fps) sequence known to cause flickering for most iTMOs. This sequence is converted to HDR Table 4.1 Mean, SD, and 95% CI of the average ΔL for different values of the saturation factor in our proposed color adjustment method ΔL  s = 0.8 s = 1 s = 1.2 Mean 0.0889 0.0080 0.0761 SD 0.0363 0.0131 0.0351 95% CI [0.0853,0.0925] [0.0067,0.0093] [0.0727,0.0795]   Figure 4.12 Average of ΔL along with its 95% CI over our dataset of 400 SDR frames for three different values of s.     Table 4.2 Mean, SD, and 95% CI of the average ΔL for different values of the saturation parameter in our proposed color adjustment method ΔL  s = 0.8 s = 1 s = 1.2 Mean 0.0889 0.0080 0.0761 SD 0.0363 0.0131 0.0351 95% CI [0.0853,0.0925] [0.0067,0.0093] [0.0727,0.0795]   88  using our proposed iTMO once with the entropy-based segmentation method described in Chapter 2 and the second time with our entropy-based segmentation method presented in this chapter.  For both generated HDR sequences we calculated the absolute difference for Y1 and Y2 (see Figure 4.15) for all 120 frames in each sequence as follows: 1 1, 1, 1,   ,   1,2,...,119i iY i F FDiff Y Y i                                                                                       (4.10) 2 1, 2, 2,   ,   1,2,...,119i iY i F FDiff Y Y i                                                                                      (4.11) where 1YDiff  and 2YDiff  are the absolute difference for Y1 and Y2, respectively, and Fi denotes the i-th frame in the sequence. Table 4.2 Mean, SD, and 95% CI of the average ΔL using our proposed iTMO with three different color adjustment methods ΔL  Proposed Mantiuk Schlick Mean 0.00808 1.9×10-14 0.53034 SD 0.013134 2.1×10-14 0.77058 95% CI [0.006,0.009] [1.7×10-14, 2.1×10-14] [0.4548,0.6059]   Figure 4.13 Average of ΔL along with its 95% CI over our dataset of 400 SDR frames using our proposed iTMO for three different color adjustment methods.     Table 4.6 Mean, SD, and 95% CI of the average ΔL using our proposed iTMO with three different color adjustment methods ΔL  Proposed Mantiuk Schlick Mean 0.00808 1.9×10-14 0.53034 SD 0.013134 2.1×10-14 0.77058 95% CI [0.006,0.009] [1.7×10-14, 2.1×10-14] [0.4548,0.6059]   89    Figure 4.15 An example of our mapping curve. X1, and X2 are the boundaries for dark, and bright regions, respectively while Y1, and Y2 are their corresponding PQ values in HDR domain, respectively.   Table 4.3 Mean, SD, and 95% CI of the average ΔL using Kim et al.’s iTMO with three different color adjustment methods ΔL  Proposed Mantiuk Schlick Mean 0.0079 1.5×10-14 0.4261 SD 0.0130 1.6×10-14 0.6295 95% CI [0.0068,0.009] [1.3×10-14, 1.6×10-14] [0.364,0.487]   Figure 4.14 Average of ΔL along with its 95% CI over our dataset of 400 SDR frames using Kim et al.’s iTMO for three different color adjustment methods.     Table 4.10 Mean, SD, and 95% CI of the average ΔL using Kim et al.’s iTMO with three different color adjustment methods ΔL  Proposed Mantiuk Schlick Mean 0.0079 1.5×10-14 0.4261 SD 0.0130 1.6×10-14 0.6295 95% CI [0.0068,0.009] [1.3×10-14, 1.6×10-14] [0.364,0.487]   90  Flickering happens when a noticeable brightness variation occurs between two consecutive frames, meaning the brightness difference between the two consecutive frames exceeds the Just Noticeable Difference (JND) threshold [46]: JND   ,   1,2iYDiff i                                                                                                                                                 (4.12) Figures 4.16 (a) and (b) show 1YDiff  and 2YDiff  when we employ the entropy-based segmentation method in Chapter 2, respectively. Figures 4.16 (c) and (d) show 1YDiff   and 2YDiff   when we use the entropy-based segmentation method presented in this chapter, respectively. The horizontal red-dotted line in Figures 4.16 (a) – (d) denotes the JND value [46]. From Figures 4.16 (a) and (c) we observe that the absolute difference for Y1 does not exceed the JND value, meaning that our eyes will not notice these changes. However, Figure 4.16 (b) shows that the entropy-based                                                                (a)                                                (b)                                                                (c)                                                (d) Figure 4.16 (a) and (b) Absolute difference for Y1 and Y2 generated by our proposed iTMO using the segmentation method in Chapter 2, respectively, compared to (c) and (d) the proposed segmentation method in this Chapter.                                                                 (a)                                                (b)                                                                (c)                                                (d) Figure 4.30 (a) and (b) Absolute difference for Y1 and Y2 generated by our proposed iTMO using the segmentation method in Chapter 2, respectively, compared to (c) and (d) the proposed segmentation method in this Chapter.  91  segmentation in Chapter 2 causes flickering, as the absolute difference for Y2 passes the JND threshold. On the other hand, Figure 4.16 (d) demonstrates that our proposed iTMO coupled with the new entropy-based segmentation prevents flickering, as the changes fall well below the JND threshold. We evaluated the visual quality of the generated HDR videos using our proposed iTMO against Endo et al.’s, Luzardo et al.’s, Bist et al.’s, iTMO in Chapter 3, and Kim et al.’s iTMO through a subjective test. For all of the iTMOs, we used their default parameters as mentioned in their respective papers, except the maximum brightness that was set to 1000 nits. For the Endo et al.’s iTMO, we used the open source code available in [65]. We implemented Luzardo et al.’s, and Bist et al.’s iTMOs based on the description in their respective papers. Authors of [10] generously provided us with the implementation of their iTMO.  In this test, we asked our subjects to compare the HDR videos generated by our proposed iTMO against other methods and choose the one that they believe looks more visually pleasing to them. For this test, we used nine 4K SDR videos [59] (each ten seconds long) representing dark, normal, and bright scenes to evaluate the performance of our iTMO against others under different brightness conditions. Figure 4.17 depicts one frame of each of the nine 4K SDR videos used in                          (a) Desert                 (b) Mall                  (c) Boat                  (d) Paris               (e) Crowd                                   (f) Performance            (c) Wall                 (d) Bridge              (e) Brunch Figure 4.17 One frame of each of the nine 4K SDR videos used in our subjective test.                           (a) Desert                 (b) Mall                  (c) Boat                  (d) Paris               (e) Crowd                                   (f) Performance            (c) Wall                 (d) Bridge              (e) Brunch Figure 4.32 One frame of each of the nine 4K SDR videos used in our subjective test.  Table 4.13 Mean and SD of MOS scores for the six iTMOs based on the dataset in Figure 4.17                          (a) Desert                 (b) Mall                  (c) Boat                  (d) Paris               (e) Crowd                                   (f) Performance            (c) Wall                 (d) Bridge              (e) Brunch Figure 4.17 One frame of each of the nine 4K SDR videos used in our subjective test.                           (a) Desert                 (b) Mall                  (c) Boat                  (d) Paris               (e) Crowd 92  our subjective test, which include Desert, Mall, Boat, Paris, Crowd, Performance, Wall, Bridge, and Brunch. Eighteen subjects including 12 males, and 6 females participated in our test. Subjects were between twenty-two and thirty-three years old. All subjects were screened for color blindness and visual acuity using the Ishihara and Snellen charts, respectively. Prior to our test, we performed a training session to familiarize subjects with the test procedure. Two HDR videos were shown on two identical Sony BVM-X300 professional monitors. All different combinations of the test videos were randomly included in our subjective test resulting in 45 HDR videos that were evaluated by the subjects. During the test, subjects were asked to choose one of the two HDR videos (“A” or “B”) that looked more visually pleasing. We also provided the subjects with the “equal” choice in case the two HDR videos appeared to have the same level of visual quality. The position of HDR videos were randomly changed between the left and right displays to remove any effect caused by differences between the two displays and their physical location. The subjective test results were analyzed based on the methodology proposed in [53]. We used the outlier detection methodology in [53] and found two outliers and excluded their results from our analysis.  Figure 4.18 depicts the average of times subjects preferred the visual quality of the HDR videos generated by our proposed iTMO against Endo et al.’s, Luzardo et al.’s, Bist et al.’s, iTMO in Chapter 3, and Kim et al.’s operator. We observe that, on average, our iTMO outperforms Endo et al.’s by 85% ,Luzardo et al.’s by 94%, Bist et al.’s by 85%, iTMO in Chapter 3 by 67%, and Kim et al.’s by 77% of the times in terms of generating HDR videos that are more visually pleasing to the viewers. We further analyzed the MOS scores of the six iTMOs in this subjective based on F-value and p-value of one-way analysis of variance (ANOVA) to determine whether there any 93  statistical significance between any of the iTMOs. To do this, we performed the Levene’s test [61] to guarantee the homogeneity of variances of the gathered data and we concluded that one-way ANOVA can be applied. Table 4.4 presents the results of the one-way ANOVA test including the mean and standard deviation (SD) of the MOS scores for each approach. The ANOVA test  Figure 4.18 Preference scores of the subjects in mean opinion score (MOS) when comparing the visual quality of the HDR videos generated by our iTMO over that of Endo et al.’s, Luzardo et al.’s, Bist et al.’s, iTMO in Chapter 3, and Kim et al.’s iTMOs.   Table 4.17 Mean and SD of MOS scores for the six iTMOs based on the dataset in Figure 4. Figure 4.18 Preference scores of the subjects in mean opinion score (MOS) when comparing the visual quality of the output HDR video generated by our iTMO over that of Endo et al.’s, Luzardo et al.’s, Bist et al.’s, iTMO in Chapter 3, and Kim et al.’s iTMOs.    Table 4.4 Mean and SD of MOS scores for the six iTMOs based on the dataset in Figure 4.17 iTMO Mean SD Endo et al.’s 1.1875 1.1672 Luzardo et al.’s 0.5 0.7302 Bist et al.’s 1.25 1 iTMO in Chapter 3 2.4375 1.5478 Kim et al.’s 1.875 1.2041 Proposed 35.0625 3.2139 F-value (p-value) 6.8270 (1.9290×10-5)   Table 4.14 Mean and SD of MOS scores for the six iTMOs based on the dataset in Figure 4.17 iTMO Mean SD Endo et al.’s 1.1875 1.1672 Luzardo et al.’s 0.5 0.7302 Bist et al.’s 1.25 1 iTMO in Chapter 3 2.4375 1.5478 Kim et al.’s 1.875 1.2041 Proposed 35.0625 3.2139 94  concluded that the null hypothesis (i.e., there are no statistical significance between the six iTMOs in this subjective test) probability (p-value) for the dataset in this subjective test is 1.92×10-5. Since the p-value is less than the significance level α=0.05 we reject the null hypothesis and conclude that the difference in MOS values obtained by the six iTMOs in this subjective test is statistically significant.  To evaluate the MOS-based performance difference between any two of the six iTMOs, we performed Tukey’s post-hoc analysis for their MOS scores. Figure 4.19 shows the results of this test when α=0.05. Methods in the same group are the ones with statistically insignificant difference. As it can be seen, Endo et al.’s, Luzardo et al.’s, Bist et al.’s, iTMO in Chapter 3, and Kim et al.’s fall within the same group while our proposed iTMO falls in a separate one, meaning our iTMO yield statistically distinct HDR videos in terms of subjective quality assessment. We also compared the performance of our proposed iTMO against Endo et al.’s, Luzardo et al.’s, Bist et al.’s, iTMO in Chapter 3, and Kim et al.’s iTMOs using HDR VDP-2.2 and PU-SSIM objective quality metrics. To this end, we used nine original HDR frames randomly chosen from the dataset in [58] representing dark, normal, and bright scenes. Figure 4.20 shows the tone mapped version of the nine HDR frames which include View, Arch, Sunlight, Desk, Frontier, Building, Office, Hut, and Stadium. As can be seen, our dataset includes dark, normal, and bright scenes to evaluate the performance of our iTMO against others under different brightness conditions. We tone mapped these HDR frames using Reinhard et al.’s tone mapping operator [37]  Figure 4.19 Post-hoc analysis using Tukey’s method for mean values of MOS scores and α=0.05. Methods within the same group generate statistically insignificant HDR output.    Figure 4.19 Post-hoc analysis using Tukey’s method for mean values of MOS scores and α=0.05. Methods within the same group generate statistically insignificant HDR output.    Figure 4.19 Post-hoc analysis using Tukey’s method for mean values of MOS scores and α=0.05. Methods within the same group generate statistically insignificant HDR output.    Figure 4.19 Post-hoc analysis using Tukey’s method for mean values of MOS scores and α=0.05. Methods within the same group generate statistically insignificant HDR output.  95  and used the generated SDR frames as input for the six iTMOs. Finally, we evaluated the quality of the generated HDR frames with respect to the original HDR frame using HDR VDP-2.2 and PU-SSIM. Tables 4.5, 4.6, and 4.7 show the results of HDR VDP-2.2 for the six iTMOs for dark (Desk, Frontier, Building), normal (Office, Hut, Stadium), and bright (View, Arch, Sunlight) frames, respectively. In addition, Tables 4.8, 4.9, and 4.10 show the results of PU-SSIM for the six iTMOs for dark, normal, and bright frames, respectively. As it can be seen from the results of both metrics, on average, our proposed iTMO achieves the highest visual quality for all types of SDR frames, reaffirming the result of our subjective test. We also evaluated the performance of our proposed iTMO against Endo et al.’s [9], Luzardo et al.’s [8], Bist et al.’s [7], the iTMO in Chapter 3, and Kim et al.’s [10] iTMO in terms of maintaining the color accuracy between the input SDR and generated HDR frames by means of another subjective test. For all these iTMOs, we used their default parameters as mentioned in their respective papers, except the maximum brightness that was set to 1000 nits. For the Endo et al.’s iTMO, we used the open source code available in [65]. We implemented Luzardo et al.’s, and Bist et al.’s iTMOs based on the description in their respective papers. Authors of [10] generously                                  (a) View               (b) Arch            (c) Sunlight            (d) Desk             (e) Frontier                                              (f) Building           (g) Office              (h) Hut            (i) Stadium Figure 4.20 Nine original tone mapped HDR frames randomly chosen from the dataset in [58].    Table 4.19 Visual quality scores by HDR VDP-2.2 for dark frames in Figure 4.                                 (a) View               (b) Arch            (c) Sunlight            (d) Desk             (e) Frontier                                              (f) Building           (g) Office              (h) Hut            (i) Stadium Figure 4.20 Nine original tone mapped HDR frames randomly chosen from the dataset in [].    Table 4.20 Visual quality scores by HDR VDP-2.2 for dark frames in Figure 4.20                                 (a) View               (b) Arch            (c) Sunlight            (d) Desk             (e) Frontier                                              (f) Building           (g) Office              (h) Hut            (i) Stadium Figure 4.20 Nine original tone mapped HDR frames randomly chosen from the dataset in [58]. 96  provided us with the implementation of their iTMO. Our proposed iTMO uses the proposed color adjustment method in this chapter, while Luzardo et al.’s, Bist et al.’s, Endo et al.’s, and the iTMO in Chapter 3 all use Mantiuk et al.’s color adjustment method. In addition, Kim et al.’s iTMO uses Schlick et al.’s color adjustment method. Table 4.5 Visual quality scores by HDR VDP-2.2 for dark frames in Figure 4.20 for six different iTMOs Frame Proposed iTMO in Chapter 3 Kim Endo Bist Luzardo Desk 52.2767 49.58417 46.748792 45.866851 49.49307 52.69622 Frontier 47.67912 45.002 42.651174 42.721272 44.87382 45.7905 Building 44.78351 41.23503 39.487692 37.408875 41.15466 44.5629 Average 48.24644 45.27373 42.962552 41.999 45.17385 47.68321  Table 4.6 Visual quality scores by HDR VDP-2.2 for normal frames in Figure 4.20 for six different iTMOs Frame Proposed iTMO in Chapter 3 Kim Endo Bist Luzardo Office 47.03192 43.7328 41.546788 39.761681 43.33167 43.41994 Hut 47.87282 45.49702 43.632633 44.234949 43.51107 45.96083 Stadium 52.49312 49.69019 47.158808 47.541001 47.79539 50.22695 Average 49.13262 46.30667 44.112743 43.845877 44.87938 46.53591  Table 4.7 Visual quality scores by HDR VDP-2.2 for bright frames in Figure 4.20 for six different iTMOs Frame Proposed iTMO in Chapter 3 Kim Endo Bist Luzardo Lake 47.52308 45.5694 43.640389 45.751585 43.74832 43.93695 Arch 47.88094 47.32708 44.844326 46.533 44.06702 51.2192 Sunlight 47.88069 44.45322 41.976296 44.6211 43.90446 43.03713 Average 47.76157 45.7832 43.487004 45.6352 43.9066 46.06443   Table 4.22 Visual quality scores by PU-SSIM for dark frames in Figure 4.Table 4.23 Visual quality scores by HDR VDP-2.2 for dark frames in Figure 4.20 Frame Proposed iTMO in Chapter 3 Kim Endo Bist Luzardo Desk 52.2767 49.58417 46.748792 45.866851 49.49307 52.69622 Frontier 47.67912 45.002 42.651174 42.721272 44.87382 45.7905 Building 44.78351 41.23503 39.487692 37.408875 41.15466 44.5629 Average 48.24644 45.27373 42.962552 41.999 45.17385 47.68321  Table 4.24 Visual quality scores by HDR VDP-2.2 for normal frames in Figure 4.20 97  In this test, we asked our subjects to compare the HDR frames generated by our proposed iTMO against other state-of-the-art methods and choose the one that they think is closer to the original SDR frame in terms of its colors. For this test, we used ten SDR frames randomly chosen from [36] covering a wide range of color hues and saturation levels representing human faces, indoor, and outdoor scenes. Figure 4.21 shows the dataset used in this subjective test which Table 4.8 Visual quality scores by PU-SSIM for dark frames in Figure 4.20 for six different iTMOs Frame Proposed iTMO in Chapter 3 Kim Endo Bist Luzardo Desk 0.996626 0.995117 0.9822295 0.9355952 0.984618 0.984865 Frontier 0.986346 0.983639 0.9664061 0.9268817 0.966728 0.967879 Building 0.983923 0.97933 0.9582654 0.918452 0.959809 0.960335 Average 0.988965 0.986028 0.968967 0.9269763 0.970385 0.971027  Table 4.9 Visual quality scores by PU-SSIM for normal frames in Figure 4.20 for six different iTMOs  Frame Proposed iTMO in Chapter 3 Kim Endo Bist Luzardo Office 0.976591 0.960288 0.95844 0.9385459 0.9595202 0.964176 Hut 0.984207 0.969201 0.967841 0.9462454 0.9702934 0.975921 Stadium 0.993044 0.985961 0.985215 0.9639887 0.9851512 0.990104 Average 0.984614 0.971816 0.970499 0.9495933 0.9716549 0.976734  Table 4.10 Visual quality scores by PU-SSIM for bright frames in Figure 4.20 for six different iTMOs Frame Proposed iTMO in Chapter 3 Kim Endo Bist Luzardo Lake 0.980239 0.970378 0.972178 0.958125 0.9713273 0.971805 Arch 0.982488 0.983075 0.974917 0.9612105 0.975587 0.975587 Sunlight 0.986377 0.971541 0.973628 0.9465461 0.9728668 0.980015 Average 0.983035 0.974998 0.973574 0.9552939 0.9732604 0.975802   Table 4.34 Visual quality scores by PU-SSIM for dark frames in Figure 4.20 Frame Proposed iTMO in Chapter 3 Kim Endo Bist Luzardo Desk 0.996626 0.995117 0.9822295 0.9355952 0.984618 0.984865 Frontier 0.986346 0.983639 0.9664061 0.9268817 0.966728 0.967879 Building 0.983923 0.97933 0.9582654 0.918452 0.959809 0.960335 Average 0.988965 0.986028 0.968967 0.9269763 0.970385 0.971027  Table 4.35 Visual quality scores by PU-SSIM for normal frames in Figure 4.20 Frame Proposed iTMO in Chapter 3 Kim Endo Bist Luzardo 98  includes Powder, Autumn, Mix, Pepper, Lake, Legos, Laughter, Bird, Park, and Jenga. Eighteen subjects (4 females and 14 males) between the ages of twenty-three and thirty-four participated in our test. The subjects who participated in this test were different than the ones in our previous subjective test. All subjects were screened for color blindness and visual acuity using the Ishihara and Snellen charts, respectively. Prior to the test, subject were familiarized with the test procedure through a training session. The original SDR frames were shown on a Sony BVM-X300 professional monitor [60], while the HDR frames were shown on another identical Sony BVM-X300 professional monitor in a side-by-side manner, using a 14-pixel wide black stipe between them. All different combinations of the test frame pairs were included randomly in the experiment, resulting in 50 overall frames that were evaluated by the subjects. Each SDR-HDR pair was shown for 10 seconds on each display. During the test, subjects were asked to choose one of the two HDR frames (“A” or “B”) that they think its colors more closely represent the ones in the original SDR frame. Subjects were also provided with the “equal” choice in case the colors in both HDR frames appeared equally close to the ones in the original SDR frame.                  (a) Powder                    (b) Autumn                     (c) Mix                   (d) Pepper                 (e) Lake                   (f) Legos               (g) Laughter                  (h) Bird                      (i) Park                        (j) Jenga Figure 4.21 Ten SDR frames randomly chosen from [49] for our subjective test.                             (a) Powder                    (b) Autumn                     (c) Mix                   (d) Pepper                 (e) Lake                   (f) Legos               (g) Laughter                  (h) Bird                      (i) Park                        (j) Jenga Figure 4.21 Ten SDR frames randomly chosen from [47] for our subjective test.                             (a) Powder                    (b) Autumn                     (c) Mix                   (d) Pepper                 (e) Lake      99  The position of HDR frames were randomly switched between the left and right side of the display to remove any effect caused by any variations of brightness across the display. Analysis of the results followed the process described in [53]. We used the outlier detection methodology in [53] and found two outliers and excluded their results from our analysis.  Figure 4.22 shows the percentage of how many times, on average, subjects decided that the colors generated by our proposed iTMO more closely represent the ones in the original SDR frame over that of Endo et al.’s, Luzardo et al.’s, Bist et al.’s, the iTMO in Chapter 3, and Kim et al.’s iTMOs. As it can be observed, on average, our proposed iTMO outperforms Endo et al.’s by 79%, Luzardo et al.’s by 68%, Bist et al.’s by 96%, the iTMO in Chapter 3 by 72% and Kim et al.’s by 65% of the times in terms of how closely the colors of the output HDR frame represent the ones in the original SDR frame. We further analyzed the MOS scores of the six iTMOs in this subjective based on F-value and p-value of one-way analysis of variance (ANOVA) to determine  Figure 4.22 Preference scores of the subjects in mean opinion score (MOS) when comparing the color accuracy of HDR frame generated by our proposed iTMO over that of Endo et al.’s, Luzardo et al.’s, Bist et al.’s, iTMO in Chapter 3, and Kim et al.’s operator.    Figure 4.22 Preference scores of the subjects in mean opinion score (MOS) when comparing the color accuracy of output HDR frame generated by our proposed iTMO over that of Endo et al.’s, Luzardo et al.’s, Bist et al.’s, iTMO in Chapter 3, and Kim et al.’s operators.    100  whether there any statistical significance between any of the iTMOs. To do this, we performed the Levene’s test [61] to guarantee the homogeneity of variances of the gathered data and we concluded that one-way ANOVA can be applied. Table 4.11 presents the results of the one-way ANOVA test including the mean and SD of the MOS scores for each approach. The ANOVA test concluded that the null hypothesis (i.e., there is no statistical significance between the six iTMOs in this subjective test) probability (p-value) for the dataset in this subjective test is 4.184×10-6. Since the p-value is less than the significance level α=0.05 we reject the null hypothesis and conclude that the difference in MOS values obtained by the six iTMOs in this subjective test is statistically significant.  To evaluate the MOS-based performance difference between any two of the six iTMOs, we performed Tukey’s post-hoc analysis for their MOS scores. Figure 4.23 shows the results of Table 4.11 Mean and SD of MOS scores for the six iTMOs based on the dataset in Figure 4.21 iTMO Mean SD Endo et al.’s 1.9375 0.9287 Luzardo et al.’s 3 0.9660 Bist et al.’s 0.3125 0.6020 iTMO in Chapter 3 2.5625 1.3149 Kim et al.’s 3.3125 0.9464 Proposed 36.625 3.7925 F-value (p-value) 7.7570 (4.1840×10-6)   Table 4.44 Execution times of six iTMOs (in seconds) for a 4K SDR frameTable 4.45 Mean and SD of MOS scores for the six iTMOs based on the dataset in Figure 4.21 iTMO Mean SD Endo et al.’s 1.9375 0.9287 Luzard  et al.’s 3 0.9660 Bist et al.’s 0.3125 0.6020 iTMO in Chapter 3 2.5625 1.3149 Kim et al.’s 3.3125 0.9464 Proposed 36.625 3.7925 F-value (p-value) 7.7570 (4.1840×10-6)  Figure 4.23 Post-hoc analysis using Tukey’s method for mean values of MOS scores and α=0.05. Methods within the same group generate statistically insignificant HDR output.    Figure 4.23 Post-hoc analysis using Tukey’s method for mean values of MOS scores a d α=0.001. Methods within the same group generate statistically insignificant HDR output.   Table 4.43 Mean and SD of MOS scores for the six iTMOs based on the dataset in Figure 4.21  Figure 4.23 Post-hoc analysis using Tukey’s method for mean values of MOS scores and α=0.05. Methods within the same group generate statistically insignificant HDR output.   101  this test when α=0.05. Methods in the same group are the ones with statistically insignificant difference. As it can be seen, Endo et al.’s, Luzardo et al.’s, Bist et al.’s, the iTMO in Chapter 3, and Kim et al.’s iTMOs fall within the same group while our proposed iTMO falls in a separate one, meaning that it yields statistically distinct HDR colors in terms of subjective quality assessment. 4.3 Conclusions In this Chapter, we proposed a new content adaptive iTMO designed for video applications that can effectively deal with all types of SDR content from dark, to normal and bright scenes. Our iTMO works in the perceptual domain, allowing mapping to take advantage of the human eye’s sensitivity to brightness changes in different regions of the scene. A segmentation method that is based on maximum entropy thresholding is used to effectively segment the frame into dark, normal, and bright regions in order to better control the overall impression of each region and maintain the artistic intent, while also preventing flickering. The challenge of color shifting in inverse tone mapping is addressed by the introduction of a perception-based color adjustment method that preserves the hue of colors with insignificant changes in brightness, producing HDR colors that are faithful to their SDR counterparts. Subjective and objective evaluations showed that our proposed iTMO outperforms other state-of-the-art operators in terms of overall visual quality of the generated HDR video, while producing HDR colors that are faithful to their SDR counterparts. 102  Chapter 5: Conclusions and Future Work 5.1 Summary of the Contributions In this thesis, we proposed novel iTMOs capable of generating high visual quality HDR content. We also proposed a novel perception-based color adjustment method that prevents color shifts in SDR to HDR mapping and is capable of generating HDR colors that are faithful to the original SDR counterparts. Our subjective and objective evaluations demonstrate that our proposed iTMOs outperform other state-of-the-art methods in terms of overall visual quality and maintaining the color accuracy between input SDR and generated HDR content.  In Chapter 2, we proposed an entropy-based iTMO that can deal with all types of SDR content, generating high visual quality HDR frames. Working in perceptual domain allowed us to model the sensitivity of our eyes to brightness changes in different regions of a scene. To keep the dark areas dark and bright areas bright without darkening and brightening up the frame, we proposed an entropy-based segmentation method capable of dividing the SDR frame into dark, normal, and bright regions. Then, we employed a novel piece-wise linear mapping curve with two breaking points to map the three brightness regions to the HDR domain. The slopes for each segment of the mapping curve were chosen based on comprehensive subjective tests. Our subjective evaluations demonstrated that our proposed iTMO outperformed other state-of-the-art methods in terms of generating high visual quality HDR images. Despite its good performance, this proposed iTMO is only capable of generating 4000 nits HDR frames and is not completely content adaptive as the slopes of the mapping curve were fixed for all types of SDR content. Another limitation is that the application of this iTMO is limited to SDR images, as the proposed segmentation method will cause flickering in video applications due to its high sensitivity to small changes in pixels distribution. 103  In Chapter 3, we proposed a perceptually optimized iTMO that addressed the shortcomings of the method in Chapter 2 while generating high visual quality HDR videos for any arbitrary dynamic range supported by the target HDR display. To address the high sensitivity of the segmentation method proposed in Chapter 2 to small variations in pixel distribution, we proposed fixed boundaries for the dark, normal, and bright regions of the SDR frame. Similar to Chapter 2, a piece-wise linear mapping curve with two breaking points was used to convert SDR video to HDR format. However, to overcome the issue of having fixed slopes for all types of SDR content, we calculated the slope of each segment of the mapping curve using a hybrid approach that optimizes the trade-off between contrast and brightness in the generated HDR frame. Our subjective and objective evaluations demonstrated that this proposed iTMO outperforms other state-of-the-art iTMOs in terms of generating high visual quality HDR videos. Despite that, the proposed iTMO is not fully content adaptive, as it uses fixed boundaries for detecting the dark, normal, and bright regions of an SDR frame. In Chapter 4, we proposed a fully content adaptive iTMO that addressed the shortcomings of the methods in Chapters 2, and 3 while generating high visual quality HDR videos for any dynamic range supported by the target HDR display. To overcome the issue of non-content adaptive limitation of the iTMO in Chapter 3 that results from using fixed boundaries for detecting dark, normal, and bright regions, we proposed a novel entropy-based segmentation method. We also demonstrated that unlike the segmentation method in Chapter 2, this cumulative segmentation is less sensitive to small changes in pixels distribution and prevents flickering. Similar to Chapters 2 and 3, a piece-wise linear mapping curve with two breaking points was used to convert SDR video to HDR format. The slope of each segment of the mapping curve is optimized by finding the best trade-off between contrast and brightness throughout the entire dynamic range. We also 104  addressed the issue of color shifts in SDR to HDR mapping by introducing a perception-based color adjustment method that preserves the color accuracy between input SDR and generated HDR frames that maintain the hue levels while yielding negligible changes in luminance. Our subjective and objective evaluations demonstrated that our proposed iTMO outperforms other state-of-the-art methods in terms of generating high visual quality HDR videos while also producing HDR colors that closely follow their SDR counterparts. 5.2 Significance and Potential Applications of the Research While HDR video technology has a disruptive impact on the capturing and display industries, it also opens new opportunities for SDR content owners as well as broadcasters that will continue producing real-time events in SDR format for the foreseeable future. For the latter case, SDR content will have to be efficiently converted to HDR format, taking advantage of the advanced visual quality of HDR displays to offer viewers a much better quality of experience than what the original SDR content would ever do. In fact, iTMOs have widespread applications including but not limited to TV manufacturing, broadcasting, video gaming, security, HDR cinema and post-production. The proposed iTMO is an ideal solution, as it is capable of generating high visual quality HDR videos, surpassing in performance every other existing approach. It is worth noting that the work presented in this thesis has been patented and is already being employed by TELUS Communications Inc. to convert their existing SDR content to HDR format. In the case of video gaming, there has been a growing interest in the market towards HDR streaming. Recently, gaming consoles such as Microsoft’s Xbox One S [66], and Sony’s PS4 Pro [67] support HDR streaming. However, although there exist several HDR monitors such as Samsung KS9800 [68], LG C8 [69], and Sony X800E [70], not all video games are or will be in HDR format. Hence, the proposed iTMOs can be implemented in the game engines or the HDR 105  monitors to convert existing SDR gaming content to HDR format, offering a higher visual quality and wider range of color and brightness.  In the case of security and surveillance, there has been a growing demand for HDR support as it will drastically improve contrast, improve details in the dark regions and even prevent images from being washed out by harsh sunlight. Therefore, our proposed iTMOs in this thesis can be implemented into the security camera hardware to offer higher visual quality, and visible details in dark and bright regions of the scene.   Another application for the research presented in this thesis is HDR cinema. HDR technology has not yet reached the cinema industry, and movies are still in the SDR format with the maximum brightness of 48 nits [71]. SDR cinema content quality has been widely perceived as better than SDR Home image quality, yet HDR cinema has not yet been established while HDR home content is becoming widely available [71]. Our proposed method can be implemented for converting legacy cinema SDR content to HDR format for theater levels of HDR, providing cinemagoers with life-like viewing experience.  Another important application of the research presented in this thesis is post-production. To enable a more pleasing and immersive viewing experience in the home and cinema applications a range of post-production operations are adopted by content owners. In post-production, color grading will happen manually on HDR display with fixed capabilities. The process of color grading varies depending on the application. For example, color grading for cinema and home viewing requires a separate process since the viewing environment in cinema is dark, while home environment is usually dim. In addition to the viewing environment, capabilities of home displays vary widely. This means that the combination of target display and target viewing environment is unlikely to match the grading environment produced by post-production studios. The manual 106  process is extremely time consuming. Tests with many studios have shown that our method outperforms manual approaches and TELUS has already replaced such internal process with our iTMO solution. 5.3 Future Work Future work may include extension of the final proposed iTMO described in Chapter 4 to Light Field (LF) technology and more specifically conversion of SDR LF content to HDR format in order to increase its overall visual quality. Although LF technology allows capturing richer visual information than traditional cameras by capturing light intensity and its direction [72], existing LF capturing technologies are limited to SDR format. Our proposed iTMO can help adjust the overall brightness and contrast along with color information of the light field content for a number of applications such as entertainment, autonomous driving, robotics, human monitoring and endoscopy.  Another possible topic for further research is to develop a fully-automatic content adaptive Tone Mapping Operator (TMO) based on the principals we used in designing our iTMO. Tone Mapping is the process of converting the HDR content to SDR format. To preserve the artistic intent of the original HDR content we can divide it into three brightness regions using our proposed entropy-based segmentation in Chapter 4. Furthermore, to map each of the three HDR region we can find the slopes based on minimizing our proposed contrast and brightness difference functions in Chapter 3. We expect that the values of optimization weights w1 and w2 need to be adjusted through objective assessment for Tone Mapping applications.107  Bibliography [1] R. Boitard, M. T. Pourazad, P. Nasiopoulos, and J. Slevinsky, “Demystifying High-Dynamic-Range Technology: A new evolution in digital media.,” IEEE Consum. Electron. Mag., vol. 4, no. 4, pp. 72–86, 2015. [2] A. O. Akyüz, R. Fleming, B. E. Riecke, E. Reinhard, and H. H. Bülthoff, “Do HDR displays support LDR content?,” ACM Trans. Graph., vol. 26, no. 3, p. 38, 2007. [3] L. Meylan, S. Daly, and S. Süsstrunk, “The reproduction of specular highlights on high dynamic range displays,” in Final Program and Proceedings - IS and T/SID Color Imaging Conference, 2006, vol. 2006, pp. 333–338. [4] R. P. Kovaleski and M. M. Oliveira, “High-quality brightness enhancement functions for real-time reverse tone mapping,” Vis. Comput., vol. 25, no. 5–7, pp. 539–547, 2009. [5] R. P. Kovaleski and M. M. Oliveira, “High-quality reverse tone mapping for a wide range of exposures,” in Brazilian Symposium of Computer Graphic and Image Processing, 2014, pp. 49–56. [6] C. Bist, R. Cozot, G. Madec, and X. Ducloux, “Style aware tone expansion for HDR displays,” in Proceedings - Graphics Interface, 2016, pp. 57–63. [7] C. Bist, R. Cozot, G. Madec, and X. Ducloux, “Tone expansion using lighting style aesthetics,” Comput. Graph., vol. 62, pp. 77–86, 2017. [8] G. Luzardo, J. Aelterman, H. Luong, W. Philips, D. Ochoa, and S. Rousseaux, “Fully-Automatic Inverse Tone Mapping Preserving the Content Creator’s Artistic Intentions,” in 2018 Picture Coding Symposium, PCS 2018 - Proceedings, 2018, pp. 199–203. [9] Y. Endo, Y. Kanamori, and J. Mitani, “Deep reverse tone mapping,” in ACM Transactions on Graphics, 2017, vol. 36, no. 6. 108  [10] D. E. Kim and M. Kim, “Learning-based Low-Complexity Reverse Tone Mapping with Linear Mapping,” IEEE Transactions on Circuits and Systems for Video Technology, 2019. [11] J. Zhang and J. F. Lalonde, “Learning High Dynamic Range from Outdoor Panoramas,” in Proceedings of the IEEE International Conference on Computer Vision, 2017, vol. 2017-Octob, pp. 4529–4538. [12] G. Eilertsen, J. Kronander, G. Denes, R. K. Mantiuk, and J. Unger, “HDR image reconstruction from a single exposure using deep CNNs,” in ACM Transactions on Graphics, 2017, vol. 36, no. 6. [13] D. Marnerides, T. Bashford-Rogers, J. Hatchett, and K. Debattista, “ExpandNet: A deep convolutional neural network for high dynamic range expansion from low dynamic range content,” Comput. Graph. Forum, vol. 37, no. 2, pp. 37–49, 2018. [14] L. Meylan, S. Daly, and S. Süsstrunk, “Tone mapping for high dynamic range displays,” in Human Vision and Electronic Imaging XII, 2007, vol. 6492. [15] S. Lee, G. H. An, and S. J. Kang, “Deep chain HDRI: Reconstructing a high dynamic range image from a single low dynamic range image,” IEEE Access, vol. 6, pp. 49913–49924, 2018. [16] Y. Kinoshita and H. Kiya, “ITM-Net: Deep Inverse Tone Mapping Using Novel Loss Function Considering Tone Mapping Operator,” IEEE Access, vol. 7, pp. 73555–73563, 2019. [17] P. Mohammadi, M. T. Pourazad, and P. Nasiopoulos, “A high contrast video inverse tone mapping operator for high dynamic range applications,” in Proceedings - IEEE International Symposium on Circuits and Systems, 2019, vol. 2019-May. [18] P. Mohammadi, M. A. Hashemi, and M. T. Pourazad, “An Entropy-Based Inverse Tone 109  Mapping Operator with Improved Color Accuracy for High Dynamic Range Applications,” in 2019 IEEE International Conference on Consumer Electronics, ICCE 2019, 2019. [19] P. Mohammadi, M. T. Pourazad, and P. Nasiopoulos, “An Entropy-based Inverse Tone Mapping Operator for High Dynamic Range Applications,” in 2018 9th IFIP International Conference on New Technologies, Mobility and Security, NTMS 2018 - Proceedings, 2018, Jan. 2018, pp. 1–5. [20] T. H. Wang et al., “Pseudo-multiple-exposure-based tone fusion with local region adjustment,” IEEE Trans. Multimed., vol. 17, no. 4, pp. 470–484, 2015. [21] F. Banterle, P. Ledda, K. Debattista, and A. Chalmers, “Inverse tone mapping,” Proc. - Graph. 2006 4th Int. Conf. Comput. Graph. Interact. Tech. Australas. Southeast Asia, pp. 349–356, 2006. [22] F. Banterle, P. Ledda, K. Debattista, A. Chalmers, and M. Bloj, “A framework for inverse tone mapping,” in Visual Computer, 2007, vol. 23, no. 7, pp. 467–478. [23] F. Banterle, P. Ledda, K. Debattista, and A. Chalmers, “Expanding low dynamic range videos for high dynamic range applications,” in Proceedings - SCCG 2008: 24th Spring Conference on Computer Graphics, 2010, pp. 33–42. [24] F. Banterle, M. Dellepiane, and R. Scopigno, “Enhancement of Low Dynamic Range Videos using High Dynamic Range Backgrounds,” in Eurographics 2011 Area papers on HDR, 2011. [25] Y. Huo, F. Yang, and V. Brost, “Inverse tone mapping based upon retina response,” Sci. World J., vol. 2014, 2014. [26] Y. Huo, F. Yang, L. Dong, and V. Brost, “Physiological inverse tone mapping based on retina response,” Vis. Comput., vol. 30, no. 5, pp. 507–517, 2014. 110  [27] A. G. Rempel et al., “Ldr2Hdr: On-the-fly reverse tone mapping of legacy video and photographs,” in Proceedings of the ACM SIGGRAPH Conference on Computer Graphics, 2007. [28] F. Banterle, A. Artusi, K. Debattista, and A. Chalmers, Advanced High Dynamic Range Imaging. 2011. [29] A. Artusi, F. Banterle, T. O. Aydin, D. Panozzo, and O. Sorkine-Hornung, Image content retargeting: Maintaining color, tone, and spatial consistency. 2016. [30] R. Mantiuk, R. Mantiuk, A. Tomaszewska, and W. Heidrich, “Color correction for tone mapping,” Comput. Graph. Forum, vol. 28, no. 2, pp. 193–202, 2009. [31] T. Pouli, A. Artusi, F. Banterle, A. O. Akyüz, H. P. Seidel, and E. Reinhard, “Color correction for tone reproduction,” in Final Program and Proceedings - IS and T/SID Color Imaging Conference, 2013, pp. 215–220. [32] A. Artusi, T. Pouli, F. Banterle, and A. Oğuz Akyüz, “Automatic saturation correction for dynamic range management algorithms,” Signal Process. Image Commun., vol. 63, pp. 100–112, 2018. [33] E. Šikudová et al., “A gamut-mapping framework for color-accurate reproduction of HDR images,” IEEE Comput. Graph. Appl., vol. 36, no. 4, pp. 78–90, 2016. [34] International Telecommunication Union, “Parameter values for the HDTV standards for production and international programme exchange BT Series Broadcasting service,” Recomm. ITU-R BT.709-5, vol. 5, pp. 1–32, 2002. [35] International Telecommunication Union - ITU-R, “Parameter values for ultra-high definition television systems for production and international programme exchange,” Recomm. ITU-R BT.2020, vol. 1, pp. 1–8, 2014. 111  [36] “Pixabay.” [Online]. Available: https://pixabay.com/. [37] E. Reinhard, M. Stark, P. Shirley, and J. Ferwerda, “Photographic tone reproduction for digital images,” in Proceedings of the 29th Annual Conference on Computer Graphics and Interactive Techniques, SIGGRAPH ’02, 2002, pp. 267–276. [38] P. Debevec, “A median cut algorithm for light probe sampling,” in ACM SIGGRAPH 2005 Posters, SIGGRAPH 2005, 2005, p. 66. [39] M. J. Mayer and J. E. Dowling, “The Retina: An Approachable Part of the Brain,” Am. J. Psychol., vol. 101, no. 4, p. 602, 1988. [40] E. Eisemann and F. Durand, “Flash photography enhancement via intrinsic relighting,” in SIGGRAPH 2004, 2004, pp. 673–678. [41] G. Petschnigg, M. Agrawala, H. Hoppe, R. Szeliski, M. Cohen, and K. Toyama, “Digital photography with flash and no-flash image pairs,” in SIGGRAPH 2004, 2004, pp. 664–672. [42] L. Color, “Explanation of the LAB Color Space,” Int. Soc. Opt. Photonics, p. 126, 2004. [43] P. E. Debevec and J. Malik, “Recovering high dynamic range radiance maps from photographs,” in Proceedings of the 24th Annual Conference on Computer Graphics and Interactive Techniques, SIGGRAPH 1997, 1997, pp. 369–378. [44] J. N. Kapur, P. K. Sahoo, and A. K. C. Wong, “A new method for gray-level picture thresholding using the entropy of the histogram.,” Comput. Vision, Graph. Image Process., vol. 29, no. 3, pp. 273–285, 1985. [45] International Telecommunication Union, “Recommendation ITU-R BT.1886 (03/2011) Reference electro-optical transfer function for flat panel displays used in HDTV studio production,” Recomm. ITU-R BT.1886, vol. 1886, no. 1–7, 2011. [46] S. Miller, M. Nezamabadi, and S. Daly, “Perceptual signal coding for more efficient usage 112  of bit codes,” in SMPTE Annual Technical Conference and Exhibition 2012, SMPTE 2012, 2012, pp. 163–171. [47] D. G. Brooks, “The art of better pixels,” SMPTE Motion Imaging J., 2015. [48] S. J. Robinson and J. T. Schmidt, “Fluorescent Penetrant Sensitivity and Removability - What the Eye Can See, a Fluorometer Can Measure.,” Mater. Eval., vol. 42, no. 8, pp. 1029–1034, 1984. [49] C. E. Shannon, “A Mathematical Theory of Communication,” Bell Syst. Tech. J., vol. 27, no. 3, pp. 379–423, 1948. [50] “Sim2 Hdr Tv.” [Online]. Available: http://www.sim2.com/HDR/. [51] P. Kellnhofer, T. Ritschel, K. Myszkowski, E. Eisemann, and H. P. Seidel, “Modeling Luminance Perception at Absolute Threshold,” in Computer Graphics Forum, 2015, vol. 34, no. 4, pp. 155–164. [52] D. G. Curry, G. L. Martinsen, and D. G. Hopper, “Capability of the human visual system,” in Cockpit Displays X, 2003, vol. 5080, p. 58. [53] P. Hanhart, P. Korshunov, and T. Ebrahimi, “Subjective evaluation of higher dynamic range video,” in Applications of Digital Image Processing XXXVII, 2014, vol. 9217, p. 92170L. [54] A. Adams and R. Baker, The Negative. Boston: Little Brown and Company, 2002. [55] P. Saengudomlert, Convex Optimization. Cambridge University Press, 2011. [56] M. Narwaria, R. K. Mantiuk, M. P. Da Silva, and P. Le Callet, “HDR-VDP-2.2: a calibrated method for objective quality prediction of high-dynamic range and standard images,” J. Electron. Imaging, vol. 24, no. 1, p. 010501, 2015. [57] T. O. Aydın, R. Mantiuk, and H.-P. Seidel, “Extending quality metrics to full luminance range images,” in Human Vision and Electronic Imaging XIII, 2008, vol. 6806, p. 68060B. 113  [58] M. D. Fairchild, “The HDR photographic survey,” Final Program and Proceedings - IS and T/SID Color Imaging Conference, 2007. [Online]. Available: http://rit-mcsl.org/fairchild//HDRPS/HDRthumbs.html. [59] “4K Media.” [Online]. Available: https://4kmedia.org/. [60] “Sony BVM-X300.” [Online]. Available: https://pro.sony/en_CA/products/broadcastpromonitors/bvm-x300-v2. [61] H. Levene, “Robust tests for equality of variances,” Contrib. to Probab. Stat. Essays …, 1960. [62] C. Schlick, “Quantization Techniques for Visualization of High Dynamic Range Pictures,” in Photorealistic Rendering Techniques, 1995, pp. 7–20. [63] F. Ebner and M. D. Fairchild, “Development and testing of a color space (IPT) with improved hue uniformity,” in Final Program and Proceedings - IS and T/SID Color Imaging Conference, 1998, pp. 8–13. [64] S. Shen, “Color difference formula and uniform color space modeling and evaluation,” Rochester Inst. of Technology, 2009. [65] Y. Endo, Y. Kanamori, and J. Mitani, “Deep reverse tone mapping,” ACM Transactions on Graphics, 2017. [Online]. Available: http://www.npal.cs.tsukuba.ac.jp/~endo/projects/DrTMO/. [66]  “Xbox One S,” Wikipedia, 2018. [Online]. Available: https://es.wikipedia.org/wiki/Xbox_One#/media/File:Microsoft-Xbox-One-S-Console-wController-L.jpg. [67] Sony, “PS4 Pro,” Playstation.Com, 2017. [Online]. Available: https://www.playstation.com/es-es/explore/ps4/ps4-pro/. 114  [68] “SAMSUNG KS9800.” [Online]. Available: https://www.samsung.com/sg/tvs/suhd-ks9800/. [69] “LG C8.” [Online]. Available: https://www.lg.com/us/tvs/lg-OLED55C8PUA-oled-4k-tv. [70] “Sony X800E.” [Online]. Available: https://www.sony.ca/en/electronics/televisions/xbr-x800e-series. [71] R. Boitard, M. Smith, M. Zink, G. Damberg, and A. Ballestad, “Predicting HDR cinema requirements from HDR home master statistics,” SMPTE Motion Imaging J., vol. 128, no. 3, 2019. [72] G. Wu et al., “Light Field Image Processing: An Overview,” IEEE Journal on Selected Topics in Signal Processing, vol. 11, no. 7. pp. 926–954, 2017.  

Cite

Citation Scheme:

        

Citations by CSL (citeproc-js)

Usage Statistics

Share

Embed

Customize your widget with the following options, then copy and paste the code below into the HTML of your page to embed this item in your website.
                        
                            <div id="ubcOpenCollectionsWidgetDisplay">
                            <script id="ubcOpenCollectionsWidget"
                            src="{[{embed.src}]}"
                            data-item="{[{embed.item}]}"
                            data-collection="{[{embed.collection}]}"
                            data-metadata="{[{embed.showMetadata}]}"
                            data-width="{[{embed.width}]}"
                            data-media="{[{embed.selectedMedia}]}"
                            async >
                            </script>
                            </div>
                        
                    
IIIF logo Our image viewer uses the IIIF 2.0 standard. To load this item in other compatible viewers, use this url:
https://iiif.library.ubc.ca/presentation/dsp.24.1-0388672/manifest

Comment

Related Items