CANCELLATION/REDUCTION AND MEASUREMENTS OF SOME IMPAIRMENTS IN CABLE TELEVISION PICTURES by JOSE ALBERTO LAU B.Sc(EE), Seattle University, 1991 A THESIS SUBMITTED IN PARTIAL FULFILLMENT OF T H E REQUIREMENTS FOR T H E DEGREE OF MASTER OF APPLIED SCIENCE in T H E FACULTY OF GRADUATE STUDIES DEPARTMENT OF ELECTRICAL ENGINEERING We accept this thesis as conforming to the required standard T H E UNIVERSITY OF BRITISH COLUMBIA July 1995 © Jose Alberto Lau, 1995 In presenting this thesis in partial fulfilment of the requirements for an advanced degree at the University of British Columbia, I agree that the Library shall make it freely available for reference and study. I further agree that permission for extensive copying of this thesis "for scholarly purposes may be ;'granted by the head of department or by his or her representatives. It is ; understood that copying my or publication of this thesis for financial gain shall not be allowed without my written permission. Department of ; EuEcT/H'dH- The University of British Columbia Vancouver, Canada Date DE-6 (2/88) ^VoVA^/MS- Abstract As technology improves, customers' expectations become higher every day. With such technologies as the Super-VHS video recording and laserdisc player, customers compare these against the quality of the video received over the cable system. Distortions in the image are mainly caused by the nonlinearity of the underground cascaded amplifiers in the cable TV system. Moreover, the addition of new channels on the congested transmission system will increase present interference levels or create new interferences causing picture quality degradation. There are two approaches for combatting impairments. First, these can be prevented by modifying hardware components in the cable TV system such as amplifiers, converters among others. The other approach is to deal with the distortion itself through the use of image processing and filtering techniques. The idea of this approach is, if not to totally cancel the distortion, to reduce it until it becomes less visible to the human eye. This thesis deals with the second approach that consists of reducing/cancelling and measuring impairments such as the single frequency intermodulation, narrowband and wideband CSO (Composite Second Order) distortion. This would complement the automatic detection for a complete monitoring process of impairments. Our approach would fit in an automatic system since it employs non-intrusive methods that do not involve the use of a test signal and hence, no interruption in the transmission would be needed. In this thesis, several filtering methods for cancellation and reduction of single frequency intermodulation, narrowband and wideband CSO are studied and compared. Among these methods are the FIR Linear Filtering, Interpolation in the frequency domain and the Alpha Trimmed Filter. Results show that a combination of the last two methods reduces the single frequency intermodulation considerably. In addition, results also show that the interpolation in the frequency domain perform better than the other methods for narrowband CSO cancellation and reduction. In the case of wideband CSO, a combination of the interpolation in the frequency domain and the Alpha Trimmed filter reduces the impairment considerably. ii In the area of SNR measurements, different algorithms are proposed for each impairment. For the single frequency intermodulation, the interpolation in the frequency domain algorithm provides acceptable SNR measurements of less than 0.5 dB margin error. For the narrowband CSO, algorithms consisting of the interpolation in the frequency domain and the block averaging are proposed. These two algorithms provide acceptable results. For the wideband CSO, several algorithms that involve the one-dimensional and two-dimensional Discrete Fourier transform are proposed. The two-dimensional algorithms provide more accurate results than the one-dimensional approach giving a margin error of less than 2 dB for SNR measurements of up to 53 dB. iii Table of Contents Abstract ii List of Figures x List of Tables xiv Acknowledgment xv 1. Introduction 1 1.1. Motivation and Overview of the Problem 1 1.2. Scope of Thesis 2 1.3. Organization of Thesis 2 2. Cable TV System Overview 4 2.1. Introduction 4 2.2. Overview of the Television System: 4 2.2.1. The Picture 4 2.2.2. The Scanning Process 4 2.2.3. The Interlacing Process 5 2.3. Overview of the Cable TV System 2.3.1. Picture Quality 6 7 2.3.1.1. Combatting Picture Quality Degradation 7 2.3.1.1.1 Preventing Picture Quality Degradation 7 2.3.1.1.2 Dealing with the Picture Quality Degradation 9 iv 2.3.2. Classification and Types of Impairments 9 2.3.2.1. Classification by Origin 9 2.3.2.2. Classification by Patterns 9 2.3.2.3. Classification by Amplifier's Effects 12 2.3.2.3.1 Thermal Noise 12 2.3.2.3.2 Modulation Distortions 12 2.3.2.3.3 Discrete Intermodulation Distortions 13 2.3.2.3.4 Composite Intermodulation Distortions 13 2.3.3. Detection, Cancellation/Reduction and Measurements of Impairments 14 2.3.3.1. Automatic Detection 15 2.3.3.2. Cancellation/Reduction 15 2.3.3.3. Measurements 15 3. Theoretical Concepts Review 16 3.1. Review of the Discrete Fourier Transform 16 3.1.1. Theoretical Development 16 3.1.2. Relationship between the Discrete and the Continuous Fourier Transform 20 3.1.2.1. Case I : Truncation interval T is equal to the signal's period T 0 (To=T ) p 20 p 3.1.2.2. Case II : Truncation interval T is not equal to the period 0 (WT ) P 22 3.1.3. Application of Discrete Fourier Transform to Images 24 3.1.3.1. One-Dimensional Discrete Fourier Transform (1-D DFT) . . . . 24 3.1.3.2. 1-D DFT Representation of an Image 26 3.1.3.3. Two-Dimensional Discrete Fourier Transform (2-D DFT) . . . . 28 3.1.3.4. 2-D DFT Representation of an Image 3.1.4. Summary of Pitfalls from using DFT's 28 29 3.1.4.1. Aliasing 29 3.1.4.2. Leakage 30 3.1.4.3. Picket-fence Effect 32 3.2. Review of Used Techniques 33 3.2.1. Window-Based FIR Linear Filter 33 3.2.2. Median Filter 35 3.2.3. Frame Averaging 36 3.2.4. Zero Padding 36 3.2.4.1. Interpolation with Zero Padding 4. Single Frequency Intermodulation: Cancellation and Measurements 37 38 4.1. Introduction 38 4.2. Characteristics and Causes 38 4.3. Mathematical Description 38 4.3.1. Discrete Version of a TV Picture 39 4.3.2. DFT of a Portion of a Digitized Impairment Picture 41 4.4. DFT of a Line of the Digitized Impairment Picture 46 4.5. Filtering Methods 48 4.5.1. FIR Linear Filtering 48 4.5.2. Interpolation in the Frequency Domain 49 vi 4.6. Experimental Filtering Results 52 4.6.1. Interpolation 52 4.6.2. Combination: Interpolation and Alpha-Trimmed Filter 56 4.7. Objective Evaluation 59 4.7.1. Definitions 59 4.7.1.1. Spatial Domain 59 4.7.1.2. Spectrum Domain 60 4.7.1.3. C/N Ratio 61 4.8. Evaluation of Experimental Results 62 4.9. Algorithm for Measurements 62 5. Narrowband Composite Second Order Beats: Cancellation and Measurements 63 5.1. Characteristics and Causes 63 5.1.1 Harmonically Related Carriers (HRC) 64 5.1.2 Incremental or Interval Related Carriers (IRC) 64 5.2. Mathematical Description 64 5.3. Fourier Model 65 5.4. Filtering Methods 66 5.4.1. Interpolation in the Frequency Domain Algorithm 66 5.4.1.1 Importance of Zero-Padding 70 5.4.1.2 Interpolation by Averaging: Algorithm #1 73 5.4.1.3 Interpolation by Complete Removal: Algorithm #2 73 5.4.2. Alpha-Trimmed Filter 73 vii 5.5. Experimental Filtering Results 74 5.5.1. Interpolation by Averaging: Results 74 5.5.2. Interpolation by Complete Removal: Results 77 5.5.3. Alpha Trimmed Filter: Results 79 5.6. Measuring Methods 81 5.6.1. Interpolation 81 5.6.2. Block Averaging 81 5.7. Evaluation of Experimental Results 85 5.7.1. Interpolation via 1-D DFT: Results 85 5.7.2. Block Averaging: Results 85 6. Wideband Composite Second Order Beats: Cancellation and Measurements 87 6.1. Characteristics and Causes 87 6.2. Mathematical Description 87 6.3. Fourier Model 88 6.4. Filtering Methods 92 6.5. Experimental Filtering Results 95 6.5.1. 1-D DFT Interpolation: Results 95 6.5.2. Alpha Trimmed Filter: Results 97 6.5.3. Combination: Results 99 6.6. Measuring Methods 101 6.6.1. 1-D DFT Algorithm 101 6.6.2. 2-D DFT Algorithm: Pattern #1 101 6.6.3. 2-D DFT Algorithm: Pattern #2 102 6.6.4. 2-D DFT Algorithm: Pattern #3 103 viii 6.6.5. 2 - D DFT Algorithm: Pattern #4 . . . . 6.7. Experimental Measurements Results 103 105 6.7.1. Results of the 1-D DFT Algorithm 105 6.7.2. Results of the 2 - D DFT Algorithm — Pattern #1 107 6.7.3. Results of the 2 - D DFT Algorithm — Pattern #2 108 6.7.4. Results of the 2 - D DFT Algorithm — Pattern #3 109 6.7.5. Results of the 2 - D DFT Algorithm — Pattern #4 (16x16 and 32x32) 110 6.8. Observations from the Measurement Results 111 6.8.1. 1-D DFT Interpolation 111 6.8.2. 2 - D DFT — Pattern #1 112 6.8.3. 2 - D DFT — Pattern #2 112 6.8.4. 2 - D DFT — Pattern #3 112 6.8.5. 2 - D DFT — Pattern #4 113 6.8.6. Final Observations 113 7. Discussion 114 7.1. Summary 114 7.2. Conclusions 114 7.2.1. On Cancellation 114 7.2.2. On Measurements . 115 7.3. Future Work 115 Bibliography 116 Appendix A. Harmonic Related Channels 120 IX List of Figures Figure 2.1 Scanning process 5 Figure 2.2 The odd field of the interlacing process 6 Figure 2.3 Basic distribution system for cable television 8 Figure 2.4 Thermal noise 12 Figure 2.5 Hum modulation 13 Figure 2.6 Discrete Distortions 13 Figure 2.7 Composite Distortions 14 Figure 3.1 DFT of a band-limited periodic waveform when truncation interval is equal to one period Figure 3.2 DFT of a band-limited periodic waveform when the truncation interval is not equal to one period Figure 3.3 22 24 Time-domain signal and 1-D DFT of a line of a 512 by 512 image 27 Figure 3.4 2-D DFT of a 512 by 512 image 28 Figure 3.5 Illustration of aliasing caused by a low sampling rate 30 Figure 3.6 Use of rectangular data window for data analysis 31 Figure 3.7 Leakage effect from one discrete frequency into adjacent frequencies 32 Figure 3.8 Bandstop filter designed with different types of windows . . . 33 Figure 3.9 Designed notch filter of order 2N=2x84 35 Figure 4.1 Interlaced Scanning 39 Figure 4.2 Variation of e in C(l) 44 Figure 4.3 Amplitude spectrum of an intermodulation beat 45 Figure 4.4 Sketch of the 1-D DFT representation of single frequency Figure 4.5 intermodulation 47 Designed notch filter of order 2N=2x84 49 X Figure 4.6 1-D DFT of a typical line of an impaired image 50 Figure 4.7 1-D DFT of different lines of an impaired image 51 Figure 4.8 Noisy picture shopping.pgm with single frequency intermodulation 52 Figure 4.9 Noisy picture shopping.pgm after the notch filter 53 Figure 4.10 Noisy picture shopping.pgm after three zeros padded and one point removed Figure 4.11 54 Noisy picture shopping.pgm after three zeros padded and one point averaged Figure 4.12 54 Noisy picture shopping.pgm after eight zeros-padded and one point removed Figure 4.13 55 Noisy picture shopping.pgm after eight zeros and one point averaged Figure 4.14 55 Noisy picture shopping.pgm after three zeros-padded and one point removed followed by the alpha trimmed f i l t e r . . . . 56 Figure 4.15 Noisy picture shopping.pgm after three zeros-padded and one point averaged followed by the alpha trimmed filter . . . 57 Figure 4.16 Noisy picture shopping.pgm after eight zeros-padded and one point removed followed by the alpha trimmed filter Figure 4.17 58 Noisy picture shopping.pgm after eight zeros-padded and one point averaged followed by the alpha trimmed filter 59 Figure 5.1 Sketch of the narrowband C S O 65 Figure 5.2 1-D DFT of the first line of the impaired image: cso_467 . . 66 Figure 5.3 Noisy picture: CSO_467 Figure 5.4 2-D DFT Brightness level of the noisy image CSO_467 . . . 69 Figure 5.5 1-D DFT of the first line with no zeros being padded 70 Figure 5.6 1-D DFT of the first line with one zero being padded 71 xi 68 Figure 5.7 1-D DFT of the first line with two zeros being padded . . . . 71 Figure 5.8 1-D DFT of the first line with three zeros being padded . . . 72 Figure 5.9 1-D DFT of the first line with four zeros being padded . . . . 72 Figure 5.10 Noisy picture CSO_467 (Figure 5.3) after 1-D DFT interpolation by averaging Figure 5.11 75 2-D DFT Brightness level of the supposedly clean image after 1-D DFT interpolation by averaging shown in Figure 5.10 . . 76 Figure 5.12 Noisy picture CSO_467 (Figure 5.3) after 1-D DFT interpolation by complete removal Figure 5.13 77 2-D DFT Brightness level of the supposedly clean image after interpolation by complete removal shown in Figure 5.12 . . . 78 Figure 5.14 Noisy picture CSO_463 after 5-points alpha trimmed filtering . 79 Figure 5.15 Noisy picture CSO_463 after 7-points alpha trimmed filtering . 80 Figure 5.16 Basic illustration of used approach 81 Figure 5.17 Image from the chase_sequence 82 Figure 5.18 Steps of the block averaging algorithm 84 Figure 6.1 Sketch of the wideband C S O 88 Figure 6.2 2-D DFT Characteristics of wideband C S O 91 Figure 6.3 Noisy picture drink.pgm with wideband C S O of 35 dB . . . . 93 Figure 6.4 Noisy picture julie.pgm with wideband C S O of 35 dB Figure 6.5 Noisy picture meeting.pgm with wideband CSO of 35 dB . . 94 Figure 6.6 Filtered drink.pgm with one zero-padded and three points removed Figure 6.7 95 Filtered julie.pgm with one zero-padded and four points removed Figure 6.8 93 96 Filtered meeting.pgm with one zero-padded and three points removed 96 xii Figure 6.9 Julie.pgm after the alpha-trimmed filter (7-points window) . . 97 Figure 6.10 Drink.pgm after the alpha-trimmed filter (7-points window) Figure 6.11 Meeting.pgm after the alpha-trimmed filter (7-points window) . 98 98 Figure 6.12 Picture processed by interpolation in the frequency domain . 99 Figure 6.13 Picture processed by interpolation followed by the 5-points window alpha-trimmed filter 100 Figure 6.14 Illustration of the 2-D DFT algorithm: pattern #1 102 Figure 6.15 Illustration of the 2-D DFT algorithm: pattern #2 102 Figure 6.16 Illustration of the 2-D DFT algorithm: pattern #4 104 xiii List of Tables Table 2.1 Effects on distortions by the amplifier's parameters 14 Table 4.1 Experimental results for shopping_int 62 Table 5.1 Experimental results of measured Narrowband C S O 85 Table 6.1 Experimental results for measurements using 1-D DFT interpolation by averaging Table 6.2 Experimental results of measurements using the 1-D DFT interpolation by removal Table 6.3 107 Experimental results of measurements using the 2 - D DFT algorithm: pattern #2 Table 6.5 Table A. 1 109 Experimental results of measurements using the 2 - D DFT algorithm: pattern #4 (16x16) Table 6.7 108 Experimental results of measurements using the 2 - D DFT algorithm: pattern #3 Table 6.6 106 Experimental results of measurements using the 2 - D DFT algorithm: pattern #1 Table 6.4 105 110 Experimental results of measurements using the 2 - D DFT algorithm: pattern #4 (32x32) 111 Harmonic Related Channels 116 xiv Acknowledgment I would like to express my sincere gratitude to my research supervisor, Dr. Rabab Ward, for her constant support, suggestions and guidance throughout the course of this thesis. I am extremely grateful to my family for their encouragement and support throughout my entire student life. I would like to thank my friends Dr. Pingnan Shi, Dr. Qiaobing Xie and Mr. Yuen-Wen Lee for their assistance and their insightful discussions. I would also like to thank Rogers Cable Inc. for their financial support in the form of a research assistantship. Special thanks are extended to all my friends, colleagues and non-colleagues, for their words of advice, encouragement at all times, especially in the moments of struggles. I am especially very grateful to God, without him I would not be where I am now. XV Chapter 1 Introduction 1.1 Motivation and Overview of the Problem Advances in technology have been tremendous and so has been the expectations of people toward what technology can offer. New technologies as well as improvements on existing technologies and systems are constantly being developed. Cable TV (CATV) systems fall into such a category where the industry has grown rapidly in recent years and thus, it is constantly being improved to meet viewers' expectations. Among the needs.that Cable TV companies have to deal with is the addition of new channels to the existing ones. This will certainly increase present interference levels or perhaps create interferences at certain positions in the C A T V band causing picture quality degradation received by customers. Specifically, the addition of channels causes an increase in intermodulation distortion and other harmonics because the added channels are usually carried at higher frequencies. Large [1] states that intermodulation products increase rapidly in number and amplitude as the number of carriers increases. In addition, the cascaded amplifiers in the cable TV system introduce harmonics types of impairments referred to as beats. Cable TV companies realize that customers today have become more critical and sensitive to the quality of the received image sent over the cable system. With references such as the Super-VHS video recording technology and laserdisc player, cable subscribers constantly compare these against the quality of the video over the cable system. The recent introduction of digital broadcast service (DBS) with its high quality pictures further allows a subscriber to differentiate between picture qualities. These are some signs of a very demanding market where expectations are needed to be fulfilled. To ensure high quality TV pictures reception, cable TV companies need to find efficient methods to cancel or reduce the impairments arising from the cable. To be able to ensure 1 Chapter 1. Introduction 2 picture quality, there is also a need for monitoring and controlling these impairments as well as measuring the signal-to-noise levels. Currently, there is no such system that can cancel or reduce the impairments following their automatic detection. There is also no presently known methods to measure some impairments in a non-intrusive fashion. 1.2 Scope of Thesis Our main goal is to develop algorithms that eventually could be part of a system with capabilities of detecting, cancelling or reducing, and measuring the different possible existing impairments. As the competition gets larger, the need for such system is a must. There are several types of impairments that deserve to be dealt with carefully. However, we will only deal with two kinds of impairments, the single frequency intermodulation and the two types of composite second order (CSO) beats: narrowband and wideband. The reduction and measurement of these impairments are the focus of this thesis. The algorithms we propose are applied in the frequency domain, time domain or the spatial domain. These algorithms are applied to grabbed and digitized images (frames) of the analog video signal. In some instances, the impairments are simulated and added to the images while in some cases we deal with real noisy images. Image processing is then performed either in the frequency, spatial or time domain. The necessary conversion is then performed leaving the end result as an analog signal. A l l these simulations are performed in Matlab with some source code written in C language. 1.3 Organization of Thesis Chapter 2 introduces to the reader an overview of the television and cable T V systems. This overview covers the scanning and interlacing process, the picture quality and the types of impairments. Chapter 1. Introduction 3 Chapter 3 provides a review of basic theoretical concepts. This includes a review of the discrete Fourier transforms including its relationship with the continuous Fourier transform, its pitfalls and its applications to images. In addition, a review of filtering techniques such as the band-rejection FIR linear filter, frame averaging, the median filter and the interpolation with zero-padding is included. Chapter 4 deals with the single frequency intermodulation impairment and its cancellation and measurements. Filtering and measuring methods are compared and their results are presented. Chapters 5 and 6 deal with the narrowband and wideband composite second order distortions, and their cancellations and measurements. Filtering and measuring methods are compared and their results are presented. Chapter 7 presents the conclusions from the proposed algorithms in both cancellation and measurement areas. Chapter 2 Cable TV System Overview 2.1 Introduction This chapter provides a brief overview of some topics in the cable T V systems that are relevant to our work. When dealing with the cable T V system, it is necessary to understand some important concepts related to the television system such as the scanning and interlacing processes. In the cable T V system overview, issues such as picture quality, types of impairments and the detection, cancellation/reduction and measurements of these impairments are covered. 2.2 Overview of the Television System: 2.2.1 The Picture As we know, through television still pictures are shown one after the other with enough speed to give the illusion of motion. Each still picture when digitized is composed of many small areas. Each of these small areas of light or shade is a picture detail, or picture element, also called pixel or pel. A l l the picture elements together contain the visual information in the scene. 2.2.2 The Scanning Process Most T V sets paint the television image on the screen, as a series of horizontal lines using a cathode ray tube, CRT. The CRT produces an electron beam which falls on a screen coated with chemical substance. The screen glows white when struck by the electrons. The lines are painted onto the screen from left to right and drawn from top to bottom. Since each line in the image is swept from the left to the right side of the screen, at the end of the sweep the CRT beam returns rapidly to the left side of the screen. The return line is called the horizontal retrace time or the horizontal blanking 4 interval. Chapter 2. Cable TV System Overview 5 Once the cathode ray tube has drawn the last line at the bottom of the screen, it begins to draw the next image. Since the image is always scanned from top to bottom, it is necessary that the CRT returns to its initial point at the top of the screen. The interval in which the CRT beam moves back to the initial point is called the vertical retrace time. The final lines in the image are not painted on the screen and, consequently, the 525 lines resolution of the NTSC system is slightly reduced. Only 483 of these lines are scanned on the screen. Figure 2.1 illustrates the concepts of the horizontal and vertical retrace (blanking) intervals. Scan line Horizontal retrace Vertical/ retrace Figure 2.1 Scanning process 2.2.3 The Interlacing Process The NTSC (National Television System Committee) and other standards require that the television image be drawn in an interlaced format i n order to produce a better quality picture. This means that after the first line is drawn, the CRT draws the third line instead of the second line. After the odd lines are drawn, the CRT returns to the top of the screen and draws the even lines starting with the second line until the picture is completely drawn. Each such scan of odd or even lines is called a field, i.e. a picture or a frame is composed of two fields. The NTSC system has a rate of 30 frames per second. It was found that by interlacing the image, the flicker is significantly reduced. If the image were not interlaced, each area of the screen would fade before the CRT beam returned Chapter 2. Cable TV System Overview 6 to trace this region. The effect of this would be to produce a 30-cycle flicker which would be quite visible to the human eye [2]. The horizontal scanning frequency is 525 lines which when divided by 1/30 s/scan gives a total of 15,750 lines per second or a horizontal sweep interval of 63.5 //s/line. Of this 63.5 US, about 11 us are allowed for the horizontal retrace or blanking interval. Line #1 Line #3 Line #5 Line #7 Line #L-4 Line #L-2 Line #L Figure 2.2 The odd field of the interlacing process 2.3 Overview of the Cable TV System Cable T V systems obtain their T V programs from different sources such as local T V stations, satellite transmission (from fixed or broadcasting satellites services) and microwave networks. Each program is assigned to a different channel and these are combined at the headend of the cable system resulting in a wide band multichannel T V signal [3]. This wide band signal is transmitted to millions of viewers through a carefully designed cable network. In the cable network, broad band amplifiers are cascaded every 1/4 to 1/2 mile for the purpose of keeping the power in the signal above a certain level. If the amplifiers are perfectly linear, the output at the end of the cable network will not be distorted by the amplifiers. In reality, the imperfection of amplifiers might cause higher order harmonics to appear in the output signal of the amplifier. With a cascaded system stretching into several Chapter 2. Cable TV System Overview 7 miles, these impairments become too difficult to control. Moreover, the addition of new channels to the present congested transmission system will increase the existing interference levels as well as create interferences at new positions in the C A T V band [4]. 2.3.1 P i c t u r e Q u a l i t y The television signal may be quite weak unless the viewer lives within the station transmitter's range, thus several cable companies provide an amplified signal to far located homes for a monthly fee. These companies are able to provide additional important services, such as more channels, pay T V and information channels. The signals which are collected at the cable headend are then transmitted through the cable network to the cable subscribers. The malfunctioning of any point of the cable network will certainly degrade the image quality. Recently, the quality of the picture has become more important (critical) to cable subscribers. Most cable companies are aware of this factor as is reflected by the constant improvements in their services. Ideally, the problem should be detected and repaired before any interruption to the subscriber's television signal takes place. 2.3.1.1 Combatting Picture Quality Degradation There are two approaches that can be taken to improve the quality of the picture in Cable TV. The first deals with the prevention of picture quality degradation. This approach is hardware-oriented and consists of making improvements in the components of the existing cable design such as amplifiers, converters, etc. The second deals with post-processing the picture to reduce any degradations if present. This approach consists of applying image processing and filtering techniques to a single frame or image or to a sequence of images grabbed from the cable. In this thesis we follow the second approach. 2.3.1.1.1 Preventing Picture Quality Degradation To make the cable T V system more efficient and economical and to provide an improved quality picture, hardware implications in the cable system network are constant topics of study and research [5]. Chapter 2. Cable TV System Overview 8 A typical cable T V system is composed of the following elements: Local origination (studio) Receiving antenna Satellite receiver Directional coupler Trunk amplifier Coaxial cable \ Trunk amplifier Bridging amplifier -c Branch line 3 (> Drop to subscriber Line extension amplifier Converter To TV set Drop Figure 2.3 Basic distribution system for cable television Large [1] suggests using video timing techniques to reduce distortions in cable TV. Specifically, he compares several approaches and suggests the power spreading technique to reduce system non-linearities as opposed to methods such as H R C (Harmonic Related Carriers) or video synchronization which, at best, hide distortions rather than reducing them. As widely known, converters deployed in subscriber's homes are considered to be detrimental to picture quality by subscribers. Moloney [5] suggests that one of such improvements will be through the use of low noise-noise pre-amplification in the converters. These types of preamplifiers provide a means by which the C/N of the system will not be degraded through a converter. This, coupled with enhacement to the demodulator/modulator sections of the converter, will allow subscribers to receive the maximum quality picture available from today's NTSC system. In addition, Slater [6] focuses his attention on types of Chapter 2. Cable TV System Overview 9 preamplifiers that could be used to reduce interference, more specifically, composite second order beats. 2.3.1.1.2 Dealing with the Picture Quality Degradation Improving picture quality by image processing techniques has been a widely studied topic [7], [8]. The application of single and combined linear filters and non-linear filters such as the median filter to television images have been dealt with profound interests. 2.3.2 Classification and Types of Impairments There are different ways to classify the existing types of impairments. These are impairments by origin, by patterns, and by amplifier's effects. 2.3.2.1 Classification by Origin Impairments may be classified as the following: external and internal interferences. External interferences are types of impairments that affect a single subscriber due to a faulty television receiver or to a possible interference with some other operating electronic appliance or perhaps by another receiver. Internal interferences are those caused by cable circuitry or by a faulty cable amplifier or some other cable components. The failure of such components due to extremely poor conditions could affect an entire region. 2.3.2.2 Classification by Patterns When classified by their patterns, the impairments may be one of these two types: coherent and non-coherent. The coherent type consists of impairments with recognizable interference patterns or pictures. Those are known to be more objectionable than non-coherent impairments of the same noise strengths. Impairments that fall into the coherent category are ghosting, crossmodulation, herringbone type, broadband and impulse electrical interference, diathermy, hum, and beats caused by intermodulation distortion. 1. On the other hand, snow noise falls into the non-coherent category. Ghosting is a phenomenon in which a delayed version of the image appears on the screen. This delayed image could appear to the right or to the left of the original image. It is Chapter 2. Cable TV System Overview 10 called normal ghosting if the delayed version appears to the right. On the other hand, if it appears to the left, it is called leading ghosting since this delayed version will be stronger than the original signal. This phenomenon is caused by a strong reflected signal at the antenna site. 2. Crossmodulation is characterized by the presence of a faint image on the screen due to another channel. It is mainly caused by either overloaded converters or overloaded headend/distribution amplifiers. 3. Herringbone type is named after the pattern it produces in the image. Horizontal herringbone bands appear moving across the entire image. The cause for such impairment depends on the number of channels being affected. If only a few channels are affected, then the cause could be due to interference with lower adjacent sound frequencies. However, if all channels are affected, then the cause may be due to a mistuned television set or converter. 4. Broadband and Impulse electrical interference: The broadband type is characterized by a moving interference pattern in which tearing of some lines and peppery interference may occur across the entire image. Its cause may be due to fluorescent lighting fixtures or to power line interference. The impulse type is characterized by random burst of interference for only a brief duration and for a small number of lines in the image. Its cause may be due to electrical motors, faulty lighting fixtures, dimmer controls or by the ignition systems in internal combustion engines. 5. Diathermy is characterized by intermittent patterns of herringbones shapes, i.e. resembling the letter "S". This impairment tends to affect certain channels such as channel two, six, seven, thirteen and sometimes three. It is caused mainly by interference with other appliances such as microwave ovens. 6. Hum is characterized by faint bands moving along the image on the T V screen. These will move slowly and will appear as a single band or two bands depending on the frequency of the impairment. For the 60 Hz hum modulation, a moving-up single band will appear Chapter 2. Cable TV System Overview U on the screen meaning that a subsequent band will appear once the previous one has disappeared. For the 120 Hz hum modulation, two bands will be visible on the screen as they move upward along the image. The possible causes for hum modulation are defective T V set power supplies, overloaded power supplies or low A C power voltage in the cable distribution systems. 7. Beats caused by intermodulation distortion: Intermodulation is classified according to how the distortion affects the image. Intermodulation distortion causes single intermodulation such as picture carrier beats, color beats and also causes composite second order beats and composite triple beats. • Picture carrier beats are characterized by horizontal or diagonal intensity bands across the T V screen. These beats could be either narrow or wide depending on how close the interference frequency is to the picture carrier. The closer to the carrier, the wider is the band. • Low frequency color beats result in broad bands of color variation across the screen. Color beats may cause noticeable changes in color, i.e. from green to yellow or from red to orange. A color beat occurs when the interference frequency falls close to the color carrier of the victim channel. Picture carrier beats and low frequency color beats may be due to overdriven cable amplifiers or to interference in the television signal with the radio frequencies of taxicabs. • Composite second order beats have the appearance of diagonal bands across the image. These are caused by second order distortion of amplifiers in the coaxial cable or by lasers in the fibre cable distribution networks. • Composite triple beats have the video characteristics of graininess or a texture effect over the entire picture. This impairment is due to the cumulative effect of hundreds of third and fifth order intermodulation beats. 8. Snow noise falls into the non-coherent type of impairments. It is named after the snowstorm visual effect that it creates in the image. It causes random variations in Chapter 2. 12 Cable TV System Overview the cable signal that produce uniform effects over the entire screen by adding a random Gaussian value to each pixel. Its causes range from a faulty television set R F amplifier or a broken television matching transformer, faulty cable splices or badly coupled "F" type connector to poor weather conditions. In this thesis we deal with single intermodulation beats and composite second order beats. 2.3.2.3 Classification by Amplifier's Effects According to Poirier [9], distortions are a natural by-product of the amplifier process. In addition, he states that the type and amount of distortion generated is a function of the amplifier technology and how closely to the amplifier linearity limit the system operates. He classifies distortions into four categories: thermal noise, modulation, discrete intermodulation and composite intermodulation distortions. 2.3.2.3.1 Thermal Noise It is defined as radio energy generated as a result of random motions of electrons within the amplification system [9]. Electron Motion -59 dBm/4 MHz Bandwidth Figure 2.4 Thermal noise 2.3.2.3.2 Modulation Distortions It is commonly characterized in cable television by hum modulation and defined as the transference of unwanted modulation to the signal being amplified. As shown in Figure 2.5, the modulation comes from the power supply ripple Chapter 2. Cable TV System Overview 13 which causes periodic changes to the amplifier's performance at the ripple rate. The hum modulation normally occurs at a very small frequency. Amplifier Unmodulated Signal Power Supply Ripple Figure 2.5 Hum modulation 2.3.2.3.3 Discrete Intermodulation Distortions The process of passing two or more carri- ers through an R F amplifier causes certain amount of interaction between them. The amplifier, depending upon its non-linearity, will generate multiples of the products being amplified, i.e. it may act as a R F mixer. F2 F1 Amplitude F1-F2 2F2 F1+F2 2F1 Frequency (Hz) Figure 2.6 Discrete Distortions 2.3.2.3.4 Composite Intermodulation Distortions When more signals are carried through an amplifier, many discrete intermodulation distortion frequencies result. In many instances, these appear in the same position in the R F spectrum, where they cluster around certain Chapter 2. Cable TV System Overview 14 frequencies. Composite triple beat and composite second order distortion fall into this category. Amplitude F1 F2 F3 F4 F5 Frequency (Hz) Figure 2.7 Composite Distortions Type of impairment System Operating levels System channel loading Number of amplifiers in cascade Thermal noise is affected by is not affected by is affected by Modulation distortion is not affected by is not affected by is affected by Discrete Intermodulation is affected by is litde affected by is affected by Composite Intermodulation is affected by is affected by is affected by Table 2.1 Effects on distortions by the amplifier's parameters 2.3.3 Detection, Cancellation/Reduction and Measurements of Impairments Dealing with impairments such as intermodulation involves researchers' attention in three different areas. These are automatic detection, cancellation (or reduction) and measurements of these impairments. Chapter 2. Cable TV System Overview 2.3.3.1 Automatic Detection 15 Automatic detection of intermodulation has been recently a topic of research [2], [10], [3]. These studies deal with detection schemes of symmetric pulses obtained from the two-dimensional Fourier transform of the image. 2.3.3.2 Cancellation/Reduction There have been studies on noise suppression in television images through a variety of filters such as temporal linear filters combined with median filters [7] and non-linear temporal recursive filters [8]. However, there is not any method for intermodulation noise suppression in the current cable T V system. 2.3.3.3 Measurements This is an important issue since it can tell us how much noise is present in the cable T V images. Measurements on noise have been a subject of many studies. The test measurements for noise has been performed subjectively [11], [12], [13], [14], [15] and objectively [16]. The current cable T V system uses a method called the "quiet line" that involves the measurement of noise in areas that do not contain part of the picture signal. Chapter 3 Theoretical Concepts Review 3.1 Review of the Discrete Fourier Transform The impact of the discrete Fourier transform in many applications in image processing has been tremendous. With present day digital computers, it is possible to go back and forth between waveform and spectrum with enough speed and economy to create a whole new range of applications in a more broad perspective. The Fourier transform has long been used for characterizing linear systems and for identifying the frequency components making up a continuous waveform. However, to make the system amenable to digital computer computation, the continuous waveform is sampled and rendered finite. The discrete Fourier transform (DFT) is obtained as a close approximation to the continuous Fourier transform (CFT). The DFT retains most of the properties of the CFT, however, some differences among them result from the constraint that the DFT must operate on sampled waveforms defined overfiniteintervals. 3.1.1 Theoretical Development The Fourier transform pair for a continuous waveform h(t) can be written in the form oo H{f)= J h(t)e- * dt j2 ft (3.1) —oo oo h(t)= J Htfy^df (3.2) —oo for — oo < / < oo and — oo < t < oo. The function H(f) represents the Fourier transform of the time-domain continuous function h(t). To develop the Discrete Fourier transform pair, it is needed to derive the mathematical relationships from three modifications: time-domain sampling, truncation and frequencydomain sampling. The derivation of the discrete Fourier transform from the continuous Fourier transform is presented next. 16 17 Chapter 3. Theoretical Concepts Review Consider the continuous waveform h(t) and its Fourier transform H(f). After sampling, the sampled waveform becomes h(t)A (t), where A (t) is the time domain sampling function 0 0 with a sampling interval of T. This can be written as h(t)A (t) = h(t) - 0 oo ) kT k=—oo k = -°° (3.3) k=—oo where the value of T will determine the amount of aliasing effects. In the next step, the sampled function is truncated by multiplying with the rectangular function x(t) defined as x(t) = 1 = 0 W T T - - < t < T - 2 2 0 (3.4) otherwise where T is the duration of the truncation function. This process yields the following 0 expression, h(t) AoCO x(t) = h{kT)6(t-kT) :{t) Lfc=—oo N-1 (3.5) HkT) S(t - kT) k=0 where N=T /T, i.e. there are N equidistant impulse functions lying within the truncation 0 interval. As a final step in the process of modifying the original Fourier transform (CFT) pair to a discrete Fourier transform (DFT) pair is to sample the Fourier transform of the last equation. In the time domain, this process is equivalent to convolving the sampled truncated waveform given in the last equation and the time function Ai(t) that can be written as oo Ai(t)= To Y 6(t-rT ) 0 (3.6) Chapter 3. Theoretical Concepts Review 18 The resulting waveform h(t) becomes N-l [[/*(*) A„(t) x(t)] * Aj(i)] h(kT) S(t - kT) = Lfc=o oo To *(*-r?b) r=—oo = ... + T (kT) 6(t + T - kT) h 0 0 (3.7) ifc=0 N-l J2 (kT) 8{t - kT) +T h 0 k=0 N-l + ToY, (kT)S(t-T -kT) + ... h 0 k=0 or 7V-1 h(t) = To £ ^ /i(JfeT) r=—oo Lk=0 6(t-kT-rT ) (3.8) 0 i.e. /i(£) is an approximation to the function h(t) and is a periodic function with period T 0 and with N samples in each period. The Fourier transform of h(t) (an approximation of h(t)) is developed next. First, let us recall that the Fourier transform of a periodic function h(t) is a sequence of equidistant impulses written as follows **\T~) where ^ = a n 8 ^ ~ n ^ (3.9) where /o = - 7 - To-T/2 dt n = 0,±1,±2,... (3.10) -T/2 and after substituting (3.8) in (3.10), we obtain T -T/2 0 «» = Y / ° E T 0 -T/2 ^ r=-oo k=0 Y. ( ) ( - - ») ~ ™ ' ° h kT 8 t kT rT e 32 i T (3.11) dt / Chapter 3. Theoretical Concepts Review 19 Since integration is only over one period, °-T' N-1 2 T Y a= n J 1 -T/2 k ( ^ h kT)e- 8 l dt j27rnt Ta ~ ° °~P N-1 T 2 / = Y2 K T) k 1 ^ ~ kT r\ ]2 ni (3.12) n = 0, ± 1 , ± 2 , . . . (3.13) Ta J n k-0 = 8{t - kT) dt e- * l -T/2 Y^h(kT)e-> * / ° 2 knT T k=0 knowing that T =NT, the previous equation becomes Q N-1 a =Y K T) e-> * l k n 2 hn N k=0 Then the Fourier transform in (3.9) becomes oo (-WT)= H N-1 E E KkT)e- * ' ]2 kn N (3.14) n=—oo k=0 This transform is periodic with a period of N samples and possess only N distinct values for which it can be evaluated. Thus, it can be written as N-1 S {jf) = E KkT)e-> ™ ' ifc=o 2 n = 0,l,...,iV-l k N (3.15) This expression relates N samples of time and N samples of frequency derived from the continuous Fourier transform. The discrete Fourier transform is an approximation to the Continuous Fourier transform and thus, a special case of the CFT. In summary, the discrete Fourier transform pair is normally written as G (i^) = E*(* ) ~ r n=0 or e 32nnk/N Chapter 3. Theoretical Concepts Review 20 GW4E^) N _ u E ~ J2WNK/N J-° (3.17) N (k) = ^ G{n) g e / j2rnk N for n=0,l,...,N-l; and k=0,l,...,N-l. 3.1.2 Relationship between the Discrete and the Continuous Fourier Transform In most cases, the primary interest in the discrete Fourier transform is because it approximates the continuous Fourier transform. The validity of this approximation, however, is dependent on the waveform being analyzed. Differences between these two transforms arise because of the discrete transform requirement for sampling and truncation, issues that will be dealt with next. 3.1.2.1 Case I : Truncation interval T is equal to the signal's period T (T = T ) 0 p 0 p Consider a periodic function h(t) to be a sinusoid as shown in Figure 3.1(a). Since h(t) is a real cosine waveform, the continuous Fourier transform (CFT) consists of two impulse functions that are symmetric about the zero frequency. The periodic waveform h(t) is then sampled by multiplication with the sampling function Ao(t) as shown in Figure 3.1(b). The sampled waveform h(kT) and its Fourier transform are shown in Figure 3.1(c). The sampled waveform is then truncated by multiplication with the rectangular function x(t) as illustrated in Figure 3.1(d). The rectangular function is chosen such that the N sample values resulting from truncation equate to one period of the original waveform h(t). The Fourier transform of the finite-length sampled waveform shown in Figure 3.1(e) is obtained by convolving the frequency-domain impulse functions of Figure 3.1(c) and the [sin(f)]/f frequency function of Figure 3.1(d). From the convolution results, as observed in Figure 3.1(g), the convolved frequency function in Figure 3.1(e) is considerably distorted. To eliminate such distortion, the convolved frequency function (Figure 3.1(e)) is sampled by the Chapter 3. Theoretical Concepts Review 21 frequency-sampling function in Figure 3.1(f). The reason for that is because the equidistant impulses of the frequency-sampling function are separated by 1/T . D Because the sampled truncated waveform forms exactly one period of the original waveform h(t) and because the time-domain impulse function of Figure 3.1(f) are separated by T , the convolution yields 0 a periodic function. Sinusoids are the only kind of waveforms in which the discrete and continuous Fourier transforms are exactly the same within a seating constant [17]. The following are the conditions for these two transforms to be equivalent: 1. The time function h(t) must be periodic. 2. h(t) must be band-limited. 3. Sampling rate must be at least two times the largest frequency component of h(t). 4. The truncation function x(t) must be nonzero over exactly one period (or integer multiple period) of h(t). Chapter 3. Theoretical Concepts Review 22 H(f) A/2 0 D A (t) f 1/T -1/T i o<> A Q f k i T (b) t t 1/T -in Ah(t)A (t) A/ 4 * ° H(,) 0 A f (f) (2 A1 4 4 (c) 44 -i/r 1/T„ r j i IX(f)l x(t) To g ^ (d) -T/2 -i/r - r V 0 t J A k ~ 0 (e) i 1<> A l , f i -To v AlH(f) * A (f) * X(f)l l( > To A[h(t) 1/T 0 '"")Ao(t)x(t) "t t 1/T n [H(f)*A (f)*X(f)] A.(f) A (t)x(t)]*A (t) D AT A T Jg+N^N-fcj 0 1 A\ /A , AV(2T) u i\ \ Figure 3.1 DFT of a band-limited periodic waveform when truncation interval is equal to one period 3.1.2.2 Case H : Truncation interval T is not equal to the period ( T ^ T ) 0 0 p When the truncation of the sampled function h(t) is other than an integer multiple of the period, its discrete Fourier transform (DFT) and continuous Fourier transform (CFT) differ considerably. Consider the same sinusoidal waveform h(t) from case I, h(t) is sampled and truncated as shown in Figure 3.2(c) and Figure 3.2(e) respectively. Because the truncation interval is not Chapter 3. Theoretical Concepts Review 23 an integer multiple of the period of h(t), the sampled truncated function shown in Figure 3.2(e) results. Similarly, the Fourier transform of the sampled truncated waveform of Figure 3.2(e) is obtained from convolving the frequency-domain impulse functions in Figure 3.2(c) and the [sin(f)]/f function in Figure 3.2(d). Figure 3.2(g) illustrates the result from sampling the resulting convolution at frequency intervals of 1/T . It is observed here the presence of an 0 impulse at zero frequency. This component represents the average value of the truncated waveform and is not zero because of the uneven number of cycles in the truncated waveform. Other impulses in the frequency-domain impulses occur because the zeros of the [sin(f)]/f function are not coincident with each sample value as illustrated in Figure 3.2(g). In summary, the effect of truncation at other than a multiple of the period is to create a periodic function with sharp discontinuities. These sharp changes produce additional frequency components in the frequency domain. The additional frequency components that occur after frequency-domain sampling are called sidelobes. This effect is called leakage and is inherent in the DFT because of the required truncation in the time domain. Chapter 3. Theoretical Concepts Review 24 H(f) A/2 1/T -1/Tp W f p i o< > 1/T, A f k iI T-H t 1/T -1/T H*- t H{f)*A (f) ih(t)A (t) 0 0 A/(2T) <=> 4 A A A (c) -1/T T -f/2 0 T (d) -1/T, o A h(t)A (t)x(t) I— N - • 1/T„ "'O 0 (e) A P AlH(f) *A (f)*X(f)l 0 ii 1T IX(f)l x(t) -T/2 r AA 1/T„ 1W i ' "To t To . "t AlhWMtjxttHtA^t) (f) 1/To [H(f)*A (f).X(f)]A (f) c 1 Figure 3.2 DFT of a band-limited periodic waveform when the truncation interval is not equal to one period 3.1.3 Application of Discrete Fourier Transform to Images 3.1.3.1 One-Dimensional Discrete Fourier Transform (1-D DFT) The discrete Fourier Chapter 3. Theoretical Concepts Review 25 transform (DFT) of a sequence (u(n), n=0, ,N-1} is defined as N-l v(k) = u(n)W , n N k = 0,1, ...,N - 1 n=0 where W = exp< N (3.18) —JJ— In case of an image, each line is treated as a sequence, i.e. u(n). The inverse DFT is obtained as N-l < ) n = iv E () N > v k W n = 0,1,JV - 1 kn (3.19) In image processing, the transformation is scaled so that it is unitary and can be defined as JV-l v ^ = 7f E u ( ) N, n k = 0 , N W - 1 ;=° (3.20) <) N > < ) = 7^E k n W n= kn 0,...,N-l Among the properties of the DFT, we would like to explore two of them that will be used later. 1. The DFT is the sampled spectrum of the finite sequence u(n) extended by zeros outside the interval [0,N-1]. If we define zero-extended sequence -/ () U n \ A (u(n), = io, 0<ra<JV-l olhe'rwise « (- ) n 3 21 then its Fourier transform is oo U(w) = ^ JV-l u(n)exp(— jwn) = ^ n~—oo u(n)exp(— jwn) (3.22) n=0 Comparing to equation (3.18), its DFT is <k) = Ul— J (3.23) v(k) = - ^ U (3.24) and in case of the unitary DFT Chapter 3. Theoretical Concepts Review 26 2. The DFT or unitary DFT of a real sequence (x(n), n=0,...,N-l} is conjugate symmetric about N/2. N-1 v*(N -k)=J2 N-1 = Y, u{n)W u*{n)W~ ~ {N h)n k n n=0 ,'N N = v{k) n=0 ,\ JN v /[ - - ku N 3.1.3.2 1-D D F T Representation of an Image \ , / v[j N N (3.25) + ik Let us consider a clean image of size 512 by 512. After obtaining the 1-D DFT of every line in an image, it is observed a similarity in each of them. Figure 3.3(a) illustrates the signal in the time domain for the first line in an image while Figure 3.3(b) shows the 1-D DFT of that line. Figure 3.3(c) is the same as that of Figure 3.3(b) except that the DC term is shifted to the middle 255th index. It can also be seen a symmetry around the DC term. The spectrum is characterized by a DC component with the rest of the image contents having their frequencies concentrated around this term. Chapter 3. Theoretical Concepts Review Figure 3.3 Time-domain signal and 1-D DFT of a line of a 512 by 512 image 27 Chapter 3. Theoretical Concepts Review 28 3.1.3.3 Two-Dimensional Discrete Fourier Transform (2-D DFT) The two-dimensional DFT of an N x N image (u(m,n)} is a separable transform defined as N-l N-l »(M)=EE«( ' )W. ffl . B 0<k,l<N-l (3.26) •m—0 n—0 and its inverse as uK n) = N-l N-l Yl E (> v k l ) W N k m W N i 0 < m , n < TV - 1 n (3.27) k=0 1=0 The two-dimensional unitary D F T pair is defined as j N-l N-l ^'^ivEE^'^Wm=0 N-l u(m, n) = 1 Y 0<k,l<N-l n=0 (3.28) N-l E 0^* m W^ , B , 0 < m, n < N - 1 it=0 /=0 In terms of matrices, these expressions can be related as V= F U F or 3.1.3.4 2-D DFT Representation of an Image U = F*VF* (3.29) The two-dimensional D F T of a noise-free image is known to concentrate in the low frequency regions of the spectrum. As illustrated in Figure 3.4, the contents of an image is concentrated around the peak, i.e. the D C portion of the image. Figure 3.4 2-D DFT of a 512 by 512 image Chapter 3. 3.1.4 Theoretical Concepts Review 29 Summary of Pitfalls from using DFT's When using the discrete Fourier transform to approximate the Fourier transform of a continuous time signal, three possible phenomena arise resulting in errors between the computed and the desired transform. These are: aliasing, leakage and the picket-fence effect. 3.1.4.1 Aliasing It refers to when high-frequency components of a time function imper- sonate low frequencies if the sampling rate is too low. For example, given a continuous-time signal xi(t) as shown in Figure 3.5(a). Assuming the transform Xi(f) is bandlimited to 0 < f < fh where ft, is the highest possible frequency in the spectrum, and also assuming that the time signal is sampled at a rate f that is less than 2fh, let X2(n) represent the sampled s signal as shown in Figure 3.5(c). Its transform Xzif), as shown in part (d) of the same figure, is characterized by spectral overlap, also referred as aliasing. This effect causes the impersonation of frequencies in the overlap region for other frequencies preventing the exact recovery of the original signal. The solution to this problem is to ensure that the sampling rate is high enough for the highest frequency present to be sampled at least twice during each cycle to avoid any spectral overlap. This means f > 2fh. If the signal is known to be limited to a certain band, the s sampling rate can be determined accordingly. Normally, the signal may be filtered with a low-pass analog filter before sampling to ensure that no component above the folding frequency (also referred as the Nyquist frequency) appear. In our case, the TV picture line is sampled at a rate of 512 per active line, this means a sampling frequency f of 9.66 MHz. s Since fh < 4.5 MHz, the Nyquist criterion is satisfied. Chapter 3. Theoretical Concepts Review A lW x 30 A l( ) X f (d) Figure 3.5 Illustration of aliasing caused by a low sampling rate 3.1.4.2 Leakage This problem has been discussed i n detail in section 3.1.2. It arises because of the practical requirement that we must limit observation of the signal to a finite interval of data. This problem is inherent in the Fourier analysis of any finite length of data. As mentioned in section 3.1.2, the process of truncating the signal after a finite number of samples is equivalent to multiplying the signal by a window function, as shown in Figure 3.6. If the window function is a rectangular function, the series is abruptly terminated without modifying any coefficients within the window. The resulting transform can be considered as the convolution of the desired spectrum with the spectrum of the rectangular window. The net effect is a distortion of the spectrum. See Figure 3.7. There is a spreading or leakage of the spectral components away from the correct frequency resulting in an undesirable modification of the total spectrum. Chapter 3. Theoretical Concepts Review 21 _jl 0 1 i 10 1 i 20 31 1 i 30 1 i 40 r i 50 I 60 (C) Figure 3.6 Use of rectangular data window for data analysis The leakage effect can not always be isolated from the aliasing effect because leakage may also lead to aliasing. Since leakage results in a spreading of the spectrum, the upper frequency of the composite spectrum may move beyond the folding frequency and aliasing may then occur. The best approach for alleviating the leakage effect is to choose a suitable window function that minimizes the spreading, i.e. one that causes small sidelobes. Many window functions such as Hamming, trapezoidal and Hanning windows are popular. Chapter 3. Theoretical Concepts Review 32 F(f) "f \ A °\ / ^ W(f) F(f)-W(f) Figure 3.7 Leakage effect from one discrete frequency into adjacent frequencies 3.1.4.3 Picket-fence Effect It is produced by the inability of the discrete Fourier transform to observe the spectrum as a continuous function since computation of the discrete spectrum is bounded to integer multiples of the fundamental frequency. For instance, when observing the spectrum, the peak of a particular component may not be detected because it could lie between two of the discrete transform lines. A technique used to reduce the picket-fence effect is to vary the number of points in a time record by adding zeros to the end of the original record. This changes the period Chapter 3. Theoretical Concepts Review 33 and thus, changes the locations of the spectral lines without altering the continuous form of the original spectrum. In this way, spectral components originally hidden from view can be shifted to points where they can observed. 3.2 Review of Used Techniques 3.2.1 Window-Based FIR Linear Filter This type of filter can be designed as a lowpass, bandpass, highpass or bandstop filter. Here, we will focus our attention on the bandstop, also called band-rejection,filterusing Hamming windows. Other windows such as the Boxcar, the Bartlett and the Hanning windows are some of the most used in signal processing. Figure 3.8 illustrates the windows effects used in the design of a bandstop filter. I 1 --t-l-h 1 -\ 1 H h Bartlett window : 0.1 0.2 H 0.3 1 1 H Hamming window --i Boxcar window 0.4 0.5 0.6 0.7 0.8 0.9 Normalized Frequency (rad/s) Figure 3.8 Bandstop filter designed with different types of windows Chapter 3. Theoretical Concepts Review 34 Theoretically, a windowed FIR filter of length M with input x(n) and output y(n) can be described by the following equation y(n) = b x(n) + b\x{n - 1) + ... + bM-\x(n - M + 1) 0 M-l (3.30) where {b^} is the set of filter coefficients. The unwindowed FIR filter can be expressed as M-1 Y,Kk)x{n-k) y(n)= (3.31) fc=0 where h(k) is the unit sample response of the filter. These filters can also be characterized by their system functions M-1 H(z) = Y Kk)z- k (3.32) k=0 = h(0) + h ^ z ' 1 + h(2)z~ 2 + ... + h(M - l ) z - ( - ) M 2 and b (z) = b + hz' 1 k 0 + ... + & - i * " ( M - 1 ) M (3.33) where M-1 is the order of the filter with M coefficients. However, in the case of the bandstop filter, the order of the filter is 2(M-1). The windowed digital filter coefficients are obtained from b = w{k) k h(k) , 1 < Jb < M - 1 (3.34) where w(k) denotes the Hamming window (l<k<M-l) and h(k) represents the impulse response of the ideal filter. Figure 3.9 illustrates the design of a windowed bandstop filter of order 2x84 using the Hamming window. Chapter 3. Theoretical Concepts Review 0.91 <D 0.8 \ c 0.7 cn I I 0.6 \ 0.5 h 0.41 0.3 h 0.2 •r • i i "T" i i "T" i 1 " i" i .1. 4 — - — + • — I — h I I 13 35 1 1 | 1 -r • 1- , |. -t- -r- "T" I I T" I "T \ "T" I I .1 'I ' I I. .1. . J. L 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 Normalized Frequency (rad/s) Figure 3.9 Designed notch filter of order 2N=2x84 3.2.2 Median Filter The median filter is a non-linear type of filter commonly used in image processing and speech processing. In image processing, it is used to suppress noise while preserving the edges. This type of filter is based on selecting the median of the input samples as the output. The median P of a set of numbers is that number in the set such that half of the values in the set are greater than P and the other half are less than P. Median filters are applied to images by first selecting a window of size N by N pixels, where N must be an odd number. Then this window is scanned across the image. The center pixel at every location in the image is replaced by the median of pixel values in the N by N window. The purpose of this is to force pixels with abrupt intensities to be more like their neighboring pixels. This causes the elimination of intensity spikes or impulses which improves the signal-to-noise ratio. Chapter 3. Theoretical Concepts Review 36 3.2.3 Frame Averaging This method is used for noise suppression in video signal processing. Essentially, it is effective when the images under consideration are still. In case of motion, i.e. some parts of the image are moving, this method introduces blur. Therefore, motion detection has to be applied when frame averaging is used. Consider a noisy image g(x,y) with noise n(x,y) and f(x,y) as the original image, i.e. 9(x,y) = f(x,y)+ n(x,y) (3.35) and assume that for all pairs of coordinates (x,y), the noise are uncorrelated and have zero average value. A n image g(x,y) is formed by averaging M different noisy images, M 1 g(x,y) = M TjY9i(x,y) (3.36) i=i Then it follows that E{g(x,y)} = f(x,y) (3.37) and g(x,y) a ~ J7 n(x,y) (- ) a 3 38 where E{g(x, y)}is the expected value of g(x,y) and <T|^ ^ and <r^ ^ are the variances x x of g{x,y) and n(x,y) respectively. As M increases, the variability of the pixel values at each location (x,y) decreases. Also as M increases, ~g(x,y) approaches f(x,y). 3.2.4 Zero Padding Zero padding refers to the operation of extending a sequence of length N i samples to have a length N2, by adding N2—Ni zero samples to the given sequence. The sequence can be either in the time or the frequency domain and the zeros are normally added at the end of the sequence. Chapter 3. Theoretical Concepts Review 37 Zero padding serves two major purposes: 1. Linear Convolution: to reduce or eliminate the wraparound effects of circular convolution. 2. Interpolation: to get more samples in the other domain. 3.2.4.1 Interpolation with Zero Padding When obtaining the DFT (or IDFT) of a se- quence in one domain with zeros padded, the number of samples produced in the other domain is increased. If zeros are added in the time domain, the record length is increased, however, the sampling rate remains unchanged. Then, the number of samples in the DFT spectrum is increased, but the frequencies covered by the zero-padded DFT output is unchanged. Therefore, the additional points computed by the zero-padded DFT falls in between the original points of the unpadded spectrum. This may alleviate the picket-fence effect. There are two things we need to remember. First, zero-padding does not improve the ability to discriminate between two-closely-spaced sine waves (i.e. resolution), but rather computes in-between frequency points (i.e. spectrum interpolation). Second, the original time samples remain unchanged in the output of the zero-padded DFT. Chapter 4 Single Frequency Intermodulation: Cancellation and Measurements 4.1 Introduction Intermodulation distortion in cable T V network has been the subject of much research. These studies include the calculation of this impairment [18], [19] its reduction involving hardware modifications to certain designs [20], [1] and its automatic detection [2], [21], [3]. 4.2 Characteristics and Causes Let us consider the characteristics of an intermodulation carrier beat and the various effects this impairment has on the television picture. Intermodulation in the form of a picture carrier beat appears on the screen as a series of diagonally parallel lines, across the image. Similar patterns also appear for composite second order beat distortions. Cable companies transmit a wide band multichannel signal of up to eighty channels, to thousands of households through an elaborate network where amplifiers are cascaded about every half mile. Intermodulation implies the mixing of the desired signal with one or more unwanted signals. Impairments such as intermodulation beats are products of amplifiers, overdriven into their non-linear regions. In addition, in the case of a C A T V network, the intermodulation noise is generated in the repeaters, frequency conversion networks and even in the coaxial cables (due to corrosion and other environmental changes) [18]. 4.3 Mathematical Description The effect of an intermodulation beat on a T V signal transmitted in the cable can be modeled as a sinusoid superimposed on the signal. The resulting one-dimensional temporal signal is then mapped on the T V screen, forming a two-dimensional spatial signal (noise picture). The equations characterizing this mapping and the Fourier transform of the noise 38 Chapter 4. Single Frequency Intermodulation: Cancellation and Measurements 39 picture are provided in [3]. It is shown there that the two-dimensional Fourier transform of a beat has four pulses with distinct relationship. In [3], the continuous two-dimensional Fourier transform of the sampled version of the noise picture is derived and studied. This serves well the purpose of revealing the spectrum properties of the intermodulation beats. However, since in our canceling scheme we deal with discrete Fourier transform of parts of digitized pictures, we shall first derive the discrete Fourier transform of an arbitrary portion of the digitized noise picture. First, we will derive the discrete version of the noise picture and then we will obtain the DFT representation of an arbitrary portion of the digitized noise picture. 4.3.1 Discrete Version of a TV Picture The interlaced scanning process of a T V picture is illustrated in Figure 4.1, where X and Y are the width and height of the screen respectively. As mentioned in section 2.2.3, the odd numbered lines are painted first, and after one field (262.5 lines in NTSC system) the even numbered lines are painted. X M Figure 4.1 Interlaced Scanning Intermodulation that consists of a single beat product is represented in the time domain as f(t) = A cos fat + <j> ) t t (4.1) Chapter 4. Single Frequency Intermodulation: Cancellation and Measurements 40 with angular frequency u> . The superposition of this impairment on the victim channel's t signal f (t) results in the sum v fs(t) = f (t) + A cos (unt + <t>t) v (4.2) t When painted on the T V screen, f (t) becomes a two-dimensional spatial function f(x,y). s The variable <f>t is assumed to be zero for simplicity. Next, we explore the relationship between the time variable t and the spatial coordinate (x,y). When (x,y) lies on the odd lines, JL _ l vt = (4.3) + H) + x where v is the scanning speed, A y is the distance between two adjacent horizontal lines, and H is the distance corresponding to horizontal flyback time. When (x,y) lies on the even lines, v t ( M ± ^ = { X + H ) + V + (^A {X + H ) + (4.4) x where Mt is the number of lines per frame (525 lines for NTSC system) and V is the distance corresponding to vertical flyback time. The resulting two-dimensional spatial function, therefore, is „\ - ( K ^ ^ H * ) ) cos f ( r J [ X ' y - ) f" \ ( ( < »l »-\X+H)+$ M + + z)) A C0S Ut o d d H n e s ( A for even hues To find the discrete version of the T V picture, f(n,m), « ^ let us denote the number of samples taken for each line as N and the sampling period as r. Then we have t X - N vr (4.6) x = nvr (4.7) t and = m. The digitized picture can now Let m denotes the m* horizontal line, i.e., be expressed as f(„ J^ m m \ - )-\ fCos( (^i(jV r f)+ r)) W l C O S ( U t t + (^l(N r+f) t r e + ^+nr)) modd, m even ^ Chapter 4. Single Frequency Intermodulation: Cancellation and Measurements 41 for 1 < n < Nt and 1 < m < Mt. This equation can be rewritten as ft n _ \ m ^ ' (COs(L> (n+am-a)) m n tcos (w„(n+am+b)) ' odd, m even ^ ' ' where LOn = UHT , (4.10) - ~^r^ = a (4 n) and b=(M -l)a +— t . (4.12) VT Both a and b can be determined once N is given. t 4.3.2 DFT of a Portion of a Digitized Impairment Picture Let's consider the portion of a digitized noise picture, whose upper left corner is at {n\,m\) of the digitized picture and whose size is N x M as illustrated in Figure 4.1. This subimage can be expressed as r , •'P^ ' ' j cos ( w „ ( n + a m - a + ? i i +orai)) m+mi odd, tcos (w (n+am+b+ni m+mi even +ami)) n 1 ^ ' ' for 0 < n < N — land 0 < m < M — 1 where both M and N are assumed to be even numbers for the D F T simplicity. To obtain the two-dimensional discrete Fourier transform of f (n,m), i.e. F (l,k), we p p can write f (n,m) as p f (n,rn) = ^ 9 p g{n, m) = e + / { n , m (4.14) ) i«n(n+am+d) ( g'{n,m) = -J"n{n+am+d) 4 ( e 1 4 The parameter d varies with respect to the variable m , i.e., ^ f d i = n + a ( m — 1) m+mi odd \d2=ni+ami+b m+mi even i 1 ^ 5 1 ) 6 ) Chapter 4. Single Frequency Intermodulation: Cancellation and Measurements 42 The D F T of g(n,m) is JV-l M - 1 (i,k) G = YY, e ^" v^+T) ( n + a m + d n=0 m=0 M-1 N - l 71 = 0 m=0 1 - e^" l _ (4.18) x£(fc) , - ^ - n ) e where odd m+TTii _|_ ^ J^n{am+d ) -j^mk e 2 e m+mi euen _ M/2-1 ^ J W „ ( 2 a m ' - a m i + a + ( i i ) - i ^ ( 2 m ' - m i + l)fc_|_ e e m'=0 M/2-1 _|_ ^ e >^n(2am'-am +(i2) -i^(2m'-m )fc 1 1 e (4.19) m'=0 M/2-1 X e j(^ i m f c - u , « a m Y^ i) J '(un2a-^k) m e m'=0 / •> \ xe 1— Ju aM P n J 1 _ e -i(^*-2a«„) Here, m + m\ + 1 m (4.20) where [J is the floor operator. Combining the last two equations, we obtain 1_ G{l,k) 1_ pJUnN x i _ -K% - ») l e u jw„aM i _ e->(^*- 2 o w ») (4.21) Similarly, the D F T of g'(n,m) is I _ -ju„N jz ^_ e G'(Lk) = r X -ju aM 77 e n r X (4.22) X ^ j ( ' " ( + ) + ^ ) _|_ - ^ - . ^ j i ( f m i H w a m i ) e - a a dl ;fc e e n Chapter 4. Single Frequency Intermodulation: Cancellation and Measurements 43 Using equations 4.21, 4.22 and 4.14, we obtain the 2-D D F T of f (n,m). p This is expressed as G{l,k) + G'{l,k) 1 - e> » _ 1 ~ 2 u N ]_ _ - > ( $ I - « » ) X [ i _ X e -j(%k-2au ) e n (4.23) -3"nN 1 _ + e l -T^T-. T X 1 _ -;(T7'+<"») e x ^e - ( _ , W n ( _ 1 _ -ju„aM e e - j ( % k + 2 a u » ) ) ^ ) + "-' " je (^ a + d l + f c u (i2 , roii+w e °" ) ] mi The shape of the discrete amplitude spectrum \F (l, k)\ is now studied. We shall show p that it is comprised of four pulses of distinguishing characteristics. The amplitude of G(l,k) is |l _ !<?(/, *)| J»nN\ 1 e i _ -;($'-«») ju aM a n I X i _ -i(^*-2««») e e X 2 x , l _ ( 2 )| sin sin(f/-^)| cos 0 = 2 x C(/) x (4.24) l«n x |sin(^fc-au; )| X B + 4 - * ) - £ * ) x £(jfe) Thus |G(Z, A;) | is a separable function where C(l) varies only in the horizontal direction, and D(k) and E(k) vary only in the vertical direction. Usually 4 ^ is not an integer. Let ~27 h+t (4.25) where h is an integer and e is a real number in the interval |e| < 0.5. Then the denominator of C{1), sin (—1 \N ^) 2J sin - h - e) (4.26) Chapter 4. Single Frequency Intermodulation: Cancellation and Measurements has a global minimum at / = h / = [-^p* + (mod N). {mod N). Thus, the term C(l) 44 has a global maximum at A typical plot of C(l) is shown in Figure 4.2. For the special case e = 0, the nominator of C(l), (4.27) sin(/i7r)| , sin is equal to zero, but the denominator is equal to zero only when / = h limiting value of C(l = h (mod (mod N). The can be shown to be N. That is, in this case C(l) JV)) is an impulse equal to N. To analyze the term D(k), we let Mau> n (4.28) = 9 + t 2TT where g is an integer and £ is a real number in the interval |£| < 0.5. Then the denominator of D(k), sin —k M has a global minimum at k = g Thus, the term D(k) other at k = f + — au> (4.29) n (mod M), and another one at k = 4f + g has two pulses, one centered at k = [ + \\ (mod M a 2 ^" + i j (mod M). (mod M)and the M). -II- Figure 4.2 Variation of e in C(l) As A; varies from 0 to M-1, E(k) <p = ^ ( a + di — d2). That is, E(k) varies from |cos (cp)\ to around |cos (if — ir)\, where varies almost half a cycle over the whole spectrum Chapter 4. Single Frequency Intermodulation: Cancellation and Measurements 45 domain. In the neighborhood of any of the two pulses of D(k), the variation in E(k) is relatively very small, thus the effect of E(k) can be considered as a multiplying constant on one pulse and a different multiplying constant on the other pulse. When E(k) happens to be zero, one of the two pulses will disappear. However, the other pulse at a vertical distance of f - away will get its maximum value since E(k + f ) = |cos (y> — -j^k — -|) | = 1. In general, \G(l,k)\ has two pulses with the same horizontal coordinate but separated by f - indices in the vertical direction. One pulse is at ( [ ^ ^ n ] jy, [ ^ r e ] ^ ) acj other is at ( f i w j j y , [ f + S a w n ] J . where [x] = [x + (mod Y). Y \G'(l,k)\ also has two pulses, one at (\N — ^u ~\ , [M — j^au ] ) n N n M m ^ m e Similarly, and the other at Therefore, the amplitude spectrum of an intermodulation beat has four pulses which are at the above mentioned locations. That is, there are four pulses grouped into two pairs which are symmetrical around the origin of the spectrum domain, and the two pulses in either pair are separated by exactly f indices in the vertical direction. We refer these pulses as intermodulation pulses. Figure 4.3 Amplitude spectrum of an intermodulation beat Chapter 4. Single Frequency Intermodulation: Cancellation and Measurements 46 4.4 DFT of a Line of the Digitized Impairment Picture Single Frequency intermodulation, as indicated previously, can be characterized by the function f(n) = cos [ton) (4.30) where 27rra (4.31) 2 7 r m N where m is an integer in the range of [0, N-1], and (4.32) 2N ~ * ~ 2N In the following, we find the DFT of the sampled version of the cosine function and show that as expected it is not a pair of impulses because of the leakage problem associated with the DFT. We can express this sampled version as follows 1 {cos (ton)} = { - {jun) _|_ e -(jun) e (4.33) Dividing this equation into two parts allows us to derive the DFT of e^n) ^ N-1 9 n=0 N-1 (4.34) 1 _ J(<*N) e _l_ -i(^-H_ c Similarly, the DFT of the sampled second term in 4.33 is c>j -i(wn) j e ^< e _ 1_ -i(»N) e 1- e^ -«) l 1 _ -i(«JV) e 1 _ -i(«-^0 c (4.35) Chapter 4. Single Frequency Intermodulation: Cancellation and Measurements Al Substituting Equations (4.34) and (4.35) into Equation (4.33), 1 _ 1 _ eJ(»N) 9{cos (um)} = — 1 _ e + ->(fcl-«) 1 - -j("N) e e (4.36) W^ 1 Thus the 1-D N-point Fourier transform of the function cos (un) is 1 - e' uN 1- + *X0 = e~' uN (4.37) This equation is characterized by two pulses, as sketched in Figure 4.4, one pulse located ^ + iJ ^ d the o t h e r ^ iV - [If + ij. A Amplitude -ll- -fi -ih ^ -ll- frequency Figure 4.4 Sketch of the 1-D DFT representation of single frequency intermodulation The amplitude spectrum, for instance, at + \\ is K sin yu> W)\ |sin (±u,-f/)| (4.38) where K is a positive constant. By substituting (4.31) into (4.38), we obtain |sin (/-m)-£)| (4.39) From this equation, it can be deduced that \F(l = m)\ = m x{\F(l)\} f From equation 4.39, it can be seen that when £ = 0, \F(l = m ) | = i f Af and (4.40) ^ m ) | = 0, i.e., F(l) is an impulse; when £ = 0, F(Z) is a pulse that spreads out around / = m ; when £ = ± 2 ^ , the pulse is exactly symmetric around m±\. Chapter 4. Single Frequency Intermodulation: Cancellation and Measurements When £ ^ 0, F(l) 48 is not symmetrical. As an example, consider the five pixels centered at / = m. The values of \F(l)\ for these pixels (m-2, m-1, m, m+1, m+2) are respectively equal to A A -|£|)|' A |sin(-*-KQ|' A A |sin(|f|)|' | s i n ( ^ - | f | ) | ' |sin ( f > (4.41) where A = K\sm N£\. Since |£| < ^jy and N is large, the above values can be approximated as A A A i.e. these values are not symmetrical around I = m. denominator decreases, then F(l) A A (4.42) As N increases, A increases and the becomes flatter near / = m as expected. 4.5 Filtering Methods To filter out single frequency intermodulation, we study different methods: FIR linear filtering, interpolation in the frequency domain and non-linear filtering in the space domain such as the median filter. The picture shown as example is shopping.pgm (Figure 4.8) with simulated single frequency distortion added. 4.5.1 FIR Linear Filtering The FIR filter design technique used here utilizes Hamming windows. To remove the impairment caused by the intermodulation phenomena, a notch, bandstop or band rejection filter has been designed. The coefficients of this windowed digital filter are obtained from equations 3.33 and 3.34 after selecting parameters such as the order of the filter 2(M-1) and the frequency width of the filter response. The goal is to design a very narrow notch filter so that when filtering the impairment from the noisy picture, no intended information is lost in the process. The notch filter is applied to the time-domain signal of every line. The overall results show that for the images filtered with the notch filters of order of 2x48 and 2x52 very little presence of noise is seen, however, the presence of artifacts can be observed. Chapter 4. Single Frequency Intermodulation: Cancellation and Measurements 1.1 1 1 1 1 1 1 I 1 1 H 1 1 1 I I TJ 'c CD CO 0.7 r i i 0.6 l\ \\ 1 1 1 4 - - i f" I I - H i i n 1 0.2 l\ T i i i 0.4 1 n i f 1 1 I T i i 0.5 . 1 i i i i i i 1 0.1i i 0.2 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 T I f If v -| "- r "1 "" r 1 ' J . i i 1 1 1 1 1 1 1 1 1 1 j - i ~ 1 0.3 0.4 1 1 1 1 1 1 t 1 1 -•t- i 1 i i — H— 1 1 r h" i 1 1 1 r |_1L. L i 1 1 49 0.5 0.6 I I r i i r i i I i i i i I i i i 0.7 i r ~r i ~r i j i 0.8 0.9 1 Normalized Frequency (rad/s) Figure 4.5 Designed notch filter of order 2N=2x84 4.5.2 Interpolation in the Frequency Domain Let us consider the 1-D D F T representation of line i where i=l,...,512 shown below for the indices between 0 to 80. In this case, the frequencies of the intermodulation fall near the index 53 and 461. It is desired to remove the interference at points P2 and P3 corresponding to the index 53 and 54. This may be accomplished by padding with the appropriate number of zeros until point P2 reaches its maximum amplitude value while the amplitude of P3 decreases, and then replacing the value of P2 by the average of the neighboring points P i and P3. The DFT before averaging resembles that of Figure 5.6 . We might also completely remove the points between P i and P3 (after P2 reaches its maximum), i.e. making their amplitude values equal to zero. This process is done for every line in the image. Because of the picket-fence effect, the algorithm involves zero-padding every line in the image with zeros on each side prior to finding the line's DFT. After padding with 3 zeros, Chapter 4. Single Frequency Intermodulation: Cancellation and Measurements 50 15000 10000H 5000 0 10 20 30 40 50 60 70 80 Figure 4.6 1-D DFT of a typical line of an impaired image it is observed that the noise will fall in one point at its maximum. These peaks fall in the 54th and 466th samples. Chapter 4. Single Frequency Intermodulation: Cancellation and Measurements 1-D DFT of the first line of an impaired image Figure 4.7 1-D DFT of different lines of an impaired image 51 Chapter 4. Single Frequency Intermodulation: Cancellation and Measurements 52 Figure 4.8 Noisy picture shopping.pgm with single frequency intermodulation 4.6 Experimental Filtering Results 4.6.1 Interpolation Figure 4.9 shows the noisy image in Figure 4.8 after the notch filter is applied. The result shows artifacts all over the image. Figure 4.10 and Figure 4.11 show the results after the noisy image is filtered through the interpolation by the one-point removal and by the one-point averaging algorithms respectively, both with three zeros padded. It can be seen that both these algorithms provide good visual results in the sense that most of the single frequency intermodulation is removed. Figure 4.12 and Figure 4.13 illustrate the results from interpolation but this time with eight zeros padded and one point removed and one point averaged respectively. The results are not much different from those obtained by padding three-zeros. Chapter 4. Single Frequency Intermodulation: Cancellation and Measurements 53 Figure 4.11 Noisy picture shopping.pgm after three zeros padded and one point averaged Chapter 4. Single Frequency Intermodulation: Cancellation and Measurements 55 Chapter 4. Single Frequency Intermodulation: Cancellation and Measurements 56 4.6.2 Combination: Interpolation and Alpha-Trimmed Filter A n Alpha-Trimmed filter is applied to the previous results, i.e. following the interpolation with three and eight zeros padding. B y smoothing areas in the image where the remaining noise is visible, better visual pictures are obtained. This combination of interpolation and alpha-trimmed filter provides the best visual results among the tested algorithms for combatting single frequency intermodulation. These results are shown in Figures 4.14, 4.15, 4.16 and 4.17. Figure 4.14 Noisy picture shopping.pgm after three zeros-padded and one point removed followed by the alpha trimmed filter Figure 4.15 Noisy picture shopping.pgm after three zeros-padded and one point averaged followed by the alpha trimmed filter Figure 4.16 Noisy picture shopping.pgm after eight zeros-padded and one point removed followed by the alpha trimmed filter Chapter 4. Single Frequency Intermodulation: Cancellation and Measurements 59 Figure 4.17 Noisy picture shopping.pgm after eight zeros-padded and one point averaged followed by the alpha trimmed filter 4.7 Objective Evaluation To evaluate the effectiveness of the interpolation in the frequency domain methods, we use the signal-to-noise ratio (SNR). The S N R may be obtained from the intensities in the spatial domain, the amplitudes of the 1-D D F T or the 2 - D D F T of the picture. These criteria are dependent on the working chosen domain. 4.7.1 Definitions 4.7.1.1 Spatial Domain Ideally, after correcting/cancelling the noise from a contaminated image, evaluation of this noise is desired. Normally, there are two common criteria for expressing the amount of noise present in an image: the carrier-to-noise ratio (C/N) and the signal-to-noise ratio ( S N R or S/N). Let the contaminated image be s(m,n) and the estimated clean image after interpolation be i(m,n), then for any value of m and n where m=0,...,484 and n=0,...,645; the noise can Chapter 4. Single Frequency Intermodulation: Cancellation and Measurements 60 be found as n(m, n) = s(m, n) — 3"(m, n) (4.43) The total amount of noise energy between two images can be determined as M-l N-1 E E m=0 n=0 i( i) (4.44) ~ s{m,n)] s m n 2 where M by N correspond to the size of the image. The signal-to-noise ratio is E{s (m,n)} 2 SNR = E | n ( m , n) j 2 M-l WW E m=0 SNR M-l N-1 m=0 n=0 WW E N-1 (4.45) s(m,n) E n=0 E [s(m,n)-s(m,n)]' In other cases, the peak SNR is considered as a reference for determining the amount of noise present in a contaminated image. For the case when the intensity of every bit is digitized by an 8-bit digitizer, the peak SNR is defined as / PSNR = 10* log dB 10 1 \ (255)' n M-l y 4.7.1.2 Spectrum Domain V WW E m=0 (4.46) N-1 E (n(m,n)Y n=0 The equations for SNR and PSNR, are used when the pixels intensities are known. These equations can also be expressed in terms of the 2-D D F T and the 1-D DFT. Using Parseval's Theorem, M-l N-1 £ ^ > m=0 2 M-l ( m , n ) = N-1 ^ E E l ^ ) ! u=0 7i=0 <- ) 2 4 47 v=0 the SNR expression, when the 2-D D F T of an image is known, can be written as N-1 MNN E SNR dB = 10 * log10 u=0 M-l V WW E ( WW E E \S{u,v)\ 2 / v=0 N-1 [WW E \N(u,v)\' \ (4.48) Chapter 4. Single Frequency Intermodulation: Cancellation and Measurements 61 The peak SNR can be expressed as \ (255) PSNR ,„ = IQ* log dB 10 M-1 y M 2 (4.49) N-l E A E \N(u,v)\< JJ=0 u=0 / In the case when the 1-D DFT is known for every line in an image, the amount of noise in the frequency domain is calculated for every line. Its power for every line is computed and then added. The SNR is obtained by M-1 / SNR dB i E = 10* log 10 / JV-1 E 15,(01' iijr M-1 \ (4.50) N-l / p=0 1=0 where p is the number of lines in the image. The peak SNR can be expressed as \ PSNR (255) 10 * log 10 dB 2 M-1 / N-l n=0 \ 1=0 (4.51) 4.7.1.3 C/N Ratio Another common way to measure the severity of noise in a contaminated image is via the carrier-to-noise ratio (C/N) which is defined as follows E{s (m,n)} 2 C/N 2 iB = 10* log 1.6 * 10 2 E{n (m,n)} 2 (4.52) In other words, the C/N ratio will be related to the S/N or SNR in the following way: ClN iB = 20 * / f (16) + S/N Ofl dB 10 (4.53) « 4.08d£ + S/N dB To the human eye's sensitivity, 45 dB of C/N or 41 dB of S/N is acceptable. Chapter 4. Single Frequency Intermodulation: Cancellation and Measurements 62 4.8 Evaluation of Experimental Results Evaluation of the SNR's are performed after the methods of interpolation in the frequency domain are applied. These are displayed in Table 4.1. It can be seen here that the interpolation by either removal or by averaging provides good results. In fact, there is only less than 0.12 dB difference between the experimental values and the true SNR value. Picture shopping_int 1-D DFT SNR Measurements Methods Complete removal Averaging 21.77 dB 21.61 dB True value Error 21.73 dB < 0.12 dB Table 4.1 Experimental results for shopping_int The excellent results in the SNR's obtained in the above example suggest that these filtering methods could be used for measuring the noise level of the impairment. The proposed algorithm for measurement is summarized in the following section. 4.9 Algorithm for Measurements Following the evaluation in the previous section, the algorithm that could be used for the measurements of single frequency intermodulation would consist of a follow up of the noise cancellation. After the noise is cancelled, measurements are obtained via the 1-D DFT by the removal or by the averaging methods. The resulting images obtained from the interpolation methods (filtered images) are subtracted from the noisy image. The results would be the estimated noise. SNR measurements are then computed by either using Equation 4.50 or Equation 4.51. Chapter 5 Narrowband Composite Second Order Beats: Cancellation and Measurements 5.1 Characteristics and Causes The last few years have seen cable television systems rebuilding and upgrading their networks in order to have the capacity to carry extra channels. Carrying more channels causes an increase in intermodulation distortion. For a cable system that carries more than fifty channels, composite second order CSO distortion can be the limiting distortion [22]. The CSO distortion may have similar patterns to those of single frequency intermodulation. These are characterized as a series of diagonally parallel lines in the image. The causes for such distortion are amplifiers overdriven into their non-linear regions and possible malfunctioning of some other cable components. For optical fibre system, laser induced clipping caused by the modulating signal going below the laser threshold makes the CSO impairment as the limiting distortion. A composite second order beat is a cluster of hundreds of second order single frequency intermodulation products. A second order product is caused by the second order distortion of amplifiers or lasers in the cable distribution networks and is referred to as the beating of one signal carrier fj with another signal carrier fj resulting in a product sinusoid of frequency fi + fj or fi — fj. The cause and effect of beats have been extensively studied but the automatic detection and correction of these beats have not. CSO beats have the visual effects of diagonal lines over the entire picture. The CSO beat usually has its frequencies cluster around 1.25 M H z from the visual carrier (and less frequently around the 2.50 M H z from the visual carrier). The range of frequencies varies between +15 kHz around the 1.25 MHz, i.e. falls within a 30 kHz slot for the majority of cable systems, however, for systems with harmonically or incremental (interval) related systems, the CSO frequencies range between ± 150 Hz around the 1.25 M H z frequency, i.e. falls within a 300 Hz slot. We shall denote 63 Chapter 5. Narrowband Composite Second Order Beats: Cancellation and Measurements 64 the first type with the 30 kHz range by the term wideband CSO and the second type with the 300 Hz range by the term narrowband CSO. In a cable system, the frequencies of the visual signal carriers are not fixed. They randomly shift within a few kilohertz around their assigned frequencies. This randomness causes the CSO to spread almost ± 15 kHz around its supposed frequency. This randomness in the frequencies of the visual carriers is close enough to allow T V receivers and converters to tune to the required frequencies. There are two channeling systems: the harmonically related carriers (HRC) and the incremental or interval related carriers. 5.1.1 Harmonically Related Carriers (HRC) For harmonically related channels (HRC's) the range of the variations in the visual carrier frequencies are considerably narrowed. For HRC, the picture carrier frequencies are integral multiples of 6 M H z to within 150 Hz as shown in Table A.1, Appendix A. In this system, all visual carriers are phase locked to a 6-MHz master oscillator. The purpose for this is to ensure that all the carriers are harmonically related to 6 M H z even if a shift to the oscillator occurs. As a result, the second- and third-order intermodulation products resulting from any two carriers fall exactly on the visual carrier frequencies causing a reduction of distortion in the television pictures. 5.1.2 Incremental or Interval Related Carriers (IRC) In the IRC system, the cable channels start with frequencies at 55.25 M H z with increments of 6 MHz. 5.2 Mathematical Description The CSO beat can be represented by the sum of sinusoids k (5.1) p=Q with angular frequency LO PS 1.25 M H z from the carrier's frequencies and where k is around P a few hundreds of sinusoids. The superposition of this impairment and the victim signal f (t) t Chapter 5. Narrowband Composite Second Order Beats: Cancellation and Measurements 65 results in the sum shown below (5.2) p=0 5.3 Fourier Model For sampled pictures, the Fourier model of the CSO of a picture line can be expressed as ( * } < Y^ A v COS ( LOpTl) > j (P=O I, 2 = k (5.3) oo p-0 p=0 where n is the number of samples. Similar to section 4.4, we can show that the above equation is equal to 1 1 - 1 _ e' uN 1 - ->(^-«) e when tOpT -juN e (5.4) + = wwith the amplitude spectrum of N sin y a ; (5.5) \™(h»-w)\ Since u lies within + 150 Hz of the 1.25 M H z frequency and since the spacing between p two bins in the D F T is equal to 190467 Hz, the frequencies of the CSO cluster and fall almost totally at the 1.25 MHz. Thus, the 1-D D F T of the CSO appears similar to that of the D F T of a single frequency. Therefore, similar techniques as those utilized for reducing the single frequency impairment are utilized for reducing the CSO impairment. A Amplitude *1 -7/J 300 Hz lh~ 190467 H z frequency Figure 5.1 Sketch of the narrowband CSO Chapter 5. Narrowband Composite Second Order Beats: Cancellation and Measurements 66 5.4 Filtering Methods 5.4.1 Interpolation in the Frequency Domain Algorithm Dealing with a picture's size of 485 by 646 pixels, let us consider the 1-D DFT representation of line i where i=l,...,485 shown below for the D F T bin indices 0 to 130 of line i . The frequencies of the CSO mainly fall at indices 68 and 580 and spread narrowly around these indices (bins). BOOOI -r BO00 Figure 5.2 1-D DFT of the first line of the impaired image: cso_467 It is desired to filter out the frequencies of the CSO interference, which is mainly shown as a spike at point P2 in Figure 5.2. This may be done by smoothing the amplitudes of the frequencies around P2 or by completely removing P , i.e. substituting the value zero 2 for its amplitudes. Figure 5.3 shows a frame of a video sequence impaired by CSO. This video sequence was obtained from the U.S. cable labs testing facility in Alexandria, Virginia. The 2-D D F T of the luminance component of this image is shown in Figure 5.4. The four distinctive spots of this figure correspond to the amplitudes of the CSO impairment, i.e. i f these spots were not present the picture would be free of the CSO impairment. Chapter 5. Narrowband Composite Second Order Beats: Cancellation and Measurements One way to rid the picture of this impairment is to filter out the 4 spots in its 2-DFT (Figure 5.4). This method however is not efficient because of the following two reasons: 1. Obtaining the 2-D DFT requires double the computation load required in obtaining the 1-D DFT 2. of every line of the picture. Obtaining the 2-D DFT requires waiting until the signal of the whole frame has been received, i.e. 1/30 sec while obtaining the 1-D DFT of a line requires a waiting period of only 1/(30x525) sec and this is much faster to execute. 67 Chapter 5. Narrowband Composite Second Order Beats: Cancellation and Measurements Figure 5.3 Noisy picture: CSO_467 68 Chapter 5. Narrowband Composite Second Order Beats: Cancellation and Measurements 5.4.1.1 Importance of Zero-Padding Since the interpolation algorithm is performed in the frequency domain, the 1-D DFT of every line is needed. However, prior to obtaining the DFT of every line, it is needed to have the sequence zero-padded. The purpose for this is to make the maximum amplitude of the impairments appear in the DFT. The DFT values are samples of the Fourier transform and by padding with different number of zeros, the maximum value of the CSO may eventually appear. By filtering out this maximum value, we hope that the majority of the impairment's energy is filtered out while only one index of the true intended information is affected. When the frequencies (or indices) of the true image are affected, artifacts or "ghost" traces may appear in the filtered image. Since the sinusoids of the impairment do not vary, the number of zeros that each line is padded with should not vary from one line to another. The number of zeros needed for padding each line will vary according to the size of the lines. The results of padding with different number of zeros are shown in Figures 5.5, 5.6, 5.7, 5.8 and 5.9. 6000, 1000 Figure 5.5 1-D DFT of the first line with no zeros being padded 70 Chapter 5. Narrowband Composite Second Order Beats: Cancellation and Measurements 6000, 3000 1 2000 Figure 5.6 1-D DFT of the first line with one zero being padded 4000 3000 1000 90 100 Figure 5.7 1-D DFT of the first line with two zeros being padded 71 Chapter 5. Narrowband Composite Second Order Beats: Cancellation and Measurements 60001 5000k K « * ""•"«" «_" at 1 a «h 10 20 30 x 40 50 60 70 80 i 90 100 Figure 5.8 1-D DFT of the first line with three zeros being padded 4000h * U 2000h • • 1000H • 10 1 20 •V/30 40 50 60 rVAyrilJW 70 80 90 100 Figure 5.9 1-D DFT of the first line with four zeros; being padded From these graphs, we can observe that when using a different N for zero-padding, we see the effect of converting a three points noise components into a single point as it is seen when using N=2. On the other hand, when using N=4, the process reverses to three points. 72 Chapter 5. Narrowband Composite Second Order Beats: Cancellation and Measurements 73 It is always better to try to eliminate a one-point noise than three or more noise points for less error. 5.4.1.2 Interpolation by Averaging: Algorithm #1 This consists of finding the peak at point P2 and replacing it by the average of its neighboring point P i and P 3 . 1. Obtain the DFT of every line in the image after performing zero-padding. 2. Average P i and P3 and place that value into P2 keeping the other two values unchanged. 3. Find the inverse D F T of every line. 4. Get rid of the extra samples as a result from zero-padding. The result would be the filtered picture. 5.4.1.3 Interpolation by Complete Removal: Algorithm #2 This algorithm is similar to the interpolation by averaging except instead of averaging the neighboring points of the single peak component after performing the correct zero-padding, it replaces the amplitude of that peak by the value zero. The role of zero-padding is very crucial for both of these algorithms to work. 5.4.2 Alpha-Trimmed Filter The above filtering methods are found to yield very good results. However, the above methods require the computation of the DFT of every line of the image and then its inverse DFT. We have investigated applying spatial filters to filter the CSO impairment. The advantage of using a spatial filter is the computational efficiency. The disadvantage is that blurring or smoothing of the intended information is inevitable. Amongst many median and average-based filters tried, the following non-linear filter gave the best results in that the resulting artifacts and blurring of the intended information were minimal. For every pixel, (i,j), we consider the five pixels i n the window l,j),(i,j),(i+l,j),(i+2,j)]. [(i-2,j),(i- Among these pixels, the maximum (P x) and minimum (Pmin) ma intensity values are found. If the relationship P x ma — Pmin > certain threshold, then the Chapter 5. Narrowband Composite Second Order Beats: Cancellation and Measurements intensity value of the pixel (i,j) remains unchanged. Otherwise, its intensity is replaced by the average intensity value of the 3 remaining pixels, i.e. excluding P m a x and Pmin. This process is repeated for the same pixel but this time it is applied on the vertical window [(i,j-2),(i,j-2),(i,j),(i,j+l),(i,j+2)]. Instead of using a window of size 5 pixels, a window of size 7 may be used, i.e. three neighboring points on each side. In this case, the noise reduction increases but this is at the expense of an increase in the smoothing of the original image. A window size of 5 pixels yields the best trade-off. 5.5 Experimental Filtering Results 5.5.1 Interpolation by Averaging: Results The interpolation in the frequency domain almost completely removes the CSO noise as it can be seen from the 2-D brightness level of the filtered image, Figure 5.11. This algorithm smoothes the noise in such a way that it is hardly visible to the human eye. This can be seen by comparing the images shown in Figures 5.3 and 5.10, the images before and after executing the algorithm. Here, it is observed that the CSO frequencies have almost been eliminated. 74 Figure 5.10 Noisy picture CSO_467 (Figure 5.3) after 1-D DFT interpolation by averaging Chapter 5. Narrowband Composite Second Order Beats: Cancellation and Measurements 76 Chapter 5. Narrowband Composite Second Order Beats: Cancellation and Measurements 77 5 . 5 . 2 Interpolation by Complete Removal: Results Figure 5.12 shows the noisy image in Figure 5.3 filtered by the interpolation by complete removal method. This algorithm eliminates the noise in the cable television pictures in such a way that it is not visible to the human eye. This can be seen again in Figure 5.12 as it is compared to the noisy picture shown in Figure 5.3. The 2 - D D F T of Figure 5.12 is shown in Figure 5.13. Figure 5.12 Noisy picture CSO_467 (Figure 5.3) after 1-D DFT interpolation by complete removal 78 Chapter 5. Narrowband Composite Second Order Beats: Cancellation and Measurements 100 200 300 400 500 Figure 5.13 2-D DFT Brightness level of the supposedly clean image after interpolation by complete removal shown in Figure 5.12 600 Chapter 5. Narrowband Composite Second Order Beats: Cancellation and Measurements 79 5.5.3 Alpha Trimmed Filter: Results The results after using the alpha trimmed filter with 5 and 7 points manipulation are shown in Figures 5.14 and 5.15. These results are not as good as the ones obtained via 1-D D F T interpolation since they show artifacts and blurring in certain areas of the images. Figure 5.14 Noisy picture CSO_463 after 5-points alpha trimmed filtering Chapter 5. Narrowband Composite Second Order Beats: Cancellation and Measurements Figure 5.15 Noisy picture CSO_463 after 7-points alpha trimmed filtering 80 Chapter 5. Narrowband Composite Second Order Beats: Cancellation and Measurements 5.6 Measuring Methods 5.6.1 Interpolation Figure 5.3 shows the image cso_467. This and other pictures of the same video sequence that are used for the SNR measurement test covered in this section contain real narrowband CSO. The methods of interpolation for SNR measurements are the same methods used for narrowband CSO cancellation. The 1-D DFT method consists of obtaining the 1-D DFT for every line in the image. Then, frequencies in the 1-D DFT of every line are filtered by either complete removal or average interpolation. This allows us to isolate the estimated noise in the image by subtracting the filtered image from the noisy image as shown in Figure 5.16. Noisy Image Filtered Image — Estimated Noise Image Figure 5.16 Basic illustration of used approach 5.6.2 Block Averaging This algorithm is based on a traditional method used for picture enhacement called image averaging. Using the image averaging method, we are able to obtain a better estimate of the clean picture. Therefore, the noise obtained by subtracting the filtered image from the noisy image will represent a good estimate of the noise. This will certainly improve our measurements' accuracy. Aspects that are considered in this algorithm are: 1. Determining the minimum number of images in a sequence that are needed to achieve a certain level of accuracy in SNR values. Obviously this method will not be suitable for single noisy frame processing. The possession of single noisy frames will require an alternate method that will still provide less accuracy in measurements. 8_1 Chapter 5. Narrowband Composite Second Order Beats: Cancellation and Measurements 2. 82 Developing a motion detection, i.e. block matching, that can be used to determine the areas where there is no motion. Essentially, we search for the biggest area that contains little or no motion. A requirement for such area is that it should form either a square or a rectangle for fast and simple computation. 3. Performing block averaging for such area in all images in the sequence. This will give us the estimated clean block or subimage. 4. Subtracting these results from the first noisy image in the sequence. This w i l l give us the estimated noise that later can be used in the computation of S N R . We just need to be careful using the correct size of the block or subimage when computing the S N R value. We have used two sequences with real narrowband C S O . The sequences are taken from the movie Indiana Jones, we call them the beach_sequence and the chase_sequence. Some of the images from the beach_sequence were already shown in Figure 5.3. One of the images from the chase_sequence is shown in Figure 5.17. Figure 5.17 Image from the chase_sequence Chapter 5. Narrowband Composite Second Order Beats: Cancellation and Measurements After detecting motion, we select the areas where there is little or no motion. In the beach_sequence, two areas are selected. One falls in the area where the rocks are and the other falls where the sand is. In the chase_sequence, the selection was a little more difficult because of the abundant motion in the image. However, the two areas that are selected are the ones that fall on the clouds and on the grass. Figure 5.18 shows the steps of this algorithm. 83 Chapter 5. Narrowband Composite Second Order Beats: Cancellation and Measurements NOISY SUPPOSEDLY CLEAN • Estimated clean picture after block averaging three frames Portion of cso25663.pgm (Beach_sequence: rocks) NOISE . ...... ... 10 20 30 40 M O W .< »" 50 - 60 70 % 80 90 100 40 80 100 Noise removed &used in SNR measurement calculations Figure 5.18 Steps of the block averaging algorithm 84 Chapter 5. Narrowband Composite Second Order Beats: Cancellation and Measurements 85 5.7 Evaluation of Experimental Results 5.7.1 Interpolation via 1-D DFT: Results The results after measuring SNR values via 1-D DFT by removal and by averaging methods are shown in Table 5.1. It is observed that both interpolation by removal and by averaging provide good SNR measurement results. In both cases, the SNR values are less than 3 dB error. The results obtained from the alpha-trimmed filtering are close to the values from the 1-D D F T methods. Pictures 1-D D F T by 1-D DFT by a-Trimmed a-Trimmed True removal (dB) averaging (5-points (7-points value (dB) (dB) computation) computation) PSNR Error PSNR Error PSNR Error PSNR Error SNR cso_463 34.60 0.40 34.52 0.48 35.44 0.44 34.41 0.59 "35 cso_467 32.20 2.80 32.15 2.85 33.75 1.25 32.43 2.57 "35 Table 5.1 Experimental results of measured Narrowband CSO 5.7.2 Block Averaging: Results The SNR values obtained from averaging blocks with different numbers of images came close to the expected values within 3 dB and i f these values are averaged, the error falls within less than 2dB. This can be clearly seen in the following plot. Chapter 5. Narrowband Composite Second Order Beats: Cancellation and Measurements Plot of images vs. SNR values found by block averaging 50 r 45 Expected SNR values (True SNR values) 20 * beach_sequence (sand portion) + beach_sequence (rock portion) 15 X chase_sequence (clouds portion) o chase_sequence (grass portion) 10 L 5 6 7 Number of images 10 86 Chapter 6 Wideband Composite Second Order Beats: Cancellation and Measurements 6.1 Characteristics and Causes The characteristics of the wideband CSO are similar to the narrowband CSO. Its presence is determined by the diagonal or horizontal lines covering the entire image. They are also caused by amplifiers overdriven into their non-linear regions. 6.2 Mathematical Description Similar to the narrowband CSO, the wideband CSO is represented by h 53 A i cos ( ( w + <*>,•)< + ( & + & ) ) c or h 53 M cos ( u t + (j) ) p v (6.2) where u> = ui + u>i p c with a fixed angular frequencies u> (center frequency of the beat from the visual carrier c frequency) and u^'s are the frequencies of the beat that fall next to co . The number of c sinusoids h is around hundreds of frequencies. The superposition of this impairment and the victim signal ft(t) results in the sum h f(t) = f {t) t + 53 M ° c s (K + wi)t + (<j>c + <f>i)) (6.3) The difference between the two types of CSO (the narrowband and the wideband) is that in the narrowband CSO, the noise frequencies are limited within + 150 hz while for the wideband CSO, these are limited within ± 15 khz. Graphically speaking, the noise frequencies in the narrowband case fall very close to each other and cluster around four points in the 2-D D F T spectrum, while in the wideband case, the frequencies are spread along two lines. These can be seen in the Figures 5.4 and 6.2. 87 Chapter 6. Wideband Composite Second Order Beats: Cancellation and Measurements 6.3 Fourier Model For sampled pictures, the Fourier model of the CSO of a picture line can be expressed as k ^ < 53 A p cos ( LJ n) p [p=0 1, 2 k (6.4) oo A e^^ p + A e-( ^ jw p p=0 [p=0 where n is the number of samples. Similar to section 4.4, we can show that the above equation is equal to F(h - V J -( 4- - " ' " 1 e , Jy ^ (6.5) when u T = wwith the amplitude spectrum of p • N 1-^(01 = sin 53^1 (6.6) y C J |sin | Since u> lies within +15 kHz of the 1.25 M H z frequency and since the spacing between p two bins in the DFT is equal to 190467 Hz, the frequencies of the CSO cluster at the 1.25 MHz. The spreading caused by the presence of several frequency components introduced by the sinusoids is much wider than the narrowband since the spacing is much larger and the noise frequencies are limited within ± 1 5 kHz instead of ± 150 Hz. AAmplitude -f, *1 -If 190467 Hz 15000 H z 7/-*- frequency V Figure 6.1 Sketch of the wideband CSO Chapter 6. Wideband Composite Second Order Beats: Cancellation and Measurements 89 The 2-D D F T expression for the wideband CSO is developed next. Knowing N-1 M-l • 2TT 27T JN n=0 (6.7) JM o m=0 and f{rn,n) = Y^A cos {to {n + m)) p p p-0 h (6.8) h Y, A e( "r( » j + Y n+m p p=0 A e-( r( V ju n+m p p=0 we can find the 2-D D F T by dividing equation (6.7) into two parts, F{k,l) = F (k,l) (6.9) +F (k,l) 1 2 Then, the 2-D D F T of the first term becomes Fi{Kl) h . N-1 M-l h = E f EEE " eW 'p(n-f-m)) p=0 n=0 m=0 -j-§-nl E e p=0 \n=0 1_ £ = Et p=0 p p=0 = E? E E " p=0 -j-^mh e ^ro=0 e » M - W 1 _ £ e 7 ' K M - ^ f c ) p=0 1_ £ 1 _ £ e»K-*0 (6.10) eJ(»>-W p-0 Et p=0 1- £ 1 _ eJ'K^) i{upM) £ e p=0 i _ £ c ;(«,-*0 i _ £ p=0 j( p-W u e p-0 Similarly, 1_ g C ->K^) i _ j2 -j(<»pM) e p-0 p=0 i _ £ -;(«,-so e i _ £ p=0 e -;K-&*) (6.11) Chapter 6. Wideband Composite Second Order Beats: Cancellation and Measurements 90 Then, the 2-D DFT representation of the wideband CSO can be expressed as the sum of Fi(k,l) and F (k,l) . That is 2 F(k,l) = 1 _ J2 e^ " "> w N p=0 ,p=0 1 _ e- '( P ) ? i(»p-¥ ) £ l e E ,p=0 u k p=0 e-J'M) 1_ £ t 1 _ M ei( "-% ) 1_ £ (6.12) 1 - £ ri + w p=0 p=0 A J2 1 _ £ e - i ( « P - ^ l ) p=0 p=0 and its graphical representation is shown in Figure 6.2. e - i ^ p M ) Chapter 6. Wideband Composite Second Order Beats: Cancellation and Measurements 91 Chapter 6. Wideband Composite Second Order Beats: Cancellation and Measurements 92 6.4 Filtering Methods Wideband CSO was simulated and added to clean pictures. In this filtering test, we have three images: julie.pgm, meeting.pgm and drink.pgm. The noisy pictures with simulated CSO of 35dB are shown in Figures 6.3, 6.4 and 6.5. The methods that are used here consist of the Alpha Trimmed Median Filter, Interpolation in the frequency domain using the 1-D DFT by both removal and averaging and a combination of these two methods. The description of these methods have been described in the previous chapter. Chapter 6. Wideband Composite Second Order Beats: Cancellation and Measurements Figure 6.4 Noisy picture julie.pgm with wideband C S O of 35 dB 93 Chapter 6. Wideband Composite Second Order Beats: Cancellation and Measurements Figure 6.5 Noisy picture meeting.pgm with wideband CSO of 35 dB 94 Chapter 6. Wideband Composite Second Order Beats: Cancellation and Measurements 95 6.5 Experimental Filtering Results 6.5.1 1-D DFT Interpolation: Results Results after filtering the images by interpolation are shown in Figures 6.6, 6.7 and 6.8. In all these three images, one zero has been padded and three or four points removed. The results are not quite satisfactory even though most of the noise has been removed. The presence of artifacts and some remaining noise are quite visible in every image. The cause for the artifacts is that some of the information in the image has been lost during the filtering process. Figure 6.6 Filtered drink.pgm with one zero-padded and three points removed Figure 6.8 Filtered meeting.pgm with one zero-padded and three points removed Chapter 6. Wideband Composite Second Order Beats: Cancellation and Measurements 97 6.5.2 Alpha Trimmed Filter: Results As an alternative method to interpolation in the frequency domain, we use the alphatrimmed filter using 5 or 7 point window computation. A visual comparison between Figures 6.9, 6.10,6.11 and the ones obtained by the interpolation method indicates that the alpha trimmed filter shows slightly better results than the interpolation method. Figure 6.9 Julie.pgm after the alpha-trimmed filter (7-points window) Figure 6.11 Meeting.pgm after the alpha-trimmed filter (7-points window) Chapter 6. Wideband Composite Second Order Beats: Cancellation and Measurements 99 6.5.3 Combination: Results Combining both of the previous methods gives another alternative to combat wideband CSO. The results are much better than the ones obtained by each method. This approach allows us to eliminate most of the noise by interpolation and then smoothing the remaining noise and artifacts. Figure 6.12 Picture processed by interpolation in the frequency domain Chapter 6. Wideband Composite Second Order Beats: Cancellation and Measurements 100 Figure 6.13 Picture processed by interpolation followed by the 5-points window alpha-trimmed filter Chapter 6. Wideband Composite Second Order Beats: Cancellation and Measurements 101 6.6 Measuring Methods The pictures that are used for this test are original clean pictures with added simulated wideband CSO noise. The images are: julie.pgm, meeting.pgm and drink.pgm. The methods that were used here follow the interpolation in the frequency domain using both the 1-D DFT and 2-D D F T approaches. As the narrowband CSO case, noise is estimated by subtracting the estimated clean image from the noisy image. There are five approaches used in the attempt to measure wideband CSO that are presented next. 6.6.1 1-D DFT Algorithm 1. Obtain the 1-D D F T of every line in the image. 2. Perform interpolation by averaging resulting into the filtered image. 3. Subtract the filtered image from the noisy image to estimate the noise in the image. 4. Find the average power of the estimated noise in the image. 5. Find the peak SNR according to equation 4.51. 6.6.2 2-D DFT Algorithm: Pattern #1 1. Find the 2-D D F T of the noisy image. 2. Remove one or two lines as shown in Figure 6.14. This will result into the filtered image. 3. Subtract the filtered image from the noisy image to estimate the noise. 4. Find the average power of the estimated noise in the image. 5. Find the peak SNR according to equation 4.49. Chapter 6. Wideband Composite Second Order Beats: Cancellation and Measurements 102 O Lines whe re the peaks fall Figure 6.14 Illustration of the 2-D DFT algorithm: pattern #1 6.6.3 2-D DFT Algorithm: Pattern #2 1. Find the 2-D DFT of the noisy image. 2. Remove two lines as shown in Figure 6.15. This pattern prevents the removal of information that normally falls near the DC component of the image, i.e. around the f x axis. 3. Subtract the filtered image from the noisy image to estimate the noise. 4. Find the average power of the estimated noise in the image. 5. Find the peak SNR according to equation 4.49. 50 pixels 50 pixels{ o } 5 0 pixels D < 50 pixels where A, B, C and D are areas being removed Figure 6.15 Illustration of the 2-D DFT algorithm: pattern #2 Chapter 6. Wideband Composite Second Order Beats: Cancellation and Measurements 103 6.6.4 2-D DFT Algorithm: Pattern #3 The steps are the same as in Pattern #2 except that here, we take into account the thermal noise. The reason for that is to improve accuracy in our measurements. Thermal noise is defined as the energy generated as a result of random motions of electrons within the amplification system of the cable network. In terms of network components, thermal noise performances can be improved by using low noise amplifier technologies [9]. In this algorithm, thermal noise is taken into consideration when calculating its estimate. The estimated value is found by averaging the thermal noise located at high frequencies where the effect of the impairment is minimal. 6.6.5 2-D DFT Algorithm: Pattern #4 1. Divide the image into even sizes of subimages, i.e. 8 x 8 . 2. Find the 2-D D F T for every block or subimage. 3. Interpolate the points that may retain most of the noise resulting into the estimated clean subimage. 4. Subtract the estimated clean subimage from the noisy subimage, isolating the estimated noise. 5. Find the average power of the noise for every subimage. 6. Find the peak SNR according to equation 4.49 for every subimage. 7. Select the maximum value of the peak SNR since this will reflect the removal of less information. For this algorithm, several tests are performed by dividing the image into 4096 of 8x8, 1024 of 16x16, 256 of 32x32, 64 of 64x64, 16 of 128x128 and 4 of 256x256 subimages. Chapter 6. Wideband Composite Second Order Beats: Cancellation and Measurements Dividing the image of size 512x512 into 16 subimages of size 128x128 Figure 6.16 Illustration of the 2-D DFT algorithm: pattern #4 104 Chapter 6. Wideband Composite Second Order Beats: Cancellation and Measurements 6.7 Experimental Measurements Results 6.7.1 Results of the 1-D DFT Algorithm True Image: julie.pgm Image: drink.pgm Image meeting .pgm Measured Error Measured Error Measured Error SNR (dB) (dB) SNR (dB) (dB) SNR (dB) (dB) 65.66 45.19 20.07 49.27 16.39 42.83 22.83 59.64 45.16 14.48 49.11 10.53 42.80 16.84 56.11 45.07 11.04 48..82 7.29 42.73 13.38 53.61 44.94 8.67 48.44 5.17 42.64 10.97 51.68 44.77 6.91 47.99 3.69 42.52 9.16 50.09 44.55 5.54 47.49 2.6 42.38 7.71 48.75 44.31 4.44 46.97 1.78 42.21 6.54 47.59 44.03 3.56 46.43 1.16 42.02 5.57 46.57 43.74 2.83 45.89 0.68 41.82 4.75 45.66 43.43 2.23 45.35 0.31 41.61 4.05 44.83 43.10 1.73 44.82 0.01 41.38 3.45 41.57 41.43 0.14 42.43 -0.86 40.13 1.44 39.21 39.82 -0.61 40.43 -1.22 38.86 0.35 35.83 37.08 -1.25 37.33 -1.50 36.50 -0.67 33.40 34.89 -1.49 35.01 -1.61 34.50 -1.10 31.50 33.10 -1.60 33.16 -1.66 32.83 -1.33 29.95 31.60 -1.65 31.63 -1.68 31.39 -1.44 28.63 30.31 -1.68 30.32 -1.69 30.15 -1.52 value SNR (dB) Table 6.1 Experimental results for measurements using 1-D DFT interpolation by averaging 105 Chapter 6. True Wideband Composite Second Order Beats: Cancellation and Measurements Image: julie.pgm Image: drink.pgm Image meeting .pgm Measured Error Measured Error Measured Error SNR (dB) (dB) SNR (dB) (dB) SNR (dB) (dB) 65.66 46.86 18..80 49.52 16.14 44.97 20.69 59.64 46.87 12.77 49.47 10.17 44.97 20.69 56.11 46.84 9.27 49.34 6.77 44.93 11.18 53.61 46.76 6.85 49.12 4.49 44.86 8.75 51.68 46.63 5.05 48.84 2.84 44.76 6.92 50.09 46.46 3.63 48.51 1.58 44.63 5.46 48.75 46.26 2.49 48.13 0.62 44.48 4.27 47.59 46.02 1.57 47.72 -0.13 44.30 3.29 46.57 45.76 0.81 47.29 -0.72 44.11 2.46 45.66 45.48 0.18 46.85 -1.19 43.90 1.76 44.83 45.18 -0.35 46.41 -1.58 43.68 1.15 41.57 43.58 -2.01 44.27 -2.7 42.43 -0.86 39.21 41.99 -2.78 42.39 -3.18 41.13 -1.92 35.83 39.23 -3.40 39.36 -3.53 38.71 -2.88 33.40 37.02 -3.62 37.06 -3.66 36.67 -3.27 31.50 35.22 -3.72 35.21 -3.71 34.96 -3.46 29.95 33.70 -3.75 33.68 -3.73 33.51 -3.56 28.63 32.40 -3.77 32.37 -3.74 32.25 -3.62 106 value SNR (dB) Table 6.2 Experimental results of measurements using the 1-D DFT interpolation by removal Chapter 6. Wideband Composite Second Order Beats: Cancellation and Measurements 6.7.2 Results of the 2-D DFT Algorithm — Pattern #1 True Image: julie.pgm Image: drink.pgm Image:meeting.pgm Measured Error Measured Error Measured Error SNR (dB) (dB) SNR (dB) (dB) SNR (dB) (dB) 65.66 43.99 21.67 47.10 18.56 42.44 23.22 59.64 43.93 15.71 46.97 12.67 42.38 17.26 56.11 43.82 12.29 46.73 9.38 42.28 13.83 53.61 43.66 9.95 46.42 7.19 42.15 11.46 51.68 43.46 8.22 46.03 5.65 41.99 9.69 50.09 43.22 6.87 45.60 4.49 41.80 8.29 48.75 42.96 5.79 45.14 3.61 41.59 7.16 47.59 42.66 4.93 44.66 2.93 41.36 6.23 46.57 42.35 4.22 44.17 2.40 41.11 5.46 45.66 42.03 3.63 43.68 1.98 40.85 4.81 44.83 41.70 3.13 43.20 1.63 40.58 4.25 41.57 40.00 1.57 40.94 0.63 39.17 2.40 39.21 38.39 0.82 39.01 0.20 37.77 1.44 37.36 36.95 0.41 37.37 -0.01 36.47 0.89 35.83 35.67 0.16 35.97 -0.14 35.28 0.55 34.53 34.52 0.01 34.75 -0.22 34.21 0.32 33.40 33.50 -0.10 33.67 -0.27 33.23 0.17 32.40 32.57 -0.17 32.71 -0.31 32.35 0.05 value SNR (dB) Table 6.3 Experimental results of measurements using the 2-D DFT algorithm: pattern #1 107 Chapter 6. Wideband Composite Second Order Beats: Cancellation and Measurements 6.7.3 Results of the 2-D DFT Algorithm — Pattern #2 True Image: julie.p gm Image: drink.pgm Measured Error Measured Error Measured Error SNR (dB) (dB) SNR (dB) (dB) SNR (dB) (dB) 65.66 51.77 13.89 52.95 12.71 48.35 17.31 59.64 51.43 8.21 52.45 7.19 48.16 11.48 56.11 50.88 5.23 51.72 4.39 47.86 8.25 53.61 50.19 3.42 50.87 2.74 47.48 6.13 51.68 49.43 2.25 49.97 1.71 47.03 4.65 50.09 48.65 1.44 49.07 1.02 46.54 3.55 48.75 47.87 0.88 48.20 0.55 46.03 2.72 47.59 47.12 0.47 47.38 0.21 45.50 2.09 46.57 46.39 0.18 46.60 -0.03 44.97 1.60 45.66 45.70 -0.04 45.87 -0.21 44.45 1.21 44.83 45.05 -0.22 45.18 -0.35 43.94 0.89 41.57 42.25 -0.68 42.30 -0.73 41.60 -0.03 39.21 40.07 -0.86 40.08 -0.87 39.64 -0.43 37.36 38.30 -0.94 38.30 -0.94 37.99 -0.63 35.83 36.82 -0.99 36.81 -0.98 36.59 -0.76 34.53 35.55 -1.02 35.53 -1.00 35.36 -0.83 33.40 34.43 -1.03 34.42 -1.02 34.28 -0.88 32.40 33.44 -1.04 33.43 -1.03 33.32 -0.92 Image:meeting.pgm value SNR (dB) Table 6.4 Experimental results of measurements using the 2-D DFT algorithm: pattern #2 108 Chapter 6. Wideband Composite Second Order Beats: Cancellation and Measurements 109 6.7A Results of the 2-D DFT Algorithm — Pattern #3 True Image: julie.pgm Image: drink.pgm Image :meeting.pgm Measured Error Measured Error Measured Error SNR (dB) (dB) SNR (dB) (dB) SNR (dB) (dB) 65.66 52.12 13.54 53.33 12.33 48.55 17.11 59.64 51.75 7.89 52.79 6.85 48.34 11.30 56.11 51.15 4.96 52.01 4.10 48.03 8.08 53.61 50.43 3.18 51.10 2.51 47.63 5.98 51.68 49.63 2.05 50.15 1.53 47.17 4.51 50.09 48.81 1.28 49.22 0.87 46.67 3.42 48.75 48.01 0.74 48.32 0.43 46.14 2.61 47.59 47.23 0.36 47.48 0.11 45.60 1.99 46.57 46.49 0.08 46.68 -0.11 45.06 1.51 45.66 45.78 -0.12 45.94 -0.28 44.53 1.13 44.83 45.12 -0.29 45.24 -0.41 44.01 0.82 41.57 42.29 -0.72 42.33 -0.76 41.64 -0.07 39.21 40.09 -0.88 40.10 -0.89 39.66 -0.45 37.36 38.31 -0.95 38.31 -0.95 38.01 -0.65 35.83 36.83 -1.00 36.82 -0.99 36.60 -0.77 34.53 35.55 -1.02 35.54 -1.01 35.37 -0.84 33.40 34.44 -1.04 34.43 -1.03 34.29 -0.89 32.40 33.45 -1.05 33.44 -1.04 33.33 -0.93 value SNR (dB) Table 6.5 Experimental results of measurements using the 2-D DFT algorithm: pattern #3 Chapter 6. Wideband Composite Second Order Beats: Cancellation and Measurements 110 6.7.5 Results of the 2-D DFT Algorithm — Pattern #4 (16x16 and 32x32) True Image: julie.pgm Image: drink.pgm Image:meeting.pgm Measured Error Measured Error Measured Error SNR (dB) (dB) SNR (dB) (dB) SNR (dB) (dB) 65.66 50.47 15.19 50.25 15.41 50.08 15.58 59.64 50.20 9.44 50.37 9.27 49.99 9.65 56.11 49.68 6.43 50.42 5.69 49.70 6.41 53.61 49.76 3.85 49.88 3.73 49.24 4.37 51.68 49.60 2.08 49.71 1.97 48.66 3.02 50.09 49.21 0.88 49.33 0.76 48.00 2.09 48.75 48.64 0.11 48.79 -0.04 47.38 1.37 47.59 47.96 -0.37 48.15 -0.56 47.08 0.51 46.57 47.22 -0.65 47.60 -1.03 46.72 -0.15 45.66 46.46 -0.80 47.38 -1.72 46.31 -0.65 44.83 45.98 -1.15 47.09 -2.26 45.87 -1.04 44.07 45.51 -1.44 46.75 -2.68 45.41 -1.34 43.38 45.04 -1.66 46.36 -2.98 44.94 -1.56 42.73 44.57 -1.84 45.95 -3.22 44.46 -1.73 42.13 44.11 -1.98 45.51 -3.38 43.99 -1.86 41.57 43.78 -2.21 45.07 -3.50 43.53 -1.96 41.05 43.44 -2.39 44.63 -3.58 43.08 -2.03 40.55 43.10 -2.55 44.18 -3.63 42.63 -2.08 value SNR (dB) Table 6.6 Experimental results of measurements using the 2-D DFT algorithm: pattern #4 (16x16) Chapter 6. Wideband Composite Second Order Beats: Cancellation and Measurements True Image: julie.pgm 111 Image: drink.pgm Image:meeting.pgm Measured Error Measured Error Measured Error SNR (dB) (dB) SNR (dB) (dB) SNR (dB) (dB) 65.66 52.42 13.24 50.98 14.68 50.54 15.12 59.64 52.13 7.51 50.71 8.93 50.43 9.21 56.11 51.47 4.64 50.28 5.83 50.10 6.01 53.61 50.59 3.02 49.65 3.96 49.60 4.01 51.68 49.61 2.07 48.91 2.77 48.97 2.71 50.09 48.62 1.47 48.10 1.99 48.28 1.81 48.75 47.66 1.09 47.37 1.38 47.56 1.19 47.59 46.75 0.84 46.91 0.68 46.83 0.76 46.57 46.05 0.52 46.41 0.16 46.12 0.45 45.66 45.47 0.19 45.90 -0.24 45.44 0.22 44.83 44.92 -0.09 45.38 -0.55 44.99 -0.16 44.07 44.39 -0.32 44.87 -0.80 44.55 -0.48 43.38 43.90 -0.52 44.36 -0.98 44.11 -0.73 42.73 43.43 -0.70 43.86 -1.13 43.68 -0.95 42.13 42.97 -0.84 43.37 -1.24 43.26 -1.13 41.57 42.53 -0.96 42.91 -1.34 42.85 -1.28 41.05 42.11 -1.06 42.45 -1.40 42.45 -1.40 40.55 41.71 -1.16 42.01 -1.46 42.06 -1.51 value SNR (dB) Table 6.7 Experimental results of measurements using the 2-D DFT algorithm: pattern #4 (32x32) 6.8 Observations from the Measurement Results 6.8.1 1-D DFT Interpolation Table 6.1 presents the results of the SNR measurements obtained after using the 1-DFT interpolation by averaging algorithm. As it is observed, the use of this algorithm can provide SNR measurements values with less than 2 dB error for wideband CSO of 44.83 dB for Chapter 6. Wideband Composite Second Order Beats: Cancellation and Measurements julie.pgm, of 48.75 dB for drink.pgm and of 41.57 dB for the meeting.pgm image. Since these images differ from each other in motion and color contrast, we can safely deduce that this algorithm can provide good SNR measurements up to 41.57 dB with less than 2 dB error. Table 6.2 presents an alternative method, interpolation by removal. These results show that this algorithm can provide good results within 2 dB error when the wideband CSO is less than 45.66 dB. In the three images, level of SNR measurements are 47.59 dB for julie.pgm, 50.09 dB for drink.pgm and 45.66 dB for the meeting.pgm image. 6.8.2 2-D DFT — Pattern #1 It can be observed from Table 6.3 that this algorithm can provide good results for up to 39.21 dB wideband CSO. The level of SNR measurements that the images julie.pgm, drink.pgm and meeting.pgm can be measured up to are 41.57 dB, 45.66 dB and 39.21 dB respectively. 6.8.3 2-D DFT — Pattern #2 This algorithm has offered a major improvement for wideband CSO SNR measurements. It can be observed from Table 6.4 that it can measure up to 50 dB. The level of SNR measurements that the images julie.pgm, drink.pgm and meeting.pgm can be measured up to are 50.09 dB, 51.68 dB and 47.59 dB respectively. 6.8.4 2-D DFT — Pattern #3 A n attempt to improve even further the SNR measurements obtained from pattern #2, this algorithm shows that taking into account the thermal noise can only make some improvements in the SNR measured values by giving less error. As in pattern #2, the level of SNR measurements that these images can be measured up to by using this algorithm are 50.09 dB, 51.68 dB and 47.59 dB for the julie.pgm, drink.pgm and meeting.pgm respectively. 112 Chapter 6. Wideband Composite Second Order Beats: Cancellation and Measurements 6.8.5 2-D DFT — Pattern #4 The level of SNR measurements values by using a size of 16x16 subimages are 50.09 dB, 51.68 dB and 48.75 dB for julie.pgm, drihk.pgm and meeting.pgm respectively. When using a size of 32x32 subimages, the results do not reflect any improvement. In fact, the level of SNR measurements is 50.09 dB for the three images. 6.8.6 Final Observations According to the results shown for the different patterns used for SNR measurements of wideband CSO, it can be observed that patterns #2, #3 and #4 are acceptable since they can measure SNR's values up to 52 dB within less than 2dB error. 113 Chapter 7 Discussion 7.1 Summary As technology improves, customers' expectations become higher every day. With such technologies as the Super-VHS video recording and laserdisc player, customers compare these against the quality of the video received over the cable system. Distortions in the image are mainly caused by the nonlinearity of the underground cascaded amplifiers in the cable TV system. Moreover, the addition of new channels on the congested transmission system will increase present interference levels or create new interferences causing picture quality degradation. There are two approaches for combatting impairments. First, these can be prevented by modifying hardware components in the cable TV system such as amplifiers, converters among others. The other approach is to deal with the distortion itself through the use of image processing and filtering techniques. The idea of this approach is, if not to totally cancel the distortion, to reduce it until it becomes less visible to the human eye. This thesis deals with the second approach that consists of reducing/cancelling and measuring impairments such as the single frequency intermodulation, narrowband and wideband CSO (Composite Second Order) distortion. This would complement the automatic detection for a complete monitoring process of impairments. Our approach wouldfitin an automatic system since it employs non-intrusive methods that do not involve the use of a test signal and hence, no interruption in the transmission would be needed. 7.2 Conclusions 7.2.1 On Cancellation Several algorithms that cancel and reduce the single frequency intermodulation, narrowband and wideband CSO have been presented. Some of these algorithms are applied in the 114 Chapter 7. Discussion 115 frequency domain using the discrete Fourier transform while others are applied in the spatial domain. For these impairments, the algorithms used for cancellation/reduction are based on interpolation in the frequency domain, a type of median filter and a filter that combines these two methods. The results of the latter algorithm show more noise reduction than when using the methods individually. In the case of single frequency intermodulation and the wideband CSO, the combination method provides the best visual results while for the narrowband CSO, the interpolation in the frequency domain reduces the impairments considerably. 7.2.2 On Measurements For measurements, the algorithm used for single frequency intermodulation is based on the interpolation in the frequency domain. This provides reasonable results since it gave SNR measurements within less than 0.5 dB error. The algorithms used for narrowband CSO measurements are based on the interpolation algorithm using the 1-D DFT, and the block averaging. In both cases the SNR measurements are within less than 3 dB. The block averaging algorithm provides better results than the first method giving a an error of less than 2 dB. For the wideband CSO, the algorithms used for measurements consist of 1-D D F T interpolation, and 2-D DFT features extraction. The results from the 2-D D F T are better than the 1-D DFT approach since they can measure up to 50-53 dB of SNR within an error of less than 2 dB. 7.3 Future Work Work on the quiet line method is needed so that comparison between proposed and existing methods could be established. Experimental testing with more real noisy images for all cases should be performed in both cancellation/reduction and measurements. Bibliography [1] D. Large, "Reducing distortions using video timing techniques," Communications Engineering and Design, vol. 15, Iss. 7, pp. 114-119, June 1989. [2] R. Gresseth, "The Automatic Recognition of Intermodulation, Hum and Snow Noise in Cable Television Systems," Master's thesis, The University of British Columbia, 1992. [3] R. Ward and Q. Zhang, "Automatic Identification of Impairments Caused by Intermodulation Distortion In Cable Television Pictures," IEEE Transactions on Broadcasting, vol. 38, no. 1, pp. 60-68, Mar. 1992. • [4] P. Mambo and D. Coll, "Perceived Picture Quality in C A T V Systems with Impairments," IEEE Transactions on Cable Television, vol. CATV-4, no. 1, pp. 10-16, Jan. 1979. [5] D. M. Moloney, "Improving picture quality," Communications Engineering and Design, vol. 16, Iss. 8, pp. 60-65, July 1990. [6] J. N. Slater, Cable Television Technology. Chichester, West Sussex, England: Ellis Horwood Limited, 1988. [7] J. Salo, Y. Neuvo, and V. Hameencho, "Improving T V Picture Quality with LinearMedian Type Operations," IEEE Transactions on Consumer Electronics, vol. 34, no. 3, pp. 373-379, Aug. 1988. [8] T. Dennis, "Nonlinear temporal filter for television picture noise reduction," IEE Proc, vol. 127, Pt. G, no. 2, pp. 52-56, Apr. 1980. [9] K. Poirier, "Inside a C A T V amplifier," Communications Engineering Design, vol. 15, Iss. 7, pp. 64-76, June 1989. [10] R. Ward and P. Shi, "Analysis and Detection of Composite Triple Beats," CCTA 35th Annual Convention and Cablexpo, pp. 103-109, May 31-June 3, 1992. [1.1] B. L. Jones and J. A. Turner, "Subjective Assessment of Cable Impairments On Television Picture Quality," IEEE Transactions on Consumer Electronics, vol. 38, no. 4, pp. 850-861, Nov. 1992. [12] R. A. Bednarek, "On Evaluating Impaired Television Pictures by Subjective Measurements," Transactions on Broadcasting, vol. BC-25, no. 2, pp. 41-46, June 1979. 116 Bibliography 117 [13] B. Caron, "Facility for subjective evaluation of advanced television in North America," Fourth International Conference on Television Measurements, pp. 6-12, 20-21 June 1991. [14] B. Osborne, "The Assessment of Picture Quality on Cable Television Systems by Means of Engineering Audits," IEEE Transactions on Cable Television, vol. CATV-2, no. 2, pp. 95-98, Apr. 1977. [15] D. Wood, "Recent developments in picture quality evaluation," Fourth International Conference on Television Measurements, pp. 1-5, 20-21 June 1991. [16] C. Van Rensburg, G. D. Jager, and A. Curie, "Measurement of Signal to Noise Ratio of a Television Broadcast Picture," IEEE Transactions on Broadcasting, vol. 37, no. 2, pp. 35^13, June 1991. [17] E. O. Brigham, The Fast Fourier Transform and Its Applications. New Jersey, USA: Prentice-Hall, Inc., 1988. [18] R. Joshi, J. C. Arnbak, and R. Prasad, "A method for intermodulation noise calculations in a cable television network with HD-MAC, PAL and F M radio signals," IEEE Transactions on Broadcasting, vol. 38, no. 3, pp. 177-186, Sept. 1992. [19] K. Stokke, "Noise and intermodulation in cable distribution networks," EBU Technical Review, vol. no. 251, pp. 50-54, Spring 1992. [20] B. Arnold, "Third Order Intermodulation Products In A C A T V System," IEEE Transactions on Cable Television, vol. CATV-2, no. 2, pp. 67-80, Apr. 1977. [21] R. Ward and P. Shi, "Automatic Recognition of Intermodulation in Cable Television Pictures," IEEE Transactions on Broadcasting, vol. 39, pp. 318-326, Sept. 1993. [22] N. J. Slater and D. J. McEwen, "Composite Second Order Distortions," pp. 129-134. [23] J. M. Hood and R. B. Poirier, "Perceptibility of Composite Triple Beat," IEEE Transactions on Cable Television, vol. CATV-5, no. 3, pp. 117-123, July 1980. [24] J. Lourens, "Objective image impairment detection using image processing," Fourth International Conference on Television Measurements, pp. 13-21, June 1991. [25] B. Grob, Basic Television and Video Systems. New York, USA: McGraw Hill, Inc., 5th. ed., 1984. [26] P. Shi, R. Ward, and Q. Zhang, "Automatic recognition of intermodulation beat products in cable television pictures," IEEE Conference on Circuits and Systems, pp. 1660-1663, May 1992. 118 Bibliography [27] Q. Zhang and R.K.Ward, "Automatic monitoring of cable tv pictures," ICASSP, pp. I l l 549-552, M a r . 1992. [28] E . R. Bartlett, Cable Television Technology and Operations: HDTV and NTSC Systems. New York, U S A : M c G r a w - H i l l Publishing Company, 1990. [29] G . Bergland, " A guided tour of the fast Fourier transform," IEE Spectrum, pp. 41-52, July 1969. [30] H . Blinchikoff and A . I. Zverev, Filtering in the Time and Frequency Domains. N e w York, U S A : John Wiley & Sons, Inc., 1976. [31] J. V . Candy, Signal Processing: The Modern Approach. N e w York, U S A : M c G r a w - H i l l , 1988. [32] E . R. Dougherty and J. Astola, An Introduction to Nonlinear Washington, U S A : SPIE Optical Engineering Press, 1994. Image Processing. [33] E . R. Dougherty and C . R. Giardina, Image Processing - Continuous to Discrete, Volume 1: Geometric, Transform and Statistical Methods. Englewood Cliffs, N e w Jersey, U S A : Prentice-Hall, 1987. [34] R. C . Gonzales and R. E . Woods, Digital Image Processing. Addison-Wesley Publishing Company, 1992. [35] T. Huang, Picture Processing and Digital Filtering. N e w York, U S A : Springer-Verlag, 1979. [36] T. Huang, Two-Dimensional Digital Signal Processing II: Transforms and Median Filters. N e w York, U S A : Springer-Verlag, 1981. [37] L . B . Jackson, Digital Filters and Signal Processing. K l u w e r Academic Publishers, 1989. [38] A . K . Jain, Fundamentals of Digital Image Processing. N e w Jersey, U S A : Prentice- H a l l , 1989. [39] S. K . M i t r a and J. F. Kaiser, Handbook for Digital Signal Processing. N e w York, U S A : John Wiley & Sons, Inc., 1993. [40] A . V . Oppenheim and R. W . Schafer, Discrete-Time Signal Processing. N e w Jersey, U S A : Prentice-Hall, Inc., 1989. [41] J. G . Proakis and D . G . Manolakis, Digital Signal Processing: Principles, Algorithms, and Applications. N e w York, U S A : Macmillan Publishing Company, 2nd. ed., 1992. 119 Bibliography [42] J. Slater, "Cable Television: Part 1," Electronics pp. 20-24, A p r . 1990. Today International, vol. 19, Iss. 4, [43] J. Slater, "Cable Television: Part 2," Electronics Today International, vol. 19, Iss. 5, pp. 13-15, M a y 1990. [44] W . D . Stanley, G . R. Dougherty, and R. Dougherty, Digital Signal Processing. Jersey, U S A : Prentice-Hall, Inc., 2nd. ed., 1984. New [45] R. D . Strum and D . E . K i r k , First Principles of Discrete Systems and Digital Signal Processing. Reading, Mass., U S A : Addison-Wesley Publishing Company, Inc., 1988. Appendix A Harmonic Related Channels CHANNEL NUMBER VIDEO CARRIER CHANNEL NUMBER MHz VIDEO CHANNEL VIDEO CARRIER NUMBER CARRIER MHz MHz 00 108.00 20 156.00 40 318.00 01 114.00 21 162.00 41 324.00 02 54.00 22 168.00 42 330.00 03 60.00 23 216.00 43 336.00 04 66.00 24 222.00 44 342.00 05 78.00 25 228.00 45 348.00 06 84.00 26 234.00 46 354.00 07 174.00 27 240.00 47 360.00 08 180.00 28 246.00 48 366.00 09 186.00 29 252.00 49 372.00 10 192.00 30 258.00 50 378.00 11 198.00 31 264.00 51 384.00 12 204.00 32 270.00 52 390.00 13 210.00 33 276.00 53 396.00 14 120.00 34 282.00 54 72 15 126.00 35 288.00 55 90 16 132.00 36 294.00 56 96 17 138.00 37 300.00 57 102 18 144.00 38 306.00 58 402 19 150.00 39 312.00 59 408 Table A.l Harmonic Related Channels 120
- Library Home /
- Search Collections /
- Open Collections /
- Browse Collections /
- UBC Theses and Dissertations /
- Cancellation/reduction and measurements of some impairments...
Open Collections
UBC Theses and Dissertations
Featured Collection
UBC Theses and Dissertations
Cancellation/reduction and measurements of some impairments in cable television pictures Lau, Jose Alberto 1995
pdf
Page Metadata
Item Metadata
Title | Cancellation/reduction and measurements of some impairments in cable television pictures |
Creator |
Lau, Jose Alberto |
Date Issued | 1995 |
Description | As technology improves, customers' expectations become higher every day. With such technologies as the Super-VHS video recording and laserdisc player, customers compare these against the quality of the video received over the cable system. Distortions in the image are mainly caused by the nonlinearity of the underground cascaded amplifiers in the cable TV system. Moreover, the addition of new channels on the congested transmission system will increase present interference levels or create new interferences causing picture quality degradation. There are two approaches for combatting impairments. First, these can be prevented by modifying hardware components in the cable TV system such as amplifiers, converters among others. The other approach is to deal with the distortion itself through the use of image processing and filtering techniques. The idea of this approach is, if not to totally cancel the distortion, to reduce it until it becomes less visible to the human eye. This thesis deals with the second approach that consists of reducing/cancelling and measuring impairments such as the single frequency intermodulation, narrowband and wideband CSO (Composite Second Order) distortion. This would complement the automatic detection for a complete monitoring process of impairments. Our approach would fit in an automatic system since it employs non-intrusive methods that do not involve the use of a test signal and hence, no interruption in the transmission would be needed. In this thesis, several filtering methods for cancellation and reduction of single frequency intermodulation, narrowband and wideband CSO are studied and compared. Among these methods are the FIR Linear Filtering, Interpolation in the frequency domain and the Alpha Trimmed Filter. Results show that a combination of the last two methods reduces the single frequency intermodulation considerably. In addition, results also show that the interpolation in the frequency domain perform better than the other methods for narrowband CSO cancellation and reduction. In the case of wideband CSO, a combination of the interpolation in the frequency domain and the Alpha Trimmed filter reduces the impairment considerably. In the area of SNR measurements, different algorithms are proposed for each impairment. For the single frequency intermodulation, the interpolation in the frequency domain algorithm provides acceptable SNR measurements of less than 0.5 dB margin error. For the narrowband CSO, algorithms consisting of the interpolation in the frequency domain and the block averaging are proposed. These two algorithms provide acceptable results. For the wideband CSO, several algorithms that involve the one-dimensional and two-dimensional Discrete Fourier transform are proposed. The two-dimensional algorithms provide more accurate results than the one-dimensional approach giving a margin error of less than 2 dB for SNR measurements of up to 53 dB. |
Extent | 20107505 bytes |
Genre |
Thesis/Dissertation |
Type |
Text |
File Format | application/pdf |
Language | eng |
Date Available | 2009-01-21 |
Provider | Vancouver : University of British Columbia Library |
Rights | For non-commercial purposes only, such as research, private study and education. Additional conditions apply, see Terms of Use https://open.library.ubc.ca/terms_of_use. |
DOI | 10.14288/1.0065245 |
URI | http://hdl.handle.net/2429/3830 |
Degree |
Master of Applied Science - MASc |
Program |
Electrical and Computer Engineering |
Affiliation |
Applied Science, Faculty of Electrical and Computer Engineering, Department of |
Degree Grantor | University of British Columbia |
Graduation Date | 1995-11 |
Campus |
UBCV |
Scholarly Level | Graduate |
Aggregated Source Repository | DSpace |
Download
- Media
- 831-ubc_1995-0372.pdf [ 19.18MB ]
- Metadata
- JSON: 831-1.0065245.json
- JSON-LD: 831-1.0065245-ld.json
- RDF/XML (Pretty): 831-1.0065245-rdf.xml
- RDF/JSON: 831-1.0065245-rdf.json
- Turtle: 831-1.0065245-turtle.txt
- N-Triples: 831-1.0065245-rdf-ntriples.txt
- Original Record: 831-1.0065245-source.json
- Full Text
- 831-1.0065245-fulltext.txt
- Citation
- 831-1.0065245.ris
Full Text
Cite
Citation Scheme:
Usage Statistics
Share
Embed
Customize your widget with the following options, then copy and paste the code below into the HTML
of your page to embed this item in your website.
<div id="ubcOpenCollectionsWidgetDisplay">
<script id="ubcOpenCollectionsWidget"
src="{[{embed.src}]}"
data-item="{[{embed.item}]}"
data-collection="{[{embed.collection}]}"
data-metadata="{[{embed.showMetadata}]}"
data-width="{[{embed.width}]}"
async >
</script>
</div>
Our image viewer uses the IIIF 2.0 standard.
To load this item in other compatible viewers, use this url:
http://iiif.library.ubc.ca/presentation/dsp.831.1-0065245/manifest