A N A D A P T I V E S U B B A N D - B A S E D J O I N T S O U R C E - C H A N N E L C O D E R F O R I M A G E T R A N S M I S S I O N B y A m i n A lav i B.Sc . (Electrical Engineering), University of Tehran, Tehran, Iran, 1990 A THESIS SUBMITTED IN PARTIAL FULFILLMENT OF T H E REQUIREMENTS FOR T H E D E G R E E OF M A S T E R OF APPLIED SCIENCE in T H E FACULTY OF GRADUATE STUDIES DEPARTMENT OF ELECTRICAL AND COMPUTER ENGINEERING We accept this thesis as conforming to the required standard T H E UNIVERSITY OF BRITISH COLUMBIA June 1999 © A m i n A l a v i , 1999 In presenting this thesis in partial fulfilment of the requirements for an advanced degree at the University of Br i t i sh Columbia, I agree that the Library shall make it freely available for reference and study. I further agree that permission for extensive copying of this thesis for scholarly purposes may be granted by the head of my department or by his or her representatives. It is understood that copying or publication of this thesis for financial gain shall not be allowed without my written permission. Department of Electr ical and Computer Engineering The University of Br i t i sh Columbia 2075 Wesbrook Place Vancouver, Canada V 6 T 1W5 Date: Abstract A n adaptive subband image coding system is proposed that uses D P C M and P C M codecs for source encoding the individual subbands, and a family of variable rate channel codes for forward error correction. A low resolution family of trellis coded modulation codes and a high resolution family of punctured convolutional channel codes are implemented and investigated. Implementation of unequal error protection among the subbands, and within the subbands, is examined for the two separate versions of our system. Simulations are performed on the A W G N channel and comparisons are made wi th corresponding systems that use only a single channel code, or that use the same family of codes for an adaptive equal error protection scheme. Our proposed adaptive systems outperform any of the systems that use only a single channel code greatly, extending the useful S N R range by an amount that depends on the code family. As well, our systems outperform the corresponding adaptive equal error protection systems by as much as 2 dB P S N R . This is pr imari ly due to the use of unequal error protection among the subbands, as surprisingly l i t t le additional gain is achieved by applying unequal error protection wi thin the subbands. We use and improve a known modelling technique which enables the system to configure itself optimally for the transmission of an arbitrary image, by only measuring the mean of lowest frequency subband and variance of each of its subbands. Because our systems require an estimate of the channel S N R , we have investigated the sensitivity to poor channel S N R estimation, and have found negligible performance loss for channel mismatches up to 1.5 d B . i i Table of Contents Abstract i'i List of Figures vj List of Tables x Acknowledgments xi 1 Introduction 1 2 Source Coding 6 2.1 Subband Coding 6 2.2 Subband Fi l ter ing 6 2.3 Implementation on Images 8 2.3.1 Circular Convolution Method 10 2.3.2 Symmetric Extension Method 10 2.4 D P C M Encoding System 11 2.4.1 Prediction in D P C M 14 2.5 Quantization 16 2.5.1 Uniform Quantizer . 17 2.5.2 Non-Uniform Quantizer 17 2.6 B i t Al locat ion 18 2.7 Simulation Results 24 i i l 3 Channel Coding 28 3.1 Trellis Coded Modulat ion 28 3.2 Pragmatic Trellis Codes 29 3.3 Decoding T C M Codes 32 3.4 Performance Results 34 3.5 Punctured Convolutional Codes 38 3.6 Decoding P C Codes 39 3.7 Performance Results 40 4 Joint Source-Channel Coding 45 4.1 Subband Image Modell ing 46 4.2 B i t Al locat ion and Code Mapping 51 4.2.1 Computat ion of Rate-Distortion Functions 53 4.3 Side Information Overhead Computation 55 4.4 Simulation Results 57 4.4.1 Performance Evaluation of the Proposed Systems Based on the T C M Code Family 57 4.4.2 Performance Evaluation for Systems Based on the P C Code Fami ly 61 4.4.3 Universality and Effectiveness of the System 62 4.4.4 Performance Evaluation of the Systems Using a More Noise Sensi-tive D P C M Encoder 63 4.4.5 Effect of Channel Mismatch 65 5 Conclusions and Recommendations for Future Research 70 Glossary 73 Bibliography 77 iv A Description of Computer Simulation 81 A . l Source Coding 81 A . 2 Channel Coding 81 A . 3 A W G N Channel 82 A . 4 Generation of G G D Random Variables 83 A . 5 Confidence Interval 83 B Summary of F E C Codes Used in the System 85 v List of Figures 1.1 Generic block diagram of a communication system 2 2.1 A subband coding system 7 2.2 A n 8-band filter bank 8 2.3 A n image subband encoding system 9 2.4 A simple two band subband coding system 9 2.5 Symmetric extension signals 11 2.6 Result of subband filtering applied to Lena image, (a) Original image, (b) Lena image decomposed into 16 subimages. (c) Reconstructed Lena image. 12 2.7 A basic differential encoding system 14 2.8 D P C M encoding system 14 2.9 Performance of subband coding scheme for varying bit rate in noiseless environment 26 2.10 Near perfect reconstruction in noiseless environment, (a) Original Lena image, (b) Reconstructed image using 0.5 bpp. (c) Reconstructed image using 0.75 bpp. (d) Reconstructed image using 1.0 bpp 27 3.1 Encoder/modulator for trellis coded modulator 29 3.2 Set partitioning of the 8 P S K signal set 30 3.3 A pragmatic encoder and its corresponding trellis diagrams (Kc = 3). . . 31 3.4 Mapping schemes for pragmatic T C 1 6 P S K . (a) The scheme used in [26]. (b) The scheme introduced in [27] 32 VI 3.5 (a) Signal space, (b) trellis diagram and (c) B E R performance of pragmatic rate 1/2 T C Q P S K . . ! 35 3.6 (a) Signal space, (b) trellis diagram and (c) B E R performance of pragmatic rate 2/3 T C 8 P S K 36 3.7 (a) Signal space, (b) trellis diagram and (c) B E R performance of pragmatic rate 3/4 T C 1 6 P S K 37 3.8 P C C encoding and puncturing operation 39 3.9 Trellis of the punctured rate = 2/3 code to be used for decoding, (a) Modified trellis of the rate = 1/2 mother code, (b) Conventional trellis. . 40 3.10 Q P S K signal constellation 41 3.11 (a) Modified trellis diagram, (b) perforation matr ix and (c) B E R perfor-mance of rate 3/4 P C C 42 3.12 (a) Modified trellis diagram, (b) perforation matrix and (c) B E R perfor-mance of rate 4/5 P C C 43 3.13 (a) Modified trellis diagram, (b) perforation matr ix and (c) B E R perfor-mance of rate 5/6 P C C 44 4.1 Histograms of: (a) D E M S L F S of Lena image, (b) other subbands of Lena image, (c) D E M S L F S of Couple image, (d) other subbands of Couple image, (e) D E M S L F S of M a n d image and (f) other subbands of M a n d image. 47 4.2 Probabil i ty density function of generalized Gaussian distribution wi th se-lected parameter values 48 4.3 The general block diagram of the proposed image transmission system. . 53 v i i 4.4 Rate-Distortion curves for several channel conditions, (a) Z)i(r;) , gener-ated for M S L F S using GGD-modeled samples with a = 1.70. (b) D 2 ( ? \ ) , generated for other subimages using GGD-modeled samples wi th a = 0.75. 55 4.5 Performance comparison for channel rate R = 0.25 symbol /p ixel 58 4.6 Performance comparison for channel rate R = 0.5 symbol /p ixel 58 4.7 Performance comparison for channel rate R = 1.0 symbol /pixel 59 4.8 Performance comparison for channel rate R = 0.25 s y m b o l / p i x e l , ( T C M code family used), (a) E E P system with T C 1 6 P S K and R = 0.25, P S N R = 25.7 d B . (b) E E P system with T C 8 P S K and R = 0.25, P S N R = 29.3 d B . (c) E E P system with T C Q P S K and R = 0.25, P S N R = 28.1 d B . (d) System B with R = 0.24, P S N R = 30.6 dB 60 4.9 Performance comparison for channel rate R = 0.25 symbol /pixel , ( P C code family used) 63 4.10 Performance comparison for channel rate R = 0.5 symbol /pixel , ( P C code family used) 64 4.11 Performance comparison for channel rate R = 1.0 symbol /pixel , ( P C code family used) 65 4.12 Performance comparison for channel rate R = 0.25 symbol /p ixe l , (PC code family used), (a) E E P system with rate 5/6 P C C and R = 0.225, P S N R = 22.5 d B . (b) E E P system with rate 4/5 P C C and R = 0.234, P S N R = 24.4 d B . (c) E E P system with rate 3/4 P C C and R = 0.25, P S N R = 25.8 d B . (d) System B with R = 0.245, P S N R = 27.8 dB 66 4.13 Performance comparison for channel fate R = 0.25 symbol /pixel , for Cou-ple image ( P C code family used) 67 4.14 Performance comparison for systems employing 2D prediction, channel rate R = 0.25 symbol/pixel , for Lena image ( P C code family used). . . . 68 v i i i 4.15 Channel mismatch effect 69 B . l B E R performances of the T C M and P C channel codes 85 List of Tables 2.1 Statistical properties of 16 subbands of Lena image (512x512) 24 2.2 Statistical properties of 16 subbands of Couple image (512x512) 25 4.1 Results of the Kolmogorov-Smirnov test for a few test images 49 4.2 I D prediction coefficients for the test images 51 4.3 Throughput computed for systems using T C M and P C channel codes. . . 56 x Acknowledgments I would like to thank my father A l i A l a v i Bedjestani and my sister Samin for their continuous support and encouragement. I would also like to express my appreciation to my supervisor Dr . Samir Ka l l e l for his useful suggestions and constructive criticisms, which helped me to start the research and guided me to the completion of my thesis. I am deeply indebted to Dr . Robert Link for his invaluable help throughout this work. He has been a consistent source of advice and guidance and has always been available for discussions. I am grateful to National Sciences and Engineering Research Counci l of Canada, Br i t i sh Columbia Advanced Systems Institute and Mobi le Data Solutions Inc. of Richmond, B C , for their financial support. I would also like to thank M r . Shahram Shirani and my other friends at the communication group of the department of Electr ical and Computer Engineering for their helpful comments and discussions. x i Chapter 1 Introduction The evolution of wireless communication systems from analog to digital has opened a new era in the history of mobile communication. Wireless transmission of image, video and other data has provided a wide range of services to the users of such systems. The high demand for such services, and the explosive growth in the number of users, has made the l imi ted resources that are available to digital wireless communication systems even more valuable. This has motivated researchers to modify conventional communication systems wi th the goal of optimizing the use of the wireless resources. A conventional communication system, as depicted in Figure 1.1, includes a source encoder that removes the source redundancy. Source encoding, also known as data com-pression, is intended to reduce the required bandwidth for transmission. The conventional system also includes a channel encoder that introduces a controlled amount of redundancy for error detection and correction, known as Forward Error Correction ( F E C ) . Shannon [1][2] proved that in the l imit of large coding block sizes, the separation of the source and channel coding operations does not cause any loss of optimality. This has resulted in pr imari ly independent development of source and channel codecs in the design of commu-nication systems. However, for practical cases, where complexity is l imi ted, joint design of source and channel coders is a promising approach for achieving efficient bandwidth and power l imited solutions. A large amount of work has been carried out by researchers on the joint design of source and channel coding systems; like in [3], where the source and channel coding 1 Chapter 1. Introduction 2 Information Source Source Channel Encoder Encoder o Information Sink Source Decoder Channel Decoder Figure 1.1: Generic block diagram of a communication system. operations are truly integrated. Whereas in the work of [4] and [5], the known source coders are cascaded by known channel coders, and a bit allocation scheme determines the bit rate of the source coder and the channel coder for the best attainable performance. Some other researchers have modified the source encoder or decoder to account for the presence of the channel noise [6][7]. Another idea for joint design is Unequal Error Protection ( U E P ) , where the source is broken down into several groups, each of which is given a different level of protection against the channel errors [8] [9]. Bo th fixed rate and variable rate source encoders may be used in the joint design of a source-channel encoding system. Because of error propagation, variable rate source coding tends to be catastrophic in presence of noise, therefore fixed rate source encoding is the more preferred. However, for bandlimited channels where variable rate coding could st i l l be favorable, packetization is introduced as a remedy to the error propagation problem [8]. In this thesis a joint source-channel image coding system is proposed and analyzed which makes use of a subband source encoder and a family of channel codes. A n opt imal allocation of source coding rate to the subimages and the mapping of channel codes to the source bits is determined by a rate-distortion analysis which takes into account Chapter 1. Introduction 3 both the effects of lossy source compression and the effects of channel errors. In the proposed system, an image is decomposed into 16 subimages by using a bank of 8-tap Quadrature Mirror Filters ( Q M F ) , and Differential Pulse Coded Modulation ( D P C M ) and Pulse Coded Modulation ( P C M ) encoders with L loyd-Max quantizers are used to encode the subimages. Source bits are mapped to channel codes from a family of Trellis Coded Modulation ( T C M ) codes (or a family of Punctured Convolutional Codes ( P C C ) ) , to protect the source bits against the channel errors. Both the source encoding and the channel encoding rates are determined subject to a constrained overall channel use per pixel rate, such that the reconstruction quality is maximized, for any given but assumed known channel Signal-to-Noise Rat io (SNR) . The main objective of this thesis is the introduction of two adaptive systems A and B . The relative importance of the subimages is the determining factor for the bit allo-cation and the channel code mapping in system A . Whi le in system B , the positioning importance of the source bits to the reconstruction quality is an additional criteria for the opt imal allocation of source bits and the channel code mapping. The systems found in [8] and [9] are analogous to our system A in concept; while the system found in [10] is analogous to our system B in concept. The system in [8] uses a rate-distortion analysis similar to ours, however channel errors are not correctly taken into account when computing the contribution to the overall distortion introduced by the D P C M encoded lowest frequency subband. Using a more correct approach has led us to a slightly more complicated but also a more accurate modelling of the lowest frequency subband. Another difference between our system and that of [8] is that we use Lloyd-Max quantizers instead of uniform quantizers followed by entropy coders. As shown in [11] for very noisy channels it is better to avoid entropy coding due to its associated catastrophic error propagation. In [9] a subband based video coding scheme with .Rate Compatible Punctured Convolutional ( R C P C ) codes is Chapter 1. Introduction 4 analyzed. They use a slightly different subband decomposition and vector quantization for the individual subbands. In [10] P C M and D P C M encoders are used for the individual subbands; however, the bit allocation used there requires an arbitrary weighting factor to take into the fact that errors in D P C M encoded bits are more severe than errors in P C M encoded bits, whereas this difference is taken into account by our algorithm automatically. In this work Lloyd-Max quantization is used as in our work, however there a B C H code family is employed. Final ly, we have performed a channel S N R mismatch sensitivity analysis for Additive White Gaussian Noise ( A W G N ) channel, which is missing in the above cited works. Therefore, the contributions of this thesis are summarized as follows: • Introduction of a generalized combined bit allocation-channel code mapping algo-r i thm which allows the use of multiple channel codes per subimage. • Modification of the above to allow the use of multiple channel codes between subim-ages, such that no more than one channel code per subimage is permitted. • Study of the performance of the system for two different family of the channel codes. • Investigation of the use of the noise robust one-dimensional ( ID) and the less noise robust two-dimensional (2D) D P C M source encoders. • Introduction of an improved modelling technique to provide a universal image in-dependent system. • Examinat ion of the channel mismatch effect for the proposed systems. The outline of this thesis is as follows: Background knowledge of the source encoding systems that are used in the proposed Chapter 1. Introduction 5 system are introduced in Chapter 2. Design issues pertaining to the source encoding technique to be used for the individual subbands are studied. Channel coding systems of interest are reviewed in Chapter 3. The encoding and decoding operation is briefly explained and the error correction capabilities of such codes are presented in terms of the Bit Error Rate ( B E R ) performance. In Chapter 4, two versions of the proposed system are introduced. The modelling technique is covered, and a generalized rate distortion analysis is performed. Performance of the proposed systems are compared to those not using U E P . Chapter 5, summarizes the findings, states the conclusions and and suggests some areas related to this work for future research. Chapter 2 Source Coding 2.1 Subband Coding There is a large variety of source coding schemes, each one of them showing advantages for data sources with certain characteristics. For example, as indicated in [12], a vector quantization scheme is the most effective for a source with a highly clustered nature. A data source with small sample-to-sample differences is best compressed wi th a differential encoding method, while a source containing random samples wi l l be best compressed wi th a scalar quantizer. In cases where the source does not show any specific characteristic, it is advantageous to decompose it into sub-components whose statistical properties are such that each component merits its own encoding type. This technique, known as subband coding, is used in many applications such as speech [13], image [8][10][14][15] and video [9], and is explained in detail in the following sections. A general block diagram of the subband coding system is shown in Figure 2.1. 2.2 Subband Filtering To decompose a data source, it is first passed through a C-band tree structured filter bank, where the frequency band of the sequence is divided into C narrower bands. A n 8-band filter bank is shown in Figure 2.2. Quadrature mirror filters [16] [17] are used because they have a very sharp cutoff frequency and thus minimize aliasing between neighboring bands. These Fini te Impulse Response (FIR) filters are designed wi th the 6 Chapter 2. Source Coding 7 Analysis 1 filter 1 \ \ Encoder 1 Decoder 1 / ) \ \ Synthesis I filter 1 Analysis / filter 2 \ \ Encoder 2 Decoder 2 I CD { ) Synthesis I filter 2 Analysis / filter 3 \ \ Encoder 3 O Decoder 3 / I \ \ Synthesis I filter 3 Analysis / filter M \ Encoder M Decoder M I ) • \ ) Synthesis 1 filter M Figure 2.1: A subband coding system. sum of \HLP(W)\2 + \HHP(W)\2 made as near to 1 as possible, for the whole range of frequencies, to be able to reconstruct the signal with the least possible distortion. Note that HLP(W) and HJJP{W) are the frequency responses of the low-pass and the high-pass filter, respectively. Because of the Nyquist theorem which states that only twice as many samples per second as the range of frequencies are needed for signal reconstruction, the output of the filter banks are downsampled by a factor of C . This means every C t h sample is kept and the rest are discarded. This downsampling by a factor of C is denoted by CJ,. To reconstruct the source symbols from the decomposed subbands, they are passed through the same filters which now act as synthesis filters. Subtracting the output of the high pass filter from the output of the corresponding low pass filter, one forms the reconstructed signal at each stage. Chapter 2. Source Coding 8 Low-pass filter Low-pass filter Low-pass filter High-pass filter High-pass filter Low-pass filter High-pass filter High-pass filter Low-pass filter Low-pass filter High-pass • filter High-pass filter Low-pass filter High-pass filter Figure 2.2: A n 8-band filter bank. 2.3 Implementation on Images To perform subband coding on a source with more than one dimension like an image, two dimensional filters are needed to divide the frequency bands of each dimension. Bo th horizontal and vertical frequencies must be divided into smaller bands. In most cases, a product of two one-dimensional filters can be used as a two-dimensional filter. Rows of the image are first filtered and downsampled. A t the next step, columns of the image are passed through filters and downsampled. The result is C2 subimages of size ^ x 7^. Obviously the decomposition process could be continued to obtain smaller subimages. Decomposition of a N x N image into 4 subimages using one dimensional filters is illustrated in Figure 2.3. There exists an additional requirement in the design of Chapter 2. Source Coding 9 Original Image NxN Low-pass filter High-pass filter (hi) N/2 ». N/2 X ^ X N/2 N/2 (Ih) (hhj ^ _ N / 2 N/2 X h x N/2 N/2 Figure 2.3: A n image subband encoding system. subband image coding systems: the sum of the number of pixels in the subimages must be equal to the number of pixels of the original image. This along wi th the fact that convolution of the spatially l imited images broaden the region of support of the filtered image, creates a problem for. perfect reconstruction of the edges of subband coded images. There have been a few solutions to this problem proposed, with the two most effective ones being the circular convolution method [14] and the symmetric extension method [18]. For the explanation of these two methods, consider the classical two-band system of Figure 2.4. ho(n) Vo(n) \2 h,(n) v,(n) analysis section • y.(n) ~%(n) ' I ^ V - x'(n) synthesis section-Figure 2.4: A simple two band subband coding system. Chapter 2. Source Coding 10 2.3.1 Circular Convolution Method The distinguishing feature of the circular convolutional method is that al l the convolution operations are N-point circular instead of linear. This results in the outputs of the analysis section to be: y„(n) = v0(2n), (2.1) y1(n)=v1(2n), (2.2) where, L-l vo(n) = ^2 ho(m)[x(n — m) + x{n -f- N — m)], (2-3) m=0 L-l vi(n) = ^2 hi{m)[x{n — m) + x{n + N — m)], (2.4) m=0 and 0 < n < N. Note that the sequences yo(n) and yi(n) are both of length N / 2 . This method approaches near perfect reconstruction while preventing image size expansion. 2.3.2 Symmetric Extension Method This approach also satisfies the length constraint and provides near perfect reconstruction using quadrature mirror filters. For a better explanation of this method, we follow the procedure on the 8-point sample sequence of Figure 2.5(a). The high-pass and low-pass filter responses are illustrated in Figures 2.5(b) and 2.5(c), respectively. The finite length sequence x(n) is periodically extended to form a new sequence x(n) as shown in Figure 2.5(d). {x} = {x0,xu . . . ,z ;v_i} (2.5) {x} = { s_( L _i ) = X(L-2),X-{L-2) = X(L-3),...,X-i - X 0 , £n, X \ , X N - I , %N = £ ; V - 1 , X ^ + 1 = XN-2, £ ( J V + L - 2 ) = X(N-L+l)} (2-6) Chapter 2. Source Coding 11 Analysis filtering is now performed on the extended sequence x(n) L-l vo(n) = H h0(m)x(n - ra), (2.7) m=0 L - l t>i(n) = ] P hi(m)x(n — ra), (2-8) m=0 and 0 < n < N. The final downsampled outputs yo(n) and yi (n) , would be the 4-point sequences as shown in Figures 2.5(e) and 2.5(f). Although both methods show similar • ' 'I I 1 1 I I | I I I I | I ' ' I | I | | , (a> (b) (c) . .1111 11 11 1 1 i i . . . . 11 11 11 11 111 i • . (d) (e) (f) Figure 2.5: Symmetric extension signals. performance in the absence of channel coding and noise, the symmetric extension method shows less reconstruction defects on image boundaries when noise is involved [18]. Figure 2.6 exhibits the decomposition of the 512x512 Lena image into 16 subimages, as well as the reconstructed image from the decomposed elements. 2.4 D P C M Encoding System D P C M is a low complexity lossy compression scheme which takes advantage of the corre-lation in the source. Assume a sequence of source samples {xn}, from which the difference sequence is calculated as {dn} = {xn — x n _ i } . Once these samples are passed through the quantizer, they may be expressed as {dn} — Q{dn} = {dn + qn}, where qn is the Figure 2.6: Result of subband filtering applied to Lena image, (a) Original image, (b) Lena image decomposed into 16 subimages. (c) Reconstructed Lena image. Chapter 2. Source Coding 13 quantization error. To reconstruct the sample sequence at the receiver, the quantized difference sequence is added to the previous reconstructed values in a recursive manner to obtain = { £ n - i + dn}. Now assuming that the transmitter and the receiver start wi th the same value of XQ = xo, the following equation may be derived from a detailed observation of the quantization and reconstruction operations: n Xn = Xn + Y^Qi- (2-9) i=l In other words, system suffers from error propagation as the operation continues. To overcome the error propagation problem, the differencing and adding operations are modified at the encoder and the decoder so that they operate on the same piece of information, xn. A t the encoder, the difference sequence is obtained using {dn} = {xn — x n _ i } , and at the decoder, {xn} = {x„_i + dn} is used to reconstruct the samples. In this modified system, we wi l l obtain the following expression for the n th reconstructed sample: xn = xn + qn, (2.10) where there is no accumulation of errors. Figure 2.7 shows the block diagram of a simple D P C M encoding system. In order to minimize the quantization error, the values of {dn} need to be as small as possible, which may be achieved if xn could be predicted by past values of reconstructed sequence. This gives a motivation to change the block diagram of the D P C M encoding system, to the one in Figure 2.8. Proper design of the predictor block has a big impact on the performance of the D P C M system, therefore, we wi l l quickly review the design issues in the following section. Chapter 2. Source Coding 14 ^dn> Q dn r Xn-1 Xn-1 \ J Xn Delay C/n + Delay Decoder Encoder Figure 2.7: A basic differential encoding system. dn predictor Decoder Encoder Figure 2.8: D P C M encoding system. 2.4.1 Prediction in D P C M The superior performance of D P C M systems are mainly due to the reduced variance and dynamic range of the difference sequence, which is achieved through the prediction operation. A more accurate predictor design results in less variances in the difference sequence. So, the objective here is to design the best prediction function of the form: Pn XQ), (2.11) Chapter 2. Source Coding 15 that minimizes the variance of the difference sequence, ? A ( r j ) , (2.24) i=l subject to the rate constraint -i M R ' = M ^ (2-25) 1 V 1 i=i where, af is the variance of the zth subband, r\ is the rate allocated to the zth subband, and Di{r'^) is the quantization function for the ith. subband: N2/M E (xj ~ xi) E x) 3=1 where the quantizer inputs and the the quantizer outputs. It is mathemat-ically proved in [24] that this constrained problem is transformable to the unconstrained minimizat ion problem of minimizing {Dk(i) + Xi} fc = l , 2 , . . . , M , (2.27) where, i 6 Sk and Sk = {PkiQk} defines the valid range of allocatable bits for the kth. subimage. For a given A, assuming that there is only one solution, Ufc(A), for each subband which minimizes (2.27), then the sum of all the UkS is called the solution rate R'(X). Due Chapter 2. Source Coding 21 to irregularities in the rate-distortion functions, the solution •Ufc(A) may not be unique, meaning that there may be more than one solution rate for a given A. Such a value of A is defined as singular. The objective is to find the solution rate equal to or closest from the bottom to the design rate. The collection of u^s corresponding to this solution rate is the solution to the allocation problem. To find them, the authors of [24] proposed the following algorithm: • Step 1. M Set an ini t ia l value for A to be tj ^ {Dk(lk) — DkUk + -1)}, fe=i where, Ik is the nearest integer to [(pk + Ofc)/2] • Step 2. Solve (2.27) for al l k and obtain fl'(Ai). • Step 3. If R'(Xi) is greater than R', set A3 = Xi and Uk(\3) = fi^(A 1 ) for al l k. Then add a predetermined constant to A l . Go to step 2. • Step 4. If R'(Xi) is smaller than R', subtract a predetermined constant from A l . Go to step 2. • Step 5. Repeat the last two steps unt i l Ai and A 3 are found so that: iE'(Ai) 1} (2.32) Go to step 6 Chapter 2. Source Coding 23 • Step 10. If R' > i ? ' (A 3 ) , or both -R{(A3) and R'h(X5) are smaller than R', the algorithm is not able to find the optimal solution. What we have is an approximate solution which may be adjusted to the actual R' or the closest possible rate, according to the next step. • Step 11. The idea is to increment the rate to the next higher possible value for the subband which provides the greatest reduction in the reconstruction noise, one increment at a t ime, as long as the solution rate remains less or equal to R'. F i n d k' that solves min {Dk(uk + 1) - Dk(uk)}, (2.33) (keM-) and increase uk>, unless it causes the solution rate to exceed R'. Repeat this step unt i l the best accessible solution rate is reached. Chapter 2. Source Coding 24 Table 2.1: Statistical properties of 16 subbands of Lena image (512x512). Image Variance Mean Autocorrelation coefficient Original image 2296.02 122.0 0.995686 Subband # 1 (LFS) 2171.07 123.70 0.982834 Subband # 2 34.63 -0.0019 -0.135275 Subband # 3 12.72 -0.0134 -0.084897 Subband # 4 0.818 0.0082 0.099171 Subband # 5 8.052 0.0155 -0.274875 Subband # 6 2.081 0.0018 -0.004561 Subband # 7 10.64 -0.1720 0.017112 Subband # 8 0.939 0.0069 -0.187677 Subband # 9 0.577 0.0062 -0.021266 Subband # 10 4.424 -0.1301 0.517267 Subband # 11 1.925 0.0265 -0.063782 Subband # 12 0.381 0.0037 -0.026506 Subband # 13 2.856 -0.0031 0.216615 Subband # 14 0.498 0.0026 0.071791 Subband # 15 0.577 0.0045 -0.077262 Subband # 16 1.137 -0.0089 0.191592 Mean-subtracted LFS (MSLFS) 2171.07 0.0000 0.867495 2.7 Simulation Results To evaluate the performance of the subband image coding scheme, we have implemented a subband encoding scheme for a noiseless environment. The 8-tap quadrature mirror f i l -ters of [16] are used as separable filters for a decomposition of the image into 16 subbands, where the pixel values of the Lowest Frequency Subimage (LFS) are mean subtracted. The noiseless bit allocation algorithm described in the previous section is used for dis-tr ibution of the total quota of available bits into subimages. The statistics of two test images are presented in Tables 2.1 and 2.2 for review. Because the autocorrelation coefficients for the Mean Subtracted L F S ( M S L F S ) val-ues in the Tables 2.1 and 2.2 show a significant sample-to-sample correlation, while those Chapter 2. Source Coding 25 Table 2.2: Statistical properties of 16 subbands of Couple image (512x512). Image Variance Mean Autocorrelation coefficient Original image 1967.37 121.90 0.993606 Subband # 1 (LFS) 1758.67 120.41 0.985705 Subband # 2 57.33 0.0106 -0.117475. Subband # 3 47.60 0.0300 0.539605 Subband # 4 5.558 0.0383 0.288368 Subband # 5 7.252 -0.0001 -0.215934 Subband # 6 5.850 -0.0522 -0.013744 Subband # 7 24.77 -0.1527 0.054462 Subband # 8 1.410 0.0057 -0.120824 Subband # 9 1.645 -0.0049 -0.193377 Subband # 10 20.94 -0.1891 0.568118 Subband # 11 3.237 -0.0061 -0.236178 Subband # 12 0.521 0.0028 -0.090547 Subband # 13 3.509 0.0127 0.188940 Subband # 14 0.956 0.0113 0.152371 Subband # 15 0.775 -0.0081 -0.066752 Subband # 16 1.626 0.0083 0.135276 Mean-subtracted LFS (MSLFS) 1758.67 0.0000 0.869286 of the other subimages do not, we use D P C M to encode the M S L F S and use P C M to encode the other subimages. A t the decoding side each subband is decoded according to its corresponding encoder. The mean value of the L F S is added to the received symbols of the first subimage before passing the subimages through the synthesizer filters for image reconstruction. Subjective and objective measures may be used to evaluate the performance of the i m -age transmission systems. We have used the Peak Signal-to-Noise Rat io ( P S N R ) defined as E (255) 2 P S N R = 10 l o g 1 0 " (2.34) hi as the objective measure of the performance, where x,j and x,-j denote the original and Chapter 2. Source Coding 26 0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 Overall Bit Rate (bpp) Figure 2.9: Performance of subband coding scheme for varying bit rate in noiseless envi-ronment. the reconstructed pixel value, respectively. Indices i and j indicate the corresponding row and column indices of a pixel. It is evident from Figure 2.9 that for bit rates higher than 0.5 bit per pixel , this system allows for near perfect reconstruction of the image in noiseless environment. The reconstructed Lena image for several bit rates is displayed in Figure 2.10 to provide a means of subjective comparison. Chapter 2. Source Coding 27 Figure 2.10: Near perfect reconstruction in noiseless environment, (a) Original Lena image, (b) Reconstructed image using 0.5 bpp. (c) Reconstructed image using 0.75 bpp. (d) Reconstructed image using 1.0 bpp. Chapter 3 Channel Coding 3.1 Trellis Coded Modulation It is known that forward error correction increases the performance of a digital com-munication system at the price of bandwidth expansion. That is why F E C techniques are not appreciated as much for bandlimited channels. Bandwidth efficient modulation techniques are in common use for such systems to achieve high bandwidth efficiency. Ungerboeck [25] showed that it is possible to combine convolutional coding wi th a band-width efficient signaling scheme to achieve significant coding gains for the bandlimited channel. The key to this combination scheme is to use an expanded signal set which pro-vides some extra room to be filled by the redundant code bits, at the price of reducing the Euclidean Distance (ED) between signals in the constellation pattern, which in turn increases the error rate. This technique is called Trellis Coded Modulat ion, and of course, in order to design a good T C M scheme, the performance gained from the coding must overcome the performance lost from the signal expansion. It was shown that almost al l the gain that may possibly be attained through signal expansion is reached by doubling the signal space. To translate this into T C M terminology, a n / ( n +1) rate code is needed to use the extra room created by doubling the signal space. Figure 3.1 shows the general block diagram for a trellis encoder with a mapper. The mapping process used in this technique is called set partitioning and is based on successive partitioning of the expanded 2 n + 1 signal set into subsets wi th increasing 28 Chapter 3. Channel Coding 29 n parallel bits C 1 -C 2 -Cn-(n+1) parallel bits Rate n/(n+1) Convolutional Encoder M P S K Mapper/Modulator Channel symbol Figure 3.1: E n coder/modulator for trellis coded modulator. m in imum Euclidean distances. A n example of the set partitioning for the 8 P S K signal set is shown in Figure 3.2. To design a T C M code over a given signal configuration (i.e.: M P S K , A S K , ...), the first step is to define a trellis structure according to the desired constraint length. Signals from an expanded signal set are then assigned to the branches of the trellis, in a way which maximizes the free Euclidean distance of the code. A search for the best assignment is performed by computer, especially for codes with high constraint lengths. Unfortunately, this method results in different encoder/decoder designs for different signal configurations, simply because opt imum T C M codes with different rates have different code structures. Therefore Ungerboecks's family of T C M codes are not attractive for use in any adaptive system. A more practical approach to use is what are known as pragmatic trellis codes. 3.2 Pragmatic Trellis Codes The introduction of puncturing, allows the construction of any rate n / ( n +1) code from a basic rate 1/2 convolutional code, with a performance fairly close to that of an opt imum code. Vi te rb i [26] presented a pragmatic approach to trellis coded modulation based on this idea. Using a rate 1/2 convolutional encoder as the mother code, higher rate codes could be easily built and mapped into signals of the constellation pattern. This allows Chapter 3. Channel Coding 30 (000) (100) (010) (110) (001) (101) (011) (111) Figure 3.2: Set partitioning of the 8 P S K signal set. the realization of T C M codes with different rates, based on the same mother code and its associated trellis structure. Therefore it makes T C M codes more attractive for adaptive systems by reducing the implementation complexity and cost. The pragmatic encoder for the optimized convolutional code of constraint length 1 Kc = 3, and the corresponding trellis diagrams are displayed in Figure 3.3. Only one bit out of the n input bits is fed into the encoder, and the rest of the input bits are simply passed through uncoded. The number of uncoded bits determine the rate of the code as well as the number of parallel branches in the trellis of the code. In fact, there are 2™ _ 1 parallel branches connecting two states in the trellis diagram of a n/(n + 1) T C M P S K pragmatic trellis code. To perform the mapping according to the 1The definition of the constraint length is the total number of memory elements in the encoder plus one. Chapter 3. Channel Coding 31 (c )8PSK (d)16PSK Figure 3.3: A pragmatic encoder and its corresponding trellis diagrams (Kc = 3). scheme proposed by Vi te rb i [26], the two bits which are the outputs of the rate 1/2 encoder, pick one of the four signals according to Gray code, which guarantees that al l neighboring signals of a particular sector differ in only one bit. This has a major effect in error performance of the code, since most of the errors happen in the neighboring signals. The remaining (n — 1) bits are then used to choose sectors in an order according to the decimal equivalent of their binary vectors. This scheme which is referred to as sectorized gray coded mapping by Alamout i [27], is shown in Figure 3.4(a). It was further suggested by Alamout i [27], that the (n — 1) uncoded bits at the output of the pragmatic encoder be used to sefect a sector according to gray code. This new mapping approach which is illustrated in Figure 3.4(b), is called double gray coded mapping, and causes the signals mapped to parallel branches of the neighboring sectors Chapter 3. Channel Coding 32 01OQ -0010 * ^ « P 0 1 1 01 oa .0010 0111 ,0001 011 kOOOO 1000 1010 1001 1011 1010 1100 1110 1000 (a) (b) Figure 3.4: Mapping schemes for pragmatic T C 1 6 P S K . (a) The scheme used in [26]. (b) The scheme introduced in [27]. to differ in only one bit, improving the error performance even more. Note that both mapping schemes are identical for constellations with less than 16 signals. 3.3 Decoding T C M Codes Like for convolutional codes, each sequence of transmitted T C M encoded bits is rep-resented by a path in the trellis representation of that code. The task of the T C M decoder, based on the maximum likelihood criteria, is to examine the received sequence and to choose the most likely transmitted sequence based on this examination. Assume S = ( s i , 5 2 , s i ) is the sequence of transmitted signals and Z = (zi, z 2 , z \ ) is the sequence of received signals from the A W G N channel. We can express the relationship between the transmitted and the received sequence as: Z% — si + ni f ° r 1 ^ i ^ l-i (3.1) Chapter 3. Channel Coding 33 where n2- is a sample of the A W G N . These noise samples can be modeled by the Gaussian distribution N(0, No/2), where ^ 0 / 2 is the power spectral density of the white Gaussian noise. The likelihood function of the received sequence, given the transmitted sequence, may be expressed as: P{Z I S) = I I - 7 ^ - ^ = ( - ^ r ) ' / 2 e x P ( - ^ £ I zi ~ * | 2 ) . (3.2) Therefore, we conclude that the maximum likelihood decoding may be done by minimiz-ing the Euclidean distance between the received sequence of channel symbols and the transmitted channel sequence: r \ 1 / 2 ED(Z,S)= ( £ I " * l 2 J (3.3) Thus the Euclidean distance is used as the metric in decoding such systems using Vi te rb i algorithm [28]. Moreover, the decoding and demodulation are performed simultaneously on the soft output symbols of the channel to avoid the unrecoverable loss of hard-decision demodulation prior to decoding [29]. Decoding T C M codes containing parallel branches should be performed in two steps: 1. A branch wi th the min imum E D from the received signal, is selected from the parallel branches merging to each state. The branch label, which is the decoded bits associated with the surviving branch, and its metric are stored as the winning branch and winning metric. 2. The code trellis is searched for the most likely transmitted path using the Vi t e rb i algorithm considering only the winning branches. Note that once the parallel branches are replaced with the winning branches, the result is a normal trellis structure which the soft input Vi te rb i algorithm can easily be applied to. Chapter 3. Channel Coding 34 3.4 Performance Results In this section we report simulation results to evaluate the performances of the T C M codes that are used in our subband coding system: 1. Rate 1/2 T C Q P S K 2. Rate 2/3 T C 8 P S K 3. Rate 3/4 T C 1 6 P S K The trellis diagram, signal space and the B E R performance of each of these three codes are shown in Figures 3.5, 3.6 and 3.7. The opt imum rate 1/2 convolutional encoder of Figure 3.3(a) is used as the mother code of all these codes. The difference in the trellis diagrams of these codes are the number of parallel branches, and thus a slightly modified Vi te rb i algorithm is used to perform the decoding as explained in the previous section. Chapter 3. Channel Coding 35 Figure 3.6: (a) Signal space, (b) trellis diagram and (c) B E R performance of pragmatic rate 2/3 T C 8 P S K . Chapter 3. Channel Coding 37 Figure 3.7: (a) Signal space, (b) trellis diagram and (c) B E R performance of pragmatic rate 3/4 T C 1 6 P S K . Chapter 3. Channel Coding 38 3.5 Punctured Convolutional Codes Simple implementation of a variable rate error control system based on a single mother code is only one of the reasons that has made the Punctured Convolutional Codes ( P C C ) so attractive. Ca in [30] was the first to show that puncturing of convolutional codes simplifies the Vi te rb i decoding for high rate convolutional codes. It was shown 2 that a Vi te rb i decoder may be used for a rate k/n punctured convolutional code wi th only two branches arriving at each node in the trellis [31], as opposed to the 2k branches of standard convolutional codes [32]. The puncturing operation is performed on a mother code which has the lowest rate in the code family, according to a matrix called the perforation matrix. For a mother code of rate l/Rp, the perforation matrix has Rp rows and Cp columns. Cp which is called the perforation period, depends on the final desired code rate. Larger values of Cp are needed to construct higher rate P C codes. Elements of the perforation matrix are zeros and ones, where a zero indicates that the corresponding output must be deleted from the mother code, and a one indicates that the bit associated wi th that .position must remain untouched. Needless to say, not every perforation matr ix produce a good 3 non-catastrophic 4 code. Various authors have reported several algorithms in the search for good P C codes [31][33][34]. For example, starting from a rate 1/2 mother code wi th constraint length 3, the perforation matrix of a rate 2/3 punctured code is given by [Pi ' l 1 ^ V l 0 ; (3.4) 2Their study was carried out only for rate 2/3 and 3/4 punctured convolutional codes which were shown to be as good as the best known codes with those) rates. 3The criteria for a good code is a high value for the minimum free distance of the code and a low value for the total number of bit errors produced by all incorrect paths with distances larger than dfree. This results in a low BER performance of the code [33]. 4Catastrophic codes may translate a small number of errors in the received codeword to an unlimited number of data errors. Chapter 3. Channel Coding 39 whereas a rate 6/7 code can be obtained using the perforation matr ix ( 1 1 0 0 0 0 ^ V 1 0 1 1 1 1 The block diagram of a P C C encoder is displayed in Figure 3.8. information bits Punctured encoded bits Rate 1/Rp Convolutional Encoder 1 « Puncturer Figure 3.8: P C C encoding and puncturing operation. (3.5) 3.6 Decoding P C Codes A n slightly modified soft input Vi te rb i algorithm can easily be implemented to decode P C codes. The decoding is performed with the trellis of the low-rate mother code. Like the case for the T C M codes, demodulation and the decoding are done together to save the information which is usually lost by making hard decisions. The only modification needed is to not account for the punctured bits in the computation of the branch metr ic 5 . This is easily accomplished by inserting dummy symbols into the puncturing positions according to the perforation matr ix of the code. In the decoding process the same metric value 6 is assigned to these dummy symbols. This procedure greatly reduces the complexity of the conventional path metric computation for the punctured convolutional code. Another modification of the mother code decoder that needs to be done to decode the codes of the P C C family, is that the truncation path length must be increased wi th the code 5The same branch metric is used as the branch metric of the T C M codes for AWGN channel 6Usually this value is chosen as zero. Chapter 3. Channel Coding 40 rate [33]. The differences between the conventional trellis and the one used for modified Vi te rb i decoding based on the mother code (rate = 1/2) is demonstrated in Figure 3.9, where X denotes a punctured bit position. Q P S K modulation is used in conjunction with the P C codes, where the code mapping is shown in Figure 3.10 3.7 Performance Results Computer simulations were performed to evaluate performances of the P C codes that are used in our subband coding system: Chapter 3. Channel Coding 41 Figure 3.10: Q P S K signal constellation. 1. Rate 3/4 P C C 2. Rate 4/5 P C C 3. Rate 5/6 P C C The perforation matrix, modified trellis diagram and the B E R performance curve of each of these three codes are shown in Figures 3.11, 3.12 and 3.13. The opt imum rate 1/2 convolutional encoder of Figure 3.3(a) is used as the mother code for al l these codes. The differences in the trellis diagrams of these codes are simply in the pattern of the punctured bit positions. Chapter 3. Channel Coding 42 Figure 3.11: (a) Modified trellis diagram, (b) perforation matr ix and (c) B E R perfor-mance of rate 3/4 P C C . Chapter 3. Channel Coding 43 Figure 3.12: (a) Modified trellis diagram, (b) perforation matr ix and (c) B E R perfor-mance of rate 4/5 P C C . Chapter 3. Channel Coding 44 Figure 3.13: (a) Modified trellis diagram, (b) perforation matr ix and (c) B E R perfor-mance of rate 5/6 P C C . Chapter 4 Joint Source-Channel Coding Consider an image source that is decomposed in to 16 subbands, and a series of channel codes available with varying rates and performances. To design a subband image coding system subject to a fixed channel symbol rate per pixel , one could choose the lowest rate (i.e.: strongest) channel code and find the source coding rate accordingly. Obviously this system which has a lot of protection against channel errors, performs quite well in a channel wi th low signal-to-noise ratio. However, to design a system which operates in higher channel S N R , a smaller portion of the available channel rate could be allocated for protection against channel errors. The amount of channel rate saved could very well be added to the source encoding rate to reduce the distortion caused by lossy source encoding. A n adaptive joint source-channel design minimizes the overall reconstruction error for any given channel condition, and thus has varying parameters based on the channel condition. Based on these observations, we have proposed and studied the following two systems: System A: The objective is to find the best possible bit distribution for each subband and to choose the best code from the family of channel codes wi th the constraint that only one channel code is permitted per subband; subject to overall constraint rate of R (channel use/pixel). System B: This system works similarly to system A , with the difference being that the use of multiple channel codes is allowed within every single subband. 45 Chapter 4. Joint Source-Channel Coding 46 System A takes advantage of only the difference in importance levels of the subbands; whereas system B also takes advantage of the positioning importance of bits wi thin pixels of each subband. Many source coders, including P C M and D P C M , produce codewords in which bits that are more important to reconstruction quality also occupy the more significant bit positions of the codeword. As in Chapter 2, we encode the L F S using D P C M , and encode the other subbands using P C M . It is imperative that the design of a joint source-channel coding system such as ours not be image dependent. Various codec parameters w i l l be image dependent, and in order that the codec be able to determine these parameters for an arbitrary image, we wi l l model all images in such a way that any particular image wi l l be fully characterized by only a few parameters. These parameters wi l l be measured by the coder and transmitted to the decoder as side information. 4.1 Subband Image Modell ing In order to design a universal system, the statistical properties of the subimages of the subband coder are studied to find the best possible mathematical model from which to bui ld the system upon. We have found the statistics of al l the subimages other than the L F S , to be sufficiently similar to allow a single model for al l of them. The histograms of all the subimages except the L F S , as well as the histograms of the Differentially-En coded Mean-Subtracted L F S ( D E M S L F S ) are plotted in Figure 4.1 for several 512x512 standard monochrome images. We wi l l model the D E M S L F S and the other subimages wi th the Generalized Gaussian Distr ibution ( G G D ) . The probability density function of the G G D is given by [35]: ^ ^ ^ e x p i - r j i a , ^ } ; (4.1) p(x) = where 2r(l/a) T/(a,^) = /3-1 r ( 3 / « ) 1 1 / 2 r ( l / a ) . (4.2) Chapter 4. Joint Source-Channel Coding 47 (e) (f) Figure 4.1: Histograms of: (a) D E M S L F S of Lena image, (b) other subbands of Lena i m -age, (c) D E M S L F S of Couple image, (d) other subbands of Couple image, (e) D E M S L F S of M a n d image and (f) other subbands of M a n d image. Chapter 4. Joint Source-Channel Coding 48 3 r 1 1 1 1 1 1 r 2.5 h Figure 4.2: Probabil i ty density function of generalized Gaussian distribution wi th selected parameter values. a > 0, which is called the shaping factor is an indication of exponential decay rate, j3 is the standard deviation of the distribution, and F is the standard gamma function. Typ ica l behavior of the unit variance (/? = 1) G G D pdf wi th a few different values of the shaping factor is shown in Figure 4.2. Note that a G G D with a = 1 is a Laplacian distribution, and that wi th a = 2, is a Normal distribution. The Kolmogorov-Smirnov (KS) test [36] is used as a measure of distance between the variance-normalized empirical distribution and the probability distribution of the G G D with (3 — 1. For a given set of data samples X = (xi, X2,£jv), the Kolmogorov-Smirnov test measures the distance between the Cumulat ive Distr ibution Function ( C D F ) of the samples Fx(xi) and a known Chapter 4. Joint Source-Channel Coding 49 Table 4.1: Results of the Kolmogorov-Smirnov test for a few test images. Best value of a for Image Size DEMSLFS other subbands Lena 512 x 512 1.45 0.75 Couple 512 x 512 1.50 0.75 M a n d 512 x 512 1.95 0.80 Peppers 512 x 512 2.00 0.75 Grandma 512 x 512 1.90 0.65 Bridge 256 x 256 1.95 0.95 Einstein 256 x 256 1.05 0.65 crowd 256 x 256 2.10 0.50 Tank 256 x 256 1.75 0.80 Kodak 256 x 256 1.60 0.80 Average 1.70 0.75 distribution function F(-). The K S test statistic, dks , is defined by dka = max\Fx(xi)-F(xi)\, i = 1 , 2 , N . (4.3) Of a set of distribution models, the closest to a given empirical distribution is the one that results in the smallest dks. We have considered the unit variance generalized Gaussian distributions with a ranging from 0.3 to 2.5 in steps of 0.05, and compared them against a series of five images of size 512x512 and five images of size 256x256. Employing the K S distance test, we have found the best value of a for the D E M S L F S , as well as the best value of a for the other subimages, to be those listed in Table 4.1. A first order one dimensional linear predictor with Chang-Donaldson renormalization is used for the differential encoding operation. It is reasonable to choose the averaged a from the table and declare a = 1.70, to be a good approximation for the shape factor of the G G D for the D E M S L F S . Likewise, a = 0.75 is found to be the best shaping factor for modelling the other subimages. Chapter 4. Joint Source-Channel Coding 50 A model found in this way is more precise than the Laplacian model which was used in early subband image coding systems [14] [15]. However, the final values of a are somewhat different than the ones reported in [8] [37], because these values depend on: • filters used for the subband decomposition, • modelling technique for the L F S , and • predictor design of the D P C M encoder. Once the model is defined, a series of GGD-based synthetically generated random samples wi l l be used as the pixel values of al l the subimages except the L F S . For the L F S , the inverse of the differential encoding operation is performed on al l the synthet-ically generated samples that represent the D E M S L F S . The average of the prediction coefficient for the M S L F S pixels are computed from the ten sample monochrome images listed in Table 4.1, and is used for the inverse differential encoding operation on the syn-thetically generated samples. It is now timely to explain why the proposed system works on the mean-subtracted symbols. From the calculation of the prediction coefficient in Chapter 2, it is easy to see that the prediction coefficient of a sequence of pixels changes as the mean is shifted, while it remains unchanged as the variance is varied. The inverse of the differential operation on the unit-variance zero-mean samples results in zero-mean samples wi th a variance not equal to one. The differentially decoded synthetically gener-ated pixels can be variance normalized without any change in the interpixel correlation characteristics. These new synthetically generated pixels are then stored and used as the model for M S L F S . The synthetically generated samples representing the zero-mean, unit-variance subimages form the data sets upon which we design our system. The sys-tem parameters for actual transmission of a particular image wi l l be determined by the mean and variance of the L F S and the variances of the other subimages of the subband decomposition. Chapter 4. Joint Source-Channel Coding 51 Table 4.2: I D prediction coefficients for the test images. Image Size One dimensional prediction coefficient Lena 512 x 512 0.579316 Couple 512 x 512 0.581730 M a n d 512 x 512 0.519225 Peppers 512 x 512 0.634847 Grandma 512 x 512 0.738692 Bridge 256 x 256 0.589222 Einstein 256 x 256 0.489050 Crowd 256 x 256 0.429054 Tank 256 x 256 0.686670 Kodak 256 x 256 0.601806 Average 0.585 4.2 Bit Allocation and Code Mapping To determine the optimal allocation of bits between the source coding and the channel coding of al l the subimages, we need variance normalized rate-distortion (RD) curves. The rate of the R D curve for a subimage wi l l be the total rate after source and channel coding and modulation. Whi le the distortion is the total distortion incurred by the lossy source coding plus that due to uncorrected channel errors. We employ the following channel codes from the family of trellis coded modulation schemes reviewed in Chapter 3: • T C Q P S K , which produces a channel symbol for every bit, • T C 8 P S K , which produces a channel symbol for every 2 bits, and • T C 1 6 P S K , which produces a channel symbol for every 3 bits. We define the average number of channel uses required to transmit one pixel of the image to be the system constraint; call it the channel rate and denote it as R . Our goal is to find \ Chapter 4. Joint Source-Channel Coding 52 the opt imum number of bits to be allocated for encoding each subimage, and at the same time assign the best channel code for the source encoded bits, to achieve the m i n i m u m distortion after decoding at the receiver, subject to a fixed design channel rate. Hence, the task of bit allocation and channel code mapping must be performed simultaneously. To formulate the joint bit allocation-code mapping problem we write the overall Mean Squared Error ( M S E ) of the subband coding technique, D , as: M ^ ^ ( r O + E ^ Z M ^ ) , (4-4) i=2 where M is the total number of subimages, erf is the variance of the i t h subimage, and D\(ri) and / ^ ( ^ i ) denote the variance-normalized distortion-rate functions for the M S L F S and the other subimages, respectively. The overall channel rate is given by: 1 M « = S I > . . (4.5) (4.4) and (4.5) are same as (2.24) and (2.25) except that instead of bits per pixel , r; is now the number of channel uses per pixel for the z'th subimage, which is not necessarily an integer. For example, assume that a subimage is to be encoded with one bit , and the single source bit is to be channel coded using T C 1 6 P S K . Recall from Chapter 3 that this code generates one channel symbol for every three input bits. Therefore, the channel rate associated with this single bit representing one subband sample, is 1/3. Once the two rate-distortion functions are known to the transmitter, the allocation algorithm of [24], reviewed in section 2.6, is used to find the best rt- for each subimage and the appropriate channel code mapping associated with that channel rate. Likewise the same algorithm is used at the receiver, to obtain the required information for decoding. The block diagram of the system is shown in Figure 4.3. The rate distortion performance (i.e.: channel use/pixel vs. M S E ) for each subimage is obtained as follows. Chapter 4. Joint Source-Channel Coding 53 Source Encoders 1 Router ''Channel -Encoder"#.'3? :-Cliannel Encoder it 2, Channel Encoder # 3 Channel Estimation Channel Decoder#2 Channel Decoder # 2 =Channel-Decoder #3; MUX DEMUX Source Decoders Figure 4.3: The general block diagram of the proposed image transmission system. 4.2.1 Computation of Rate-Distortion Functions A t first, al l possible values of R are computed based on combinations of bit allocation and channel code mapping. Note that there might be more than one configuration leading to a single value of channel rate. For example, if two bits are allocated to each pixel of a subimage and both bits are channel coded using T C Q P S K , the total channel rate for that subimage would be 2. Nonetheless, allocating three bits to each pixel and using T C Q P S K to code the first bit and T C 8 P S K to encode the other two bits, also leads to a total channel rate of 2. For cases where multiple channel codes are to be assigned to the source encoded bits of a pixel, the order of channel code assignment is that the stronger (lower rate) codes are used for the more significant bits, and the weaker (higher rate) codes are assigned to the less significant source encoded bits. Once al l possible configurations for the points on the R-axis are computed and stored in memory, synthetically generated Chapter 4. Joint Source-Channel Coding 54 GGD-modeled samples for the M S L F S (and the other subimages) are fed to the D P C M (and the P C M ) source followed by channel encoders, and are then passed through the A W G N channel according to each configuration. After channel and source decoding, the overall normalized M S E is calculated as a measure of quality between the samples at the output of the source decoder and the original samples, as: ^2(xi,j xi,j) M S E = ^ - = n ; (4.6) 4-i xi,i where a;,-j and Xij denote the original and the reconstructed pixel at the output of the source decoder, respectively. To generate a distortion measure associated wi th any channel rate n, the smallest measured normalized M S E for al l configurations is stored, along wi th the corresponding configuration, only if M S E ( r 8 ) < M S E ( r j _ ! ) for r; > r t _ i . The M S E and the configuration associated with the previous lower channel rate wi l l be stored otherwise. Following this procedure guarantees that under no circumstances wi l l the measured M S E increase as the channel rate increases. This procedure wi l l generate the R D curve for system B , nevertheless, the only modification needed for system A , is that configurations which have more than one channel code for the subimage are ignored. This w i l l l imi t the realizable values of the channel rate. Figure 4.4 shows the lower envelope of the variance-normalized rate-distortion func-tions of system B , for several channel conditions. It is clearly seen that as the channel noise increases, more channel rate is needed to keep the distortion unchanged. These curves, generated using GGD-modeled synthetic source, are stored by both the transmit-ter and the receiver. Because the curves are actually a collection of discrete points, the storage requirement for this is reasonably low. Chapter 4. Joint Source-Channel Coding 55 For the family of DPCM eivtlecoders For the family of PCM er/decoders 10"3 10"2 10"' 10° 10"3 10"2 10"' 10° MSE (Normalized). MSE (Normalized) (a) (b) Figure 4.4: Rate-Distortion curves for several channel conditions, (a) Z)i(r;) , generated for M S L F S using GGD-modeled samples with a = 1.70. (b) Di(ri), generated for other subimages using GGD-modeled samples with a = 0.75. 4.3 Side Information Overhead Computation As discussed earlier, the mean value of the L F S must be known to the receiver for recon-struction of the L F S pixels. The variances of all subimages are required by the receiver so that it can determine the bit allocation and channel code mapping; as well , the receiver requires the reconstruction levels for the Lloyd-Max quantizers, and the prediction coeffi-cient of the M S L F S . The number of reconstruction levels changes as the source encoding Chapter 4. Joint Source-Channel Coding 56 Table 4.3: Throughput computed for systems using T C M and P C channel codes R = 0.25 R = 0.5 R = 1.0 System using T C M codes for error correction System using P C codes for error correction 87.6% 95.7% 83.9% 92.9% 84.1% 90.9% rate varies, making it impossible to compute a fixed measure of side information. How-ever, in what follows, we have considered a worst case scenario to compute a lower bound on the system throughput. It can easily be shown that the total number of reconstruction levels for source coding of all subimages according to a given quota of bits is maximized, when al l available bits are allocated to one subimage. In our model, we have l imi ted the max imum number of bits per pixel for each subimage to be 8. For a design rate of 1 channel use/pixel, and for a maximum number of reconstruction levels, we assume that the least number of bits are being assigned for forward error correction, meaning that only T C 1 6 P S K is in use for channel encoding of al l source encoded bits. It is assumed that 2 bytes are used for transmission of each real number, and it is further assumed that using the strong rate 1/4 P C code and Q P S K modulation, provides sufficient protection for transmission of the side information to the receiver. W i t h these assumptions, the lower bound for the throughput for transmission of a 512x512 image is: [512 x 512 x 8 x 6/16 x 1/3] [(6 x 2 8 + 16 + 2) x 2 x 8 x 4 x 1/2] + [512 x 512 x 8 x 6/16 x 1/3] ' { ' } The lower bound for the throughput for different design rates for our proposed system using two different family of channel codes are computed and presented in Table 4.3. It was observed that the actual system throughput during the course of simulations, did not fall below 98%. Chapter 4. Joint Source-Channel Coding 57 4.4 Simulation Results 4.4.1 Performance Evaluation of the Proposed Systems Based on the T C M Code Family To evaluate the performance of the proposed systems, the first system is implemented as follows. A pair of D P C M en coder/decoder wi th first order linear prediction and Chang-Donaldson renormalization and a series of P C M en coder/decoder pairs are implemented wi th L loyd-Max quantizers. Three channel codes from the family of the T C M codes in chapter 3 are used for channel error protection. The D E M S L F S and other subimages are modeled by a generalized Gaussian distribution with a = 1.70 and a = 0.75, respectively. Two rate-distortion functions D\{r) and D2(r) are generated according to the channel S N R , and the bit allocation and mapping algorithm explained in Section 4.2 is employed to find the opt imum bit allocation and appropriate channel code mapping for the source coded bits. The monochrome 512x512 Lena image is sent over the A W G N channel using systems A and B , wi th the results plotted for different design channel rates and varying channel conditions in Figures 4.5, 4.6 and 4.7. Note that the dashed curves indicate the performances of Equal Error Protection ( E E P ) systems, where only one channel code is available for forward error correction for all subimages. The proposed systems A and B show a performance similar to the one for the E E P system using T C Q P S K in the low channel S N R region, however, they exhibit over 3 dB performance gain for higher values of the channel S N R . Moreover, systems A and B show a similar performance to that of the E E P system wi th T C 1 6 P S K in high S N R environments, but perform extremely better ( P S N R gains of up to 17 dB) in the presence of low channel S N R . If one is to choose a single channel code to design an image transmission system over the S N R range that we consider, then the T C 8 P S K code would be the best option in the sense that the average distortion is minimized over the entire range of channel conditions. Al though our Chapter 4. Joint Source-Channel Coding 58 m •a a SP 6 -o c o J3 C/3 34 32 30 28 26 24 22 20 18 EEP system with TC16PSK - - 0 - • EEP system with TC8PSK - -A— EEP system with TCQPSK - - O - -Adaptive system B ptive s^stem^^^^^^^^^^^ ^dagti 10 12 E s / N 0 (in dB) 14 16 18 20 Figure 4.5: Performance comparison for channel rate R = 0.25 symbol /p ixel . -o T3 0) S o o J3 (1, 34 32 30 28 26 24 22 20 18 EEP system with TC16PSK - - O - • EEP system with TC8PSK - - A - • EEP system with TCQPSK - - B -Adaptive system B ^darjtive^stem^^^^^^^^^^^ 10 12 E s / N 0 (in dB) 14 16 18 20 Figure 4.6: Performance comparison for channel rate R = 0.5 symbol /p ixel . Chapter 4. Joint Source-Channel Coding 59 9 bo e C o o u J3 on Oh 36 34 32 30 28 26 24 22 20 EEP system with TC16PSK - - O - -EEP system with TC8PSK - - A - -EEP system with TCQPSK - - a - -Adaptive system B Adaptive system A 10 E s / N 0 (in dB) 12 14 16 18 20 Figure 4.7: Performance comparison for channel rate R = 1.0 symbol /p ixel . adaptive scheme behaves similarly to the T C 8 P S K system in a small range of channel SNRs in the mid region of the whole range, it exhibits a much better performance in all other channel conditions. Another possible design scenario is that al l the channel codes are available to the system, but only one could be used at a t ime according to the channel S N R . Threshold limits must be set according to the B E R performance curves of the channel codes, and the system must be allowed to switch between the channel codes as the S N R changes. The performance of such a system is an exact envelope of the three dashed curves in Figures 4.5, 4.6 and 4.7. The adaptive system A outperforms this envelope curve over the whole S N R range, with an advantage that it is as high as 2 dB P S N R . However, since system B outperforms system A by a negligible amount, it is evident from these curves that very li t t le may be gained by allowing mult iple channel codes to be used per subimage. The results of the transmission of the Lena image for Chapter 4. Joint Source-Channel Coding 60 Figure 4.8: Performance comparison for channel rate R — 0.25 s y m b o l / p i x e l , ( T C M code family used), (a) E E P system with T C 1 6 P S K and R = 0.25, P S N R = 25.7 d B . (b) E E P system with T C 8 P S K and R = 0.25, P S N R = 29.3 d B . (c) E E P system with T C Q P S K and R = 0.25, P S N R = 28.1 d B . (d) System B with R = 0.24, P S N R = 30.6 d B . Chapter 4. Joint Source-Channel Coding 61 S N R = 13 d B , are shown in Figure 4.8 for subjective comparison purposes. The multiple codes used by system A or B at a particular channel S N R are in their operational region according to the B E R performance of the code. In fact, from our results it is evident that the channel S N R is the primary factor determining which codes are selected by the system. Moreover, in practice it was found that most of the subim-ages other than the L F S are discarded, and even the ones which are not discarded are assigned a very small number of source bits, where there is a l i t t le difference between the Most Significant B i t ( M S B ) and the Least Significant B i t ( L S B ) . This explains the small performance gain of system B over system A . It is further observed that for the areas of the channel S N R where not more than one code is considered operational, the performance curves of the adaptive system converges to the performance curve obtained from the E E P system wi th best code in that region. In fact, the only gain observed in this area is due to the more precise bit allocation technique used in the adaptive system. To investigate this further, the higher resolution family of P C channel codes are considered instead for the adaptive systems in the next section. 4.4.2 Performance Evaluation for Systems Based on the P C Code Family The following P C codes, used with Q P S K modulation, that were reviewed in Chapter 3 are used: • rate 3/4 P C C with Q P S K , which produces 2 channel symbols for every 3 bits, • rate 4/5 P C C with Q P S K , which produces 5 channel symbols for every 8 bits, and • rate 5/6 P C C with Q P S K , which produces 3 channel symbols for every 5 bits. The similar performance of these codes, causes more than one code to be available to the system at each channel S N R , which in turn causes the adaptive system to show a better Chapter 4. Joint Source-Channel Coding 62 performance at every channel S N R . Codes which are weaker in terms of protection against channel errors are used to protect the L S B s . These codes consume less channel bandwidth and provide some extra room for additional protection of M S B s of the source bits. Of course there is a price to pay, and that is the reduced dynamic range of channel SNRs . If this system is to perform in the same range of channel conditions as the system using the T C M code family, more channel codes of the family must be employed. The more channel codes used, the more complex would become the task of generation of the rate-distortion curves. However, this added complication is not an issue since the computation of the R D curves are done once, and the results are stored in the memory thereafter. Figures 4.9, 4.10 and 4.11 exhibit the performance comparisons for the systems A , B , and E E P systems for three design channel rates 1. It is clearly seen that although there is an overall improvement in the performance of the adaptive scheme over the E E P systems, both of the proposed systems A and B show similar performance. This is because even though multiple codes are used in system B to differentiate the M S B from the L S B , the B E R performance of these codes are so close that there is actually not much difference in the amount of protection assigned to M S B s and L S B s . The results of the transmission of Lena image for S N R = 6 d B , are plotted in Figure 4.12 for subjective comparison purposes. 4.4.3 Universality and Effectiveness of the System In order to evaluate the universality of our system, we have performed the simulations over a different test image. This test is performed on the systems employing the family of P C codes, for the design rate R = 0.25 channel symbol /pixel . The results are pre-sented in Figure 4.13. Comparing to Figure 4.9, It is observed that the similar gains are 1In cases where the exact value of the design rate is not reachable, the largest possible value from below is considered. For example, the actual channel rate for EEP systems using the rate 4/5 PC code (rate 5/6 PC code) is 0.234, 0.469 and 0.977 (0.225, 0.488 and 0.975) in Figures 4.9, 4.10 and 4.11 Chapter 4. Joint Source-Channel Coding 63 34 32 30 28 26 24 22 20 1! -1 1 y J ><''' [ ^— ^ f i <'' / • EEPsy EEP sy stem with f stem with <• yfi PCC 5 PCC - -A— -< r S S * 1 C B f system wim t r t — u — Adaptive system B Adaptive system A 4.5 5.5 6.5 7 7.5 Es/N0 (in dB) 8 8.5 9.5 Figure 4.9: Performance comparison for channel rate R = 0.25 symbol /pixel , ( P C code family used). achieved by the adaptive system for this different test image. Furthermore, to examine the effectiveness of our modelling technique, we have used the actual subband symbols of the Lena image to obtain the required source encoding and channel code mapping parameters, for any given channel S N R . These parameters were found to be similar to the ones obtained from the model. 4.4.4 Performance Evaluation of the Systems Using a More Noise Sensitive D P C M Encoder The performance comparison of a 2D predictor with I D predictor [7] confirms that the latter is less sensitive to noise. The use of a more noise sensitive D P C M encoding scheme for encoding the L F S would make the transmission more sensitive to the channel noise, and therefore we expect the performance gain achieved by the adaptive system in this Chapter 4. Joint Source-Channel Coding 64 34 32 30 28 26 24 22 20 18 r ' 1 r . x s * s • • * f I r ' > • s ^ i s i EEP sy EEPsy stem with f stem with ^ V6 PCC /5 PCC A < * S S f cur system witn r<^^ — t j — Adaptive system B Adaptive system A 4.5 5.5 6.5 7 7.5 8 E s / N 0 (in dB) 8.5 9.5 Figure 4.10: Performance comparison for channel rate R = 0.5 symbol /pixel , ( P C code family used). case be of more significance than in the case of I D prediction. Figure 4.14 exhibits the performance comparison for the design rate of R = 0.25, performed wi th the Lena test image. Evidently, the performance curves of al l systems are degraded in the low S N R regions due to the higher noise sensitivity of the encoding scheme used for L F S . It is further noted that the adaptive systems perform slightly better than the E E P systems, as expected. Note that the change in predictor design affects the characteristics of the D E M S L F S values and therefore a new G G D model is needed. In fact, new tests were performed to find the best model for L F S of the test images, and the G G D with a = 1.40 found to be the best model for the D E M S L F S . A new series of synthetic samples are generated and used to generate a new rate-distortion function for each channel S N R . The rate-distortion functions representing other subimages remain unchanged. Chapter 4. Joint Source-Channel Coding 65 34 32 30 28 26 24 22 20 18 EEP system with PCC EEP system with 4/5 PCC EEP system with 3/4 PCC Adaptive system B Adaptive system A — -A— -- -ED— -4.5 5.5 6.5 7 7.5 8 E s / N 0 (in dB) 8.5 9.5 Figure 4.11: Performance comparison for channel rate R = 1.0 symbol /pixel , ( P C code family used). 4.4.5 Effect of Channel Mismatch In the proposed system, we assumed that the channel condition is known to the trans-mitter and the receiver. This in fact is a practical approach which can be implemented using a pilot signal to sense the channel condition. However, for a realistic evaluation of the system performance it is desirable to know how the system performs when the chan-nel S N R is poorly estimated. To study the channel mismatch effect we define P S N R p x as the average P S N R resulting from a system designed for channel S N R = F dB and applied to a channel wi th S N R = X d B . Likewise the P S N R ^ x ^ s defined as the average P S N R resulted from a system designed for and applied to channel S N R = X d B . The P S N R g x and P S N R ^ x values are plotted in figure 4.15 to show the effect of the channel mismatch. These curves are obtained for the Lena image, transmitted using R = 0.25 Chapter 4. Joint Source-Channel Coding 66 Figure 4.12: Performance comparison for channel rate R = 0.25 symbol /p ixe l , (PC code family used), (a) E E P system with rate 5/6 P C C and R = 0.225, P S N R = 22.5 d B . (b) E E P system with rate 4/5 P C C and R = 0.234, P S N R = 24.4 d B . (c) E E P system with rate 3/4 P C C and R = 0.25, P S N R = 25.8 d B . (d) System B with R = 0.245, P S N R = 27.8 d B . Chapter 4. Joint Source-Channel Coding 67 . E Y- - - J y" [ i r < y y r < y y y y f EEPsy EEP sy stem with! stem with <• y«PCC .6 PCC — -A— -nur system witn .y* — t j — Adaptive system B Adaptive system A 4.5 5.5 6.5 7 7.5 E s / N 0 (in dB) 8.5 9.5 Figure 4.13: Performance comparison for channel rate R image ( P C code family used). 0.25 symbol /p ixel , for Couple channel use/pixel and the system using the P C C family and 2 D - D P C M encoding for the M S L F S . Chapter 4. Joint Source-Channel Coding 68 34 32 30 28 26 24 22 20 18 . ! 1 1 1 ' " - ' ' 1 1 s s * * i * s >' / I s — ^ s s < / s s s [ <' / s > f EEPsy EEPsy stem with f stem with <• v^PCC /5 PCC - - 0 - -— -A— -I • s net- system witn -tr-Adaptive system B Adaptive system A 4.5 5.5 6.5 7 7.5 E s / N 0 (in dB) 8.5 9.5 Figure 4.14: Performance comparison for systems employing 2D prediction, channel rate R = 0.25 symbol /pixel , for Lena image ( P C code family used). Chapter 4. Joint Source-Channel Coding 69 Chapter 5 Conclusions and Recommendations for Future Research We have proposed two versions of a joint source-channel image coding scheme which shows an excellent performance over a wide range of A W G N channel conditions. System A allows the use of al l channel codes for al l the subimages, but l imits the number of channel codes to be used per subimage to one. Whi le system B does not have the latter restriction. A D P C M codec and a series of P C M codecs are employed to encode the 16 subimages, which are obtained by a 4x4 subband decomposition. A set of channel codes are then made available to the system from one of two families of codes, either T C M or P C codes, and the performance of both of the systems are evaluated in each case. We have compared our proposed systems to the E E P systems of equivalent channel rate. It is shown in Chapter 4 that none of the E E P systems can provide a good quality reconstructed image over the range of channel SNRs for which our system can. Another design scenario could be an adaptive E E P system with selective channel coding, where the best channel code is used for the whole image at a particular channel S N R . The performance of such an adaptive system would be an exact envelope of the performance of the individual E E P systems. Our proposed systems (system A and system B) outperform the adaptive E E P system by as much as 2 dB P S N R , pr imari ly because of the unequal error protection employed among subimages. There is l i t t le difference between system A and system B for both the low-resolution T C M family of codes and the high resolution P C family of codes, which shows that unequal error protection within a subimage is relatively unimportant. For the system employing the T C M code family, the fact that 70 Chapter 5. Conclusions and Recommendations for Future Research 71 the performance of the individual members are so different, l imits the number of channel codes invoked by the system at each channel S N R . A t a given channel S N R , one code is predominantly used, and therefore the M S B and L S B are not treated very differently for error protection. On the other hand, the similar performance of the individual members of the P C code family causes many codes to be invoked at each channel condition, and therefore different channel codes are assigned to LSBs and M S B s of individual subimages. Nevertheless, the similarity of the performance of the channel codes in the family results in a similar amount of protection against channel errors for the M S B and the L S B , which causes system B to perform negligibly better than system A at any channel S N R . Since it is observed that there is very l i t t le advantage in the performance of system B over system A , for both the low resolution family and the high resolution family of channel codes, no significant gain is anticipated for any other intermediate resolution code family. Needless to say, the dynamic range of the operating channel SNRs for the family of P C codes is reduced, therefore more than three channel codes would be needed to cover a larger range of channel SNRs by this code family. Our modelling technique allowed us to characterize any arbitrary image by only a few parameters, which enabled us to reduce the amount of the required side information, and achieve a universal system design that for any particular image performs almost as well as a system tuned to that image. It was also shown that although our system needs an estimate of the channel S N R to obtain the transmission parameters, it is very robust to an inaccurate channel S N R esti-mation. The following topics for further research in this area are suggested: • Evaluate the proposed subband encoding scheme over fading channels. Chapter 5. Conclusions and Recommendations for Future Research 72 • Investigate different methods for modelling the images, like Au to Regressive M o d -elling. • Consider modification of the channel decoders to take advantage of the source statistics for decoding. • Study the performance of the system for color images. • Search for methods of estimating the channel condition. Glossary A S K Ampl i tude shift keying A W G N Addi t ive white Gaussian noise ai z'th prediction poefficient ay Horizontal prediction coefficient a 2 Vert ical prediction coefficient bpp B i t per pixel B E R B i t error rate {&;} Decision boundaries of the quantizer C Number of bands in the filter bank C p Number of columns of the perforation matr ix C D F Cumulative distribution function D The total distortion D E M S L F S Differentially-encoded mean-subtracted lowest frequency subimage Di(r^) Distortion of the ith. subimage at given r\ bit rate dkS Kolmogorov-Smirnov distance measure dn Difference sample dn Quantized difference sample D P C M Differential pulse coded modulation Di(ri) Distortion corresponding to the first subimage, given r; channel rate D