UBC Theses and Dissertations

UBC Theses Logo

UBC Theses and Dissertations

Simulation and subjective evaluation of an adaptive differential encoder for speech signals Hanson, Bruce Albert 1977

Your browser doesn't seem to have a PDF viewer, please download the PDF to view this item.

Notice for Google Chrome users:
If you are having trouble viewing or searching the PDF with Google Chrome, please download it here instead.

Item Metadata

Download

Media
831-UBC_1977_A7 H35.pdf [ 2.82MB ]
Metadata
JSON: 831-1.0094070.json
JSON-LD: 831-1.0094070-ld.json
RDF/XML (Pretty): 831-1.0094070-rdf.xml
RDF/JSON: 831-1.0094070-rdf.json
Turtle: 831-1.0094070-turtle.txt
N-Triples: 831-1.0094070-rdf-ntriples.txt
Original Record: 831-1.0094070-source.json
Full Text
831-1.0094070-fulltext.txt
Citation
831-1.0094070.ris

Full Text

SIMULATION AND SUBJECTIVE EVALUATION OF AN ADAPTIVE DIFFERENTIAL ENCODER FOR SPEECH SIGNALS by Bruce Albert Hanson B.Sc, University of Manitoba, 1975 A THESIS SUBMITTED IN PARTIAL FULFILLMENT OF "THE REQUIREMENTS FOR THE DEGREE OF MASTER OF APPLIED SCIENCE in the Department of Electrical Engineering We accept this thesis as conforming to the required standard THE UNIVERSITY OF BRITISH COLUMBIA June 1977 ^c^j Bruce Albert Hanson, 1977 In presenting th i s thes i s in pa r t i a l fu l f i lment of the requirements for an advanced degree at the Un ivers i ty of B r i t i s h Columbia, I agree that the L ibrary sha l l make it f ree ly ava i l ab le for reference and study. I further agree that permission for extensive copying of th is thesis for scho lar ly purposes may be granted by the Head of my Department or by his representat ives. It is understood that copying or pub l i ca t ion of th is thes is for f i nanc ia l gain sha l l not be allowed without my writ ten permission. Department of E l e c t r i c a l Engineering The Univers i ty of B r i t i s h Columbia 2075 Wesbrook Place Vancouver, Canada V6T 1W5 ABSTRACT This t h e s i s describes the subjective analysis of a DPCM system featur- n ing an adaptive quantizer. The system i s simulated on a d i g i t a l computer and operated under v a r i -ations i n the sampling frequency and the number of available quantizer l e v e l s . The subjective performance of the system i s judged using the isopreference method which presents t e s t r e s u l t s i n the form of isopreference contours.drawn on a plane showing sampling frequency and number of quantizer l e v e l s as axes. From these curves the minimum required channel capacity f o r a given subjective preference l e v e l i s shown to occur when sampling i s at the Nyquist rate. The previous statement applies when the quantizer output l e v e l s are nat-u r a l l y coded or entropy coded. The isopreference contours i n d i c a t e implementation tradeoffs between the number of quantizer l e v e l s and the sampling frequency. The isopreference contours also show that odd l e v e l quantizers outperform even l e v e l quantizers when entropy coding i s used. A n a l y t i c a l measures of performance i n the form of output signal-to-noise r a t i o (SNR) are obtained. Although c o r r e l a t i o n between curves of constant SNR and curves of constant subjective q u a l i t y are evident, the SNR curves do not accurately r e f l e c t the r e s u l t s of subjective evaluation. A s p e c i a l experiment i n v o l v i n g quantizer dc o f f s e t i s described which indicates that SNR could not be used to compare speech samples containing large proportions of d i f f e r e n t types of noise. Throughout the work, the d i g i t a l channel between encoder and decoder i s assumed n o i s e l e s s . i T A B L E O F C O N T E N T S A B S T R A C T i T A B L E O F C O N T E N T S i i L I S T O F T A B L E S i v L I S T O F I L L U S T R A T I O N S v A C K N O W L E D G E M E N T S v i i I I N T R O D U C T I O N 1 1 . 1 D i f f e r e n t i a l E n c o d i n g o f S p e e c h 1 1 . 2 R e v i e w o f P r e v i o u s W o r k 2 1 . 3 S c o p e o f t h e T h e s i s 3 I I D P C M S I M U L A T I O N 5 2 . 1 I n t r o d u c t i o n 5 2 . 2 D P C M R e v i e w a n d T e r m i n o l o g y 5 2 . 3 T h e C o h n a n d M e l s a D P C M - A Q B S y s t e m 8 2 . 3 . 1 I n t r o d u c t i o n 8 2 . 3 . 2 T h e P r e d i c t o r 1 0 2 . 3 . 3 T h e Q u a n t i z e r 1 0 I I I S U B J E C T I V E E V A L U A T I O N 1 7 3 . 1 I n t r o d u c t i o n 1 7 3.2 S u b j e c t i v e T e s t i n g R a t i o n a l e 1 7 3 . 3 T h e I s o p r e f e r e n c e M e t h o d 1 9 3 . 3 . 1 G e n e r a l D e s c r i p t i o n 1 9 3 . 3 . 2 S c a l i n g I s o p r e f e r e n c e C o n t o u r s 2 0 3 . 4 P r e p a r a t i o n o f S p e e c h M a t e r i a l 2 2 3 . 5 S u b j e c t i v e T e s t P r o c e d u r e 2 6 3 . 5 . 1 T e s t F o r m a t 2 6 3 . 5 . 2 O b j e c t i v e o f t h e T e s t s 2 9 I V R E S U L T S O F S U B J E C T I V E T E S T S 3 1 4 . 1 I n t r o d u c t i o n 3 1 4 . 2 D e t e r m i n a t i o n o f E x p e r i m e n t a l I s o p r e f e r e n c e C o n t o u r s . . . 3 1 4 . 3 P r e s e n t a t i o n o f T e s t R e s u l t s 3 3 4 . 4 D i s c u s s i o n o f T e s t R e s u l t s 3 5 4 . 4 . 1 D i s c u s s i o n o f I s o p r e f e r e n c e C o n t o u r s 3 5 4 . 4 . 2 E n t r o p y C o d i n g a n d M i n i m u m R e q u i r e d C h a n n e l C a p a c i t y 3 6 4 . 4 . 3 S N R C o m p a r i s o n s 4 2 i i 4 . 5 C o m p a r i s o n s o f t h e R e s u l t s w i t h P r e v i o u s W o r k 4 8 V C O N C L U S I O N 5 1 5 . 1 S u m m a r y 5 1 5 . 2 S u g g e s t i o n s f o r F u r t h e r W o r k 5 4 R E F E R E N C E S 5 5 i i i L I S T O F T A B L E S T A B L E 2 . 1 E x p e r i m e n t a l l y o b t a i n e d p a r a m e t e r v a l u e s 1 5 ( a ) Q u a n t i z e r t h r e s h o l d s 1 5 ( b ) Q u a n t i z e r s c a l i n g f a c t o r s 1 5 ( c ) E x p a n s i o n - c o n t r a c t i o n c o e f f i c i e n t s p l u s S C A L E a n d B I A S 1 5 2 . 2 O p t i m u m s c a l i n g f a c t o r s f o r C o h n a n d M e l s a ' s d e l t a m o d u l a t o r . . 1 6 4 . 1 A c o m p a r i s o n o f S N R v a l u e s a s d e r i v e d f r o m s a m p l e s p r o c e s s e d w i t h a n d w i t h o u t a d c o f f s e t 4 4 5 . 1 C o m p a r i s o n o f a p p r o x i m a t e S N R v a l u e s f o r n o n - a d a p t i v e p r e v i o u s - s a m p l e f e e d b a c k D P C M , a n d D P C M - A Q B 5 3 i v LIST OF ILLUSTRATIONS FIGURE 2.1 (a) A PCM system (b) A DPCM system 6 2.2 Block diagram of the Cohn and Melsa DPCM - AQB system. (a) Transmitter (b) Receiver 9 2.3 Quanitzer Formats, (a) even number of levels (b) odd number of levels H 3.1 (a) A typical isopreference contour. (b) A typical psychometric curve 21 3.2 (a) Normalized amplitude probability density of speech. Symetrical average of positive and negative data. 24 (b) Power density spectrum of speech. 25 3.3 Block representation of a paired comparison test set. . . . 27 4.1 (a) An experimental psychometric curve. (b) Preference in units of normal deviates plotted against a varying parameter. The line is f i t -ted by a least-squares method 32 4.2 DPCM - AQB isopreference contours. The test signals of each contour are marked "x". Two transitiv-ity test signals are marked "o". 95% confidence intervals are denoted by a bar through each ex-perimental point. SNR^^ values are given in dB as are the SNR values which appear enclosed in brackets 34 4.3 Curves of constant bit rate. Superimposed are the isopreference contours (dashed) of Fig. 4.2 38 4.4 Subjective ratings of Fig. 4.2 plotted against their respective minimum required channel capacities 39 4.5 Curves of constant bit rate using entropy coding. Superimposed are the isopreference contours (dashed) of Fig. 4.2 40 4.6 Matrix of bit rate reduction coefficients r for entropy coding 43 v F I G U R E 4 . 7 C u r v e s o f c o n s t a n t s i g n a l t o n o i s e r a t i o . S u p e r i m p o s e d a r e t h e i s o p r e f e r e n c e c o n t o u r s ( d a s h e d ) o f F i g . 4 . 2 4 6 4 . 8 S N R v s . f / f c u r v e s f o r t h e 2 , 3 , 4, 5 , 6 a n d s n 7 l e v e l q u a n t i z e r s s t u d i e d . A l s o s h o w n i s a c u r v e r e s u l t i n g f r o m J a y a n t ' s o n e - b i t mem-o r y d e l t a m o d u l a t o r 4 7 v i ACKNOWLEDGEMENTS I am g r a t e f u l to Teleglobe of Canada for support received under t h e i r fellowship program. Grateful acknowledgement i s also given for the u n i v e r s i t y research assistanships received from 1976 to 1977. Special thanks are extended to Dr. R.W. Donaldson for h i s patiencecand help throughout the l i f e of the project. Thanks are also extended to Messrs. M. Koombes and A. MacKenzie fo r t h e i r t e c h nical assistance and to Ms. M.E. Flanagan for typing the manuscript. v i i 1. I INTRODUCTION 1.1 DIFFERENTIAL ENCODING OF SPEECH The m i n i a t u r i z a t i o n of d i g i t a l components has enabled d i g i t a l e l e c t r o n i c s to enter into a l l aspects of modern industry. The f i e l d of communications i s no exception and the search to f i n d simpler and more e f f i c i e n t methods to d i g i t a l l y code speech has been continuing for several years. The system most commonly used to date i s pulse code modulation (PCM). A v a r i a t i o n of t h i s method c a l l e d d i f f e r e n t i a l PCM (DPCM) and the much more simply structured d e l t a modulator (DM) are two common techniques, being considered f o r improved A/D conversion. These schemes code the d i f -ference between the input s i g n a l and a system-generated predictor s i g n a l . Two of the major problems encountered when using DPCM or DM are the introduction of noise through quantization and the determination of optimum stepsize to minimize that noise. If the step s i z e i s too small, then the quantizer w i l l not be able to follow large changes i n the input s i g n a l , while an overly large stepsize w i l l introduce unwanted granular noise. By permitting e i t h e r , or both, of the quantizer or predictor of such systems to be adaptive, the encoder i s made s e l f - a d j u s t i n g to better s u i t the varying s t a t i s t i c s of the input s i g n a l . Another of the d i f f i c u l t i e s faced i n studying such systems i s obtaining an accurate measure of performance. The most common mathematical approach has been to use mean square error or s i g n a l to noise r a t i o s . How-ever, the f i n a l test of a system used with a human observer as a sink i s the subjective q u a l i t y of the output s i g n a l . I t i s w e l l known that purely a n a l y t i c a l measures, such as s i g n a l to noise r a t i o , do not n e c e s s a r i l y r e f l e c t system performance as perceived by human subjects. 2. The i n t e r e s t spurred by recently developed adaptive quantization schemes, along with the continuing need f o r subjective evaluation of such systems, has led to the subjective evaluation of a DPCM system featuring an adaptive quantizer presented i n th i s t h e s i s . 1.2 REVIEW OF PREVIOUS WORK Various techniques f o r analogue to d i g i t a l conversion i n the form of d e l t a modulation and d i f f e r e n t i a l PCM have been studied f o r several years. O'Neal [01, 02] has investigated the use of DM and DPCM o n t t e l e v i s i o n and Gaussian s i g n a l s . McDonald [Ml] has shown DPCM to be superior to PCM for speech a p p l i c a t i o n s . I t has been discovered that by allowing the quantizers of these systems to be self-adapting through the a p p l i c a t i o n of c e r t a i n algorithms, improvements i n s i g n a l reproduction are possi b l e . DM has been of p a r t i c u l a r i n t e r e s t to many researchers because of i t s s i m p l i c i t y . Jayant [JI] has proposed an adaptive quantizer f o r a DM encoder using a one-bit memory. He has also conducted bit-sequence c o r r e l a t i o n studies on such a system [J4]; Tazaki et a l [TI] have derived a set of equations which can repre-sent several previously published formulas i n c l u d i n g Jayant's DM. Adaptive quantizers have also been applied to DPCM systems. Cohn and Melsa [C4], Qureshi and Forney [Ql], and Cummiskey et a l [C6] have proposed and tested d i f f e r e n t algorithms f o r quantizer adaptation. The d i f f e r e n t types of encoding systems referred to above, have been brought together by N o l l [NI] who has completed a comparative study of quantizing schemes f or speech systems. Many attempts have been made to analyse and measure the perform-ance of coding schemes using a mathematical approach. Goodman [G3] has de-vised expressions for the quantizing noise i n DM and PCM systems. Green-3. s t e i n [G5] has derived equations to c a l c u l a t e slope overload noise i n d e l t a modulators. More recently, Goldstein and L u i [G2] have derived equations describing the three b a s i c types of quantization noise appearing i n a DPCM system featuring an adaptive quantizer. Others have approached the problem of performance evaluation using methods based on subjective perception. Donaldson et a l [DI, D2, C2, Yl] have used extensive subjective t e s t i n g f or evaluating systems operat-ing on speech s i g n a l s . Their method of evaluation i s based on the i s o -preference method f i r s t described by Munson and K a r l i n [M4], Grether and Stroh [G6] on the other hand have s u c c e s s f u l l y used a version of the cate-gory judgement method. 1.3 SCOPE OF THE THESIS The purpose of t h i s thesis i s to investigate various aspects of the performance of a d i f f e r e n t i a l pulse code modulator u t i l i z i n g an adap-t i v e quantizer operating on speech s i g n a l s . The two major parameters under study are the number of quantization l e v e l s and the sampling rate r e l a t i v e to the Nyquist sampling rate. The model used f o r t h i s study i s presented and discussed i n some d e t a i l i n Chapter 2. Optimization of system parameters i s also considered. For reasons of s i m p l i c i t y and r e p r o d u c i b i l i t y , simulation of the model i s accomplished using a high l e v e l programming language on an IBM 370/168 computer. Chapter 3 of the thesis describes subjective evaluation related to voice communication systems. A comparison of i n t e l l i g i b i l i t y t e s t i n g , subjective t e s t i n g , and a n a l y t i c a l measures of performance i s given. Following this an explanation of the isopreference method i s presented. The chapter concludes with a d e s c r i p t i o n of the manner i n which data was 4. prepared and then presented f o r subjective evaluation. Chapter 4 presents the r e s u l t s of the subjective t e s t s . A p l o t of the isopreference contours as determined by analysis of the subjective test r e s u l t s i s given. Following t h i s presentation i s a discussion of the contours, comparisons with s i g n a l to noise measurements and comparisons with previous relevant work. Also considered are the advantages of b i t rate reduction schemes employing entropy coding. Chapter 5 concludes the thesis with a summary of the work and i t s i m p l i c a t i o n s . 5 . II DPCM SIMULATION 2.1 INTRODUCTION This chapter presents a review of D i f f e r e n t i a l Pulse Code Modu-l a t i o n (DPCM) and explains the terminology used i n t h i s and following chapter. A d e t a i l e d d e s c r i p t i o n of the DPCM model used i n t h i s work i s then given. 2.2 DPCM REVIEW AND TERMINOLOGY The terminology presented i n t h i s section and throughout the paper w i l l follow that of N o l l [NI]. N o l l also presents a good comparison of d i f f e r e n t quantizing schemes f o r those wishing further d e t a i l . A pulse code modulation {PCM) system i s shown i n Figure 2 . 1 a . (A version of t h i s scheme using an 8 - b i t quantizer i s now being used i n the industry.) Operation of t h i s system r e s u l t s i n the input s i g n a l being band-limited,' sampled at or j u s t above the Nyquist rate, logarithmically, quantized, and then coded for transmission. The receiver performs the reverse steps using an inverted quantizer. In d i f f e r e n t i a l PCM (DPCM) (see Figure 2 .1b ) the addition of a feedback loop and adder e f f e c t i v e l y subtracts a predicted value, p^, from the input sample s^. Estimate p^ i s generally a l i n e a r sum of past quan-t i z e r outputs; thus P k - j ^ ( i ) S k_. ( 2 . 1 ) i = l The r e s u l t i n g d i f f e r e n c e or error s i g n a l , e^ i s quantized and transmitted. As the error s i g n a l i s of lower redundancy than the o r i g i n a l input s i g n a l , coding can generally be accomplished using fewer b i t s than a comparable PCM system. Conversely, q u a l i t y could be improved f o r a given b i t rate. This f a c t has been shown a n a l y t i c a l l y and s u b j e c t i v e l y i n many experiments S(t)\ PRE FILTER U>db\ QUANTIZER CODER CHANNEL DECODER POST FILTER Sft) Sft) PREFILTER Pk QUANTIZER PREDICTOR CODER \---\CHANNEL DECODER + + POST FILTER PREDICTOR h (b) F i g . 2.1 (a) A PCM system (b) A DPCM system 7. [N2, DI, D2, G6, 02]. The receiver section of the DPCM system adds the received error e^ to the predicted value to a r r i v e at the estimated sample s^. A further improvement i n s i g n a l reproduction has been introduced by using an adaptive predictor. That i s , the p r e d i c t o r c o e f f i c i e n t s are modified according to some algorithm. This system i s appropriately re-ferred to as an adaptive DPCM (ADPCM) system. I f adaptation of the pre-d i c t o r c o e f f i c i e n t s i s generated from the o r i g i n a l input s i g n a l then the scheme i s referred to as forward p r e d i c t i o n . That i s , the predictor co-e f f i c i e n t s must be transmitted forward to the receiver as i t has no know-ledge of the adaptation strategy. On the other hand i f p r e d i c t o r c o e f f i c i e n t s are generated from the quantizer output the scheme i s c a l l e d backward p r e d i c t i o n . In t h i s , case parameter transmission i s not required because the receiver has a l l the information needed to reproduce the required c o e f f i c i e n t s . Although t h i s l a t t e r method has the a t t r a c t i v e q u a l i t y of not increasing the b i t rate to accommodate pre d i c t o r c o e f f i c i e n t adaptation, i t has been shown to be unsuitable when used on channels with high channel b i t error p r o b a b i l i t y [N2]. Reconstructed s i g n a l q u a l i t y can be improved or the b i t rate made lower by making the quantizer adaptive. This new system has been c a l l e d a r e s i d u a l coder by Cohn and Melsa [05]. However i n keeping with the terminology of t h i s thesis and that of N o l l i t w i l l be referred to as an ADPCM - adaptive quantizer (ADPCM - AQ) . Both.ibackward adaptive quantiz-ation schemes (ADPCM - AQB) and forward (ADPCM - AQF) schemes are possible and d i f f e r e n t algorithms have been proposed and studied [C5, C7, J2, J3, M3, Q l ] . 8. A special case of the systems described above is the delta mod-ulator (DM). The quantizer of this system contains only two levels and is of special interest because of its simplicity. . It too has been improved through the use of adaptive quantizers (DM - AQ) [C8, JI, J3, SI]. Many studies concerning delta modulators have been carried out [Cl, G3, G5, TI]. The system under study in this thesis is a DPCM - AQB incorpor-ating an adaptive quantizer algorithm as derived by Cohn and Melsa [C5]. It represents one of the few systems devised to date which comes close to fi l l i n g the three basic requirements; low bit-rate, good quality speech and relative simplicity of implementation. 2.3 THE COHN AND MELSA DPCM - AQB SYSTEM 2.3.1 Introduction The system used in this work is modelled after the one presented by Cohn and Melsa [C5]. Their system is an ADPCM - AQB which attempts to estimate the standard deviation of the input signal and normalize i t before quantization. The system is depicted in Figure 2.2. Note that a l l receiver variables maintain the same values as their transmitter counterparts as long as the channel is error free. To simplify system implementation Cohn and Melsa's adaptive pre-dictor has been replaced in our study with a linear time invariant pre-dictor based on the immediately preceeding receiver output s^. Most pre-dictor adaptation algorithms including the one presented by Cohn and Melsa involve much calculation. Furthermore, Qureshi [Ql] has shown that a sys-tem with a fixed predictor performs only 1 to 2 dB worse than the same sys-tem with an adaptive predictor. Although Cohn and Melsa reported a more appreciable difference of 4 to 5 dB i t should be noted that the author's results without an adaptive predictor came to within 2-3 dB of Cohn and QUANTIZER CODER TO CHANNEL INVERSE QUANTIZER PREDICTOR 3 THRESHOLD FACTOR z k (a) FROM CHANNEL DECODER \ INVERSE QUANTIZER r V F i g . 2.2 Block diagram of the Cohn and Melsa DPCM - AQB system. (a) Transmitter (b) Receiver THRESHOLD FACTOR r? PREDICTOR (b) 10. Melsa's published results obtained using the adaptive predictor. The bit rate reduction which is obtained by source coding the quantizer output signal has also been examined. 2.3.2 The Predictor Assnoted earlier the predictor output p^ is formed from a linear combination of previous receiver outputs. p k = ? *k ( i ) § k - i ( 2 - 2 ) 1=1 In our study n = 1 and a^ was set to an experimentally determined optimum value of 0.8. Computer simulation of the model necessitated the threshold .fac-tor in Figure 2.2 being placed before the delay element of the predictor in odd level quantizers. The decision element's output §, = 0 i f z. < .01 k k s. = zr i f z. > .01 k k k — (2.3) forced a l l low level outputs to equal zero. This threshold1 rule prevented s, from exponentially approaching zero in the case of a very low or zero level input. The problem was also solved by adding a small amount of noise to the input signal however, this solution was not used in this study. 2.3.3 The Quantizer Two, three, four, five, six and seven output level quantizers were tested. Therefore, both odd and even level formats were needed (see Figure 2.3). For the quantizers considered a l l parameter values were symmetric. Unlike standard PCM systems, the process of quantization must be broken into two sections, a quantization and an inverse quantization -0~ T k'2 <rkF(5) ( r k F ( 3 ) <rkF(l) rk F ( 2 ) crkF(4) crkF(6) ( a ) °-kF(5) °~kF(3) <k= n —r <7, k = 2 %=4 ."kill | o | \ _ / (b) F i g . 2.3 Quantizer Formats (a) even number of l e v e l s (b) "odd number of l e v e l s 12. (see Figure 2.2). Operation of the quantizer i s s t r a i g h t forward. The input sample e^ . i s compared to the quantizer thresholds,o^T^ and the range into which the sample f a l l s i s determined. This range s p e c i f i e s quantizer and transmitter output, q^. The inverse quantizer receives the l e v e l q^ as i t s input and produces an output e^ as defined by the product of the sc a l i n g factor ^(1^) and the state v a r i a b l e which i s described at a l a t e r point. As the n o n l i n e a r i t i e s of the system makes mathematical optimiz-ation d i f f i c u l t , a random search was used to determine the optimum thresh-old and s c a l i n g factor values. The s i g n a l to noise r a t i o of the ent i r e DPCM - AQB model was used as the optimization c r i t e r i o n I t i s t y p i c a l l y measured i n decibels (dB) and i s computed by 2 SNR = 10 log E [ S ] 9 (2.4) E [(s-an 2 Cohn and Melsa on the other hand optimized over the value E[(e -e ) ] . As k k the SNR approach produced the same optimum values f o r the f i v e and seven l e v e l quantizers as those published by Cohn and Melsa i t suggests that both methods are equally v a l i d . Tables 2.1a and 2.1b give the experimentally determined optimum values.^ As the system was to operate under a range of sampling frequencies above the Nyquist rate, the quantizer parameters were reoptimized at two other sampling frequencies. Values obtained were very close to those obtained at the Nyquist rate. Also any change i n SNR which resulted i n using a reoptimized parameter was quite small, usually not much greater than 0.1 dB. I t was therefore concluded that the model was stable over the range of sampling frequencies chosen and the i n i t i a l values given i n Tables 2.1a and 2.1b were maintained f o r a l l sampling frequencies under study. _ A l l parameter values and data f o r graphs were obtained during or by repeating the stage of s i g n a l processing which resulted i n the formation of the sample data base as described i n Section 3.4. The exception to the above statement occurred i n the case of the two l e v e l quantizer or d e l t a modulator. The optimum value of the s c a l i n g factor did vary with frequency and reoptimization was necessary at a l l sampling frequencies (see Table 2 . 2 ) . Adaptation of the quantizer i s baseddon an estimate of the stand-ard deviation of the quantizer input s i g n a l e^ .. As the optimum threshold for quantizing a given v a r i a b l e varies l i n e a r l y with the standard devia-t i o n of that v a r i a b l e [C5], d i v i d i n g the input s i g n a l e^ by i t s standard deviation w i l l - r e s u l t i n a normalized s i g n a l with a standard deviation of unity. A quantizer with f i x e d thresholds can then be designed. A l t e r n a t e l y one can view the process as an attempt to keep the quantizer within operating range of the d i f f e r e n c e s i g n a l by a s e r i e s of expansions and contractions. The algorithm used for estimating the standard deviation of e^ i s that described by Cohn and Melsa. I t operates on two l e v e l s . For periods of unvoiced speech or s i l e n c e a moving average of s^ i s used to estimate the standard deviation of e^. As e^ i s not a v a i l a b l e at the re-ceiver an alternate s i g n a l must be used. Signal 3^ * s used rather than §, as i t s SNR i s better while at the same time i t s envelope tends to be k very s i m i l a r to that of e^. For voiced speech the standard deviation of e^ i s very large at the beginning of a p i t c h period and the moving scaled average i s no longer a good estimate. Therefore, a feature has been included to allow f a s t adaptation. Whenever eit h e r of the outermost quantizer l e v e l s occurs i n -d i c a t i n g a sharp increase i n s i g n a l magnitude, the d i s c r e t e standard de-v i a t i o n a, i s s i g n i f i c a n t l y increased by a f a c t o r a(outermost l e v e l ) . This e f f e c t i v e l y pushes the quantizer l e v e l s out to accommodate high l e v e l 14. s i g n a l s . I f no further outer l e v e l s occur decays back to the scaled average. Accordingly i s calculated by the equation: a k = MAX (a(q k) • , E [ | s k | ] / SCALE + BIAS) (2.5) where: a(q^) are the expansion-contraction c o e f f i c i e n t s , q^ i s the quantizer output, E[|§ k|] / SCALE i s the moving scaled average of and BIAS maintains a, at a minimum or base value for low l e v e l k s i g n a l s . The expansion-contraction c o e f f i c i e n t s , a(<l k) were obtained i n the same way described e a r l i e r f o r the thresholds and s c a l i n g f a c t o r s . Again r e s u l t s f or the f i v e and seven l e v e l cases matched those of Cohn and Melsa and remained e s s e n t i a l l y constant with changes i n sampling frequency, (Table 2.1c). As d e l t a modulation involves only two l e v e l s the expansion-contraction c o e f f i c i e n t s have been set equal to zero. Thus adaptation i n the case of d e l t a modulation depends only on the estimate of the standard deviation of e, . k The values selected f or BIAS and SCALE are also given i n Table 2.1c. The diff e r e n c e between these and Cohn and Melsa's values may be ex-plained by differences i n the i n i t i a l stages of data preparation, i n par-t i c u l a r the analogue to d i g i t a l conversion. The average E [ | s ^ j ] i s calculated from a moving window covering the one hundred samples preceding the one currently being processed. N J l § k - i i E[|S k|] = , N = 100 (2.6) The above model was simulated on a d i g i t a l computer and used to process a l l data for the t h e s i s . 15. Table 2.1 Experimentally obtained optimum parameter values (a) quantizer thresholds (b) quantizer scaling factors (c) expansion-contraction coefficients plus SCALE and BIAS NUMBER OF QUANTIZATION LEVELS 2 3 4 5 6 7 1 0 1.4 0 1.0 0 .5 T2 - - 3.0 3.5 1.5 1.5 T 3 (a) 3.5 3.5 F(l) F(2) Table 2222 -Table 2.2 0 -3.0 1.25 -1.25 0 -2.0 .75 -.75 0 -1.0 F(3) - 3.0 5.0 2.0 2.25 1.0 F(4) - - -5.0 -5.25 -2.25 -2.0 F(5) - - - 5.25 5.0 2.0 F(6) - - - - -5.0 -4.5 F(7) (b) 4.5 a(l) 0 .6 .30 .40 .50 .70 a(2) 0 1.35 .30 .80 .50 .80 a(3) - 1.35 1.30 .80 .90 .80 a(4) - - 1.30 2.20 .90 .90 a(5) - - - 2.20 1.70 .90 a(6) - - - - 1.70 2.30 a(7) - - - - - 2.30 BIAS 1.0 1.0 1.0 1.0 1.0 1.0 SCALE 5.0 5.0 5.0 5.0 5.0 5.0 (c) f g / f n F ( l ) F(2) 1 4.5 -4.5 1.25 4.0 -4.0 1.5 3.5 -3.5 1.75 3.5 -3.5 2 3.0 -3.0 2.25 3.0 -3.0 2.5 3.0 -3.0 2.75 2.75 -3.0 3 2.75 -2.75 3.25 2.75 -2.75 3.5 2.75 -2.75 Table 2.2 Optimum s c a l i n g factors f o r Cohn and Melsa's d e l t a modulators. 17. I l l SUBJECTIVE EVALUATION 3.1 INTRODUCTION This chapter deals with subjective evaluation r e l a t e d to voice communication systems. A b r i e f comment i s f i r s t offered i n Section 3.2 on the differences between evaluation of systems by the means of a r t i c u -l a t i o n and i n t e l l i g i b i l i t y t e s t s , a n a l y t i c a l means such as s i g n a l to noise power r a t i o s , and subjective tests based on preference, such as the i s o -preference method. Section 3.3 of t h i s chapter outlines isopreference t e s t i n g as used i n th i s paper. Section 3.4 then describes the phase of data preparation. The sentence used f o r evaluation purposes i s presented, with arguments f or i t s choice, followed by a d e s c r i p t i o n of the method involved i n producing the test material. The section 3.5 which concludes the chapter-describes the tests themselves. 3.2 SUBJECTIVE TESTING RATIONALE A b r i e f comment i s i n order concerning the evaluation of a sys-tem using a r t i c u l a t i o n and i n t e l l i g i b i l i t y tests as opposed to using sub-j e c t i v e tests based on preference, such as the isopreference method. A r t i c u l a t i o n t e s t i n g pertains to the comprehension of units of speech material c o n s i s t i n g of meaningless s y l l a b l e s or fragments of speech. I n t e l l i g i b i l i t y t e s t i n g r e f e r s to the comprehension of phonetically b a l -anced units of speech material such as meaningfull words, phrases or sentences [M2]. The two terms however are often confused as i s the term ' a r t i c u l a t i o n index'. For th i s reason the term a r t i c u l a t i o n w i l l be avoided. Instead the ' i n t e l l i g i b i l i t y index' w i l l be defined as the percentage of units of speech c o r r e c t l y i d e n t i f i e d during an i n t e l l i g i b i l i t y t e st. An i n t e r e s t i n g observation to make i s that subjective tests do not n e c e s s a r i l y r e f l e c t i n t e l l i g i b i l i t y . That i s , as long as a high q u a l i t y s i g n a l i s being used, the subjective t e s t s , s i m i l a r to the one described i n this work, are e s s e n t i a l l y independent of i n t e l l i g i b i l i t y . This s t a t e -ment can be v e r i f i e d by considering the elements a f f e c t i n g i n t e l l i g i b i l i t y of a processed s i g n a l , the main two elements being f i l t e r i n g , and d i s t o r -t i o n caused by system noise. Concerning the tests c a r r i e d out i n t h i s thesis a s i g n a l bandlimited from 200 to 3200 Hz retains an i n t e l l i g i b i l i -ty index of approximately 90% [C6, K l ] , I t i s also i n t e r e s t i n g to note that by using a closed set of test samples the e f f e c t of i n t e l l i g i b i l i t y losses due to system noise can be ignored. This i s due to the f a c t that complete knowledge of the test material by the l i s t e n e r s removes the s t i p -u l a t i o n that the threshold of recognition of a word heard i n noise be i n -versely proportional to the logarithm, of i t s frequency of occurrence [Wl], Knowledge of the t e s t material therefore has the e f f e c t of t e s t i n g with samples having an apparent i n t e l l i g i b i l i t y index of 100% even though i t may be some what l e s s . The preceeding discussion i s presented not to cast doubt on the worthiness of subjective tests but rather to c l a r i f y the difference be-tween a subjective r a t i n g such as an isopreference contour and an i n t e l l i -g i b i l i t y score. The converse of the above discussion states that while a s i g n a l may be 100% i n t e l l i g i b l e i t may not possess, from a subjective point of view, the q u a l i t y or naturalness of the o r i g i n a l s i g n a l . I t i s f o r t h i s reason that methods such as isopreference tests are necessary. Although points along an isopreference contour may conceivably possess d i f f e r e n t intelligibility scores, one would expect that highly rated signals will reflect relatively high intelligibility indices in which case limits on intelligibility would be set by factors such as audio bandwidth. A second argument for subjective tests arises from the inability of analytical methods, for example signal to noise power ratios, to reflect the signal quality as perceived by human subjects [C6, G4]. Increasingly, the practice has been to include some form of subjective testing of a system in its analysis. Various methods of subjective evaluation have been proposed and tested [M2, G6, M4]. The three main methods have been presented and discussed in "IEEE Recommended Practices for Speech Quality Measurements" [II]. 3.3 THE ISOPREFERENCE METHOD 3.3.1 General Description Originally proposed by Munson and Karlin [M4] the isopreference method has been studied and applied by numerous researchers [DI, RI, T2, Yl]. Because the method has been adequately described in numerous papers, only a brief description is given here. The isopreference method assumes that the speech signals under test can be judged on the unidimensional scale of preference. This assump-tion allows a series of isopreference contours to be drawn on a plane whose axes are measures of the parameters under test. Points that l i e on the contours are determined by a series of paired comparison tests pre-sented in random order. A test signal, for example point A in Figure 3.1 a, is initial l y picked to define the subject quality of one curve. This signal, whose parameters are held constant is then compared to another signal with one varying parameter. As the parameter is varied a value is obtained for which a l l listeners show an equal preference for both signals. 20. The two signals are then declared to be isopreferent. Whether parameter a or parameter (3 of Figure 3.1a i s varied depends on the expected shape of the contour. Since the parameters are varied by d i s c r e t e amounts the r e s u l t s of the comparisons are generally expressed i n proportions of subjects not p r e f e r r i n g the test s i g n a l . The r e s u l t s are then plotted against the varying parameters. A smooth psychometric curve i s then drawn through the experimental points as shown i n Figure 3.1b. From this curve the abscissa corresponding to a proportion of one-half defines the value of the varying parameter that defines the isopreferent point. Repeating th i s process using various values of a and g r e s u l t s i n an isopreference curve being drawn through the o r i g i n a l test s i g n a l . 3.3.2 Scaling Isopreference Contours As isopreference curves generally include points possessing d i f -ferent s i g n a l to noise r a t i o s i t i s desirable to attach a common standard of q u a l i t y to each curve. Various speech r a t i n g standards have been pro-posed and tested [Dl, HI, R l , S2, S3]. The method of s c a l i n g generally used i s that of comparing a test s i g n a l on each contour to a family of standard reference s i g n a l s . These reference signals are generated by adding varying amounts of a degrada-t i o n s i g n a l to a high q u a l i t y s i g n a l . Paired comparison tests are then used to determine which reference signals are isopreferent to the test s i g n a l s . The amount of degradation i n the isopreferent reference s i g n a l s , given by subjective s i g n a l to noise measure, i s then attached to the i s o -preference contours from which the respective test signals were taken. In t h i s study the method of generating a family of reference signals introduced by Schroeder [S2] i s used. The method produces r e f -21. fb) F i g . 3.1 (a) (b) A t y p i c a l isopreference contour A t y p i c a l psychometric curve e r e n c e s i g n a l s d e f i n e d b y t h e e q u a t i o n _ 1 r a ( t k ) = ( 1 + a 2) 2 [ s ( t k ) + a • n ( t k > ] ( 3 . 1 ) w h e r e a d e f i n e s t h e s i g n a l t o n o i s e r a t i o , S N R , ., i n d B v i a : ° s u b j S N R , . = 1 0 l o g i n a " 2 ( 3 . 2 ) s u b j ° 1 0 T h e n o i s e s a m p l e n ( t j c ) i s o b t a i n e d b y m u l t i p l y i n g t h e s i g n a l s a m p l e s C ^ ) b y a z e r o m e a n d e s c r e t e s t o c h a s t i c p r o c e s s e ( t ^ ) = ± 1 w h i c h i s u n c o r r e -c t e d w i t h t h e s i g n a l . S u c h a m e t h o d w a s c h o s e n o v e r d e g r a d a t i o n u s i n g w h i t e g a u s s i a n n o i s e , s i n c e t h e n o i s e i n t r o d u c e d b y D P C M c o d i n g i s s i g n a l d e p e n d e n t . A s n o t e d b y S c h r o e d e r , d e g e n e r a t i o n w i t h w h i t e g a u s s i a n n o i s e d o e s n o t r e s u l t i n t h e s a m e s u b j e c t i v e q u a l i t y d e g r a d a t i o n , a s q u a n t i z a t i o n n o i s e t h e r e b y m a k i n g c o m p a r i s o n m o r e d i f f i c u l t w h e n t h e t w o t y p e s o f n o i s e a r e c o m p a r e d . 3 . 4 P R E P A R A T I O N O F S P E E C H M A T E R I A L T h e s p e e c h m a t e r i a l c h o s e n f o r t h e s u b j e c t i v e e v a l u a t i o n o f C o h n a n d M e l s a ' s D P C M - A Q B s y s t e m w a s t h e p a i r o f s e n t e n c e s " J o e t o o k f a t h e r ' s s h o e b e n c h o u t . S h e w a s w a i t i n g a t my l a w n " . T h e p a i r w a s n o m -i n a l l y l o w p a s s f i l t e r e d a t 3 2 0 0 H z i n k e e p i n g w i t h C o h n a n d M e l s a ' s s t u d y a n d h i g h p a s s f i l t e r e d a t 2 0 0 H z t o e l i m i n a t e a n y l o w f r e q u e n c y n o i s e s u c h a s 6 0 c y c l e h u m . T h e s e n t e n c e " J o e . . . l a w n " w a s s p o k e n b y a t h i r t y - e i g h t y e a r o l d m a l e w i t h a w e s t e r n C a n a d i a n a c c e n t . T h e s e n t e n c e w a s r e p e a t e d i n a n I n d u s t r i a l A c o u s t i c s C o m p a n y m o d e l 1 2 0 5 - A q u i e t r o o m . A f u l l b a n d w i d t h r e c o r d i n g w a s t h e n o b t a i n e d u s i n g a s i n g l e t r a c k S c u l l y 2 8 0 r e c o r d e r o p e r -a t i n g a t 1 5 i . p . s . w i t h l o w n o i s e A m p e x 4 3 4 a u d i o t a p e a n d a B r u e l a n d K j o e r T y p e 2 8 0 1 p o w e r a m p l i f i e r a n d m i c r o p h o n e s e t . 23. Speech statistics of the sentence "Joe ... lawn" are given in Figure 3.2a. Chan and Donaldson [C2] have shown that the amplitude prob-ability of the test sentence normalized with respect to its R.M.S. value is reasonably close to that of both Gaussian and Laplacian distributions, both of which have been suggested as models of the amplitude density of speech. Benson and Hirch [BI] have compared the spectrum of the sentence to samples of news and technical material and found them to be not signifi-cantly different (Figure 3.2b). The sentence "Joe ... lawn" can therefore be regarded as a reasonable representation of conversational speech. Once recorded the sentence was played back and digitized using the system described by Chan [C3]. Eleven master samples were produced by repeatedly playing back the sentence and adjusting the effective sam-pling rate from 6,400 to 22,400 Hz at intervals of 3200 Hz. The sampled signals were uniformly quantized to twelve bits and stored on nine - track IBM compatible digital magnetic tape. The tapes were then transported to the IBM 370/168 facilities where processing was accomplished. A l l samples were initially normalized to a mean .of zero thereby eliminating any d.c. bias introduced by the band-pass filters. Simulation of the DPCM - AQB algorithm was carried out in the PL/1 programming language. Each of the eleven master samples was processed six times using each of the six quan-tizers described in Chapter 2. This produced sixty-six samples which com-p r i s e d ' the main data base. The Nyquist - sampled master signal was then processed using the Schroeder algorithm described in Section 3.3.2. A family of standard ref-erence signals were thereby obtained with SNR^^j values ranging from -2 to 34 dB in steps of 2 dB. These were added to the data base. Samples to be used in the subjective listening tests were then transferred from the data base to other nine - track tapes in the order in i o r 24. i • • I F i g . 3.2 (a) Normalized amplitude p r o b a b i l i t y density of speech. Symmetrical aver-age of p o s i t i v e and negative data. CO C l K » CQ o Q: CL U I Q ^ » i— — i U. •001 0 1 2 3 4 5 6 I N S T A N T A N E O U S S P E E C H A M P L I T U D E R E L A T I V E TO R M S ,, -(a) which they were to appear on the analogue test tapes. These nine - track tapes were then returned to the facilities previously mentioned and passed back through digital to analogue converters and filters to produce the analogue test tapes. During analogue to digital conversion loudness was controlled by monitoring the record amplifier of the Scully tape deck. 3.5 SUBJECTIVE TEST PROCEDURE 3.5.1 Test Format A total of seven one-half hour sessions were used to accumulate the data analyzed in Chapters 4. Each session consisted of paired com-parison tests. The sentence "Joe took father's shoe bench out. She was waiting at my lawn" was used in a l l cases. A l l tests were conducted with the guidelines of the IEEE recommendations in mind [II]. Each paired comparison was presented in a set as shown in Fig-ure 3.3. The first speech sample of a pair, designated as A, and the second, as B, were immediately repeated to form one set. Each sentence of a set was preceeded by a one second pause and each set was followed by a three second pause during which time the subjects could mark their de-cision, or preference, on supplied answer forms. A tone indicated the beginning of a new set. In the course of the tests each pair was present-ed a second time with samples A and B appearing in reverse order. The half hour sessions were divided into two parts. Thirty-one sets were presented during the first fifteen minutes. A five minute break followed during which limited discussions were held on the topic under study.in an attempt to increase interest and eliminate fatigue. Twenty more sets were presented in the remaining ten minutes. The first set of each session was a familiarization set and was not included in the results. During this set participants were allowed to adjust their volume controls O 0 SPEECH SAMPLE Al SPEECH SAMPLE BI 10~ SPEECH SAMPLE Al 75 SPEECH SAMPLE BI ~20 Lu 25 SPEECH SAMPLE A2 to 35~ TIME - SECONDS F i g . 3.3 Block representation of a paired comparison t e s t set. 28. from a preset medium as chosen from the r e s u l t s of the p i l o t t e s t (see Section 3.5.2). No further adjustment i n volume was allowed. The sessions were conducted i n a quiet classroom. The tapes were played back on the S c u l l y 280 tape recorder, through an a m p l i f i e r to nine i n d i v i d u a l volume controls. Sharpe HA - 10 - MK - II and Jensen model 220 stero headsets were used for the test s . Both headsets demon-strated s i m i l a r frequency response curves and possessed -40 dB i s o l a t i o n c h a r a c t e r i s t i c s . P r i o r to the l i s t e n i n g sessions the l i s t e n e r s were read the following i n s t r u c t i o n s : "The following speech samples are the r e s u l t of a sentence which has been processed by a communications systems algorithm i n which several parameters have been v a r i e d . The samples w i l l be presented i n p a i r s . The f i r s t speech sample of each p a i r w i l l be designated as A, the second as B. Each p a i r w i l l be immediately repeated. A three second pause w i l l follow to allow you to mark on the answer sheet which of the two speech samples you would prefer to l i s t e n to. A 'tone' w i l l i n d i c a t e the beginning of the next set of p a i r s . In making your decision please ignore any c l i c k s that may occur immediately before or a f t e r each speech sample. Also please try to ignore any volume d i f f e r e n c e s . Please be as at t e n t i v e as possible f o r a lack of concentration w i l l lead to confusion. In the case of two samples being of equal preference, i n your opinion, choose the second sample. The sentence you w i l l hear i s 'Joe took father's shoe bench out. She was waiting at my lawn.'" A t o t a l of eighteen l i s t e n e r s , f i f t e e n male and three female, p a r t i c i p a t e d i n the t e s t s . A l l were u n i v e r s i t y students ranging i n age from eighteen to thirty-two years and representing various cultural back-grounds. A l l had no previous experience in listening test and exhibited no hearing abnormalities as tested for in the pilot test, or other hearing abnormalities known to themselves. All listeners participated in every test. 3.5.2 Objective of the Tests The seven test sessions were divided into three test aspects: a pilot test, a test for determining isopreference contours, and a test for rating the isopreference contours. The three tests comprised one, four, and two sessions respectively and spanned a period of four weeks. The pilot test was run with three objectives in mind. The first objective was to choose a set of points that could be used for sub-sequent measurement of the subjective quality of the proposed isoprefer-ence contours. One of these points would appear in each of the paired comparisons of the next .two test aspects. The second objective was to ensure that listeners were capable of consistently discerning speech quality of signals whose SNR values were within 3-6 dB of each other. The third objective was to permit subjects to select their individual volume settings. A l l volume controls would be initially preset to a single value as determined by the mean of these settings for a l l ensuing tests. The test for determining the isopreference curves covered four sessions. The first session was used to discover the general characteris-tics of curves defined by the points chosen from the pilot test. The re-maining three sessions were dedicated to defining precisely the isopref-erence contours. Results are presented in Chapter 4. The third test aspect involved two sessions and utilized Schroeder' reference signals. As long as transitivity of subjective preference can be assumed along the curves, and t h i s i s one of the basic assumptions of the isopreference method, then r a t i n g any point on a contour i s equivalent to r a t i n g the whole contour. On t h i s assumption, each of the te s t points as arriv e d at i n the p i l o t t e s t was compared to the reference signals to de-termine i t s isopreferent "mate". The value of SNR , . of the reference subj s i g n a l was then attached to the respective curve. Included i n these tests were two extra points to test the . t r a n s i t i v i t y assumption. In t h i s way a complete set of data was c o l l e c t e d to which the isopreference analysis method could be applied. I V R E S U L T S O F S U B J E C T I V E T E S T S 4 . 1 I N T R O D U C T I O N T h e l i s t e n i n g t e s t s d e s c r i b e d i n C h a p t e r 3 r e s u l t e d i n a n a c c u m -u l a t i o n o f d a t a b a s e d o n p r e f e r e n c e . T h i s d a t a w a s a n a l y z e d i n o r d e r t o d e t e r m i n e i s o p r e f e r e n c e c o n t o u r s . S e c t i o n 4 . 2 o u t l i n e s t h e m e t h o d u s e d t o d e t e r m i n e t h e v a l u e o f a p a r a m e t e r w h i c h r e s u l t s i n o n e s i g n a l b e i n g i s o -p r e f e r e n t t o a t e s t s i g n a l . T h e i s o p r e f e r e n c e c o n t o u r s a s d e t e r m i n e d b y t h e d a t a c o l l e c t e d i n t h i s t h e s i s a r e p r e s e n t e d i n S e c t i o n 4 . 3 . A c o m p r e -h e n s i v e d i s c u s s i o n o f t h e r e s u l t s a n d t h e i r i m p l i c a t i o n s i s g i v e n i n S e c -t i o n 4 . 4 a n d 4 . 5 . 4 . 2 D E T E R M I N A T I O N O F E X P E R I M E N T A L I S O P R E F E R E N C E C O N T O U R S T h e m e t h o d o u t l i n e d i n S e c t i o n 3 . 3 . 1 w a s u s e d t o d e t e r m i n e t h o s e v a l u e s o f t h e i n d e p e n d e n t s y s t e m p a r a m e t e r s f o r w h i c h a r e c o n s t r u c t e d s p e e c h s i g n a l i s i s o p r e f e r e n t t o a t e s t s i g n a l . A f t e r p l o t t i n g t h e p r o -p o r t i o n o f l i s t e n e r s n o t p r e f e r r i n g t h e t e s t s i g n a l a s m o o t h c u r v e c a n b e d r a w n t h r o u g h t h e p o i n t s w h i c h i s a s s u m e d t o b e a c u m m u l a t i v e n o r m a l c u r v e r e l a t i n g t h e p r o p o r t i o n , p , t o t h e p a r a m e t e r v a l u e s , L . ( S e e F i g u r e 4 . 1 a . ) T h e K o l m o g o r o v - S m i r n o v ( K - S ) g o o d n e s s o f f i t t e s t [ L I ] w a s u s e d o n a l l d a t a t o t e s t t h e a s s u m p t i o n o f n o r m a l i t y . T h e s t a t i s t i c u s e d i s t h e m a x i m u m a b s o l u t e d e v i a t i o n o f t h e e x p e r i m e n t a l c u r v e ? n ( x ) t o f o r m t h e h y p o t h e s i z e d c u r v e E ^ ( x ) r e p r e s e n t e d b y D ^ i n e q u a t i o n 4 . 1 . D = S U P | F ( x ) - F ( x ) j ( 4 . 1 n x 1 n o 1 A l l b u t a f e w o f t h e c u m m u l a t i v e c u r v e s r e s u l t i n g f r o m t h e t e s t d a t a p a s s e d t h e K - S t e s t a t a s i g n i f i c a n c e l e v e l o f . 0 1 . P R O P O R T I O N O F L I S T E N E R S N O T P R E F E R R I N G T E S T S I G N A L r t It X H* fl> H •'O CO D. fD (D 3 H l-h PJ f> H-H- 0? (C S 5 & U N I T N O R M A L D E V I A T E S O F P R O P O R T I O N O F L I S T E N E R S » < . f f ? A/ 0 7 " P R E F E R R I N G T E S T SIGNAL Once the normality c r i t e r i o n had been j u s t i f i e d the p values were transformed i n t o measures of unit normal deviates, z. (See Figure 4.1b.) An approximately l i n e a r r e l a t i o n s h i p between z and L resulted. A l e a s t -square s o l u t i o n using Muller-Urban weights was used to f i t a s t r a i g h t l i n e y = a + bx (4.2) to the data points. The estimated mean x and the estimated standard devi-ation S could then be obtained from (4.3) and (4.4). X x = ~ (4.3) S = h (4-4) X b This mean was then taken as the value of L which produced the isopreferent s i g n a l . The standard deviation was inserted i n (4.5) to calculate the 95% confidence i n t e r v a l f o r the mean [LI]. Let u represent the population mean of which x i s the estimate, where S S x - t x < u < x + t — — (4.5) /n-1 /n-1 The s i z e of the confidence i n t e r v a l i s given by 100(1 - a) % where a i s the s i g n i f i c a n c e l e v e l , t ^ i s a tabulated value corresponding to a t - d i s t r i b u -t i o n , and n i s the sample s i z e . 4.3 PRESENTATION OF TEST RESULTS The isopreference contours as determined from the isopreference points obtained using the method described i n Section 4.2 are presented i n Figure 4.2. The two parameters which define the plane are the number of quantization l e v e l s , L, used i n the DPCM - AQB quantizer, and the r a t i o of sampling frequency to the Nyquist frequency, f / f .. s n 34. 0* 1 1 1 : 1 : I I 2 3 4 5 6 7 N U M B E R O F Q U A N T I Z A T I O N L E V E L S L F i g . 4 . 2 D P C M - A Q B i s o p r e f e r e n c e c o n t o u r s . T h e t e s t s i g n a l s o f e a c h c o n t o u r a r e m a r k e d " x " . T w o t r a n s i t i v i t y t e s t s i g n a l s a r e m a r k e d " o " . 9 5 % c o n f i d e n c e i n t e r v a l s a r e d e n o t e d b y a b a r t h r o u g h e a c h e x p e r i m e n t a l p o i n t . S N R g u ^ j v a l u e s a r e g i v e n i n d B a s a r e t h e S N R v a l u e s w h i c h a p p e a r e n c l o s e d i n b r a c k e t s . T h e t e s t s i g n a l s a s d e t e r m i n e d b y t h e p i l o t t e s t a n d u s e d i n t h e p a i r e d c o m p a r i s o n t e s t s t o e s t i m a t e t h e i s o p r e f e r e n c e c o n t o u r s a r e m a r k e d b y a n " X " . B e s i d e e a c h o f t h e s e p o i n t s i s g i v e n t h e e s t i m a t e d S N R ^ ^ j , a n d i t s a s s o c i a t e d 9 5 % c o n f i d e n c e l i m i t s . B e n e a t h t h e s e v a l u e s i n p a r e n -t h e s e s a r e t h e c o m p u t e d S N R ' s o f t h o s e p o i n t s . Because of the assumption o f t r a n s i t i v i t y a l o n g t h e i s o p r e f e r e n c e c o n t o u r s t h e S N R g u ^ j o f a t e s t s i g -n a l a p p l i e s t o a l l p o i n t s o n t h a t c u r v e . T h e S N R ^ ^ j o f t w o o t h e r p o i n t s m a r k e d " 0 " h a s a l s o b e e n d e t e r m i n e d . T h e i r v a l u e s s u p p o r t t h e a s s u m p t i o n o f t r a n s i t i v i t y a n d a l s o i n d i c a t e l i s t e n e r j u d g e m e n t c o n s i s t e n c y . T h e p o i n t s d e t e r m i n e d e x p e r i m e n t a l l y a s b e i n g i s o p r e f e r e n t t o t h e t e s t s i g n a l s a r e m a r k e d A b a r t h r o u g h e a c h p o i n t i n d i c a t e s t h e 9 5 % c o n f i d e n c e i n t e r v a l o f t h e m e a n c a l c u l a t e d u s i n g ( 4 . 5 ) . T h e u n i t o f m e a s u r e f o r e a c h i n t e r v a l i s d e f i n e d b y t h e a x i s t o w h i c h i t i s p a r a l l e l . T h e c u r v e s t h e m s e l v e s w e r e b a s e d o n b e s t v i s u a l f i t s t o t h e d a t a p o i n t s . C o n s t r a i n t s a f f e c t i n g t h e i r p o s i t i o n i n g w e r e t h a t t h e y h a v e t h e s a m e g e n e r a l s h a p e a s n e i g h b o r i n g c u r v e s , a n d t h a t t h e y b e d r a w n c l o s e t o p o i n t s p o s s e s s i n g s m a l l c o n f i d e n c e i n t e r v a l s . 4 . 4 D I S C U S S I O N O F T H E T E S T R E S U L T S 4 . 4 . 1 D i s c u s s i o n o f I s o p r e f e r e n c e C o n t o u r s S e v e r a l f a c t s c a n b e d e d u c e d f r o m F i g u r e 4 . 2 . T h e s e w i l l b e p r e s e n t e d i n t h i s a n d f o l l o w i n g s e c t i o n s . A s t h e s a m p l i n g f r e q u e n c y i s i n c r e a s e d t h e c o r r e l a t i o n b e t w e e n s a m p l e s i s a l s o i n c r e a s e d . T h i s i n c r e a s e d c o r r e l a t i o n a l l o w s t h e a d a p t a -t i o n s t r a t e g y o f t h e q u a n t i z e r t o b e t t e r f o l l o w t h e i n p u t s i g n a l s t a t i s t i c s r e s u l t i n g i n a h i g h e r q u a l i t y o u t p u t s i g n a l . F i g u r e 4 . 2 s h o w s h o w e v e r , that most of t h i s improvement takes place i n the f i r s t stages of frequency increase. Beyond this a saturation zone i s encountered i n which increasing the sampling frequency has a l e s s e r e f f e c t on s i g n a l q u a l i t y . The cause of t h i s may be a t t r i b u t e d to quantization noise. That i s , any further gain made by increasing sample c o r r e l a t i o n i s masked by the dominant quantizing noise. To the designer, s a t u r i z a t i o n zones of t h i s type mean a l i m i t a -t i o n or lower bound on parameter values. For example, to obtain a subj-e c t i v e q u a l i t y of 25 dB only quantizers with f i v e or more l e v e l s need be considered. The p l o t also shows that c e r t a i n tradeoffs are possible between the number of quantization l e v e l s and the sampling rate. For example, the much more e a s i l y implemented d e l t a modulator could replace a f i v e l e v e l quantizer simply by oversampling at 3.25 times the Nyquist rate to obtain a subjective performance of approximately 13.5 dB. One consideration that may detract from carrying out such a replacement may be b i t rate con-si d e r a t i o n s . This.aspect i s presented i n the following section. 4.4.2 Entropy Coding and Minimum Required Channel Capacity Most studies to date on DPCM and ADPCM systems involve quantizers f o r which L = 2 b (4.6) where L denotes the number of quantization l e v e l s and b i s normally an i n -teger equal to the number of b i t s required to code each input sample. For equiprobable quantizer output l e v e l s , such a scheme r e s u l t s i n a minimum b i t rate or minimum required channel capacity of C = b • f b i t s per second (4.7) 3 7 . O n t h e b a s i s o f ( 4 . 7 ) t h e b i t r a t e r e s u l t i n g f r o m a l l c o m b i n a t i o n s o f q u a n t i z a t i o n l e v e l s a n d s a m p l i n g f r e q u e n c y h a s b e e n c a l c u l a t e d . T h e s o l i d c u r v e s o f F i g u r e 4 . 3 r e p r e s e n t p a t h s o f e q u a l b i t - r a t e . S u p e r i m p o s e d o n t h e s e c u r v e s a r e t h e i s o p r e f e r e n c e c o n t o u r s o f F i g u r e 4 . 2 . A s a n y t w o c u r v e s , o n e f r o m e a c h s e t , i n t e r s e c t a t o n l y o n e p o i n t i t b e c o m e s o b v i o u s t h a t t h e m i n i m u m b i t r a t e f o r a g i v e n p r e f e r e n c e l e v e l o c c u r s w h e n s a m p l i n g i s a t t h e N y q u i s t r a t e , o r j u s t f a r e n o u g h a b o v e t h e N y q u i s t r a t e t o e n s u r e i n t e g e r v a l u e s o f L . T h e o n l y e x c e p t i o n t o t h i s r u l e m a y a p p l y i n v e r y l o w q u a l i t y r e g i o n s w h e r e t h e t w o s e t s o f c u r v e s b e c o m e a l m o s t p a r a l l e l . T h e s e m i n i m u m r e q u i r e d c h a n n e l c a p a c i t i e s a s e s t i m a t e d f r o m F i g . 4 . 3 a r e p l o t t e d a g a i n s t S N R g u b _ . v a l u e s i n F i g . 4 . 4 . T h e b a r s t h r o u g h , t h e p o i n t s i n d i c a t e t h e 9 5 % c o n f i d e n c e i n t e r v a l s . T h e s h a d e d r e g i o n o f t h e g r a p h i s b o u n d e d o n o n e s i d e b y a l i n e f i t t e d t o t h e l o w e r f o u r p o i n t s b y t h e l e a s t - s q u a r e s m e t h o d , a n d o n t h e o t h e r s i d e b y t h e m i n i m u m o b t a i n a b l e b i t - r a t e f o r t h e v a l u e o f p a r a m e t e r s c o v e r e d b y t h i s s t u d y . I f b i t r a t e a n d p r e f e r e n c e l e v e l a r e h e l d c o n s t a n t i n F i g u r e 4 . 3 , t h e o n l y d e s i g n i m p l e m e n t a t i o n t r a d e o f f s a p p e a r i n t h e l o w q u a l i t y r e g i o n o f t h e p l o t . O u t s i d e o f t h i s r e g i o n i t w o u l d b e n e c e s s a r y t o d e s i g n a r o u n d a q u a n t i z e r w h i c h i s p o s i t i o n e d a t t h e i n t e r s e c t i o n o f t h e g i v e n b i t r a t e a n d p r e f e r e n c e c u r v e s . I t i s p o s s i b l e t o r e d u c e b i t r a t e s b y e m p l o y i n g c o d i n g s c h e m e s w h i c h t a k e a d v a n t a g e o f t h e f a c t t h a t q u a n t i z e r o u t p u t l e v e l s a r e n o t n o r m -a l l y e q u i p r o b a b l e . C o d i n g s c h e m e s s u c h a s t h e s e , d e v e l o p e d a r o u n d q u a n -t i z e r s t a t i s t i c s , a r e r e f e r r e d t o a s e n t r o p y c o d i n g . C o h n a n d M e l s a h a v e p r o p o s e d s u c h a s o u r c e c o d i n g s c h e m e t o r e d u c e t h e b i t r a t e o f t h e i r s y s t e m . 38. F i g 4.3 Curves of constant b i t rate. Superimposed are the isopreference contours (dashed) of F i g . 4.2. 3 9 . 2 5 r Q\ L _ . — i : , 0 5 10 7 5 20 2 5 3 0 M I N I M U M R E Q U I R E D C H A N N E L C A P A C I T Y K B I T S / S E C F i g . 4.4 Subjective ratings of F i g . 4.2 p l o t t e d against t h e i r respective minimum required channel c a p a c i t i e s . 4 0 . F i g . 4 . 5 C u r v e s o f c o n s t a n t b i t r a t e u s i n g e n t r o p y c o d i n g . S u p e r i m p o s e d a r e t h e i s o p r e f e r e n c e c o n t o u r s ( d a s h e d ) o f F i g . 4 . 2 . 41. The lower l i m i t on the number of b i t s required to encode a quan-t i z e r output l e v e l can be obtained by c a l c u l a t i n g the entropy of the quan-t i z e r output samples [ G l ] . N 1 H(L) = I p(L ) log ( - p i y ) (4.8) i = l 1 2 P C V where p(L^) i s the p r o b a b i l i t y of occurrence of output l e v e l L^ .?" Note that c o r r e l a t i o n s i n adjacent output samples have been ignored, since such co-r r e l a t i o n s are minimized by quantizing the difference s i g n a l s . Contours of constant b i t rate as defined by the entropy of the source are indicated by the s o l i d curves of Figure 4.5. Superimposed are the isopreference contours from Figure 4.2. As with the curves of constant b i t rate without entropy coding, i t i s seen that for a given preference l e v e l the minimum b i t rate occurs when sampling at or near the Nyquist rate minimum required channel capaci-t i e s have been p l o t t e d i n Figure 4.5 as previously described. Comparison of t h i s l i n e and the l i n e without entropy coding indicates that f o r SNR g^ values greater than 4 dB entropy coding w i l l r e s u l t i n a saving of b i t rate and that the magnitude of th i s saving increases with increasing SNR g u^ . Entropy coding presents the designer with several options. For example, a system requiring an SNR^^j °f approximately 16 dB with a b i t rate of 16 k bps r e s u l t s from using e i t h e r a three- or f i v e - l e v e l quantizer. This example also reveals that f or a given b i t rate, entropy coding produ-ces better speech q u a l i t y when implemented with quantizers having an odd number of l e v e l s than with neighboring even l e v e l quantizers. To give a c l e a r e r p icture of the amount of b i t rate reduction possible through the use of entropy coding techniques f o r t h i s system, a "*"The frequency with which quantizer l e v e l s occurred during the processing of the master samples to form the data base (Section 3.4) were used to ca l c u l a t e the entropy of the source. m a t r i x o f r e d u c t i o n c o e f f i c i e n t s , r , h a s b e e n c a l c u l a t e d a n d i s s h o w n i n F i g u r e 4 . 6 w h e r e b i t r a t e ( e n t r o p y c o d i n g ) „\ r ~ b i t r a t e D M h a s t h e s p e c i a l p r o p e r t y o f a l w a y s b e i n g e n t r o p y c o d e d a n d t h e r e f o r e p o s s e s a c o e f f i c i e n t o f u n i t y . D e l e t i n g t h e u n i q u e c a s e o f DM, d i v i d i n g t h e r e m a i n i n g m a t r i x i n t o t w o s e c t i o n s a n d a v e r a g i n g o v e r e a c h s e c t i o n y i e l d s : E [ r ] = .7 f o r f / f > 2 . 0 L 1 s n ( 4 . 1 0 ) E [ r ] = .8 f o r f / f . < 2 . 0 s n — I n o t h e r w o r d s t h e b i t r a t e , o n t h e a v e r a g e , c a n b e r e d u c e d f r o m b e t w e e n 2 0 % t o 3 0 % b y e m p l o y i n g c o d i n g b a s e d o n t h e e n t r o p y o f t h e q u a n t i z e r o u t -p u t l e v e l s . M e l s a a n d C o h n h a v e r e c o r d e d a n e n t r o p y o f 1 . 3 7 b i t s u s i n g N y q u i s t r a t e s a m p l i n g a n d a f i v e l e v e l q u a n t i z e r w i t h a n a d a p t i v e p r e -d i c t o r . T h e r e s u l t w a s a r e d u c t i o n c o e f f i c i e n t o f r = . 6 2 c o m p a r e d t o r = . 7 8 w i t h o u t t h e a d a p t i v e p r e d i c t o r . I t s e e m s t h a t a f u r t h e r s a v i n g s i n b i t r a t e c a n b e a c c o m p l i s h e d a t t h e e x p e n s e o f t h e c o m p l e x i t y i n v o l v e d i n a d a p t i v e p r e d i c t i o n . A g e n e r a l t r e n d i n d i c a t e d b y t h e r e d u c t i o n c o e f f i c i e n t m a t r i x i s a n i n c r e a s e o f m a g n i t u d e o f r w i t h i n c r e a s i n g f / f . s n. 4 . 4 . 3 S N R C o m p a r i s o n s T h e u l t i m a t e m e a s u r e o f p e r f o r m a n c e o f a s p e e c h d i g i t i z a t i o n s c h e m e i s t h e s u b j e c t i v e q u a l i t y a s p e r c e i v e d b y a h u m a n l i s t e n e r . O t h e r m e a s u r e s o f p e r f o r m a n c e c a n o n l y b e u s e d t o i n d i c a t e s u b j e c t i v e q u a l i t y . O n e o f t h e m o s t c o m m o n o f t h e s e i s t h e s i g n a l t o n o i s e p o w e r r a t i o g i v e n b y _fs_ fn 3.5r 3.0 2.5 2.0 1.5 10 0 © 7.0 1 o 1 .62 © .58 .60 e .77 a .72 7*0 e 9 7 0 .63 o .73 9 -75 e 1.0 e 1 .65 9 71 9 .64 • .75 « .77 9 10 « 1 .65 9 7 2 9 •55 .75 e .77 > 9 1.0 1 0 1 .68 e 7 4 .55 e .75 •78 | 10 1 © 9 74_ _ • • _ - Z 7 _ e J 9 10 e 1 .70 9 7 6 • .58 .7$ 9 1.0 e 1 73 7 8 •77 .80 9 .87 , • 1,0 © 1 .76 .80 • .73 •S3 .83 > 0 7.0 ' .77 • 9 •89 » .87 • .91 9 .90 9 7.0 l 1 .80 i 9 •87 i « .78 i 9 •88 i .85 2 3 : 4 5 5 7 ^ C 0 = -7 N U M B E R O F Q U A N T I Z A T I O N LEVELS L Fig. 4.6 Matrix of b i t rate reduction c o e f f i c i e n t s r for entropy coding. u> 4 4 . M s 2 ] S N R = 1 0 l o g i n ± U E [ ( s - s ) Z ] ( 4 . 1 0 ) w h e r e s r e p r e s e n t s t h e o r i g i n a l i n p u t s i g n a l a n d § i s t h e o u t p u t o r r e -c o n s t r u c t e d s i g n a l . A l t h o u g h a u s e f u l g u i d e t o m e a s u r i n g r e l a t i v e p e r -f o r m a n c e b e t w e e n s i g n a l s c o n t a i n i n g v a r y i n g a m o u n t s o f a c h a r a c t e r i s t i c n o i s e t y p e , S N R d o e s n o t n e c e s s a r i l y r e f l e c t a s y s t e m ' s s u b j e c t i v e p e r -f o r m a n c e w h e n d i f f e r e n t n o i s e t y p e s a r e p r e s e n t . A n e x p e r i m e n t h a s b e e n c o n d u c t e d t o c l e a r l y i n d i c a t e t h i s i n -c o n s i s t a n c y . T a b l e 4 . 1 p r e s e n t s t h e S N R v a l u e s c o m p u t e d f o r t w o s e t s o f d a t a l a b e l l e d A a n d B . T h e m a s t e r s a m p l e s w e r e p r o c e s s e d t o p r e s e n t a d i a g o n a l c r o s s e c t i o n o f t h e p l a n e d e f i n e d b y t h e t w o p a r a m e t e r s L a n d f ' / f . s **• L f If s n S N R A d c = 0 S N R B d c = 2 0 2 1 . 0 4 . 9 1 5 . 1 4 3 1 . 5 1 2 . 6 6 1 2 . 7 1 4 2 . 0 1 5 . 9 5 1 5 . 9 6 5 2 . 5 1 7 . 9 1 1 7 . 9 4 6 3 . 0 2 1 . 1 7 2 1 . 1 9 7 3 . 5 2 3 . 3 2 2 3 . 4 0 T a b l e 4 . 1 A c o m p a r i s o n o f S N R v a l u e s a s d e r i v e d f r o m s a m p l e s p r o c e s s e d w i t h a n d w i t h o u t a d c o f f s e t . T o p r o d u c e t h e r e s u l t s u n d e r s e c t i o n B e a c h s i g n a l w a s p r e c o n d i t i o n e d b y a d d i n g a d c o f f s e t o f 2 0 u n i t s ' ' " b e f o r e p r o c e s s i n g . T h o s e s i g n a l s i n g r o u p A w e r e n o t a l t e r e d b e f o r e p r o c e s s i n g b y t h e D P C M - A Q B s y s t e m . T h e S N R v a l u e s w o u l d i n d i c a t e t h a t n o d i f f e r e n c e e x i s t e d b e t w e e n t h e t w o s e t s o f 2 p r o c e s s e d d a t a . T h e d c o f f s e t , h o w e v e r , p r o d u c e d r e l a t i v e l y h i g h e n e r g y ^ " T h e s e u n i t s a r e d e f i n e d b y t h e 1 2 b i t q u a n t i z a t i o n p r o c e s s d e s c r i b e d i n C h a p t e r 3. ^ T h e r e c o r d i n g p r o c e s s r e m o v e s a n y d c o f f s e t p r e s e n t i n t h e s i g n a l s o t h a t a l l s i g n a l s a r e c l e a r e d o f d c o f f s e t s b e f o r e s u b j e c t i v e e v a l u a t i o n . l i m i t cycles [C4] i n the signals of set B r e s u l t i n g i n an audible and often disturbing r i n g i n g sound. Results of comparisons included i n the subjective tests previously described revealed a strong preference f o r those signals not including the dc b i a s . These findings confirm that SNR does not always r e f l e c t a signals's subjective q u a l i t y , p a r t i c u l a r l y when more than one form of noise i s present. SNR values have been determined f o r the parameters under study to allow a more d e t a i l e d comparison with subjective preference. Curves of constant SNR are drawn i n Figure 4.7. Superimposed are the i s o p r e f e r -ence contours of Figure 4.2. Certain s i m i l a r i t i e s and differences are evident. F i r s t , both sets of contours have the same general shape, i n d i -cating some degree of c o r r e l a t i o n . Second, the largest proportion of i n -crease with respect to sampling rate both i n preference and SNR l e v e l s , occurs i n the bottom, portion of the plane. In f a c t Figure 4.8 suggests that 75% of observed improvement i n SNR occurs by the time the sampling rate has doubled the Nyquist rate. T h i r d l y , both sets of contours show a p o s i t i v e change i n q u a l i t y with increasing sampling rate, f g . Here the s i m i l a r i t y ends. The f l a t t e r isopreference curves ind i c a t e that more subjective gain i s possible by increasing f than the SNR curves would i n -dicate. The slope of the SNR curves steepens quickly as soon as f /f^ i s increased beyond 2, while the slope of the preference curves change more gradually. The difference may be explained by the ears s e n s i t i v i t y to the type of noise being eliminated at t h i s l e v e l . The question l e f t unanswered i s "When can SNR be used and how e f f e c t i v e i s i t as a measure of subjective q u a l i t y ? " SNR i s always a good measure of s i g n a l reproduction. When the signals are produced f o r human F i g . 4.7 Curves of constant signal to noise r a t i o . Superimposed are the isopreference con-tours (dashed) of Fig. 4.2. S N R dB 25 2 0 15 10 0 ) D P C M - A Q B J I L 1.0 15 2.0 2 5 3.0 3.5 S A M P L I N G F R E Q U E N C Y / N Y Q U I S T R A T E - f s / f n F i g . 4.8 SNR vs. f /fa curves f o r the 2, 3, 4, 5, 6 and 7 l e v e l quantizers studied. Also shown i s a curve r e s u l t i n g from Jayant's one-bit memory d e l t a modulator. 4 8 . l i s t e n i n g i t w o u l d a p p e a r t h a t s o m e d i s t o r t i o n s a r e m o r e d i s t u r b i n g t h a n o t h e r s , s o t h a t S N R c o u l d o n l y b e u s e d a s a r e l a t i v e m e a s u r e , a n d t h e n o n l y w h e n i t i s j u d g e d t h a t t h e d o m i n a n t n o i s e t y p e s p e r t u r b i n g t h e s i g n a l s b e i n g c o m p a r e d a r e o f t h e s a m e g e n e r a l t y p e . 4 . 5 C O M P A R I S O N S O F T H E R E S U L T S W I T H P R E V I O U S WORK T h e p u r p o s e o f p r e v i o u s s e c t i o n s h a s b e e n t o s t u d y a D P C M - AQB s y s t e m u t i l i z i n g t h e M e l s a a n d C o h n a d a p t i v e q u a n t i z e r . I t w o u l d b e u s e f u l a t t h i s p o i n t t o m a k e c o m p a r i s o n s b e t w e e n t h e p r e s e n t w o r k a n d t h a t o f o t h e r r e s e a r c h e r s . I s o p r e f e r e n c e c u r v e s f r o m C h a n a n d D o n a l d s o n [ C 2 ] i n d i c a t e t h a t f o r a D P C M s y s t e m u t i l i z i n g a 2 b i t q u a n t i z e r , a n S N R ^ ^ j r a t i n g n e a r 2 d B c a n b e e x p e c t e d w h i l e f o r a 3 - b i t q u a n t i z e r , a v a l u e o f a b o u t 8 d B c a n b e r e a c h e d . T h e s e v a l u e s w e r e t a k e n w h e n s a m p l i n g a t t h e N y q u i s t r a t e . F i g u r e 4 . 2 r e v e a l s l o w e r l i m i t s o f 4 a n d 1 8 d B f o r t h e t w o c a s e s s i t e d , t h e r b y i n -d i c a t i n g a d e f i n i t e i m p r o v e m e n t i n s u b j e c t i v e q u a l i t y b e t w e e n D P C M c o d i n g w i t h a n d w i t h o u t a d a p t i v e q u a n t i z a t i o n . I t s h o u l d b e p o i n t e d o u t t h a t t h e i r r e f e r e n c e s i g n a l " J o e . . . l a w n " w a s e f f e c t i v e l y b a n d l i m i t e d t o 4 k H z a n d s a m p l e d a t 8 k H z w h i l e o u r t e s t s u t i l i z e d t h e s a m e s i g n a l b a n d l i m i t e d t o 3.2 k H z a n d s a m p l e d a t 6 . 4 k H z . T h e l o w e r b a n d l i m i t e d r e f e r e n c e s i g n a l w a s u s e d i n o u r w o r k t o g i v e i t t h e s a m e c h a r a c t e r i s t i c s a s t h e s i g n a l s u n d e r t e s t . T h i s w a s d o n e t o s i m p l i f y t h e t a s k o f s i g n a l c o m p a r i s o n . A n i n t e r e s t i n g o b s e r v a t i o n f r o m F i g u r e 4 . 2 i s t h a t t h e S N R v a l u e s a s s o c i a t e d w i t h t e s t s i g n a l s s a m p l e d c l o s e t o 6 . 4 k H z c o m p a r e s v e r y c l o s e l y t o t h e S N R g u ^ _ . v a l u e s o f t h e i r i s o p r e f e r e n t r e f e r e n c e s i g n a l s . T h i s w o u l d t e n d t o c o n f i r m t h a t s i g n a l d e g e n e r a t i o n b y S c h r o e d e r ' s t e c h n i q u e ( C h a p t e r 3 ) r e p r e s e n t s q u i t e w e l l t h e n o i s e i n t r o d u c e d 49. by DPCM coding. Furthermore the consistency of SNR^^j values i n the over-sampled region, where SNR i s no longer a good i n d i c a t o r of subjective q u a l i -ty, supports the assumption that Schroeder's reference signals present a v a l i d means of comparing the subjective q u a l i t y of d i f f e r e n t processing systems which introduce s i g n a l dependent noise. Another study, by Goldstein and L u i [G2], has investigated the operating c h a r a c t e r i s t i c s of a DPCM - AQB system using an adaptive quanti-zation, scheme s i m i l a r to the one described by Cummiskey et a l [C7]. Gold-s t e i n and Lui's system operating on a f l a t band-limited Gaussian s i g n a l displayed a l i n e a r r e l a t i o n s h i p between SNR and log ( f ^ . / f n ) . For R-C shaped Gaussian signals t h e i r mathematically derived equations again predicted a l i n e a r r e l a t i o n s h i p while simulation r e s u l t s suggest a s l i g h t l e v e l i n g o f f at high SNR values. Jayant's one-bit memory DM [JI] operates using an adaptation algo-rithm s i m i l a r to the one used by Goldstein and L u i . Operation of Jayant's DM when applied to voice signals supports Goldstein's r e s u l t s and indicates that the general c h a r a c t e r i s t i c s displayed by Goldstein and L u i may hold f o r speech samples. The SNR vs. log (f / f n ) p l o t f or Jayant's DM - AQB over the operating range considered i n t h i s t h e s i s , has been determined and i s also presented i n Figure 4.8. Although better performance can be expected at low frequencies, the Cohn and Melsa curve quickly f l a t t e n s while Jayant's curve continues to demonstrate a l i n e a r r e l a t i o n s h i p . We note that the difference i n behaviour between our r e s u l t s and those of Jayant [JI] and Goldstein and L u i [G2] are probably due to differences i n the quantizer adaptation algorithms. Although performing w e l l both at and j u s t above the Nyquist rate, Cohn and Melsa's DPCM - AQB system seems not to allow f o r optimum performs ance at highly oversampled rates. I t i s suggested that speech studies on adaptive quantization schemes such as the one proposed by Cummiskey et a l . may r e s u l t i n much f l a t t e r isopreference curves thus r e s u l t i n g i n improved performance and i n t e r e s t i n g design tradeoffs between the quantizer s t r u c -ture and sampling frequencies. 51. V CONCLUSION 5.1 SUMMARY In t h i s thesis the subjective q u a l i t y of a DPCM system featuring a quantizer adaptation algorithm proposed by Cohn and Melsa [C5] has been investigated. The system operated on high q u a l i t y speech samples and was subject to c o n t r o l l e d v a r i a t i o n s i n the sampling frequency r e l a t i v e to the Nyquist rate, and i n the number of quantization l e v e l s . Because low b i t rate applications were of p a r t i c u l a r i n t e r e s t , the sampling rate and quan-t i z e r structure were bounded at 3.5 times the Nyquist rate and at seven l e v e l s , r e s p e c t i v e l y . Simulation of the system was c a r r i e d out on a d i g i t a l computer while a l l subjective test r e s u l t s were evaluated according to the isopreference method. The subjective tests resulted i n a p l o t of isopreference contours being drawn on a plane whose ordinate was defined by the r a t i o of sampling frequency to the Nyquist frequency, and whose abcissa was defined by the number of quantization l e v e l s . The curves revealed that increases i n sub-j e c t i v e q u a l i t y r e s u l t i n g from increases i n sampling frequency r e l a t i v e to the Nyquist frequency became minimal a f t e r a r a t i o of two had been reached. This r e s u l t i n d i c a t e d that gains made by the r e s u l t i n g increase i n sample c o r r e l a t i o n were being masked by other elements such as quantizer noise. Plots of b i t rate r e s u l t i n g from various combinations of sampling frequency and quantizer structures were obtained, both with and without en-tropy coding. I t was determined that the minimum required channel capacities for a given subjective preference l e v e l occurred when sampling at the Ny-quist rate. Implementation tradeoffs between the number of quantization l e v e l s and the sampling frequency became apparent from the isopreference contours. However, no r e a l design options r e s u l t e d from coding schemes which assigned an equal number of b i t s to each quantizer l e v e l under constant b i t -rate constraints. On the other hand, use of entropy coding showed that at suboptimal b i t rates a given subjective preference l e v e l could be attained using d i f f e r e n t combinations of sampling frequency and quantizer s t r u c t u r e s . Also, f o r a given b i t rate, i t was found that odd-level quantizers out per-formed even l e v e l quantizers when entropy coding was employed. Comparisons with c a l c u l a t e d SNR values i n d i c a t e d a general cor-r e l a t i o n between curves of constant subjective q u a l i t y and curves of con-stant SNR. The SNR curves did not however, accurately describe the sub-j e c t i v e test r e s u l t s and the conclusion was drawn that SNR could not gen-e r a l l y be used as a precise measure of subjective q u a l i t y . A s p e c i a l ex-periment was conducted to show that i n p a r t i c u l a r , SNR could not be used to compare speech samples containing large proportions of d i f f e r e n t types of noise. The subjective test r e s u l t s were also compared with r e s u l t s of others' work. On the basis of Schroeder's [S2] speech q u a l i t y standard signals i t was determined that DPCM using an adaptive quantizer out per-formed a fi x e d quantizer DPCM system. This improvement can e a s i l y be seen i n Table 5.1 where SNR . . values for PCM and DPCM have been taken from subj Chan and Donaldson [C2] and Yan and Donaldson [ Y l ] . The performance of another adaptive quantization scheme studied by Goldstein and L u i [G2] was also compared to the r e s u l t s of th i s work. I t was suggested that the adaptation scheme of t h i s t h e s i s , although performing very w e l l at low ra t i o s of sampling rate to Nyquist rate, did not perform optimally at higher r a t i o values. DPCM DPCM - AQB - CHAN YAML. YAN //of N F Quantization W>.'= 3.2 W = 4 W = 4 W = 4 W=3.2 W=3.2 Bit s f = 6.4 f = 8 f = 8 f = 8 f = 6.4 f = 8 s s s s s s 2 2 4 (1.68) (5.07) 4 r 6.5 3 8 100 12.5 11 18 20 4 13 16 18 17 5 20 24 24 25 6 25 — (25.05) (27.3) Table 5.1 Comparison of approximate SNR^^j values f o r non-adaptive previous-sample feedback DPCM, and DPCM - AQB. Results for DPCM are from Chan and Donaldson [C2] and Yan and Donaldson [YI]. (Values i n brackets represent the l i m i t i n g values of the respective graphs. N - natural binary coding. F - folded binary coding. W - bandwidth.) 5.2 SUGGESTIONS FOR FURTHER WORK The r e s u l t s of th i s study further confirm the economy-efficiency compromise obtainable using a d i f f e r e n t i a l encoder with an adaptive quan-t i z e r . Further t e s t i n g and development i s necessary before such systems are proven acceptable f o r use i n modern communications systems. E f f e c t s of d i g i t a l channel transmission errors must be consid-ered both from the point of view of o v e r a l l subjective e f f e c t s as w e l l as system error propagation. I t remains to optimize the quantizer adaptation strategy f o r various values of f s / r n a n d L. This need for optimization suggests that other adaptive algorithm, such as the one presented by Goldstein and L u i , should be su b j e c t i v e l y tested on speech samples to f u l l y understand t h e i r operating c h a r a c t e r i s t i c s . 5 5 . R E F E R E N C E S B I R.W. B e n s o n a n d I . J . H i r s h , " S o m e v a r i a b l e s i n a u d i o s p e c t r o m e t r y " , J. A c o u s t . S o c . Am., V o l . 7 5 , p p . 4 9 9 - 5 0 5 , M a y 1 9 5 3 . C I J . C . C a n d y , " L i m i t i n g t h e P r o p a g a t i o n o f E r r o r s i n O n e - B i t D i f f e r e n -t i a l C o d e c ' s " BSTJ, V o l . 5 3 , p p . 1 6 6 7 - 1 6 7 7 , O c t o b e r 1 9 7 4 . C 2 D . C h a n a n d R.W. D o n a l d s o n , " S u b j e c t i v e E v a l u a t i o n o f P r e a n d P o s t F i l t e r i n g i n P A M , P C M a n d D P C M V o i c e C o m m u n i c a t i o n S y s t e m s " , I E E E  T r a n s , o n C o m m u n i c a t i o n s T e c h n o l o g y , V o l . 1 9 , p p . 6 0 1 - 6 1 2 , O c t o b e r 1 9 7 1 . C 3 D . C h a n , " O p t i m a l p r e - a n d p o s t f i l t e r i n g o f n o i s y s a m p l e d s i g n a l s -p a r t i c u l a r a p p l i c a t i o n s t o P A M , P C M a n d D P C M c o m m u n i c a t i o n s y s t e m s " , P h . D . T h e s i s , D e p a r t m e n t o f E l e c t r i c a l E n g i n e e r i n g , U n i v e r s i t y o f B r i t i s h C o l u m b i a , V a n c o u v e r , B . C . , C a n a d a , A u g u s t 1 9 7 0 . C 4 T . A . C . M . C l a s s e n , W . F . G . M e c k l e n b r a n k e r , a r i d J . B . H . P e e k , " E f f e c t s o f q u a n t i z a t i o n a n o v e r f l o w i n r e c u r s i v e d i g i t a l f i l t e r s " , I E E E  T r a n s , o n A S S P , V o l . 2 4 , p p . 5 1 7 - 5 2 9 , D e c e m b e r 1 9 7 6 . C 5 D . L . C o h n a n d J . L . M e l s a " T h e r e s i d u a l e n c o d e r - a n i m p r o v e d A D P C M s y s t e m f o r s p e e c h d i g i t i z a t i o n " i n C o n f . R e c . I E E E I n t . C o n f . C o m m u n . , S a n F r a n c i s c o , C a l i f . , p p . ( 3 0 - 2 6 ) - ( 3 0 - 3 0 ) , 1 9 7 5 . C 6 R . E . C r o c h i e r e , S . A . W e b s t e r , and J . L . F l a n a g a n , " D i g i t a l c o d i n g o f s p e e c h i n s u b b a n d s " , BSTJ, V o l . 5 5 , p p . 1 0 6 9 - 1 0 8 5 , O c t o b e r 1 9 7 6 . C 7 P . C . C u m m i s k e y , N . S . Jayant, a n d J . L . F l a n a g a n , " A d a p t i v e q u a n t i z a -t i o n i n d i f f e r e n t i a l P C M c o d i n g o f s p e e c h " , BSTJ, V o l . 5 2 , p p . 1 1 0 5 -1 1 1 8 , S e p t e m b e r 1 9 7 3 . C 8 P . C u m m i s k e y , " S i n g l e i n t e g r a t i o n a d a p t i v e d e l t a m o d u l a t i o n " , BSTJ, V o l . 5 4 , p p . 1 4 6 3 - 1 4 7 4 , O c t o b e r 1 9 7 5 . D I R.W. D o n a l d s o n , a n d D . C h a n , " A n a l y s i s a n d s u b j e c t i v e e v a l u a t i o n o f d i f f e r e n t i a l p u l s e c o d e m o d u l a t i o n v o i c e c o m m u n i c a t i o n s y s t e m s " I E E E T r a n s , o n C o m m u n i c a t i o n T e c h n o l o g y , V o l . 1 7 , p p . 1 0 - 1 9 , F e b 1 9 6 9 . D 2 R.W. D o n a l d s o n a n d R . J . D o u v i l l e , " A n a l y s i s , s u b j e c t i v e e v a l u a t i o n , o p t i m i z a t i o n a n d c o m p a r i s o n o f t h e p e r f o r m a n c e c a p a b i l i t i e s o f P C M , D P C M , AM, a n d P M v o i c e c o m m u n i c a t i o n s y s t e m s . " I E E E T r a n s , o n C o m - m u n i c a t i o n T e c h n o l o g y , V o l . 1 7 , p p . 4 2 ^ 4 3 1 , A u g u s t 1 9 6 9 . F I J . L . F l a n a g a n , S p e e c h A n a l y s i s , S y n t h e s i s , a n d P e r c e p t i o n , S p r i n g e r -V e r l a g , B e r l i n - H e i d e l b e r g * N e w Y o r k , 1 9 7 2 . G I R . G . G a h l a g e r , I n f o r m a t i o n T h e o r y a n d R e l i a b l e C o m m u n i c a t i o n , W i l e y , 1 9 6 8 . G 2 L . H . G o l d s t e i n a n d B . L u i , " Q u a n t i z a t i o n N o i s e i n A D P C M S y s t e m s " , I E E E T r a n s , o n C o m m u n i c a t i o n s , V o l . 2 5 , p p . 2 2 7 - 2 3 7 , F e b r u a r y 1 9 7 7 . 5 6 . G 3 D . J . G o o d m a n a n d L . J . G r e e n s t e i n , " Q u a n t i z i n g n o i s e o f A M / P C M e n -c o d e r s " , B S T J , V o l . 5 2 , p p . 1 8 3 - 2 0 4 , F e b r u a r y 1 9 7 3 . G4 D . J . G o o d m a n , B . J . M c D e r m a t t a n d L . H . N a k a t a n i , " S u b j e c t i v e e v a l u a -t i o n o f P C M c o d e d s p e e c h " , B S T J , V o l . 5 5 , p p . 1 0 8 7 - 1 1 1 0 , 0 c t o l 9 7 6 . G 5 L . J . G r e e n s t e i n , " S l o p e o v e r l o a d n o i s e i n l i n e a r d e l t a m o d u l a t i o n w i t h g a u s s i a n i n p u t s " , B S T J , V o l . 5 2 , p p . 3 8 7 - 4 2 2 , M a r c h 1 9 7 3 . G 6 C . B . G r e t h e r a n d R.W. S t r o h , " S u b j e c t i v e e v a l u a t i o n o f d i f f e r e n t i a l p u l s e c o d e m o d u l a t i o n u s i n g t h e s p e e c h g o o d n e s s r a t i n g s c a l e " , I E E E  T r a n s a c t i o n o n A u d i o , V o l . 2 1 , p p . 1 7 9 - 1 8 4 , J u n e 1 9 7 3 . G 7 J . P . G u i l f o r d , P s y c h o m e t r i c M e t h o d s , M c G r a w - H i l l , 1 9 3 6 . H I M . M . L . H e c k e r a n d C . E . W i l l i a m s , " C h o i c e o f r e f e r e n c e s c o n d i t i o n s f o r s p e e c h p r e f e r e n c e t e s t s " , J . A c c o u s t . S o c . Am., V o l . 3 9 , p p . 9 4 6 - 9 5 2 , M a y 1 9 6 6 . I I " I E E E R e c o m m e n d e d p r a c t i c e f o r s p e e c h q u a l i t y m e a s u r e m e n t s " , I E E E  T r a n s . o n A u d i o E n g . , V o l . 1 7 , p p . 2 2 7 - 2 4 6 , S e p t e m b e r 1 9 6 9 . J I N . S . J a y a n t , " A d a p t i v e D e l t a M o d u l a t i o n w i t h a o n e - b i t m e m o r y " , B S T J , V o l . 4 9 , p p . 3 2 1 - 3 4 2 , M a r c h 1 9 7 0 . J 2 N . S . J a y a n t , " A d a p t i v e Q u a n t i z a t i o n w i t h a o n e - w o r d m e m o r y " , B S T J , V o l . 5 2 , p p . 1 1 1 9 - 1 1 4 4 , S e p t e m b e r 1 9 7 3 . J 3 , N . S . J a y a n t , " S t e p s i z e t r a n s m i t t i n g d i f f e r e n t i a l c o d e r s f o r m o b i l e t e l e p h o n y " , B S T J , V o l . 5 4 , p p . 1 5 5 7 - 1 5 8 2 , N o v e m b e r , 1 9 7 5 . J 4 . N . S . J a y a n t , " O n t h e c o r r e l a t i o n b e t w e e n b i t s e q u e n c e s i n c o n s e c u t i v e d e l t a m o d u l a t i o n o f a s p e e c h s i g n a l , B S T J , V o l . 5 3 , p p . 9 3 7 - 9 4 9 , M a y - J u n e 1 9 7 4 . K l K . D . K r y t e r , " M e t h o d s f o r t h e c a l c u l a t i o n a n d u s e o f a r t i c u l a t i o n i n d e x " , J . A c o u s t . S o c . Am., V o l . 3 4 , p p . 1 6 8 9 - 1 6 9 7 , 1 9 6 2 . L I B.W. L i n d g r e n a n d G.W. E l r a t h , I n t r o d u c t i o n t o P r o b a b i l i t y S t a t i s t i c s , T h e M a c M i l l a n C o . , 1 9 6 9 . M l R . A . M c D o n a l d , " S i g n a l t o N o i s e a n d I d l e c h a n n e l p e r f o r m a n c e o f d i f -f e r e n t i a l p u l s e c o d e m a o d u l a t i o n s y s t e m s - p a r t i c u l a r a p p l i c a t i o n s t o v o i c e s i g n a l s " , B S T J , V o l . 5 4 , p p . 1 1 2 3 - 1 1 5 2 , S e p t e m b e r 1 9 6 6 . M2 P . M i l n e r , " A d v a n t a g e s o f e x p e r i e n c e d l i s t e n e r s i n i n t e l l i g i b i l i t y t e s t i n g " , I E E E E T r a n s . o n A u d i o E l e c t r o a c o u s t i c s , V o l . 2 1 , p p . 1 6 1 -1 6 5 , J u n e 1 9 7 3 . M 3 0 . M i t r a , " M a t h e m a t i c a l a n a l y s i s o f a n a d a p t i v e q u a n t i z e r " , B S T J , V o l . 5 3 , p p . 8 6 7 - 8 9 8 , M a y - J u n e 1 9 7 4 . M4 W.A. M u n s o n a n d J . E . K a r l i n , " I s o p r e f e r e n c e m e t h o d s f o r m e a s u r i n g s p e e c h q u a l i t y " , J . A c o u s t . S o c . Am., V o l . 3 4 , p p . 7 6 2 - 7 7 4 , J u n e 1 9 6 2 . 5 7 . N I P . N o l l , " A c o m p a r a t i v e s t u d y o f v a r i o u s q u a n t i z i n g s c h e m e s f o r s p e e c h e n c o d i n g " , B S T J , V o l . 5 4 , p p . 1 5 9 7 - 1 6 1 4 , N o v e m b e r 1 9 7 5 . N 2 P . N o l l , " E f f e c t s o f c h a n n e l e r r o r s o n t h e s i g n a l - t o - n o i s e p e r f o r m -a n c e o f s p e e c h e n c o d i n g s y s t e m s " , B S T J , V o l . 5 4 , p p . 1 6 1 5 - 1 6 3 6 , N o v 1 9 7 5 . 0 1 J . B . O ' N e a l , J r . " D e l t a m o d u l a t i o n q u a n t i z i n g n o i s e a n a l y t i c a l a n d c o m p u t e r s i m u l a t i o n r e s u l t s f o r g a u s s i a n a n d t e l e v i s i o n i n p u t s i g -n a l s " , B S T J , V o l . 4 5 , p p . 1 1 7 - 1 4 2 , J a n u a r y 1 9 6 6 . 0 2 J . B . O ' N e a l , J r . " P r e d i c t i v e q u a n t i z i n g s y s t e m s ( d i f f e r e n t i a l P C M ) f o r t h e t r a n s m i s s i o n o f t e l e v i s i o n s i g n a l s " , B S T J , V o l . 4 5 , p p . 6 8 9 -7 2 2 , M a y - J u n e 1 9 6 6 . Q l S . U . H . Q u r e s h i a n d G.D. F o r n e y , J r . " A 9 . 6 / 1 6 k b / s s p e e c h d i g i t i z e r " i n C o n f . R e c , I E E E I n t . C o n f . C o m m u n , S a n F r a n c i s c o , C a l i f , p p . ( 3 0 - 3 1 ) - ( 3 0 - 3 6 ) , 1 9 7 5 . R l E . H . R o t h a u s e r , G . E . U r b a n e k a n d W.D. P a c h l , " I s o p r e f e r e n c e m e t h o d f o r s p e e c h e v a l u a t i o n " , J . A c o u s t . S o c . Am., V o l . 4 4 , p p . 4 0 8 - 4 1 8 , F e b r u a r y 1 9 6 8 . 5 1 H.R. S c h i n d l e r , " L i n e a r , n o n l i n e a r , a n d a d a p t i v e d e l t a m o d u l a t i o n " I E E E T r a n s , o n C o m m u n i c a t i o n s , V o l . 2 2 , p p . 1 8 0 7 - 1 8 2 2 , N o v e m b e r 1 9 7 4 . 5 2 M.R. S c h r o e d e r , " R e f e r e n c e s i g n a l s f o r s p e e c h q u a l i t y s t u d i e s " , J .  A c o u s t . Soc.:. A i d . , V o l . 4 4 , p p . 1 7 3 5 - 1 7 3 6 , D e c e m b e r 1 9 6 8 . 5 3 J . S w a f f i e l d a n d D . L . R i c h a r d s , " R a t i n g o f s p e e c h l i n k s a n d p e r f o r m -a n c e o f t e l e p h o n e n e t w o r k s " , P r o c . I E E E , V o l . 1 0 6 - B , p p . 6 5 - 7 6 , M a r c h 1 9 5 9 . T I S . T a z a k i , H . O s a w a , Y . S h i g e m a t s u , " A u s e f u l a n a l y t i c a l m e t h o d f o r D i s c r e t e a d a p t i v e d e l t a m o d u l a t i o n " , I E E E T r a n s . o n C o m m u n i c a t i o n s , V o l . 2 5 , p p . 1 9 3 - 2 0 0 , F e b r u a r y 1 9 7 7 . T 2 W.H. T e d f o r d , J r . a n d T . V . F r a z i e r , " F u r t h e r s t u d y o f t h e i s o p r e f -e r e n c e m e t h o d o f c i r c u i t e v a l u a t i o n " , J . A c o u s t . S o c . Am., V o l . 3 9 , p p . 6 4 5 - 6 4 9 , A p r i l 1 9 6 6 . W l W o o d w a r d a n d S c h l o s b e r g , E x p e r i m e n t a l P s y c h o l o g y , V o l . 1 , H o l t -R i n e h a r t W i n s t o n ( 1 9 3 8 ) 1 9 7 2 . Y I J . Y a n a n d R.W. D o n a l d s o n , " S u b j e c t i v e e f f e c t s o f c h a n n e l t r a n s m i s s i o n e r r o r s o n P C M a n d D P C M v o i c e c o m m u n i c a t i o n s s y s t e m s " , I E E E T r a n s , o n  C o m m u n i c a t i o n s , V o l . 2 0 , p p . 2 8 1 - 2 9 0 , J u n e 1 9 7 2 . 

Cite

Citation Scheme:

        

Citations by CSL (citeproc-js)

Usage Statistics

Share

Embed

Customize your widget with the following options, then copy and paste the code below into the HTML of your page to embed this item in your website.
                        
                            <div id="ubcOpenCollectionsWidgetDisplay">
                            <script id="ubcOpenCollectionsWidget"
                            src="{[{embed.src}]}"
                            data-item="{[{embed.item}]}"
                            data-collection="{[{embed.collection}]}"
                            data-metadata="{[{embed.showMetadata}]}"
                            data-width="{[{embed.width}]}"
                            data-media="{[{embed.selectedMedia}]}"
                            async >
                            </script>
                            </div>
                        
                    
IIIF logo Our image viewer uses the IIIF 2.0 standard. To load this item in other compatible viewers, use this url:
https://iiif.library.ubc.ca/presentation/dsp.831.1-0094070/manifest

Comment

Related Items