Open Collections

UBC Theses and Dissertations

UBC Theses Logo

UBC Theses and Dissertations

Musical interval perception with pulsatile electrical stimulation of profoundly deaf ears Pijl, Sipke 1994

Your browser doesn't seem to have a PDF viewer, please download the PDF to view this item.

Item Metadata

Download

Media
831-ubc_1994-894415.pdf [ 2.35MB ]
Metadata
JSON: 831-1.0099227.json
JSON-LD: 831-1.0099227-ld.json
RDF/XML (Pretty): 831-1.0099227-rdf.xml
RDF/JSON: 831-1.0099227-rdf.json
Turtle: 831-1.0099227-turtle.txt
N-Triples: 831-1.0099227-rdf-ntriples.txt
Original Record: 831-1.0099227-source.json
Full Text
831-1.0099227-fulltext.txt
Citation
831-1.0099227.ris

Full Text

MUSICAL INTERVAL PERCEPTION WITH PULSATILE ELECTRICAL STIMULATION OF PROFOUNDLY DEAF EARS  by  SIPKE PIJL  B.Ed., The University of British Columbia, 1975 M.A., Western Washington University, 1977  A THESIS SUBMITTED IN PARTIAL FULFILLMENT OF THE REQUIREMENTS FOR THE DEGREE OF DOCTOR OF PHILOSOPHY  in  THE FACULTY OF GRADUATE STUDIES (Neuroscience Program)  We accept this thesis as conforming to the required standard  THE UNIVERSITY OF BRITISH COLUMBIA February 1994  ®  Sipke Pijl,  1994  In presenting this thesis in partial fulfilment of the requirements for an advanced degree at the University of British Columbia, I agree that the Library shall make it freely available for reference and study. I further agree that permission for extensive copying of this thesis for scholarly purposes may be granted by the head of my department or by his or her representatives. It is understood that copying or publication of this thesis for financial gain shall not be allowed without my written permission.  Department of  URO(t.Lc  The University of British Columbia Vancouver, Canada  Date 1  DE-6 (2/88)  i99L.  ABSTRACT This research examines, in a musical context, the measurement of pitches heard by Nucleus cochlear implant recipients upon systematic variation of electrical pulse rates, delivered to single intracochlear electrodes at a Stimuli were configured by a comfortable listening level. computer in tandem with the Boys Town National Institute Interf ace for psychophysical research with the Nucleus Seventeen subjects participated in a 30cochlear implant. Many subjects item tune recognition test (Experiment I). identified a substantial number of items. Three subjects underwent a more detailed investigation to determine whether pitches resulting from pulse rate variation were sufficiently salient for musical interval perception. The results of a closed—set melody recognition test (Experiment II) suggested that recognition was possible on the basis of melody, i.e., even in the complete absence of rhythmical information, and that recognition was possible over However, these results did not a range of pulse rates. determine whether performance was based on ordinal properties of the pitches, or whether successive pitches defined identifiable musical intervals. Intonation quality judgements (Experiment III) of intervals ranging in size from a minor 3rd to a 5th provided evidence that the frequency ratios which characterize acoustical musical intervals also apply to electrical pulse Further evidence of musical ratio recognition was rate pitch. obtained using the method of adjustment (Experiments IV and At least 2 out of 3 subjects were able, by means of the V). adjustment of a variable pulse rate, to reconstruct selected musical intervals abstracted from melodies well—known to the Two subjects, furthermore, were able to transpose subjects. these melodic patterns to higher and lower pulse rates, in a manner similar to that demonstrated by normal—hearing subjects when listening to musical intervals. These results suggest that temporally mediated pitches are capable of conveying ratio pitch information, in the sense that equal ratios of pulse rates appear to produce equal These findings lend support to musical pitch intervals. temporal theories of musical pitch and interval perception.  ii  TABLE OF CONTENTS  ii  ABSTRACT  iii  TABLE OF CONTENTS LIST OF TABLES  V  vi  LIST OF FIGURES  viii  LIST OF ABBREVIATIONS  ix  DEFINITION OF TERMS ACKNOWLEDGEMENTS  Xiii  xiv  DEDICATION  1  INTRODUCTION METHODS:  13  General  Experimental Setup  18  Preliminary Psychophysical Measurements  22  Experimental Subjects  25  Results  28  Discussion  32  EXPERIMENT I:  Open-Set Tune Identification  38  Methods  39  Results  42  Discussion  48  iii  Closed-Set Melody Identification  EXPERIMENT II:  55  Methods  56  Results  60  Discussion  64  EXPERIMENT III:  Intonation Quality Judgements  69  Methods  70  Results  76  Discussion  90  EXPERIMENT IV: Musical Interval Reconstruction  100  Methods  101  Results  106  Discussion  111  EXPERIMENT V:  Musical Interval Transposition  122  Methods  122  Results  128  Discussion  143  GENERAL DISCUSSION  152  REFERENCES  171  APPENDICES 184  Experiment I  1.  Open-Set Tunes:  2.  Pulse Rates for 7-Note Melodies:  iv  Experiment II  187  LIST OF TABLES  Page  Table 1.  Subject Data  23  2.  points of Intonation Quality Judgements: subjective equality (PSE) and standard deviations  78  3.  4.  5.  Method of Adjustment: data  interval reconstruction 107  Method of Adjustment: interval transposition data for Subject 7  130  Method of Adjustment: interval transposition ascending intervals data for Subject 10  134  Method of Adjustment: interval transposition descending intervals data for Subject 11  137  -  6.  -  v  LIST OF FIGURES  Page  Figure 1.  Equipment set-up  21  2.  Current amplitudes at comfortable loudness: effect of pulse width  29  Current amplitudes at comfortable loudness for 4 subjects  29  Current amplitudes at comfortable loudness: Subject 10 versus Subject 11  30  Current amplitudes at comfortable loudness: replicability of measurements  30  Open-set tune identification: percent correct scores  44  Open-set tune identification: Number of positive identifications of each tune  45  Tune identification performance versus speech perception scores  47  Closed-set melody identification: percent correct scores  61  Schematic representation of intonation quality judgements  72  3.  4.  5.  6.  7.  8.  9.  10.  11.  Intonation Quality Judgements:  interval of a 5th  79  12.  Intonation Quality Judgements:  minor 3rd  80  13.  Intonation Quality Judgements:  interval of a 4th  81  14.  Intonation Quality Judgements:  major 6th  82  15.  Point of subjective equality and standard deviations for intervals from a minor 3rd to a major 6th  vi  83  Page  Figure 16.  17.  18.  19.  20.  effect of Intonation Quality Judgements: electrodes on interval of a 5th  87  effect of pulse Intonation Quality Judgements: rate of starting note on interval of a 5th  89  mean interval size Interval reconstruction: and standard deviations  109  percentage of Interval reconstruction: adjustments 0-2 semitones from target  112  mean interval size and Interval transposition: intervals of a 5th and standard deviations Subject 7 minor 3rd.  131  —  21.  22.  23.  24.  25.  Interval transposition: standard deviations Subject 7  -  Interval transposition: standard deviations Subject 10  —  Interval transposition: standard deviations Subject 10  —  mean interval size and interval of a 4th. 132 mean interval size and ascending intervals. 135 mean interval size and descending intervals. 138  differences in interval Interval transposition: size related to adjustment of upper or lower note  140  percentage of Interval transposition: adjustments 2 semitones or less from target  144  vii  LIST OF ABBREVIATIONS  ANOVA  Analysis of Variance  ANSI  American National Standards Institute  BTNI  Boys Town National Institute  E  electrode  Hz  Hertz  kHz  kiloHertz  MHz  MegaHertz  msec  milliseconds  pps  pulses per second  PSE  points of subjective equality  S  Subject  SD  standard deviation  sec  microseconds microamps  viii  DEFINITION OF TERMS  (Items arranged in logical sequence) That aspect of pitch (related to chroma, not Musical pitch: pitch height) that is capable of conveying musical The musical scale specifies interval information. exactly the frequency ratios that are to be used to These ratios are thus on establish musical intervals. the stimulus side of the “equation”, and not on the Musical intervals phenomenal side of pitch. characterized by tones in equal frequency ratios are judged to be subjectively equivalent by musicians, but not necessarily or generally by others, except as For any listeners acquire experience with music. listener, subjectively equal musical intervals do not necessarily represent equal nonmusical pitch (pitch The latter differences depend on height) differences. the relative position of the intervals on the frequency scale. The full set of discrete musical intervals Musical scale: In the music of Western culture, which are permitted. these steps are defined by frequency ratios, and are The unit derived from the octave as the basic interval. of the Western musical scale is the semitone, which for the equal temperament scale, is obtained by dividing the Pairs of tones octave into 12 equal frequency intervals. separated by a given number of semitones (and a given frequency ratio) are given the same name, such as minor 3rd, major 6th, and so forth, regardless of where they The ratios used for all the occur in the musical scale. calculations in the experiments reported in this paper are based on the equal temperament scale. Tones in one of a set of standard Musical intervals: frequency ratios which ideally characterize the frequency Intervals relationships between musical notes. characterized by identical frequency ratios are generally (at least for musicians) perceived as being musically In the music of Western European culture, equivalent. interval size is measured in semitoies. One semitone equals one—twelfth of an octave (21/12), or a frequency The intervals commonly ratio of approximately 1:1.059. in this paper, detailed experiments the in to referred ratio and number g frequency accompanyin with the together follows: as briefly described are of semitones, ix  Name of Interval  Ratio  semitone major second minor third major third fourth tritone fifth major sixth octave  21/12 22/12 23/12 24/12 25/12 26/12 27/12 29/12  qm  or or or or or or or or  1:1.059 1:1.122 1:1.189 1:1.257 1:1.335 1:1.414 1:1.498 1:1.681 1:2  -i  tric  1 2 3 4  5 6 7 9 12  For this paper, melody is defined as an ordered Melody: series of musical intervals, in the absence of rhythmical The magnitude and or phrase length information. direction (i.e., ascending or descending) of such While melodies played intervals are precisely specified. in one frequency range may be transposed to other frequency ranges, melodies retain their identity only when the sequence, the direction, and the magnitude of the intervals (i.e., the ratio relationships between the frequencies of the notes) are preserved. Tune:  For this paper, tune is arbitrarily defined as a melody (i.e., an ordered series of intervals) with a specified The preservation of rhythm rhythmical pattern. distinguishes tune from melody, as used here, in that with melody, only the series of intervals (without rhythmical or phrase-length information) is available to the listener.  A generalized form of interval information, Melodic contour: and refers to the overall pattern or shape of a melody. Contour is defined by the directional relationships between temporally adjacent notes, without precise specification of the magnitude of the pitch changes Contour, therefore, indicates (i.e., interval size). whether adjacent notes in a melody are higher or lower than contiguous notes (i.e., the sequence of ups and For example, the contour of the first downs in pitch). phrase of “Twinkle, Twinkle, Little Star” could be represented as =  +  =  +  represent unisons, ascending, and where =, +, and Melodies may share descending intervals, respectively. identical contours, but differ in the precise size of the intervals (e.g., “Twinkle, Twinkle, Little Star” versus the Andante from Haydn’s Surprise Symphony). —  x  A procedure by which a melody or other Musical transposition: musical entity is shifted up or down on the musical scale, with preservation of the constituent musical intervals, as defined by the number of semitones (the mathematical frequency relationships) between the notes. Transposed familiar melodies are readily recognized as musical equivalents, even by musically unsophisticated listeners. The adjustment of the frequency of one tone until Intonation: some specified ratio relationship to a second is in it The permissible ratios are defined by the musical tone. scale, and may represent a unison interval, a fifth, a When fourth, and octave, or any other desired interval. musicians listen for the precision of the frequency adjustment (ratio properties) of musical intervals, they are said to be performing intonation quality or In judging musical intonation accuracy judgements. interval size, musically unsophisticated listeners are thought to rely more on pitch height (the vertical axis of the pitch helix) than on tone chroma (the precise position of a tone within the octave) or the ratio The intonation properties of the stimulus frequencies. quality judgements performed by such listeners must, therefore, provide a much cruder index of target interval size than when musicians perform such tasks (Burns and Ward 1978). The helical model of pitch represents pitch Pitch height: height as vertical dimension of the spiral, and chroma (C, G, etc.) by the angle of rotation around the spiral. One rotation represents a circle consisting of the 12 Thus, two C’s an octave notes of the chromatic scale. apart have the same chroma, but differ in pitch height, whereas a C and D adjacent to each other on the musical scale are similar in pitch height, but differ in chroma. A continuous increase in frequency will, according to this representation, be heard as a continuous increase in pitch height (a nonmusical aspect of the helix model), along with repetitions of tone chroma as each octave is Pitch height is thought to be an important traversed. factor governing the perceived relations among tones when they are presented outside a musical context or when tones are presented to musically unsophisticated Chroma is thought to be relatively more listeners. musically sophisticated listeners or when to important in a musical context (Krumhansl and presented tones are Shepard 1979).  xi  A forced-choice test paradigm, in which subjects Closed-set: are provided with a limited set of stimulus—response alternatives. Open-set: A test paradigm in which subjects are not provided with any specific response alternatives.  xii  ACKNOWLEDGMENTS  The preparation of this work was greatly facilitated by the help and encouragement of the following persons, each of whom provided a unique and essential contribution to its completion: I am grateful to Dr. D.W.F. Schwarz, for serving as my dissertation advisor and Committee Chairman, for his encouragement and valuable discussions regarding this project, and for helpful comments on a preliminary version of this paper. Many thanks to Professor D. Greenwood, for his many constructive comments on an earlier draft of this thesis. I am indebted to Dr. P.J. Doyle, Professor and Head (Emeritus) of the Division of Otolaryngology at the University of British Columbia, for encouraging me to undertake this project. My deepest gratitude to Mrs. J.H. Mitchell, President of the John Hardie Mitchell Family Foundation, for her generosity in providing the funding essential for the completion of this project. A very special thanks to all the cochlear implant patients who participated in this work. Especially, I am grateful to the three patients who contributed tirelessly of their time, effort and energies. In particular, I thank Peter Laschitz, who spent many hours in our laboratory, evaluating and adjusting musical intervals, and participating in innumerable pilot experiments. Also, thanks to Lewis Dronfeld and Sonja Reid. Each of them provided some unique insights into the pitches perceived with their cochlear implant. Thanks to Dr. Bob Shannon, who devised and implemented the interface used for this research and provided us with sample software routines. My sincere appreciation to my son Michael, for his extensive technical assistance and for writing the computer programs which formed the backbone of this work. My deepest appreciation to my wife Lois, for her patience and emotional support throughout the lengthy preparation of this work, and for her diligence in overcoming numerous obstacles in the preparation of the figures and the final copy of the manuscript. Thanks to Marion Hawryzki for proofreading the manuscript. Thanks also to my daughter Emily, my family and friends for their patience, understanding and support.  xiii  Dedicated to the memory of my Father Who sparked my love for music  xiv  1  MUSICAL INTERVAL PERCEPTION WITH PULSATILE ELECTRICAL STIMULATION OF PROFOUNDLY DEAF EARS INTRODUCTION  The relative contributions of temporal and cochlear place mechanisms in the perception of pitch have been debated for well over 100 years.  Place coding of frequency refers to a  differential distribution of activity across the neuronal array,  as a function of tone frequency.  In the normal ear,  this results from the mechanical and spatial analysis of the Temporal coding of  waveform along the cochlear partition.  frequency results from the synchronization of auditory nerve fiber responses to the mechanical oscillations of the basilar membrane and its associated structures. The place theory of pitch encoding was advanced in the mid-nineteenth century by Helmholtz.  Helmholtz found support  for his theory in three, then recent, developments in auditory science.  Helmholtz theorized that the rods of Corti,  discovered by Corti in 1851, resonators,  formed a series of finely tuned  responsive to progressively lower frequencies in a  basal to apical direction.  In a later publication, Helmholtz  asserted that the transverse fibers of the basilar membrane, rather than the rods of Corti, were the actual resonating elements.  The length, tension, and mass of these fibers were  thought to vary from base to apex.  A second influence was  Ohm’s theory of complex sounds, which stated,  in essence,  that  2  any complex periodic sound could be specified as the sum of a series of sine wave components in appropriate amplitude and phase relationships.  A third major influence was Muller’s  doctrine of the specific energy of nerves.  Helmholtz extended  this doctrine, theorizing that each resolvable frequency component within a stimulus would be associated with a specific cochlear place and corresponding neuron.  Such a  neuron was thought to respond with an intensity—dependent increase in discharge rate.  Thus, the excitation of specific  neurons was thought to be associated with specific pitches. With the subsequent work of von Bekesy, beginning in 1928, the tuned—resonance concept was largely replaced by the traveling wave theory, which incorporates the notion of basilar membrane motion rather than sympathetic resonance of individual transverse fibers.  Bekesy’s observations showed  that movement of the stapes resulted in displacements of the cochlear fluids, with resultant displacements of the scala media alternately towards the scala tympani and the scala vestibuli.  Thus, the vibratory motion of the stapes initiated  a traveling wave along the basilar membrane, moving from base to apex, with a progressive phase lag at successively apical locations.  The locus of maximum displacement and the  amplitude characteristics of the traveling wave were shown to be determined by the mechanical properties of the cochlear partition,  of which the stiffness was found to decrease, and  the width to increase,  from base to apex.  More recent  physiological data have shown that the mechanical responses in  3  the intact cochlea are highly sensitive, and more sharply tuned than found by Bekesy in cadaver ears,  and that these  responses may be aided by active processes. The traveling wave is known to reach a maximum amplitude at a locus consistent with the mechanical properties of the membrane at that location.  Thus,  for high frequencies, the  maximum amplitude is realized within the basal turn, whereas for low frequencies, the maximum amplitude of displacement occurs more apically.  The locus of the population of cochlear  neurons activated at low stimulus intensities is therefore a strong function of the stimulating frequency. neurons,  Individual  therefore, possess frequency selectivity (and a  “characteristic frequency” or CF, to which the response occurs at the lowest intensity)  by virtue of their unique position  along the cochlear partition. the basal turn, regions.  High frequency fibers innervate  and low frequency fibers the more apical  While a low-intensity pure tone stimulus elicits  increased activity in a small population of cochlear neurons, increased intensity of the stimulus results in a spread of neural excitation to the increasing effective displacement of the basal portion of the displacement envelope.  This results  in a recruitment of neurons with higher characteristic frequencies. The “tonotopic” frequency organization, the cochlea,  established in  is maintained in the cochlear nuclei and at  higher levels of the auditory system.  This place-specific  frequency analysis is therefore one mechanism by which  4  frequency information can be transmitted to the central However, demonstrations that the pitch of  auditory system.  complex tones does not depend on the presence of the fundamental frequency (Fletcher, Licklider,  1954)  1938;  have presented a serious obstacle for simple  place pitch theories Miller,  1928; Schouten,  (reviewed in Greenberg,  1980; Sachs and  1985)  The second type of frequency information available to the brain is encoded within the temporal structure of neural responses.  The temporal or periodicity theory was advanced by  Seebeck in 1841, by Wundt in 1880, and later refined by Rutherford in 1886  (reviewed in Plomp 1976).  In these early  conceptualizations, the acoustic waveform was thought to undergo little,  if any, modification in the cochlea.  Thus,  the pitch of a tone was thought to be based on the frequency or rate of neural discharge, rather than on the identity of a particular set of active nerve fibers.  These early temporal  theories were limited in their ability to account for encoding of frequencies greater than several hundred Hertz, as the upper limit of firing for individual neurons is considered to be less than 500 spikes per second. Wever and Bray,  in 1930,  However, the recording by  of frequencies as high as 5 kHz, with  macroelectrodes in the auditory nerve,  led to the initial  suggestion that this potential might represent the aggregate response of a population of nerve fibers, each locked in phase to the stimulating frequency.  Although subsequent studies  have shown that the cochlear microphonic contributed to the  5  recordings of Wever and Bray, these findings nevertheless led to a revival of interest in the role of temporal mechanisms in the perception of pitch. A mechanism by which the pitch of complex tones could be encoded in the temporal responses of auditory nerve fibers was proposed by Schouten  (1940).  In this scheme, the pitch of  complex tones resulted from the preservation of the periodicity of the stimulus waveform in the neural responses, because of the interaction of high, unresolved harmonics in the basal region of the cochlea. Schouten’s proposal,  Further refinements of  suggesting that neural responses occurred  on the high-amplitude peaks of the fine structure of the waveform, were necessitated by evidence of pitch shifts, which were found to occur when the components of a complex tone were frequency—shifted by an equal amount, periodicity remained unchanged.  even though the waveform  Evidence regarding the  relatively greater importance of low order harmonics 1967)  (Ritsma  in the perception of pitch (i.e., those which are  relatively more resolved in the cochlea), has necessitated further revisions of temporal theories,  such as contributions  from units tuned to frequencies midway between contiguous resolved components (Javel 1980).  In spite of a wealth of  physiological and psychoacoustical data, the precise roles and contributions of temporal and place mechanisms in the perception of pitch remain poorly understood.  We shall now  consider some of the evidence for the importance of temporal information.  6  The precision of phase-locking and the preservation of the temporal features of the stimulus waveform in the responses of auditory nerve fibers and Hind 1967; Javel 1980; Geisler,  and Deng 1986),  (Rose, Brugge, Anderson,  Sachs and Young 1980; Greenberg,  as well as the persistence of these  features into the auditory brainstem Moiseff,  and Konishi 1984;  and Schreiner 1988;  (Langner 1983; Takahashi,  Sullivan and Konishi 1986; Langner  Schreiner and Langner 1988),  suggest that  temporal information may play an important role in providing cues regarding stimulus frequency (Javel, McGee, Horst, Farley 1988).  However, there exists,  in addition,  and  cogent  psychoacoustical evidence regarding pitch shifts which cannot easily be explained on the basis of temporal mechanisms et al.  (Davis  1950)  A variety of data from normal—hearing subjects have supported the existence of temporally based pitch.  Stimuli  consisting solely of high, unresolvable harmonics have been found to evoke residue pitches corresponding to the absent fundamental 1990).  (Moore and Rosen 1979; Houtsma and Smurzynski  Additional evidence for the importance of temporal  information in pitch perception has been obtained from experiments utilizing amplitude-modulated white noise and Taylor 1948;  Small 1955; Harris 1963; Pollack 1969;  Patterson and Johnson—Davies 1977). (1976,  1981)  (Miller  Burns and Viemeister  demonstrated that the pitches evoked by  sinusoidally amplitude-modulated noise were sufficiently salient to enable subjects to recognize simple melodies and  7  musical intervals, when the constituent “notes” corresponded to low modulation frequencies. Further evidence for temporal mechanisms in pitch perception can be found in time separation and phase—shift phenomena.  Cramer and Huggins  (1958),  for example,  showed  that dichotic white noise, of which a small frequency region presented to one ear had been phase—shifted, resulted in the perception of a faint pitch which corresponded to the frequency of the phase transition.  (Small and  Others  McClellan 1963; Warren and Bashford 1988)  have shown that when  a pulse train is complemented with a delayed replica of itself, a pitch is heard which corresponds to the reciprocal of the delay (t)  between the two pulse trains.  Pitch matches  between periodic all-positive polarity pulses and periodic patterns of positive and negative polarity pulses and Guttman 1960; Warren and Bashford 1988)  (Flanagan  further support  the existence of pitches which are based on exclusively temporal mechanisms, at low pulse rates  (below 200 pps).  At  higher pulse rates, pitch matches were made on the basis of fundamental frequency. Similarly, Pierce (1991) rates  showed that,  at low repetition  (up to about 250 Hz for bursts of a 4978 Hz tone),  sequences of tone bursts with the same sign and sequences of tone bursts with alternating sign were perceptually indistinguishable,  in spite of differences in fundamental  frequency and harmonic spacing.  The pitches resulting from  these stimuli, at low repetition rates, have been shown to be  8  sufficiently salient to convey musical interval information (Pierce 1991) With acoustical stimuli and normal-hearing listeners,  it  has been difficult to devise experiments that unequivocally dissociate place and timing information (Pierce, Lipes, Cheetham 1977). delay,  and  When noise is added to itself following a  the power spectrum displays peaks at 1/t Hz,  integral multiples of 1/t Hz.  and at  Although interrupted or  amplitude—modulated white noise has an essentially flat longterm spectrum,  fluctuations in the short—term spectrum may  permit limited spectral cues related to modulation rate which, without appropriate precautions  (filtering or masking),  contribute to pitch perception (Pierce, Lipes, 1977).  could  and Cheetham  While these alterations in the statistical properties  of the noise are thought to be small, especially with sinusoidal modulation,  and of limited significance in  demonstrations of temporally based pitch (Moore and Glasberg 1986),  these possibilities introduce uncertainties regarding  the utilization of temporal features.  Furthermore, even in  the absence of spectral peaks and valleys in the acoustic stimuli themselves, experimental findings could be contaminated by the presence of distortion products generated within the ear.  Horst, Javel, and Farley (1990)  have shown,  in the phase—locked responses of auditory nerve fibers, the presence of nonlinearities which introduce distortion products at integer multiples of the signal frequencies and at frequencies lower than those contained in the signal.  9  With direct electrical stimulation of residual auditory neurons in deaf subjects, however, analysis of the signal 1973;  there is no spectral  (Kiang and Moxon 1972; Merzenich et al.  Shannon 1983; Javel, Tong,  Shepherd,  den Honert and Stypulkowski 1987a),  and Clark 1987; van  and the locus of neuronal  stimulation is dependent exclusively on the distribution of For bipolar  current relative to the residual neuronal array.  stimulation, this distribution is a complex function of the bipolar electrode location, the electroanatoiny of the residual cochlear apparatus, duration  and of current amplitude and pulse The tight  (van den Honert and Stypulkowski l987a).  relationship between the temporal characteristics of the electrical stimulating waveform and the neuronal response (Hartmann, 1989)  Topp,  and Klinke 1984; Javel et al.  1987; Parkins  suggest that electrically stimulated subjects present an  unique opportunity to investigate periodicity-related pitch mechanisms. Although a substantial body of literature has accumulated over the past 30 years supporting the presence of rate—based pitch effects with electrical stimulation  (Eddington,  Brackmann, Mladejovsky, and Parkin l978a,  1978b;  al.  1979,  1981; Tong,  Tong and Clark 1983,  Clark,  Blamey,  Pfingst 1985; Townshend, Shallop,  Simmons et  and Dowell 1982;  1985; Hochmair-Desoyer, Hochmair,  and Stigibrunner 1983; Shannon 1983;  1987;  Busby,  Burian,  Eddington and Orth 1985;  Cotter, Van Compernolle,  Beiter, Coin,  Dobelle,  and Mischke 1990),  and White  the  qualitative aspects of these pitch percepts and their  10  relevance to communication have rarely been addressed (Muller 1983; Pfingst 1985;  Schubert 1983).  Most studies of  electrical rate pitch have employed conventional psychophysical procedures such as difference limen assessment, pitch matching, and magnitude estimation of pitch on an arbitrary scale.  While these procedures have been useful in  documenting some elementary relationships between electrical frequency and subjective pitch, they do not place pitch within a meaningful context.  Even with normal—hearing subjects,  pitch scales resulting from approaches such as magnitude estimation,  interval equisection and quasi—fractionation  (as  employed by Stevens and Volkmann in 1940), have been shown to be of little relevance to pitch perception in the musical (Ward 1970).  sense  Psychophysical studies in normal-hearing  subjects have also failed to demonstrate a correlation between the size of the just-noticeable-difference for pitch of pure tones and the ability to learn or reproduce simple tunes  (Wing  1948) Although a number of experiments between 1966 and 1979 suggested that limited recognition of common melodies could be achieved with electrical stimulation of ears without functional receptor cells 1978a,  (Simmons 1966; Eddington et al.  1978b; Chouard 1978; Moore and Rosen 1979),  investigators have failed to confirm these findings and Lansing 1991).  other (Gfeller  Thus, while it appears clearly established  that pitch increases systematically with electrical frequency or pulse rate up to a maximum of 300 to 1000 Hz,  it is not  11  clear whether these pitches permit the perception of melody or the recognition of musical intervals, as defined by electrical frequency or pulse rate ratios. In the investigations which follow,  Experiment I explored  the ability of 17 musically unsophisticated cochlear implant (Nucleus)  recipients to identify common tunes “played” by  systematic variation of pulse rate on single apical intracochlear electrodes, using low pulse rates.  Experiments  Il-V employed three musically unsophisticated Nucleus implant recipients in a more detailed examination of the musical pitch and interval properties of electrical pulse rate pitches. Experiment II assessed the ability of these subjects to identify melodies from a closed set,  in the absence of  rhythmical and melody-length information.  These brief  melodies were played on electrodes in a variety of locations along the electrode array, and over a range of pulse rates. Experiments III to V examined the relationships between the sizes of melodic pitch intervals for electrical pulse rate pitches and the frequency ratios which characterize acoustical musical intervals.  In Experiment III, this was accomplished  by means of intonation quality judgements, where the subjects rated a randomized series of pitch intervals as “flat”, tune”,  “in  or “sharp”, relative to their memory of a specific  musical interval from a well—known melody.  In Experiments IV  and V, the method of adjustment was employed to assess the ability of the same subjects to reconstruct and transpose  12  musical intervals resulting from changes in pulse rate on single intracochlear electrodes.  13  GENERAL  METHODS:  INTRODUCTION  Psychophysical results with cochlear implant patients have shown that there are several variables which influence Amongst the most widely  the pitch of electrical stimuli.  investigated are those which pertain to the locus of stimulation  (electrode position),  or repetition rate.  and to the signal frequency  These pitchlike percepts are commonly  referred to as “place pitch” and “rate pitch”,  respectively.  Many studies have confirmed that gradations of pitchli]ce or timbrelike percepts result from electrical stimulation in These sensations have been  different regions of the cochlea.  found to vary in accordance with the tonotopic organization of the cochlea Martin,  (Eddington et al.  Busby,  and Patrick 1980;  1983; Townshend et al. addition, frequency.  1978a; Tong, Millar,  1987).  Simmons et al.  Clark,  1981;  Electrode position may,  Shannon in  interact in a complex fashion with pulse rate or Shallop et al.  (1990),  for example,  reported that  increases in pulse rate had a much greater effect on pitch estimates for apically than for basally situated electrodes. Shannon  (1983)  reported pitch estimates consistent with  increases in electrical sinusoidal frequency for both basal and apical electrodes,  although the asymptotic pitch estimates  remained higher for basal than for apical electrodes.  14  Stimulus waveform and mode of stimulation (monopolar or bipolar)  have also been reported to influence perceived pitch.  Shannon (1983)  found the expected pattern (of higher pitch  estimates upon stimulation of basal electrodes, and lower pitch estimates for apical electrodes)  to be quite specific to  the mode of stimulation. Pitchlike phenomena related to the rate rather than the place of stimulation have also been described extensively (Merzenich et al.  1973; Hochmair-Desoyer et al.  1983;  Shannon 1983; Tong and Clark 1983; Townshend et al. Shallop et al.  1990).  However,  1987;  it is frequently difficult to  differentiate pitch-related phenomena from those related to timbre.  Simmons  (1966),  for example, reported that  stimulation at rates greater than 50 Hz  (both pulsatile and  sinusoidal)  resulted in auditory percepts which were described  as “rough”,  “rattling”, and “crackling”, whereas stimuli above  100 Hz were described as “smooth” and “steady”.  Between 20 Hz  and 100 Hz, the pitch changes were reported to be small, occasionally,  nonexistent.  and  For repetition rates between 100  and 400 Hz, pitch rankings were consistently in the expected direction.  Further increases in rate, beyond 400 Hz, yielded  anecdotal descriptions corresponding to pitches normally evoked by acoustic stimulation with 2—4 kHz sounds, change in pitch.  or no  Similar rapid increases in pitch with  increases in frequency or repetition rate have been reported by others  (Tong, Black, Clark,  Forster, Millar, Q’Loughlin,  15  and Patrick 1979; Shannon 1983; Tong, Blarney, Dowell,  and  Clark 1983). However,  electrical pulse rate or frequency and electrode  position are not the only stimulus parameters which have an effect on pitch or timbre.  Variation of pitch and timbre have  also been reported with changes in stimulus amplitude 1983).  Simmons et al.  (1979,  1981),  (Shannon  for example, reported  anecdotally that a low—intensity stimulus on a given electrode could sound like “a knock on a large sheet of plywood”, whereas a high—intensity stimulus on the same electrode resembled “a small triangle being struck”.  These changes were  quite specific to certain electrodes, while on other electrodes,  increases in amplitude resulted in only loudness  increases, without concomitant changes in pitch or other perceptual attributes.  Asymmetrical spread of current or  spatial asymmetry of surviving neurons relative to the stimulating electrodes have been considered as potential explanations for these phenomena. The potential effects of stimulus amplitude on pitch or timbrelike percepts necessitate a further consideration of loudness.  With electrical stimulation of hearing, the primary  physical determinant of loudness is the amount of charge delivered per unit time 1983).  (Eddington et al.  l978a; Tong et al.  When pulse width is increased, the amount of charge  delivered and the amount of time available to charge the capacitance of the neuronal membranes are also increased. Thus,  stimulus amplitude, pulse width,  and stimulus duration  16  have been shown to be the parameters which have the greatest effect on psychophysical detection thresholds and on perceived loudness  (Eddington et al.  Holloway 1991;  1978a; Pfingst,  Shannon 1992).  DeHaan,  and  Loudness has also been shown to  be an important factor in the performance of cochlear implant patients on psychophysical tasks such as the measurement of gap detection thresholds and temporal difference limens,  in  that smaller temporal differences can be appreciated at higher intensity levels  (Shannon 1989; Tyler, Moore,  Loudness judgements may,  furthermore,  be complicated by  psychological factors such as “pleasantness”, “shrillness”,  and Kuk 1989).  “noisiness”,  as well as by the probable unnaturalness of  auditory sensations that result from electrical stimulation. The lower and upper limits of the operating range of cochlear implant electrode pairs are generally defined by the thresholds of detection and of loudness discomfort, respectively  (Pfingst 1984).  Between these two extremes lies  the dynamic range for electrical stimulation.  The amount of  current required to generate hearing sensations in profoundly deaf ears has been shown to vary considerably between patients and electrode configurations,  and to correlate negatively with  both nerve fiber survival and proximity of the electrodes to the surviving nerve fiber population  (Pfingst and Sutton  1983) In spite of extensive verification of the generality of “place” and “rate” pitchlike effects with electrical stimulation of hearing,  the qualitative aspects of these  17  Part of the  phenomena remain insufficiently characterized. difficulty, no doubt,  lies in the elusive subjectivity and  complexity of the pitch percept, as well as in inadequate definitions of what is meant by pitch.  Pitch is broadly  defined by the American National Standards Institute S3.20-1973)  (ANSI  as “that attribute of auditory sensation in terms  of which sounds may be ordered on a scale from low to high.” This definition, however, properties of pitch.  addresses solely the ordinal  Furthermore, many implanted subjects  have little sophistication in describing hearing sensations, especially following prolonged auditory deprivation. Further complicating the issue is the ambiguity of the “high” and “low” descriptors generally applied to pitch. These descriptors,  in common usage,  also to loudness and timbre. can also be scaled, matched, high.  are frequently applied  These attributes, labeled,  like pitch,  and ranked from low to  It is clear that conventional psychophysical tasks such  as pitch matching, pitch ranking, and magnitude estimation are inadequate for the dissociation of pitch and timbre. in the musical sense,  Pitch,  implies more than the mere ability of  the listener to make “higher” versus “lower” judgeinents (ordinal pitch).  Musical pitch,  in addition,  implies that  musically intelligent listeners should be able to assign a tone or stimulus to a position relative to another tone on a musical scale. identify,  label,  In short,  such listeners should be able to  or adjust musical intervals.  Houtsma  (1984)  has argued that any useful definition of pitch should be based  18  on the ratio properties of the pitch percept. of this paper,  For the purpose  the operational definition of pitch will be  that adopted by Burns and Viemeister  (1981),  i.e.,  that aspect  of auditory sensation which carries melodic information. While loudness equalization is of paramount importance in the accurate measurement of pitch difference limens 1988),  (Pfingst  it is not certain whether exhaustive loudness  equalization is essential in melody recognition and musical interval adjustment tasks.  To preclude the availability of  gross loudness cues in the experiments which follow,  stimuli  were drawn from a continuum of pulse rates which were equalized for loudness.  While a procedure which randomized  stimulus amplitude within the dynamic range of individual subjects might have been more desirable, this was precluded by software limitations, as well as by potential interactions between stimulus intensity, timbre, and pitch.  A procedure  was therefore devised in which subjects, using the method of adjustment at a number of particular pulse rates,  established  the current amplitudes required for a “comfortable”  (defined  as neither too loud for extended listening, nor too soft so as to be difficult to hear)  and equal loudness over the range of  pulse rates tested.  EXPERIMENTAL SETUP  The Nucleus Ltd./Cochlear Corporation cochlear implant  delivers a pulsatile electrical signal via an array of 22  19  electrodes inserted into the scala tympani Clark,  and Seligman 1987).  (Blarney,  The electrode array  Dowell,  (Model C122M)  consists of 32 platinum bands spaced 0.75 mm apart, on the distal 25 mm of a silastic carrier.  situated  The 22 distalmost  (occupying the distalmost 17 mm of the array)  bands  the functioning electrodes.  serve as  Typical insertion distances  result in the delivery of current pulses to cochlear regions with characteristic frequencies of 1-16 kHz 1990).  (Greenwood 1961,  The electrodes are arbitrarily numbered from 1-22 in a  basal to apical direction, with the electrode number referring to the basal member of a bipolar electrode pair (active/ground).  These bands are independently connected to  and driven by the receiver—stimulator package, surgically embedded in the mastoid bone, hermetically sealed titanium capsule.  which is  and consists of a  The receiver—stimulator  contains a Complementary Metal Oxide Semiconductor  (CMOS)  integrated circuit which decodes the signal and routes the information to selected electrodes.  The data stream  originating from either the patient’s wearable speech processor or from an appropriate test device is transmitted transcutaneously to the implanted receiver—stimulator package by means of an externally worn transmitter coil, using a radio frequency  (2.5 MHz)  signal.  The clinical test device software  (DPS659A)  (Dual Processor Interf ace)  used in the routine psychophysical  assessment and programming of the processor worn by the patient does not permit detailed control over stimulus  and  20  parameters.  Therefore,  for the purpose of these experiments,  the stimulus pulses delivered to the implanted Receiver/Stimulator were configured by a special computer Interface for Psychophysical Research with the Nucleus Cochlear Implant,  designed and implemented at Boys Town The technical specifications of  National Institute (BTNI).  this interface have been detailed elsewhere Ferrel,  Palumbo,  and Grandgenett 1990).  (Shannon, Adams,  The BTNI interface  used in these experiments was an outboard version, the host 386 computer.  external to  The interface was connected to the  computer via the parallel printer port.  The host computer and the  transmitted a specified byte stream to the interface,  interface then generated the appropriate burst sequence for transmission to the internal receiver/stimulator (Figure 1). Pulse rate,  amplitude,  by the experimental setup interface,  and waveform parameters generated  (the host computer and the BTNI  and the transmission cable/coil)  were verified by a  test system consisting of a Receiver/Stimulator unit in—a—Box,  (Implant-  consisting of a Nucleus C122M internal device  identical to that implanted into the experimental subjects, a plastic enclosure)  in  and a Tektronix 2232 100 MHz Digital  Storage Oscilloscope. The sample software routines provided with the BTNI interface were rewritten and expanded,  using Pascal,  in order  to permit the delivery of sequences of pulse trains at specified rates and amplitudes.  Pulse parameters were  individualized for each subject,  so that amplitudes were  21  Transmitter  Coil  Equipment setup with the Boys Town National Figure 1. Institute interface for psychophysical research with the Nucleus cochlear implant.  22  within the linear operating range of the device, detailed in Table 1.  and are  The purpose of the preliminary  psychophysical measurements was to establish an equal and comfortable loudness level over a range of pulse rates on selected electrodes.  These amplitude values were then  accessed by all subsequent routines, which permitted the delivery of either continuously variable pulse rates,  or  specific sequences of pulse rates organized into tunes, melodies or musical intervals.  All randomization of stimuli  and scoring procedures were performed automatically by the computer.  Subjects were provided with written instructions  for each of the experiments.  For all the experimental  procedures which follow, the subjects were connected directly, via a standard tricord cable and external transmitter coil, to the BTNI Interface.  PRELIMINARY PSYCHOPHYSICAL MEASUREMENTS  The method of adjustment was used to establish a comfortable listening level on selected electrodes, approximately a 4—octave range of pulse rates, pps.  for  from 54 to 1096  Spatial separation of active and ground electrodes and  pulse widths were individualized for all subjects, due to differences in current requirements of individual subjects (Shannon 1989; Blarney, Pyman, Clark, Dowell, Gordon, and Hollow 1992).  Brown,  These parameters, once established for each  subject, were maintained constant throughout the remainder of  23  TABLE 1 SUBJECT DATA*  S  AGE DUR. AGE ONSET DEAF.  1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17  68 59 54 35 40 37 35 58 44 70 43 77 33 38 58 64 50  15 39 32 32 34 19 30 48 20 8 37 57 16 6 7 42 46  1 1 4 2 1 2 2 9 24 1 4 1 13 9 5 20 1  YRS EXP.  yr 1 yr 2 yrs 5 yrs 2 yr 5 yrs .5 yrs 2 yrs 1 yrs 4 yr 2 yrs 1 yr .5 yrs 5 yrs 2 yrs 5 yrs 5 yr 3  ETIOLOGY Unknown Unknown Labyrinthitis Unknown Trauma Unknown Unknown Trauma Trauma Otitis Media Meningitis Unknown Meningitis Unknown Otoscierosis Unknown Unknown  MUS E 0 2 2 1 0 0 0 0 0 2 2 0 2 2 2 0 2  12 18 18 20 19 18 18 18 20 18 18 18 18 19 14 20 20  PW jsec 100 205 150 150 100 100 150 100 205 250 100 100 150 100 250 150 150  ELEC. SEPAR. 1.50 2.25 .75 1.50 .75 2.25 1.50 1.50 1.50 2.25 2.25 1.50 2.25 2.25 2.25 1.50 1.50  % 73 89 83 64 96 74 91 88 23 88 84 55 82 77 50 68 88  *Legend: S: Subject. Age: Age in years at time of experiments. Age Onset: Age in years at onset of hearing loss. Dur. Deaf.: Duration of total deafness prior implantation. Yrs Exp: Duration of experience with cochlear implant. Mus: Self-rated musical ability prior to deafness: 0=No interest in music; 1=Interest in listening to music; 2=Played a musical instrument. E: Basal member of electrode pair used for testing. Electrodes are numbered in a basal to apical direction, from 1-22. PW sec: Duration (in microseconds) of each phase of biphasic current pulses. Interpulse interval 40 tsec. Elec Separ: Distance in millimeters between electrode pairs. %: Average keyword correct score on CID Everyday Sentences, Iowa Sentences, and BKB Sentences (Cochlear Corporation, Tape Recorded Version).  24  the experiments  (Table 1).  The interval between the negative  and positive polarity portions of the biphasic waveform was 40 jsec for all subjects.  This value of interpulse interval was  arbitrarily selected merely because it was the interpulse interval used for clinical applications of the device.  Trains  of pulse rates were presented at a rate of 1 per second, with a 50% duty cycle.  The relatively long duration (500 msec) was  intended to permit subjects sufficient time to estimate pitch, and was consistent with stimulus duration in acoustical studies requiring musical pitch judgements.  Subjects were  instructed to adjust the amplitude of the signal  (using the  “up” or “down” arrow keys of the computer keyboard) comfortable loudness. bracketing,  i.e.,  to a  They were instructed to do this by  by making the sound alternately too loud and  too soft before deciding on a comfortable loudness level.  The  subjects were instructed to press the <Enter> key upon completion of the trial for a given pulse rate.  The computer  then logged the appropriate amplitude stepnumber for that pulse rate,  and automatically proceeded to test the next  higher pulse rate.  Amplitude stepnumbers were converted to  microamperes by means of a table, stored in computer memory, of actual measurements made by the manufacturer on each device prior to implantation.  A total of 16 pulse rates were tested,  increasing by an arbitrary factor of 1.22 from 54 to 1096 pps. The entire set of rates was then replayed, both in sequence and in random order,  and inequalities in loudness eliminated  25  by further adjustment, until all the pulse rates in the sequence were judged to be equally loud.  EXPERIMENTAL SUBJECTS  The small number of local cochlear implant recipients precluded recruitment of subjects with documented musical competence.  However, since performance of cochlear implant  recipients on speech recognition tasks is known to vary widely,  it was conceivable that optimal results would not  necessarily be obtained from the most musical subjects. Fourteen unpaid volunteer subjects who were returning to the Laboratory for routine follow-up testing participated in the open—set tune recognition test.  Subjects were questioned  about their interest and participation in musical activities, prior to deafness. in Table 1.  Details regarding subjects are presented  All subjects  (except S-9) were reported,  at  surgery, to have a minimum of 22 intracochlear electrodes. Three subjects Experiments 11—V.  (S-7,  S-b,  S-il)  participated in  These subjects were paid,  and were not  recruited on the basis of their musical ability. subjects understood the concept of an octave. were readily available for the experiments, interest in the project.  None of the  These subjects  and had a general  All three subjects had excellent  speech perception results with the implant.  A brief musical  history of these three subjects is detailed below.  26  SUBJECT 7  (S-7)  This 35 year old male reported no particular interest in music prior to deafness.  He was, however,  variety of common tunes.  Although he had never played a  musical instrument,  or participated in any significant  sung,  familiar with a  musical activity, he was nevertheless a highly motivated and committed subject, with an interest in exploring his perceptual capabilities.  SUBJECT 10  (S—b)  This 70 year old male had sung as a cathedral chorister between the ages of 8 and 15 years, and was able to recall both the texts and tunes of a variety of classical choral selections. player. child,  His father had been a semiprofessional tuba  Subject 10 had received a few violin lessons as a but had not played any musical instruments since  childhood.  In spite of his gradually worsening hearing since  the age of 8 years, he reported having had an intense interest in classical music during most of his adult life,  and  participated extensively in ballroom dancing activities. Postoperatively, he indicated little or no enjoyment of the melodic and harmonic aspects of music, but was able to continue dancing.  Subject 10 was conversant with the terms  “sharp” and “flat” as they relate to interval size,  and  applied these terms voluntarily and readily to musical intervals played at low pulse rates.  For example, when  adjusting the lower note of a musical interval with a fixed  27  upper note, he readily described the interval as sounding progressively “sharp”, even though it was the lower note of the interval which he had adjusted “flat”. The electrode array for this subject was located in the scala vestibuli.  Extensive osteoneogenesis following the  removal of an identical previous cochlear implant from the scala tympani precluded reinsertion via the round window.  The  removal of the first device was necessitated by postoperative Except for the higher current requirements with  infection.  the scala vestibuli device, audiological results were comparable to those achieved previously in the same patient with the scala tympani implant (Pijl and Noel 1992).  SUBJECT 11  (8-11)  This 43 year old female reported a considerable interest in singing prior to the onset of deafness.  Although she did  not read music or play any musical instruments, having been able to play, by ear, of accuracy,  on a piano keyboard.  wide variety of common tunes.  she reported  simple tunes, with a degree She was familiar with a  With her cochlear implant,  she  reported little or no enjoyment of music, and indicated only occasional recognition, following a large number of repetitions, of tunes played on musical instruments by her children.  She felt that her occasional recognition of common  tunes in everyday situations was based primarily on rhythmical cues.  28  RESULTS  The results of the preliminary loudness equalization procedures for several subjects are shown in Figures 2 to 5. The current levels required to generate comfortable hearing sensations ranged from approximately 400 to 1600 )LA, depending on the subject, the spatial separation between the active and ground electrodes, and the pulse width employed.  Direct  comparison of current amplitudes for different subjects is precluded by intersubject differences in stimulation parameters  (Table 1).  Reductions in pulse width resulted in  marked elevations in the amplitudes required for comfortable hearing levels, when the active and return electrodes were kept constant  (Figure 2).  For most subjects, the amplitude  function showed a decrease as pulse rates were increased (Figures 3 and 4).  The slope of this function was steep for  some subjects  S—b), especially at low pulse rates,  (S—3,  shallow for others  (S-7,  S-14).  and  In order to maintain an equal  loudness sensation, one subject (S-il)  required higher  amplitudes as pulse rates were increased.  The current  amplitudes required for comfortable hearing sensations were found to be reasonably consistent from session to session (Figure 5). Anecdotally, pulse rates below 100 pps were often described as a “telephone ringing” or “motorboat”, depending on the basal or apical location, respectively, of the activated electrode.  Subjects reported that pulse rates above  29  800  —  700  --_  600  500  200  54  66  81  89  ili  log  11  21  20  3Q  492  601  730  897  1096  PULSE RATE PW=150  --  PW=300  -  PW=600  Current amplitudes at comfortable loudness Figure 2. from 54 to 1096 pps over a range of rates for pulse Subject 7, from 150 to 600 isec. pulse widths, 1.5 mm. = separation Electrode Electrode 18. PW = pulse width/phase.  700  620  040 0 60  460  380  300  [E1IE18_-A-- 541E20  --  S31E18  --  514/E19  Current amplitudes at comfortable loudness Figure 3. for pulse rates from 54 to 1096 pps for 4 subjects. data regarding subject and stimulation Additional Subject; E= Electrode. parameters in Table 1. S=  30  1600 1400 1200 1000 800 600 400 200  54  81 66  121 99  -.-  181  210 403 601 897 221 330 492 735 1096 PULSE RATE  148  S-11/E-I8IPWIOO —w-- S10iE-18IPW25fj  Figure 4. Current amplitudes at comfortable loudness for two subjects with electrode separation of 2.25 mm. S=Subject; E=Electrode; PW=Pulse width/phase.  1600 1400 1200 1000 800  -f I  600  I f I  400 200 54  81 66  121 99  181 148  270 601 403 897 221 330 492 735 1096 PULSE RATE  Figure 5. Current amplitudes at comfortable loudness for Subject 7 over 10 sessions. Markers and vertical error bars represent the mean and ± 1 standard deviation. Electrode 18; pulse width 150 jsec. Electrode separation 1.5 mm.  31  100 pps tended to sound more pleasing (“more musical” and “smoother”) and above,  than lower pulse rates. some subjects  At rates of 400—500 pps  (especially S—li)  frequently reported  hearing transient onset pitches followed by a sustained noise with no definite pitch.  Other subjects  (particularly S-b)  reported hearing occasional “double” pitches, which consisted of both a high-pitched and a simultaneous low-pitched component.  However, these phenomena appeared to occur  inconsistently, and were often not replicable, even upon repetition of the identical pulse rate sequence.  Other  subjects reported inconsistent decreases in pitch at pulse rates above 600 pps. Above 200 pps,  especially with wider pulse widths,  some  subjects reported hearing only a transient, rapidly disappearing sound.  One subject described this as resembling  the sound of a kettle drum, and another, as the sound of a pencil tapping.  For 1000 sec/phase pulses, this occurred at  approximately 200 pps, whereas for 300 sec/phase pulses, this limit was usually near 1000 pps.  These upper limits appeared  to be similar for all subjects and electrodes tested. Apical electrodes were reported by some subjects to resemble the sound of a tuba, while basal electrodes were described as sounding “like a telephone ringing”.  Subjects  who participated in pilot experiments described the apical electrodes as sounding “more musical” than the basal electrodes.  Two subjects  (S-2 and S-l4),  both of whom had  some musical background, volunteered that the place pitches  32  resulting from systematically sweeping across electrodes (1 electrode/sec)  in a basal—to—apical or apical—to—basal  direction at a constant pulse rate (250 pps)  resulted in “a  better musical scale” than did slow (1/sec) pulse rate sweeps on single electrodes.  Only two subjects volunteered that the  pitch changed with increases in amplitude. indicated that only at high pulse rates  Subject 7  (above 700-800 pps) Subject 5  did the pitch appear to increase with amplitude.  described a pitch decrease with increases in amplitude at all pulse rates.  DISCUSSION  The current amplitudes required for comfortable hearing sensations were comparable to those reported with similarly spaced electrodes in both humans and animals Dewhurst,  (Clark, Black,  Forster, Patrick, and Tong 1977; Tong et al.  Tong et al.  1983;  Shannon 1989; Blamey et al.  1982;  1992).  Interindividual and interelectrode differences in current amplitudes required for auditory thresholds have been reported to be associated with differences in nerve fiber survival and proximity of the electrodes to the neural targets Sutton 1983).  (Pfingst and  Increased distance between stimulating  electrodes and neural targets may explain the high current levels required by S—b, whose electrode array was located in the scala vestibuli.  Intersubject comparisons of stimulation  requirements in our data were precluded by individual  33  (Shannon  differences in electrode spacing and pulse width 1989; Blarney et al.  1992)  When pulse width equals the period of electrical sinusoids,  the current amplitudes required for detection  thresholds and for the upper limits of the dynamic range have been reported to increase systematically with increases in pulse rate, 1984).  (Shannon 1983; Pfingst  especially above 100 pps  However, with pulses of constant width,  utilized in our study,  such as those  psychophysical detection thresholds and  the upper limits of the dynamic range have been reported to decrease as pulse rate is increased Dobelle,  (Mladejovsky,  and Brackrnann 1975; Pfingst,  Spelman 1979; Tong et al.  Eddington,  Donaldson, Miller,  1983; Pfingst 1984),  and  presumably due  to an increase in charge transferred per unit time. While we did not specifically examine hearing thresholds and maximum acceptable loudness,  our results confirmed that  for most subjects, the current amplitudes required for a comfortable loudness decreased somewhat as pulse rate was increased.  The pulse rate versus amplitude function for S—b  showed a particularly steep slope.  In contrast,  an increase in current at higher pulse rates, equal loudness sensation.  S—li required  to maintain an  It is possible that the upsloping  curve for S—li was a manifestation of increasingly rapid adaptation at higher pulse rates, perhaps associated with the relatively long duration  (500 msec)  stimuli.  These  differences between subjects may reflect differences in nerve survival.  Differences in the slope of the frequency/amplitude  34  function for psychophysical detection thresholds with different degrees of cochlear pathology have been reported for implanted monkeys, (1981).  by Pfingst,  Sutton, Miller and Bohne  In spite of the subjectivity of the comfortable  loudness criterion, patients demonstrated no difficulty replicating comparable results in different sessions. The perception of intermittency for very low electrical pulse rates has been reported by other investigators 1966;  Simmons et al.  1981;  Shannon 1983),  (Simmons  and is consistent  with gap detection thresholds of 2-5 msec reported for Nucleus implant recipients  (Shannon 1989),  corresponding to interpulse  intervals at pulse rates of 500 and 200 pps,  respectively.  The relatively more pleasing, musical quality of pulse rates between 100-300 pps has also been reported previously 1966; Merzenich et al.  (Simmons  1973).  The instability of auditory percepts observed with 500 msec pulse trains at high rates may relate to the refractory period of the electrically stimulated neuronal population. Van den Honert and Stypulkowski  (1984), using both  intracellular and extracellular recordings of the cat auditory nerve activity in response to monophasic 100 sec closely spaced electrical pulses,  reported a graded decrease in the  amplitude of the neuronal response to a second pulse as interpulse intervals were decreased below 1.0 msec. Topp,  and Klinke  probability,  (1984)  Hartmann,  reported a similar decrease in firing  due to the refractory period of neurons, when  interpulse intervals were shorter than 5 msec.  These findings  35  suggest that 200 to 1000 pps may constitute an approximate upper limit for eliciting equal-amplitude responses to both pulses of a closely spaced pulse pair,  and that closer spacing  of pulses may generate only an “on” response to the first pulse of a pulse train.  This phenomenon could account for the  rapid loudness decrement noted with 500 msec stimulation at higher pulse rates.  An analogous abolition of action  potentials of auditory nerve fibers to continued  (2-5 sec)  stimulation with electrical pulse rates in excess of 600 pps was reported by Javel et al.  (1987), who attributed this  phenomenon to probable depolarization block. Alterations in pitch with changes in stimulus amplitude, such as those reported by S—5 and S—7, have also been previously reported by other investigators. for example,  Shannon  (1983),  showed an increase in pitch estimates for 1000 Hz  monopolar stimulation,  as amplitude was increased.  These  amplitude-dependent pitch shifts were reported to be more prominent than those associated with changes in the site of stimulation.  Townshend et al  increase for two patients, patient,  (1987)  reported a pitch  and a pitch decrease for a third  as intensity of a 100 and 200 Hz stimulus was  increased.  Parkins  (1989)  has suggested that changes in  firing rate which occur with changes in amplitude may provide a partial explanation for these phenomena.  It is equally  possible that the decrease in pitch with increased amplitude, such as that reported by S-5,  could result from a predominance  of place pitch information at low stimulus amplitudes,  and an  36  increasing prominence of rate—based pitches with the enhanced entrainment of spikes to the temporal pattern of the electrical stimulus, as amplitude is increased.  Amplitude—  dependent pitch shifts could also result from spatial asymmetry in current spread or asymmetry of neural survival, relative to the stimulating electrode. The decreases in pitch reported by some subjects at high pulse rates are also consistent with psychophysical findings of previous investigators (Hochmair-Desoyer et al. Shannon 1983).  Physiological data  that at low pulse rates,  1983; have shown  (Parkins 1989)  interspike intervals are determined  largely by repetition rate.  At high pulse rates  (2500 pps),  neuronal responses were shown to remain fairly periodic,  but  interspike intervals became a strong function of stimulus intensity  (with decreasing interspike intervals as intensity  is increased) rather than of interpulse intervals. Presumably,  at high pulse rates, the interspike intervals are  determined by interactions between the relative refractory status of neurons as they recover from the previous response, and the amount of charge delivered during the excitatory phase of the stimulus waveform.  Therefore, no further responses may  occur until sufficient time has elapsed for the excitation threshold to fall below the charge per phase delivered by the stimulus.  The threshold nature of these responses  the initial pulse of a high frequency burst),  (to all but  as well as the  summation of jitter and possible variations in the neuronal refractory curve preclude a high degree of synchronization to  37  the electrical waveform, rates.  such as that seen at lower pulse  38  EXPERIMENT I:  OPEN-SET TUNE RECOGNITION  Recognition of melody (a sequential pattern of pitches) requires more than simple pitch perception.  It requires  extraction, by the listener, of certain relational properties between the constituent stimulus elements in a tonal sequence. Experiment I investigates,  in a group of cochlear implant  recipients not selected for pre—deafness musical ability or postoperative speech perception results, the recognition of common tunes,  “played” by systematically varying pulse rate on  single apical intracochlear electrodes. this paper,  For the purpose of  “tune” was arbitrarily defined as a sequence of  musical intervals of which the rhythmical pattern was preserved, whereas “melody” was reserved for sequences of musical intervals without rhythmical or phrase-length information  (i.e., pitch changes only).  While it was fully  recognized that the subjects who participated in Experiment I could be basing their identifications on rhythmical cues, Experiment I nevertheless closely resembles the realistic situation of a subject listening to music, where sequences of pitches occur within a rhythmical context.  The task of  restricting discriminative cues solely to pitch changes was addressed in subsequent experiments. Variation of pulse rate was expected to induce temporally patterned firing  (Parkins 1989)  in a relatively restricted  39  (Tong et al.  1982;  van den Honert and Stypulkowski 1987a)  group of cochlear neurons, pitch  and to result in some variation of  (Simmons 1966; Merzenich et al.  1978a,  1978b;  Simmons et al.  1979,  1973;  Eddington et al.  1981; Dillier,  and Guentensperger 1983; Hochmair-Desoyer et al.  Spiliman, 1983;  Shannon 1983; Tong and Clark 1985; Townshend et al. Shallop et al.  1987;  1990).  METHODS Subjects  (Table 1)  were 17 Nucleus implant recipients,  not selected for pre—deafness musical or postoperative speech perception abilities, who were returning to the Laboratory for routine follow-up testing. participated,  in addition,  Three of the 17 subjects in Experiments II to V.  subjects ranged in age from 34 to 76  The  (mean 51.6 years).  Duration of preoperative total deafness ranged from 1 to 24 years  (mean 5.5 years). During the test session, the subjects were connected by  means of a cable and a transmission coil directly to the output of the computer and the BTNI interface.  The current  amplitudes required for a comfortable loudness at the pulse rates used in all the experiments were interpolated from the amplitudes established at discrete pulse rates during the preliminary psychophysical measurements.  The stimulus items  consisted of the first 10-20 notes of 30 well-known tunes. The melodic lines were established by a systematic variation  40  of pulse rate on single electrodes in the apical or midportion of the electrode array.  Pilot experiments suggested that  apical electrodes sounded more pleasant and musical than basal electrodes.  A typical test session,  including the preliminary  psychophysical measurements, required approximately 1—2 hours. Pulse rates for each tune were calculated by using the frequency ratios appropriate to each of the twelve equal— tempered semitones of the octave.  When the lower pulse rate  was designated as f 0 and the upper pulse rate as f, the ratio between the two pulse rates was defined by the formula f/f o=21/l2 where n equals the number of semitone steps within the specified musical interval and s is a number that determines the size of the semitone steps.  The scaling factor s, when  smaller or larger than unity, results in a compression or expansion, respectively, given melody.  of all the nonunison intervals in a  The computer was then programmed to calculate  the appropriate pulse rates for the notes of the tunes, to the nearest integer number of pulses per second, factor  once a scaling  (s) and a base pulse rate (f ) were specified. 0  Experiment I proper,  the base pulse rate was constant at 100  pps and the scaling factor (s) was constant at 1.0. rates and rhythmical patterns for each tune, subjects,  are detailed in Appendix 1.  participated,  For  in addition,  Pulse  as played to the  Two subjects  in an informal experiment which  assessed the recognition of familiar tunes when interval sizes  41  were systematically altered, scaling factors  so that n was multiplied by  (s) of 0.5 or 2.0, resulting in, respectively,  a compression or expansion of interval size. The subjects were instructed that the test would consist of the first 1-3 phrases of 30 common tunes,  and that they  were to write either the title of the tune or some of the lyrics on the response sheet.  If they did not recognize the  tune, they were to write “unfamiliar” on the response sheet. The subjects were not informed of the identity of any of the items,  or of the potential pool of tunes.  Each subject was  permitted a maximum of two repetitions of each item. feedback was provided.  Following the test session,  No subjects  were presented with a list of the titles and first lines of all the tunes, were familiar.  and asked to indicate the tunes with which they Specifically, they were asked to indicate  which tunes they thought they would have recognized, prior to deafness,  if the tune had been played on a musical instrument.  While not an objective measure, this procedure was intended to correct for the anticipated poor scores of nonmusical subjects, who might be familiar with only a small number of the test tunes.  For each subject, two percentage correct  scores were calculated.  The first was a percent correct score  based on the total 30 test items.  The second was a percent  correct score based on the number of tunes with which subjects reported themselves to be familiar.  These two scores were  arbitrarily termed the absolute and relative  (i.e.,  relative  42  to the number of tunes familiar to individual subjects) scores,  respectively.  To determine whether identification was likely solely on the basis of rhythmical information, an additional informal experiment was performed in which the 30 tunes were played to 5 normal—hearing observers, using a constant 100 Hz square wave monotone, via the acoustic monitor of the BTNI interface. only the rhythmical information of the tunes was  Thus,  preserved.  While a similar control involving electrically  stimulated subjects would have been desirable, this was precluded by a lack of time and subjects.  It is recognized  that this procedure also does not yield information regarding the influence of rhythm in the normal situation of a subject listening to music,  as rhythm may have a multiplicative or  synergistic effect on recognition, when combined with melody.  RESULTS It is difficult to specify a precise level of chance performance on open—set tune or musical phrase recognition tests, particularly when the instructions to the subjects specify “familiar” tunes.  Subjects are likely to select their  responses from their repertoire folk tunes, nursery rhyme tunes,  children’s songs or Christmas tunes,  and thereby to  limit the number of available response alternatives.  Previous  research has shown that when normal—hearing subjects are asked to identify familiar tunes which have been intentionally  43  distorted, most of the incorrect guesses tend to be drawn from the same general category of popular traditional American songs  Thus,  (Deutsch 1972).  a “familiar” tune recognition  test cannot be considered to be entirely an open-set paradigm, because of the limited number of response alternatives available to the subjects. It can be assumed, however, that a chance performance level would be close to 0%.  The results of Experiment I are  Approximately one-half of the subjects  shown in Figure 6.  scored 40% or better, and only 3 scored less than 10%. Absolute scores  (based on 30 test items)  67%, with a mean score of 33.7%. better than 40%,  ranged from 0% to  Of the 8 subjects who scored  5 had no previous experience with the stimuli  generated by the experimental setup,  except for the  preliminary psychophysical measurements.  The other three  subjects with high scores had received several hours of exposure during pilot studies.  The only subject who failed to  identify any of the test items had been deaf for 25 years prior to implantation.  The frequency of identification of  individual tunes is shown in Figure 7. Relative scores  (relative to the number of tunes the  subjects indicated they could or should have recognized prior to deafness, had the tunes been played by a musical instrument)  ranged from 0% to 84%, with a mean score of 44.1%.  The relative scores were always equal to or better than the absolute scores.  When asked about the sound quality,  some  subjects complained that the tunes sounded artificial and  44  I  C)  Ui  0 C) I  z  Ui  C) Ui  a.  1 2 3 4 5 6 7 8 91011121314151617 PATIENT NUMBER ABSOLUTE SCORE (#CORRECT/30)Xl 00  RELATIVE SCORE  (#CORRECT/#FAJAR)Xl 00  percentage of Open—set tune recognition: Figure 6. items correctly identified by each of 17 subjects. Percent correct scores were calculated both on the basis of the entire set of 30 test items (absolute score), and on the basis of the number of tunes with which the subjects reported themselves to be familiar (relative Details regarding subjects in Table 1. score).  45  0  z 0  I-  C) LI  ‘Er  I  z w 0 I  C)  w  a a  It__  .1 I  an  P.’‘yr Fr  0 C)  II.  -  0  a w D  z 4-  1. 1  2  3  4  5  6  7  8  91011 12131415161718192021 222324252627282930  TUNE NUMBER  Figure 7. Open—set tune recognition experiment: number of positive identifications for each test tune. Details regarding tunes in Appendix 1.  46  unmusical, while others reported them to sound clear and pleasing.  Four of the subjects volunteered that their  identifications were based mostly on rhythmical cues,  rather  than on pitch cues. It is obvious from an examination of the data 6 and Table 1)  (cf.  Figure  that musical interest or achievement alone,  prior to deafness,  cannot account for differences in  performance on the tune identification test.  The 3 subjects  with the highest scores reported an historical lack of interest in music.  However,  6 of the 9 subjects with a tune  identification score of 40% or greater reported having played a musical instrument.  In contrast,  five of the 7 subjects who  scored 20% or less reported no musical interest. who had played a musical instrument (N=8)  The subjects  achieved a mean  score of 44.9%, whereas the subjects who had no musical interest  (N=8)  had a mean score of 24.6%.  perception scores  (Table 1)  history were slightly higher  The speech  of the subjects with some musical (80.1%)  than those for the  subjects with no history of musical interest  (mean 70.1%).  The relationship between the tune identification scores and speech perception scores was plotted in Figure 8.  The two  sets of scores were found to be moderately correlated  (r=.53).  The tune identification experiment was repeated, informally, with two subjects, using either expanded or compressed interval sizes, with an s value of either 0.5 or 2.0 randomly assigned to each tune.  The subjects  (S-7 and S  47  100 I LU  80 I LU  0  XX  60  X  z  XXX  0 40 1  z LU  X  X X  20  X  X  LU  z  X  I-  0  I  0  I 10  I  Xi  I I I I I I I I 70 80 20 30 40 50 60 SPEECH PERCEPTION: PERCENT CORRECT  90  100  Figure 8. Scatter plot of the relationship between tune identification (Figure 6) and speech perception (Table 1) scores.  48  11)  identified the mistuned tunes as well as their properly  tuned counterparts, and responded with incredulity, when informed of the mistuning,  following completion of the  experiment. In the other informal experiment,  five normal—hearing  controls failed to identify any of the 30 tunes of Experiment I, when these tunes were played as rhythmical patterns without pitch changes, using a constant 100 Hz square wave monotone, via the acoustic monitor of the BTNI interface.  DISCUSSION  The results suggest that tune recognition in profoundly deaf subjects is possible with pulsatile electrical stimulation on single intracochlear electrodes.  These  findings are in agreement with the results of a number of previous experiments involving electrically stimulated deaf subjects,  suggesting that pitches sufficiently salient for  musical perception are possible on the basis of solely temporal information.  Eddington et al.  (1978a,  1978b)  reported limited tune recognition with pulsatile stimulation, using a single intracochlear electrode.  Their single subject  correctly identified 3 out of 5 test tunes played at low pulse rates on one electrode, but was unable to identify the tunes when they were played on other electrodes.  Chouard  (1978)  claimed that postlingually deaf implanted patients were typically able, with near 100% accuracy, to recognize popular  49  tunes.  However, the experimental protocol was poorly  documented.  Moore and Rosen (1979)  reported essentially  perfect scores for a single deaf subject on a 10—item alternative—forced-choice melody identification test, using an extracochlear electrode and an analog processing scheme.  The  16—note rhythmically identical tunes were presented live—voice and low-pass filtered at 300 Hz.  Their patient observed that  even though the sound quality resembled that of “a comb and These  paper”, the melodies were nevertheless “in tune”. findings support our conclusions that musical pitch  information sufficient for tune recognition can indeed be conveyed in the temporal discharge patterns of electrically stimulated auditory neurons. Similar data,  consistent with the existence of musical  pitches based on temporal features of acoustic waveforms, have been reported in normal—hearing subjects. Viemeister (1976)  Burns and  reported musical interval identification and  closed-set tune recognition utilizing both wide band and bandpass sinusoidally amplitude—modulated noise. subsequent experiment, Burns and Viemeister  (1981)  In a showed that  even musically naive listeners were able to identify, set, rhythmically identical,  open—  same-length melodies played by  sinusoidally amplitude-modulated white or high pass noise. While the mean number of items correctly identified was lower (13.3* to 22.5%)  than that obtained by our subjects, this  difference may well be due to our preservation of rhythmical information.  50  The observations of some subjects about the importance of rhythmical information in their tune identification performance could be due to several factors.  It is possible,  for example, that the electrical pulse rate pitches were weak and difficult to hear, at least for some subjects.  The  moderate correlation found between speech perception performance and tune identification scores supports a found also by other investigators, between the  relationship,  ability of implanted subjects to utilize temporal information and their ability to understand speech (Shannon 1989; Tyler, Moore,  and Kuk 1989).  The unnatural sound quality resulting  from electrical stimulation of the auditory system should not be surprising,  in view of the highly synchronized responses of  neurons over a range of characteristic frequencies, as well as the absence of the delays normally imposed by basilar membrane mechanics and hair cell transducer action,  and the lack of  congruence between place, period, and phase information den Honert and Stypulkowski 1987b). explain,  (van  Weakness of pitch could  for example, the failure of the subjects in the  additional informal experiment to detect gross mistuning of the melodies  (i.e., the sequence of intervals)  of the tunes.  Alternately, this failure could result from their attending solely to the identification task,  since the subjects were not  instructed to attend to the intonation quality of the intervals in the tunes.  The poor scores and predominant  reliance of some subjects on rhythmical information could also result from a musical illiteracy of the subjects or the high  51  variability of the size of the musical repertoire known to exist even among normal—hearing subjects.  It is also possible  following a period of total deafness, the memories of  that,  the melodies of common tunes for some subjects were no longer It was not possible to examine the effects of  intact.  prolonged deafness on tune identification,  as only three of  the 17 subjects had been deaf for more than 10 years. An important question remains, however, whether the subjects in Experiment I could have been responding solely to the rhythmical patterns of the tunes rather than to the melodies,  i.e.,  to the sequence of intervals.  Musically  untutored subjects reportedly listen to music in a nonanalytical way,  in which the sequence of pitches is  inextricably linked to the rhythmical structure which accompanies the melody (Cross, Howell and West 1985; Jones, Summerell,  and Marshburn 1987).  The possibility of limited  recognition on the basis of exclusively rhythmical information has been previously demonstrated by others.  Deutsch (1972),  for example, reported that 19% of subjects were able to recognize (open—set) of timed clicks.  Yankee Doodle when presented as a series  While identification of our tunes solely on  the basis of rhythmical information remains a possibility, would appear to be unlikely,  it  in view of the inability of  normal—hearing controls to identify the tunes when they were played as mere rhythmical patterns, without variation in pitch.  However, the fact that rhythm by itself was  52  insufficient for tune recognition does not mean that rhythm is also unimportant when accompanied by melody. Thus,  reliance on exclusively rhythmical information,  the absence of pitch interval or musical contour cues, be expected to result in poor recognition scores, in an open-set paradigm.  in  could  especially  While rhythmical information may  have contributed to tune identification in Experiment I, as observed by some of our subjects, our evidence suggests that melody played the greater role. Gfeller and Lansing (1991)  found that 10 Nucleus implant  recipients obtained low scores when asked to identify short taped excerpts of solo renditions of nine familiar tunes, produced on acoustic instruments.  These excerpts were played  over a soundfield speaker and processed by the body—worn device normally used by the subjects.  The Nucleus processor  performs a feature extraction upon incoming acoustic signals, in which the electrode pairs, code,  selected on the basis of a place  are activated at a pulse rate equal to the fundamental  frequency. information, recipients)  In spite of the preservation of rhythmical subjects (including,  in addition,  8 Ineraid  of Gfeller and Lansing (1991) were able to  identify the tunes for only 5% of the total number of trials. Thus,  these subjects appeared to be unable to use rhythmical  information to identify the tunes.  This inability did not  appear to be due to a general ineptitude in the use of rhythmical cues,  since these subjects reportedly achieved high  53  scores on the rhythmical subtests of the Primary Measures of Music Audiation (PMMA). The poor recognition scores of the subjects in the Gfeller and Lansing (1991) number of factors.  study may be attributable to a  With the Nucleus implant,  for example, a  complex sound delivered to the microphone of the processor results in a quasi—simultaneous activation of a number of electrodes selected on the basis of spectral peaks in the acoustic signal  (Koch,  Seligman, Daly and Whitford 1990), with  each electrode yielding a more or less distinct sound quality. It is conceivable that rate pitches resulting from the quasisimultaneous activation of a number of electrodes may be more difficult to discern than those resulting from simple variation of pulse rate on a single electrode.  Thus,  it is  possible that, under these conditions, the more complicated stimulation pattern resulting from the feature extraction process may override the temporal structure needed for pitch perception. It is noteworthy that several of our subjects also achieved relatively low scores  (e.g.,  S—i,  S—6,  S—9,  S—12),  even with simple variation of pulse rate on a single electrode.  It is possible that these subjects were less able  than the remainder to use temporal information.  It is noted  that the speech perception scores of these subjects tended to be somewhat lower than those of the remaining 13 subjects. However,  additional factors,  including the extent of pre—  deafness musical aptitude and experience,  and the size of  54  individual musical repertoires, may also be significant factors. tunes  In addition, the slow tempo of some of the test  (Appendix 1), relative to the tempo at which these tunes  are conventionally played or sung, may have degraded performance for some subjects.  In the author’s experience,  some normal-hearing subjects also have difficulty identifying tunes when the tempo differs from that to which the subject is accustomed (as for example, Lutheran chorales played or sung as a slowly moving cantus firmus in the chorale Cantatas of J.S. Bach).  It is also possible that,  following the prolonged  auditory deprivation of musically naive subjects, the internalized representations of melodies and the underlying tonal schemata are no longer intact in long—term memory. While Dowling (1978)  and others have convincingly argued that  memory for melodies and the tonal framework on which these melodies are hung are amongst the most stable of sensory schemata in cognitive psychology, the effects of long-term deafness on such memories have not been investigated.  55  EXPERIMENT II:  CLOSED-SET MELODY IDENTIFICATION  Although the results of Experiment I showed that subjects were able to identify common tunes when played as a sequence of pulse rates over single intracochlear electrodes,  it was  not clear to what extent performance was based on rhythm, on a combination of rhythm and melody.  or  Removal of rhythmical  information, while introducing a significant distortion for naive listeners, who reportedly listen to music in a nonanalytical way, which does not separate rhythm and melody, should permit the assessment of the role of pitch information. For this paper, melody has been defined as an ordered series of musical intervals,  in the absence of rhythmical or phrase  length information. Melodies may be described by the frequency intervals that separate temporally contiguous notes and by their melodic contour.  Contour is a generalized form of interval  information,  and is defined by the directional relationships  between temporally adjacent notes, without precise specification of interval magnitude.  In Experiment II,  a  closed-set melody identification paradigm was utilized to determine whether unequivocal melody identification was possible in the absence of rhythmical information. II,  in addition,  Experiment  assessed whether this information was also  56  available at higher pulse rates and on more basally situated electrodes.  METHODS  The stimulus set consisted of 7—note truncations of the initial phrases of familiar tunes. paper,  we will refer to these isorhythmical 7—note fragments  as melodies.  All the melodies were equal in length and devoid  of rhythmical cues, msec,  For the purposes of this  in that all the note durations were 500  and each note was followed by a 200 msec silent period.  While the rhythmical equalization procedure left the melody (i.e.,  the sequence of intervals)  intact, this procedure  necessarily resulted in a rhythmical distortion of some of the original tunes.  The pulse rates  (Appendix 2)  were assigned in  a manner comparable to that described for the tunes in Experiment I.  Items not familiar to individual subjects were  deleted from the test set for those subjects. familiar with all 8 items.  Subject 11 was  Subjects 7 and 10 were familiar  with only 5 or 6 of the items.  All the melodies within the  response set were constructed for similarity in respect to melodic center of gravity and pitch range.  The computer was  programmed to calculate the appropriate pulse rate values to the nearest whole number for each note of the melody, base pulse rate  ) 0 (f  had been specified.  pulse rates were randomized between 75,  once a  Melodies and base 100,  150,  200,  300,  57  400 pps by the computer.  One subject  (S-7)  was further tested  with a base pulse rate of 600 pps. Subjects 10 and 11 were not tested at higher pulse rates because at high pulse rates, they appeared to be able to achieve high recognition scores, at least in part, basis of non—pitch attributes of the sound. pulse rates above 600-800 pps,  on the  For example,  at  S-lU reported hearing only a  brief onset sound at the beginning of each stimulus note, which he described as “the sound of a kettle drum”. apparent during the initial trials that, rates,  It became  at the highest pulse  he was able to use this information to identify the  melodies. “Twinkle,  Thus,  at the highest pulse rates, he responded with  Twinkle,  Little Star” whenever he heard  brief notes on a kettle drum”)  (as “two  the two sequential A’s at the  highest point in the pitch contour of this melody.  Similarly,  at these pulse rates, he responded with “0 Suzanna” whenever he heard  (as “one short note on a kettle drum”)  note  in the contour of this melody.  (A)  similar phenomena.  Thus,  the highest  Subject 11 reported  for these two subjects, performance  at the highest pulse rates appeared to be based on non-pitch information.  For this reason,  S—b  and S—lb were not tested  at the highest pulse rates. During the pilot experiments,  all three subjects  complained about the instability of auditory percepts at higher pulse rates,  and frequently commented that melodies at  pulse rates above 200-300 pps failed to sound the same upon repetition.  Subject 11 described pulse rates above 100 pps as  58  having an onset sound with a transient, but definable pitch, which was followed immediately by a pitchiess, noiselike sound.  These instabilities were identical to those observed  during the preliminary psychophysical measurements. Closed-set melody identification at the specified base pulse rates basal,  ) was assessed for electrodes located in the 0 (f  the midportion,  and the apical regions of the array.  The sequencing of the electrodes used during the tests was counterbalanced between subjects and sessions, effects of fatigue and practice. session,  to minimize the  Thus, during a given  the sequencing of electrodes for one subject might  progress from apical to basal, while the sequence for other subjects was in a basal to apical direction.  The sequence of  electrodes used in the tests was varied from session to session.  Each subject was provided with a numbered,  individualized list of the test melodies. session,  Within each  each item on the list was presented,  order, twice at each base pulse rate.  in randomized  The subjects were  instructed to select a response from the list, and to press the corresponding number on the computer keyboard.  This  resulted in automatic scoring of responses and initiation of the next trial. permitted,  No repetition of stimulus items was  and no feedback was provided.  The percentage of  correct responses was calculated for each condition.  To  determine whether the magnitude of pulse rate musical interval in the melodies represented a significant factor in melody identification performance,  each melody was assigned to one of  59  three groups:  those with at least one interval equal to or  wider than a 4th (large intervals), those in which the largest interval was a major or minor 3rd (medium sized intervals), and those with only small intervals (major 2nd or semitone steps only). Normal—hearing subjects readily recognize transposed melodies as musical equivalents, because the transposition preserves the precise sequence and magnitude of musical intervals.  When the sizes of the musical intervals of  melodies are altered, melodies lose their identity, unless rhythmical cues  (absent in this experiment)  contour are sufficiently distinctive.  or the melodic  To determine whether S—  7 was listening to the melodies as a sequence of musical intervals,  an informal ancillary experiment was devised,  in  which this subject performed the closed-set melody recognition task with an unfamiliar melody added to the stimulus set.  The  subject was not informed of the presence of the additional melody.  It was expected that,  to the stimuli as music  if the subject were listening  (i.e., as a sequence of musical  intervals), he might readily declare the presence of an unfamiliar melody.  If he were listening only for a similarly  shaped melodic contour, he was expected to match the unfamiliar melody to a same-contour melody in the response set.  60  RESULTS  An analysis of variance  (ANOVA)  was performed on the  percentages of melodies correctly identified, with 6 levels of pulse rates,  3 electrodes, and 3 levels of magnitude of  interval size.  The mean percentage correct scores for each  subject are plotted in Figure 9.  Subjects differed  significantly in their performance [F(2,26l4)  =  186.21, and S  Subject 7 scored 3% to 56% higher than S-b  p<.0000l].  11, with the greatest performance differences occurring at the The scores of S—b  highest pulse rates.  significantly different.  and S—il were not  These findings were in sharp  contrast with the self-reported musical history of the subjects,  as both S-b  and S-li reported a longstanding  interest in music prior to deafness.  While S-7 did not report  any specific interest in music prior to deafness, he was nevertheless familiar with a considerable variety of melodies (Experiment I). The pulse rate of the starting note had a significant effect on subject performance [F(5,26l4) At low pulse rates, 100% accuracy. difficulty.  =  127.97, p<.0000l].  subjects identified the melodies with 80-  Subjects 7 and 10 did so without apparent  Subject 11,  were difficult to hear,  however,  reported that the pitches  even at low pulse rates.  The decrease  in scores at higher pulse rates was more marked for S—b lb than for S-7. 100 pps)  While mean scores at low pulse rates  and (75 and  were not significantly different from each other,  61  b  S z  7  100  150  200  300  400  600  5-10 90 80  70  b  60 50 40 30  10  ñ  10  iSO  200  300  400  600  b 0 1  I-.  z  U  a.  PULSE RATE ELECIRE  --  MIDDLO ELECTRODE  -•-  BASAL ELECTRODE  Closed—set melody recognition experiment: Figure 9. percentage of items correctly identified by Subjects 7, 10, and 11, for electrodes situated in apical, basal, and middle portions of the electrode array, over a The numbers along the horizontal range of pulse rates. axis do not represent a dimension, and represent the base pulse rate (“c”), at octave multiples of 75 and 100 Pulse rate assignments for individual notes are pps. Chance level of performance for shown in Appendix 2. each subject is indicated by a horizontal line across the panel.  62  a monotonic  scores at higher pulse rates showed,  in general,  decline with increasing pulse rate.  The superior performance  of S-7 at higher pulse rates resulted in a significant interaction between subject and pulse rate variables [F(lO,26l4)  =  18.581, p<.00001].  At low pulse rates,  Subjects  7 and 10 reported being able to hear all the pitch changes in but at higher pulse rates, they indicated that  each melody,  only some of the pitch changes were detectable.  Subjects also  volunteered that only at low pulse rates did the test items resemble music. In the informal ancillary experiment,  one of the subjects  performed the closed-set melody identification task with an unfamiliar melody added to the stimulus set.  Only when the  unfamiliar melody was played at low pulse rates did S-7 conclude that the item was not on the list of possible responses.  This suggests that, at higher pulse rates, the  subjects were perhaps simply matching melodies on the basis of pitch contour, rather than processing the pulse rates as a sequence of musical pitch intervals. remains that,  at high pulse rates,  The possibility also  subjects were using complex  cognitive strategies by attending to nonpitch attributes  (such  as “double notes”). The main effect of the electrode on which the melody was played was also significant [F(2,26l4)  =  14.944, p<.0000l].  Scores for Subjects 10 and 11 were best on apical, on basal electrodes.  and worst  The performance decrements for these two  subjects on the basal electrodes were most marked at the  63  higher pulse rates,  accounting for significant interactions  between the electrode and pulse rate variables 3.1907,  p=.00044].  [F(l0,2614)  =  There were no significant differences in  scores between electrodes in the basal portion and midportion of the array.  The significant interaction between the subject  and electrode variables  {F(4,2614)  =  6.2104,  p=.00006] was due  to differences between the performance of S—7, who demonstrated only a slight decrease in scores at the highest pulse rate tested,  and the other two subjects,  for whom the  decrease in scores at higher pulse rates was marked.  This  accounted for a 3—way interaction between subjects, and pulse rates  electrodes,  [F(20,2614)  =  1.9284, p=.00787].  All three subjects reported that the more apical electrodes yielded the more musical and pleasant sounding melodies. The size of the intervals in the melodies was not significant as a main effect [F(2,26l4)  =  2.0017,  p=.l3532],  and failed to produce significant interactions with the electrode and pulse rate variables. (such as 5ths and 4ths)  Thus,  large intervals  did not appear to make melodies more  identifiable than small intervals  (such as major 2nds and  Although a significant interaction occurred  semitones).  between subjects and interval magnitude [F(4,26l4) p=.000lO],  =  5.922,  there appeared to be no consistent pattern to this  interaction.  For example,  S—7 obtained the highest scores  with melodies consisting of small intervals, scores with large-interval melodies.  and the lowest  Subject 10 obtained the  highest scores with medium—sized intervals and the lowest  64  scores with small intervals.  Subject 11 obtained the highest  scores with large—interval melodies,  and the lowest scores  with melodies consisting of medium-sized pitch intervals.  In  view of the small number of melodies in each interval size category,  it is possible that these interactions resulted from  other, noninterval size variables,  such as a greater  familiarity of subjects with some of the melodies,  or a  greater resistance of some melodies to the temporal distortion imposed by rhythmic equalization.  DISCUSSION The findings suggest that systematic variations in pulse rate on single electrodes can result in pitch percepts sufficiently salient to enable subjects to score well on a closed—set melody recognition test, rhythmical information.  in complete absence of  At low pulse rates,  comparable for all electrodes.  scores were  At high pulse rates, the more  basally situated electrodes yielded a greater performance decrement than apically situated electrodes.  This, together  with observations of the subjects regarding the more musical quality of the apical electrodes, may reflect the importance of a congruence between place and rate information in pitch processing.  The similarity of the results for basal and  apical electrodes at low pulse rates are also in agreement with gap detection data of Shannon  (1989),  showing equivalent  65  gap detection thresholds at equal loudness levels, regardless of the intracochlear location of the stimulating electrodes. There were significant differences in the ability of subjects to utilize pitches at higher pulse rates. for example,  Subject 7,  achieved high scores even at the highest pulse  rates tested, while 5—10 and 11 showed large performance decrements at higher pulse rates. addition,  Subject 10 appeared,  in  confused by the rhythmical monotony of the melodies,  even though he reported that the pitches, especially at low pulse rates, were not difficult to hear.  Subject 11 commented  that all the pitches were difficult to hear, lowest pulse rates. pulse rate pitches,  even at the  It is possible that the weakness of the for this apparently relatively musical  subject, may relate to the distribution, number, or functional status of residual spiral ganglion cells, perhaps as a result of deafness secondary to meningitis  (Nadol and Hsu 1991).  Previous investigators have also reported large individual differences in the abilities of cochlear implant recipients to utilize temporal information (Shannon 1989; Tyler, Moore,  and  [<uk 1989) The upper limits for pitches resulting from pulsatile electrical stimulation in Experiment II appear to be comparable to the 850—1000 Hz upper limit of pitch perception observed for sinusoidally amplitude—modulated noise by Burns and Viemeister (1976,  1981),  in tasks requiring musical  interval recognition and dictation, as well as melody identification.  This upper limit was hypothesized to reflect  66  the inability of the auditory system to follow rapid temporal changes.  Even for the musically trained observers of Burns  and Viemeister  (1976,  1981),  intersubject differences at  modulation frequencies above 300 Hz were considerable. Comparable individual differences,  such as those between S—7  on the one hand, and Subjects 10 and 11 on the other, were observed in our data.  Tong and Clark (1985)  reported similar  upper limits and individual differences in the ability of implanted patients to identify electrical pulse rates. differences, Clark (1985),  These  in our subjects as well as those of Tong and appeared to be unrelated to musical experience.  For electrical stimulation rates above 500 Hz, the ability of individual neurons to fire on a cycle-for-cycle basis is limited by the neural refractory period (Javel et al. 1987; van den Honert and Stypulkowski 1987b).  While the  absolute refractory period has been estimated at approximately 300 sec, the relative refractory period may extend to at least 5 msec (van den Honert and Stypulkowski 1984). higher pulse rates,  At  interval histograms for single units have  been shown to become multimodal, with an increasing representation of multiples of the stimulus period Honert and Stypulkowski 1987b).  (van den  It is possible that the  increasing intrusion of a variety of interspike intervals, related more to multiples of the stimulus period or to the refractory period of the neurons than to the interpulse intervals, results in a weakening of pitch at higher pulse rates.  67  While results for Experiment II showed that, complete absence of rhythmical information, to identify melodies from a closed set,  even in the  subjects were able  it could not be  determined from these results whether these pitches conveyed only contour information, or both contour and interval information.  Contour refers simply to the ordinal  representations of the note frequencies, and thus designates whether individual notes are higher or lower in pitch than their contiguous counterparts, size that,  (Watkins and Dyson 1985).  independent of precise interval Previous research has shown  especially in closed—set recognition paradigms, melodic  contour can serve as a significant cue in the retrieval processes or strategies utilized by subjects. Kallmann,  and Kelly (1980)  Massaro,  and Dowling and Bartlett  (1981)  have suggested that in the closed—set format, subjects may retrieve all items in the response set from memory,  extract  the contour by covert rehearsal, and then compare the contours with those of the stimulus items.  The possibility should also  be considered that, especially at higher pulse rates, subjects  (or S-7,  our  specifically) may have been applying  nonmusical strategies,  such as listening for systematic  changes in non—pitch attributes, such as timbre, density, brightness, roughness,  or fullness.  In summary, the results of Experiments I and II suggest that melody recognition is possible with pulsatile electrical stimulation of deaf ears, using single intracochlear electrodes.  While it is possible that rhythmical cues were  68  important in the retrieval process in the open—set paradigm (Experiment I), the alternative-forced—choice melody recognition (Experiment II) that,  results for these 3 subjects show  even in the total absence of rhythmical information,  subjects were able to perform at high levels. however, was best at low pulse rates,  Performance,  and a variable but  significant deterioration occurred for all subjects at higher pulse rates.  These upper limits of performance, while  differing for different subjects,  compare favourably to the  upper limits of temporally mediated pitch in acoustic experiments  (Burns and Viemeister 1976).  69  EXPERIMENT III:  INTONATION QUALITY JUDGEMENTS  Contour information alone is known to be too imprecise to account for the accurate memory for melodies, musically untrained subjects Sloboda and Parker 1985).  (Cross, Howell,  even in and West 1985;  Precise interval size has been  shown to be important particularly in long—term memory for familiar melodies.  Dowling and Fujitani  (1971)  showed that  while melodic contour may be more important than precise interval size in short—term memory for unfamiliar melodies, well—learned melodies are stored as a precise sequence of musical intervals.  Thus,  even unsophisticated listeners are  capable of discriminating between exact transpositions and less precise imitations of familiar melodies.  Transpositions  preserve the precise sequence and sizes of musical intervals found in the original, whereas imitations preserve the contour,  but not the precise interval sizes of the original  melody. The possibility remains that, identification tests,  in the melody  our subjects were perhaps merely  recognizing the ordinal relationships among the pitches,  or  that they were applying more complex cognitive strategies, attending perhaps to nonpitch attributes of the signal.  Thus,  while the pitches heard by these subjects could perhaps be scaled or ranked by means of higher/lower, up/down,  70  same/different psychophysical procedures,  these pitches could  not be assumed to be sufficiently salient to convey information about musical intervals 1981; Houtsma 1984).  (Burns and Viemeister  Musical pitch intervals are  characterized by frequencies in very specific ratio relations (Houtsma 1984).  Ward (1970)  (1978)  and Burns and Ward  have  pointed out that the precision shown by musicians in adjusting musical intervals exceeds that shown by trained subjects in adjusting ratios of other auditory or nonauditory percepts. Nonmusicians,  however,  are known to be less precise,  and are  thought to be influenced more by pitch height rather than by complex hierarchical and ratio—governed stimulus relations (Siegel and Siegel 1977; Krumhansl and Kessler 1982; Monahan, Kendall,  and Carterette 1987).  To assess the response of cochlear implant subjects to either the ratio properties or, extent  alternatively,  (differences in pitch height)  to the pitch  of the musical intervals  which were represented by changes in pulse rate,  an intonation  quality test was devised.  METHODS  Subjects were required to label each stimulus pitch interval,  represented by changes in pulse rate on a single  intracochlear electrode,  as “flat”,  “sharp”,  or “in tune”,  relative to their memory for one of four specific intervals exemplified in well—known melodies.  For example,  to determine  71  the pulse rate ratios that would best characterize the subject’s memory of an ascending 5th, the subjects were asked to rehearse mentally the second interval nonunison interval)  of “Twinkle,  Twinkle,  (i.e.,  the first  Little Star”,  and  then to label each of 12 stimulus interval sizes as “in tune”, “flat”,  or “sharp”, relative to their memory for the interval  in question 4th,  (Figure 10).  5th, major 6th)  The 4 target intervals  (minor 3rd,  were abstracted from the initial phrases  of melodies well-known to individual subjects.  Each of these  intervals was represented by only two pitches.  In the  conventional musical scale, these four intervals consist of 3, 5,  and 9 semitones, respectively.  7,  Attempts were made to select melodies with at least some repetitive notes  (e.g.  the 5th in “Twinkle”).  the repeated lower and upper notes of Melodies used for the 4th were the  first 4 notes of “0 Christmas Tree”, “Away in a Manger”.  or the first 3 notes of  The minor 3rd was exemplified by the  first 8 notes of the chorus of “Jingle Bells”.  The major 6th  was exemplified in the first two notes of “Jingle Bells” or “My Bonnie Lies Over the Ocean”. short,  It is possible that the  two—note intervals such as those used for the major 6th  were more difficult than the 4- or 8-note sequences,  because  the subjects had less time to process the pitch information. Note duration was 500 msec,  and notes were separated by 200  msec. For a given block of stimuli, the pulse rate of the lower note remained constant.  Preliminary trials with a lower note  72  INTONATION QUALITY JUDGMENTS  aoo 1.89 1.78 1.69 1.59  TOO HIGH  1.50 IN TUNE  TWINKLE  C TWINKLE  1.41 1.33 1.26 1.19 1.12 1.06 1.00  TOO LOW  Figure 10. Intonation quality judgements: schematization of the stimuli and the task of the subjects for an interval of a 5th, as exemplified in the first nonunison interval in “Twinkle, Twinkle, Little Star”. The stimulus set consisted of 12 intervals ranging in size from a semitone (ratio of 1:1.06) to an octave (ratio of 1:2) above the lower note of the interval, in steps of one semitone. The target ratio (“in tune”) was 1:1.5. With acoustical stimuli, normal—hearing musical listeners label ratios larger than this as “sharp” (“too high”), and smaller ratios as “flat” (“too low”)  73  pulse rate which was varied from trial to trial yielded comparable results,  although requiring considerably greater  effort for the listeners.  The pulse rate of the upper note  was varied to create a set of 12 interval sizes,  each  differing from its adjacent counterpart by one semitone. resulted in one correct target interval and 11 foils,  This  ranging  in size from a minor 2nd to an octave above the lower note. The pulse rates assigned to the lower note were different for each target interval, to preclude the utilization of a constant pitch reference across the entire set of 4 intervals. Specifically, 145,  163,  the pulse rates for the lower notes were 137,  and 127 pps for the minor 3rd, the 4th,  the major 6th, respectively.  the 5th,  and  Semitone step ratios were  calculated according to the equal—temperament musical scale, where tones separated by a semitone interval are defined by a frequency ratio of 1:21/12 or 1:1.05946.  The pulse rate for the second note of the intervals was calculated by the formula  Pulse Rate of First Note where n equals integers from 1 to 12.  *  n/l2 2  For each target  interval at a specified starting pulse rate,  the 12 different  intervals were randomly presented to the subjects at amplitudes interpolated from the preliminary psychophysical measurements.  74  Subjects were informed of the musical interval under scrutiny and the tune fragment which exemplified the test interval.  Subjects were then required to label each of the  pitch intervals they heard as “in tune”, high”),  or “flat”  “sharp”  (or “too  (or “too low”), relative to their memory for  the test interval, and to respond by pressing the appropriate keys  (“F”,  “T”,  “S”)  on the computer keyboard.  of the target intervals were presented,  No exemplars  and no feedback was  provided regarding the “correctness” of the responses. A similar paradigm was used to investigate whether the ratio relationships demonstrated for a 5th at low pulse rates were transposable to lower and higher pulse rates. lower-note pulse rates  (81,  163,  326 pps),  Three  separated by  octaves, were randomized from block to block. An additional series of blocks examined whether the ability to detect gross mistuning of musical intervals at low pulse rates was perhaps restricted to apical electrodes, or whether similar results could be obtained via more basal electrodes.  Each block of trials consisted of 5 presentations  of each of the 12 intervals along the pulse rate continuum (60 judgements). in 5 blocks  Each target interval or condition was assessed (300 judgements).  Following each block of trials,  subjects received a 15 minute rest period. For each interval size, percentage of “in tune”, calculated.  condition,  and subject, the  “flat”, and “sharp” responses was  The percentages of “flat” and “sharp” judgements  for each stimulus interval size category were used to  75  calculate the mean transition points from “flat” to “not flat”,  and from “sharp” to “not sharp”  (Woodworth 1938).  The  mean of these two points was taken as the point of subjective equality  (PSE),  or the subjective musical interval  (i.e., the  interval equivalent to the memory of the subject for this interval,  as represented in the melody in question) interval size, and condition.  subject,  for each  In addition,  a  standard deviation was calculated, using the method described by Woodworth  (1938).  This computation allowed relatively  straightforward comparison of the data from this experiment, using the method of constant stimuli, with those obtained using the method of adjustment  (Experiments IV and V).  Two additional informal pilot experiments were performed. The first determined whether subjects were able to assign intonation quality labels to intervals which were represented solely by changes in cochlear place stimulation.  Thus,  or 15  electrode)  of  two subjects were asked to label the pitch  changes resulting from switching, 19 to electrode 18,  (i.e.,  for example,  from electrode  or from electrode 19 to electrode 17,  (at a constant pulse rate),  as “flat”,  tune” relative to the interval of a 5th,  “sharp”,  16,  or “in  using a paradigm  similar to that for the pulse rate musical intervals described above.  While the distances between the stimulated neural  targets is not known,  adjacent electrodes of the Nucleus  implant are spaced by 0.75 mm. 19 to 18,  from 19 to 17,  distances of 0.75,  1.5,  16, 2.25,  Thus,  switching from electrode  or 15 represented interelectrode and 3.0 mm,  respectively.  76  A second informal pilot experiment was intended to determine whether covariation of place and rate of stimulation might yield more certainly identifiable or more distinctly perceived musical pitch intervals than those resulting solely from variation of pulse rate on single electrodes. experiment, the lower pulse rate (100 pps)  In this  of a 5th was played  on an apical electrode, and the pulse rate of the upper note of the interval was played on a second electrode located 2.25 mm basalward,  a distance corresponding to approximately one—  half octave in the frequency region of 1 kHz in the normal ear (Greenwood 1961,  1990).  varied randomly,  from a semitone to an octave above 100 pps.  The pulse rate of the upper note was  The subjects were asked to label the intonation quality of the intervals as “flat”,  “in tune”, or “sharp”.  RESULTS The results for the four musical intervals for each subject were plotted in Figures 11 to 14 as 3—category complementary labelling functions, which show the percentage of items labelled as “flat”, stimulus interval size.  “in tune”,  and “sharp” for each  The ratios between the upper and  lower pulse rates of each interval are shown along the horizontal axis of each plot.  Adjacent ratios along the  horizontal axis represent a distance of one semitone. the location, along the horizontal axis,  Thus  of the maximum  percentage of “in tune” responses indicates the pitch change  77  favoured by each subject for a given pitch interval.  The  slope and the width of the “in tune” functions reflect the tolerance of each subject for varying degrees of mistuning, and the height reflects the consistency of labelling behaviour. subject,  The consistency of the judgements made by a  as well as the tolerance for mistuning, would be  expected to depend on the salience of the pitches heard, on the ability of the subjects to label musical pitch intervals, and on the accuracy of their long—term memory for the correct interval size. Musically trained subjects with normal hearing, when performing similar tasks with musical tones, produce functions analogous to those in Figures 11-14, but with very steep gradients and narrow, high maxima at very specific locations along the X-axis  (Burns and Ward 1978).  To facilitate direct comparison of the subjective sizes of the four intervals in Figures 11-14, the calculated subjective midpoints  (PSEs)  of each set of data  (Table 2) were  plotted against the physical sizes of the target intervals semitones) deviations.  (in  in Figure 15, together with the standard The bold horizontal lines in Figure 15 indicate  the correct size of each interval. At the low pulse rates tested, the interval sizes estimated by the subjects for the minor 3rd, the 4th,  5th, and  major 6th compared favourably with the frequency ratios which characterize analogous acoustical musical intervals  (vertical  lines across each plot in Figures 11-14, horizontal bold lines  78  TABLE 2 INTONATION QUALITY JUDGEMENTS* POINTS OF SUBJECTIVE EQUALITY  A.  C.  AND STANDARD DEVIATIONS  FOUR INTERVALS INTERVAL  S-7  S.D.  S-10  S.D.  S-li  S.D.  MEAN  MEAN S.D.  m3rd  3.18  (.64)  3.02  (.53)  2.90  (.91)  3.03  (.69)  4th  5.20  (1.17)  5.30  (.81)  4.50  (1.38)  5.00  (1.12)  6.55  (1.12)  9.09  (1.23)  5th  6.57  (.85)  7.16  (.81)  5.92  (1.70)  M6th  8.42  (1.03)  9.82  (1.29)  9.02  (1.37)  (1.04)  (1.34)  (.86)  (1.17)  MEAN  B.  (PSE)  5TH AT 100 PPS ON 3 ELECTRODES S-il  S.D.  MEAN  MEAN S.D.  (.75)  8.54  (1.04)  7.61  (.98)  (.57)  7.22  (1.00)  7.31  (.93)  ELECTRODE  S-7  S.D.  S-10  S.D.  18  6.64  (1.14)  7.64  12  6.48  (1.23)  7.74  5  7.18  (1.51)  7.22  (.49)  6.64  (1.31)  7.01  (1.10)  MEAN  6.77  (1.29)  7.53  (.60)  7.63  (1.12)  7.31  (1.00)  5TH AT 3 PULSE RATES PPS  S-7  S.D.  S-i0  S.D.  S-li  S.D.  MEAN  MEAN S.D.  81  6.84  (1.27)  8.10  (.91)  7.92  (.85)  7.62  (1.01)  163  6.57  (.85)  7.16  (.81)  5.92  (1.70)  6.55  (1.12)  326  5.82  (.85)  4.58  (.83)  4.88  (1.40)  5.09  (1.03)  MEAN  6.41  (.99)  6.61  (.85)  6.24  (1.32)  6.42  (1.05)  *Legend:  S= Subject; m3rd and M6th  =  minor 3rd and Major 6th.  79  60  )_  z  40  !: 100  80 z 60  0  1.12 1.19 1.26 1.33 1.41 1.50 1.59 1.  RATIO OF UPPER TO I.OWER PULSE RATE -  FLAT  -v-IN-TUNE-. SHARP  Figure 11. Intonation quality judgements: interval of a 5th. Percentage of items labelled, “flat”, “in—tune”, and “sharp”. For all stimulus items, the lower note remained fixed. The ratios between the upper and lower notes of the interval, indicated along the X—axis, progress (in semitone steps) from a semitone to an octave above the lower note. The musically correct frequency ratio is marked with a vertical line across the plot. Apical electrode. Pulse rate of lower note 163 pps. S= Subject; E= Electrode.  80  100  80  z 60  a 0 I.  z  40  U  a 20  a a a 0 I—  a  U  a  I  a a  0 D 0 I-.  a V a w  a  RATIO OF UPPER TO LOWER PULSE RATE  -6- FLAT  -v- IN-TUNE -S SHARP  Figure 12. Intonation quality judgements: minor 3rd. Pulse rate of lower note 137 pps. Legend as in Figure 11.  81  5  w C D 0 U a  w C  0 U a.  in  I  z ‘U S ‘U  ‘3 C  = U  0 I  z ‘U 1  ‘U  a  RATIO OF UPPER TO LOWER PUlSE RATE -a- FLAT  -v- IN-TUNE e- SHARP  interval of a Intonation quality judgements: Figure 13. Legend as in Pulse rate of lower note 145 pps. 4th. Figure 11.  82  160  80 I  z 6O ‘9  a a 0 4o V  20  100  80  z 60 t9 C C  40  20  100  80  60 C C C  0  40  20  0  -a FLAT  —V--IN-TUNE-S-SHARP  major 6th. Intonation quality judgements: Figure 14. Legend as in Figure 11. rate of lower note 127 pps.  Pulse  83  ,11 in u.J  z 0 I  LU V.) LU  N  7 V.)  MAJOR 6TH  5TH  4TH  MINOR 3RD  MUSICAL INTERVAL  Intonation quality judgements for four musical Figure 15. Triplets of interval sizes, from a major 6th to a minor 3rd. the results, represent size interval for each bars and markers respectively. and 11, 10 Subjects 7, for from left to right, Markers indicate subjective interval size (point of subjective equality). Vertical error bars show ± one standard deviation. Correct interval size is indicated by bold horizontal lines for each interval size.  84  in Figure 15).  For example,  the interval of a 5th is normally  characterized by a frequency difference of 7 semitones,  so  that tones comprising a 5th are related by a frequency difference of  *  27/12  —  f  0 equals the frequency or a frequency ratio of 1:1.5, where f or pulse rate of the lower note.  For S—7 and S—b,  location of the maxima of the “in tune” functions and PSEs  (Figure 15)  the  (Figure 11)  for the 5th were at or near this ratio or  number of semitones, when the pulse rate assigned to the lower These “in tune” functions showed maxima in  note was 163 pps.  the region of 6-7 semitones 7 and S-b, the maxima.  respectively),  (with PSEs of 6.57 and 7.16 for S and steep slopes on either side of  The slopes for the “flat” and “sharp” functions  were similarly steep, progressing from near—lOO% to 0% over a span of two or three semitones.  Near the target ratio,  there  was a narrow region of overlap between the “flat” and “sharp” labelling functions, reflecting a region of uncertainty. “in—tune” function for S-il  (Figure 11)  yielded a broader,  less distinct peak, with more gradual slopes, 5.92  (Figure 15),  The  and a PSE of  as well as a standard deviation  approximately twice that of the two other subjects.  This may  reflect the difficulty this subject encountered in making the judgements.  It is,  of course,  also possible that S-il would  have yielded similar results with acoustical signals, prior to deafness.  85  the results for the three subjects  For the major 6th, varied  (Figures 14 and 15).  underestimate, interval.  Subject 7 tended to  to overestimate the size of this  and S—b  Results for the latter subject showed a wide “in  tune” function spanning approximately three semitones, with occasional “in tune” labelling of octaves, This subject, however,  4ths,  and 5ths.  claimed not to remember the test tune  (the first two notes of “My Bonnie Lies Over the Ocean”) well.  This uncertainty of S-lO  very  (for whom the standard  deviations for all other intervals tended to be smaller than for the other two subjects)  appeared to be reflected in a  relatively larger standard deviation than that which this subject obtained for the other intervals. a 4th  (Figures 13 and 15),  For the interval of  5-11 underestimated the size of the  interval by approximately one semitone, while S-7 and S-ic achieved PSE5 very close to the target interval. minor 3rd  (Figures 12 and 15),  For the  all three subjects yielded PSE5  within close proximity of the target interval. Thus,  it is evident from Figures 11-14 and the PSEs  plotted in Figure 15 that the size of the subjective intervals,  for each of the musical intervals tested, was  generally within one semitone of the target size of the interval.  Subjects were more accurate in estimating the size  of smaller intervals,  such as the minor 3rd and the 4th,  of the larger intervals, Thus,  than  such as the 5th and the major 6th.  subjects showed a smaller amount of constant error  difference between the PSE and the target interval size)  (the and  86  smaller mean standard deviations for the narrower intervals (Table 2). For all intervals, subjects and conditions,  shifts in the  preferred size of the interval from one block of trials to the next were common.  Thus, while a pulse rate ratio of 1:1.5  might be considered “in tune” for one block of 60 stimuli, the next block might show a definite preference for an interval size one or two semitones larger or smaller. A comparison of the data obtained for the 5th on electrodes located in the apical, basal,  and intermediate  regions of the electrode array is shown in Figure 16 as subjective size (PSE)  of the interval, plotted against the  physical size of the interval, together with standard deviations.  The pulse rate of the lower note was 100 pps for  all three electrodes.  The size of the subjective interval of  a 5th for our subjects closely approximated the correct interval size (7 semitones)  for normal—hearing subjects  listening to acoustical stimuli.  Subjects 7 and 11 showed a  slight trend towards increased standard deviations as stimulation was moved basally, whereas S—b  achieved a lower  standard deviation on the basal, rather than the apical electrode.  Subject 11 appeared to prefer a physically larger  5th as stimulation was moved apically.  All subjects commented  that apical stimulation sounded more “musical” than basal stimulation, electrodes.  and that the tasks were more difficult on basal  87  ‘—.9 V., UI  z  0  I UI UI  SUBJECT 7  SUBJECT 10  SUBJECT 11  Intonation quality judgements for three subjects for Figure 16. of a 5th, played on three different electrodes. interval an and markers for each subject, represent the bars of Triplets left to right, for Electrodes 5, 12 and 18, from results, indicate the subjective interval size Markers respectively. Vertical error bars show ± one equality). subjective (point of size equals 7 semitones. interval Correct deviation. standard pps. 100 note: lower Pulse rate of  88  The intonation quality judgements for a 5th transposed into three different octaves, Figure 17.  Again,  from 81 to 326 pps,  are shown in  only the calculated PSEs and standard  deviations are plotted.  For all three subjects,  the  subjective interval of a 5th was within approximately one semitone of the physical 5th  (pulse rate ratio of 1:1.5,  equivalent to 7 semitones) when the lower note of the interval was 81 or 163 pps.  However, when the lower note of the  interval was 326 pps,  all subjects showed a preference for a  physically smaller pulse rate ratio. subjective interval of a 5th equalled,  For S-1O and S-li,  the  in physical terms,  approximately 5 semitones, when the lower note of the interval was 326 pps.  The preference for physically smaller intervals  at higher pulse rates was most marked for S-b  and S-il.  While subjects complained about the difficulty of assigning intonation quality labels to intervals at both the lowest and highest pulse rates,  there were no systematic shifts in  standard deviations between these conditions. The results of the two informal pilot experiments suggested that subjects were able to assign intonation quality labels also to pitch intervals resulting from changing the location of stimulation (i.e. to another, assessed. interval,  by switching from one electrode  at a constant pulse rate).  Only two subjects were  When electrode 19 served as the lower note of the the electrode yielding an optimal 5th was located  1.5-3.0 mm (mean 2.25 mm)  in a basal direction.  rate and electrode were covaried,  When pulse  intonation quality  89  LI.I  z  0  I LU LU  r%J  V•)  SUBJECT7  SUBJECT1O  SUBJECTII  Intonation quality judgements for three subjects for Figure 17. an interval of a 5th, using three different pulse rates for the Triplets of bars and markers for lower note of the interval. from left to right, for results, represent the each subject Markers 326 pps. and 81, 163, rates of pulse lower note subjective of size (point interval indicate subjective equality). Vertical error bars show ± one standard deviation. Correct interval size equals 7 semitones.  90  judgements appeared to become very difficult,  and subjects  complained that this sounded “like each note was being played on a different instrument”.  DISCUSSION  Categorization of musical interval size is a difficult task for musically untrained normal—hearing subjects, when the stimuli are musical tones.  even  Musicians have been shown  to be more proficient at making fine intonation quality discriminations than musically inexperienced subjects. Wapnick,  Bourassa,  and Sampson (1982),  for example,  demonstrated that musicians were able to categorize intervals which were only one—fifth of a semitone flat or sharp. and Ward  (1978)  Burns  showed that both the absolute values of the  difference limen for interval size and the variability of the data were much greater for musically inexperienced subjects. In spite of the difficulty of the task,  all three of our  subjects were able to categorize electrical pulse rate pitch intervals with a reasonable degree of accuracy and consistency.  The PSEs showed that the subjective intervals,  as represented in familiar tunes stored in the long—term memory of these deaf subjects, musical target intervals,  corresponded closely to the  suggesting that subjects were  responding to the ratio properties of the stimuli, merely to their ordinal relations.  and not  The fact that the PSEs did  not always coincide precisely with the correct frequency ratio  91  is not surprising, as even trained musicians judge musical intervals with a degree of constant error. Shifts in the preferred size of intervals from one block This also does  of trials to the next were common in our data. not necessarily imply weakness of pitch.  Musically untrained  subjects have no verbal structure for describing musical interval sizes  or by  (either by such names as 4th, minor 3rd,  a sol—fa scale), and are thought to use different and less efficient strategies than musically trained subjects in making frequency ratio discriminations fact,  (Siegel and Siegel 1977).  In  small intrasubject shifts in subjective response  criteria in musical interval labeling tasks have also been reported with musically sophisticated listeners Ward 1978).  (Burns and  Because musically untrained listeners judge  interval size more on the basis of pitch height than on the basis of cues associated with target frequency ratios,  such  listeners are less precise and less consistent than musically trained subjects, when judging musical interval size. Musically untrained subjects can also be expected to be more easily influenced by context effects.  For example, when  the randomization of stimulus intervals resulted in the presentation of several intervals which the subjects labelled as obviously too small  (“very flat”),  subjects appeared to be  more ready to label almost any larger interval as “in tune”. Preliminary results of Eddington et al.  (l978a,  l978b)  with melody recognition via electrical stimulation of deaf ears suggested that the frequency ratios required to elicit  92  the correct musical pitch interval might vary with the location of the stimulating electrode and hence with the The PSE5 for the  population of nerve fibers excited.  subjective interval of a 5th for electrodes in different cochlear regions, notion.  in our experiments,  fail to support this  While our subjects reported that the intervals on  basal electrodes sounded less musical and were more difficult to judge,  these difficulties were not reflected in any  consistent trends in the size of the standard deviations. two subjects  For  (S—7 and S-il), the standard deviations showed a  slight increase as stimulation moved basally, whereas for one other subject (S-b), the standard deviations showed a slight decrease.  Thus, at low pulse rates, the musical intervals  heard by these subjects appeared to correspond closely to the subjective musical intervals heard by normal—hearing subjects, regardless of the location of the stimulating electrode. A greater than normal increase in pitch for a given increase in pulse rate may be suggested by the performance of S—b  and S-il at higher pulse rates.  When the lower note of  the interval was a pulse rate of 81 pps,  all three subjects  demonstrated PSE5 within one semitone of the target interval (7 semitones).  However, when the lower note of the interval  was set to 326 pps,  the subjective interval of a 5th,  for  these two subjects, corresponded to a pulse rate ratio equivalent to approximately 5 semitones interval of approximately a 4th).  (in musical terms, an  Rapid increases in pitch  with increases in electrical pulse rate or frequency have been  93  anecdotally reported by other investigators Similarly, Tong et al.  (1979)  (Shannon 1983).  anecdotally reported that pulse  rates above 250 pps were consistently described as having pitches above that of the highest note on the piano. Thus,  the results for S-b  and S-il suggest that,  range of pulse rates from low to high,  over a  equal ratios of pulse  rates did not yield subjectively equivalent musical intervals. It is possible that,  at higher pulse rates, these subjects  were unable to extract the musical pitch and to apply their musical interval sense, perhaps because of an unpleasantness, shrill timbre, or a lack of musical quality of stimuli.  Large  increases in pitch for physically small frequency ratios were also demonstrated in normal—hearing musical subjects by Attneave and Olson (1971),  in a musical interval adjustment  task, when the frequency of the higher tone exceeded the upper limit for musical pitch (approximately 5000 Hz).  These  authors suggested that subjects were unsuccessfully attempting to apply musical standards to nonmusical tones. Musical pitch, or the capacity to identify musical intervals,  in normal—hearing ears, may have an upper limit  similar to that of neuronal phase—locking.  In the normal ear,  action potentials in auditory neurons are elicited by unidirectional deflections of the basilar membrane, and occur within a restricted time window relative to the stimulating waveform. preserve,  Thus,  cochlear nerve fibers in the cat are known to  in their temporal discharge patterns,  information  regarding the temporal fine structure of the stimulus waveform  94  (Rose et al.  for frequencies below approximately 5000 Hz 1967; Hind,  Anderson,  Brugge,  1980; Greenberg, Geisler,  and Rose 1967;  and Deng 1986).  Sachs and Young  In the cat and  squirrel monkey, phase—locking has been shown to remain good and fairly constant up to 2 kHz,  and to decline at higher  frequencies, until it is no longer detectable at 5-6 kHz. other species,  such as some rodents  (Palmer and Russell 1986),  phase—locking decreases above 600 Hz, above 3.5 kHz.  In  and is not detectable  The upper limit for phase-locking in humans  has not been established.  The saturation of firing rate of  individual fibers is reported to result from filtering by the hair cell membrane  (Palmer and Russell 1986),  from hair cell-  to—neuron transmission, and from the refractory period of the neuron  (Rose et al.  1967).  The deterioration of the  precision of neural timing information with increasing frequency has been attributed to the increasing temporal variance or standard error of neural phase—locking, to the stimulus period, 1978),  at higher frequencies  (Goldstein  or to a decrease in the alternating current  component of the hair cell response,  (a.c.)  relative to the steady  depolarization (Palmer and Russell 1986). stimulation,  relative  With acoustical  individual fibers do not fire on every cycle of  the stimulus or at precisely the same point in every effective half—cycle,  and the modal values of interspike intervals occur  at integral multiples of the stimulus period.  Individual  fibers do not generally achieve firing rates in excess of 200 spikes per second.  95  With electrical stimulation, maximum firing rates have been reported to be much higher than with acoustical stimulation frequencies,  (Moxon 1965,  1971).  At low electrical  all neurons within the suprathreshold portion of  the electrical field fire synchronously in response to every cycle of the electrical waveform (Glass 1983; Hartmann, and Klinke 1984),  up to about 500 Hz  Stypulkowski l987b).  Javel et al.  Topp,  (van den Honert and (1987)  reported saturation  discharge rates that usually equalled electrical pulse rates up to at least 800 pps.  The alternate depolarizations and  hyperpolarizations of the neuronal membrane, the electrical signal,  resulting from  are opposed by accommodation processes,  which work to restore the resting transmembrane voltage (Clopton et al.  1983),  and place an upper limit on the firing The absolute and relative  rate of individual neurons.  refractory periods have been reported to be approximately .3 insec and 5 msec,  respectively  (van den Honert and Stypulkowski  l987b). Animal data using electrical stimulation have shown a higher degree of synchronization than that observed with acoustical stimulation of the normal ear al.  1987).  (Glass 1984; Javel et  The narrow phase angle of neuronal responses to  low electrical frequencies  (Parkins 1989)  results in a smaller  degree of temporal variance of neural responses than that obtained with acoustical stimulation (Hartmann,  Topp,  Klinke 1984; van den Honert and Stypulkowski 1987b).  and The  upper limit of phase-locking to electrical signals has not  96  been determined with certainty. an upper limit of 2-3 kHz Merzenich 1983), least 12 kHz  While some studies have shown  (Loeb, White, and Jenkins 1983;  others have reported phase-locking up to at  (Glass 1984).  Although phase-locking has been  demonstrated to be statistically significant at high electrical frequencies, the firing rates at these frequencies appear to be determined not by the stimulus frequency,  but by  interactions between the refractory period of the neurons and the amount of charge delivered during the excitatory portion of the stimulus waveform.  As pointed out by Parkins  these responses at high frequencies are,  (1989),  in effect, threshold  responses, with poor synchronization to all but the first pulse of each burst. It is reasonable to assume that the properties of the electrically elicited auditory nerve fiber responses in our implanted subjects would be similar to those observed in animal experiments.  In view of the demonstrated precision of  temporal patterning of neural responses,  as shown in  physiological experiments, and the absence of a mechanical, spectral, place analysis with electrical stimulation of deaf ears,  as well as the close relationship between the timing of  electrical stimuli and the neural responses, our results suggest that the auditory nervous system is capable of basing musical interval identification on interspike interval information.  Temporally mediated pitches resulting from  pulsatile electrical stimulation thus appear to be sufficiently salient to support musical interval perception,  97  at least at low pulse rates.  Salient,  roughly defined as “noticeable”,  in this sense,  is  in that salience of a  perceptual feature is an important determinant of which features are selected or emphasized by listeners, even when other features are perfectly discriminable (Miller and Carterette 1975). Our results, while providing evidence for temporally mediated musical pitch, do not imply that place mechanisms are irrelevant in musical pitch perception.  The consistent  observations of our subjects regarding the more pleasant and musical sound quality of the apical electrodes suggest that a congruence of place and timing information may be important. The informally assessed ability of subjects to assign intonation quality labels to musical intervals which were represented solely by changes in cochlear place of stimulation also suggest a role by place mechanisms. It is recognized that the informal pilot experiment with place pitch,  by means of electrode switching,  represents but  the crudest attempt at estimating the optimal cochlear place representation for a musical interval.  Firstly,  it is not  possible to provide a continuous gradation of place of stimulation,  due to the fixed location and coarse spacing  (0.75 mm separation)  of the electrodes.  Secondly,  it is well-  known that electrical stimuli activate neurons over a considerable range of characteristic frequencies, due to current spread within the cochlea.  This current spread may be  symmetrical or asymmetrical about the stimulating electrodes,  98  depending on the electroanatomy of the residual cochlear Thirdly, the location of the responding neural  apparatus.  population cannot be determined, and may also be asymmetrical, especially in ears with irregular distributions of nerve survival. In spite of these limitations and the preliminary, informal nature of the data for these two subjects, the mean distance judged optimal for a 5th (2.25 mm)  is in fair  agreement with the Bekesy-Skarstein cochlear map,  as fitted by  the frequency-position function of Greenwood (1961,  1990).  According to this function, the interval from 1 kHz to 1.5 kHz (a 5th),  is represented by a distance of 2.58 mm.  calculations show that,  Similar  in the 1 kHz region of the cochlea,  1.5 inn represents a distance corresponding to approximately a major 3rd,  2.25 mm to approximately one-half octave,  and 3.0  mm to approximately a minor 6th (Greenwood 1994, personal communication) The substantial differences between subjects in the ability to extract musical pitch information, higher pulse rates,  is not surprising,  especially at  in view of the probable  unnaturalness of the sound quality obtained with electrical stimulation.  Large individual differences in pitch perception  have also been reported in acoustical experiments with normal hearing subjects.  Risset (1978),  for example,  showed that  sounds which increase in shrillness while decreasing in frequency may be judged as increasing or decreasing in pitch,  99  depending on the relative weights attached to each of these cues by individual observers.  100  EXPERIMENT IV:  INTERVAL RECONSTRUCTION  While psychologists of music have sometimes been skeptical about the ability of untrained subjects to participate in experiments dealing with musical pitch perception (Siegel and Siegel 1977; Burns and Ward 1978), other investigators have shown that,  given appropriate tasks,  musically unsophisticated subjects are able to perform with remarkable accuracy and consistency,  and to make valuable  contributions to the understanding of musical perception (Sloboda and Parker 1985).  Thus,  even though such subjects  are not familiar with the vocabulary for the formal naming and precise categorization of musical interval size,  this does not  necessarily mean that they have no appreciation for correct (1978)  showed that even for musically  interval size.  Dowling  naive subjects,  long—term memory of familiar tunes is based on  stable representations of interval size,  and not merely on  general melodic contour. The importance,  for nonmusical subjects,  of a familiar  melodic context was demonstrated by Attneave and Olson They showed,  in their second experiment,  (1971).  that musically naive  subjects transposed on a logarithmic or musical frequency scale when asked to reconstruct a very familiar melody presented in one octave to other frequency regions. naive subjects,  in their first experiment,  Musically  had been shown to  101  be unable to perform the transposition task with simple, artificially constructed 2—note melodies. interval adjustment task,  Elliot, Platt,  In a musical and Racine  (1987)  showed that both musically experienced and musically inexperienced subjects appeared to have internalized standards for the consistent intonation of musical pitch intervals,  but  that these standards were better—developed in the experienced subjects,  who were more accurate and more consistent in their  frequency settings.  The ability of deaf subjects to adjust  electrical pulse rates in the reconstruction of musical pitch intervals abstracted from familiar melodies, term memory,  stored in long—  has not been previously investigated,  and could  provide additional evidence of temporally mediated musical pitch.  METHODS  In the previous experiment,  subjects labelled the  intonation quality of a variety of musical intervals.  The  subjective sizes of the musical intervals, which consisted of ratios of pulse rates,  appeared to be similar to the ratios of  frequencies which characterize musical intervals heard normally.  However,  additional evidence for the salience of  musical pulse rate pitches could be obtained using the method of adjustment.  In Experiment IV,  subjects were required,  adjusting a pulse rate on a single electrode,  by  to reconstruct  102  each of three musical pitch intervals:  a 5th,  a 4th,  and a  minor 3rd. The subjects were provided with a fixed reference or anchor “note” which represented either the upper or the lower note of the interval.  When the upper note of the interval was  being adjusted, the lower note anchor, the 5th,  (93 pps)  served as the fixed  and the “correct” targets were 140, 4th,  and minor 3rd,  respectively.  125,  111 pps for  When the lower  note of the interval was being adjusted, the upper note 125,  lii pps for the 5th,  4th,  served as the fixed reference, 93 pps.  and minor 3rd,  (140,  respectively)  and the target pulse rate was  The initial pulse rate of the variable note was  selected at random by the computer,  at a pulse rate somewhere  between one octave above and one octave below the target pulse rate. The musical intervals under examination were abstracted from melodies well-known to individual subjects.  Subjects  were instructed regarding the tune and the interval under examination.  For the 5th,  the tune for all subjects consisted  of the first 4 notes of “Twinkle, Twinkle, the 4th,  Little Star”.  the tunes consisted of the first 3 or 4 notes of  “Away In A Manger” or “0 Christmas Tree”.  The minor 3rds were  abstracted from the first 3 notes of “0 Canada”, notes of the chorus of “Jingle Bells”, “Jesus Loves Me,  This I Know”.  the first 8  or the first 3 notes of  It will be noted that each of  these excerpts consisted of several notes, pitches.  For  but only two  Subjects were told to rehearse covertly these  103  excerpts,  and to take careful note of the pitch change between  the upper and lower notes of the interval.  Subjects were  informed that they would be provided with a fixed note which would have the “correct” pitch, and a variable note, they would be required to adjust the pitch.  of which  In one set of  trials, the lower note of each interval was fixed, and the upper note was the variable note.  In a second set of trials,  the upper note of each interval was fixed, was the variable pulse rate.  and the lower note  Thus, the adjustments of the  upper note and the lower note of the intervals were represented by an equal number of trials.  The fixed and the  variable notes were specified by keyboard control. were,  of course,  Subjects  informed whether the upper or the lower note They  of the interval would be placed under their control.  were instructed to adjust the pitch of the variable note, using the  “+“  and”—” keys of the computer keyboard, until the  pitch change, when switching between the fixed and the variable note,  approximately matched their memory of the size  of the musical interval in question. The subjects accessed the first note of the test interval by pressing the number “1” on the computer keyboard, second note by pressing “2”. intervals,  and the  In the case of ascending  note “1” represented the lower note of the  interval, and note “2”, the upper note.  In the case of  descending intervals, note “1” was the upper note, and note “2” the lower note.  Thus, when the upper note of an ascending  interval was to be adjusted, the subject first pressed “1”,  104  and heard a repetitive pulse train fixed at 93 pps,  and then pressed “2”.  in the subject hearing a repetitive off)  but variable stimulus,  by the computer  (500 insec on! 500 msec of f) Pressing “2” resulted  (500 msec on! 500 msec  at a pulse rate selected at random  (anywhere from an octave above to an octave  below the target pulse rate).  This rate was then adjusted by  the subject. Keyboard control,  by the examiner, permitted selection of  either note “1” or note “2” as the fixed pulse rate. note “1” was selected as the fixed pulse rate, the variable pulse rate,  and vice versa.  When  note “2” was  The pulse rate of  the variable note was adjustable within a range from one octave above to one octave below the target pulse rate. range was computed automatically by the software,  upon  specification of the target ratio by the experimenter. for the 5th, when the lower note was 93 pps,  This  Thus,  specification of  a ratio of 1:1.5 resulted in a target pulse rate of 139 pps, and a range from 70 pps to 279 pps, when the upper note of the interval was being adjusted. interval was being adjusted,  When the lower note of the calculation of the target and  range of the variable lower note of the interval was accomplished by specification of the inverse of the 1:1.5 ratio  (i.e.,  .67:1).  Subjects were encouraged to bracket the target pitch, by alternately adjusting the pitch of the variable note slightly too high and slightly too low for the interval,  before  deciding on an optimal pulse rate setting for the variable  105  note.  They were permitted,  sequence,  at any time,  to review,  in  the anchor pulse rate and current pulse rate setting  of the variable note,  and to make further adjustments to the  variable note until an optimal adjustment was achieved.  They  were also encouraged to stop stimulation for at least 5 seconds prior to reviewing their final adjustment of the variable pulse rate.  Subjects indicated their satisfaction  with the pitch adjustment by pressing the <Enter> key on the computer keyboard.  This resulted in the pulse rate value  being stored in computer memory,  and in the initiation of the  next trial. For each interval tested,  the subjects completed a  minimum of 20 adjustments of the upper note and 20 adjustments of the lower note.  A minimum of 10 practice trials preceded  formal measurements for a given target interval.  To  discourage the use of identity matches as a platform for a fixed number of increments or decrements, the magnitude of the changes in pulse rate effected by pressing the  “+“  and  “—“  keys was randomized between .5 and 1.0 semitones by the computer.  Pulse rates delivered to the subjects were rounded  off to the nearest integer value.  Subjects were unable to see  the display of stimulus parameters on the computer monitor. No exemplars were provided. stimulation at any time,  Subjects were permitted to stop  by pressing the “P”  (Pause)  key on  the computer keyboard. A typical session lasted 2-3 hours. was examined per session.  Only one interval  Mean pulse rate settings were  106  calculated, by converting each pulse rate value to a logarithm,  computing the mean logarithm for the adjustment of  the upper and lower notes of the intervals, the antilogs of these mean logarithms.  and calculating  The ratio between each  adjusted and reference pulse rate was used to compute the interval size,  in semitones,  of each adjustment.  These  numbers were then used to calculate mean interval sizes and standard deviations.  The accuracy of the adjustments was  further examined by determining the percentage of adjustments which were 2 semitones or less from the target pulse rate.  RESULTS  The mean pulse rate adjustments,  interval sizes,  and  standard deviations of the adjustments for each subject and pitch interval are presented in Table 3. target designations in Table 3  The reference and  (Ref/Target)  represent,  respectively, the fixed reference pulse rate and the target pulse rate specified by each musical interval. adjusted the upper (or lower) (upper) rates  note of the interval, the lower  pulse rate served as the reference.  (in pps)  When subjects  and mean interval sizes  The mean pulse  (in semitones)  obtained  by adjustment of the upper and lower notes of the intervals are shown in the “Upper” and “Lower” rows of Table 3, standard deviations  (in semitones)  and the  are shown in parentheses.  The data for the adjustment of the upper and lower notes of the intervals were also combined to yield an overall mean  —  CDt  JH  H  H  l’JO Ha,  WM  0  (3!  F-.)  (31  a,  -,  -  I’J  H  0” a,  (.J  C-  W  co  Olt’3  Ui  0  wo  w  o-  H (,J)  ‘.0  —  ?  .  -M O  O LJ.  W  o  .  a,  0  a,M  a, WD  ‘D o.  o  W  CD U)  C,,  H H  Cl)  Cl)  0  H  Cl)  H H H  ci  0  —  U)  —  D)  CDH  w  CD U)  U)F  1—  l--CD (-i-ti 0  l-’-CD ti0  CD  CD  Cl)  Cl)t 1 CD 0  I-, ci  Cl) CD  H  CD ti I)) H  o • H  a,  0  H  w  o  .t.  W co  a, u,  :-  a, 0l  U)  CD  0  Cl  Cl) CD  —  I-  )  ti  < CD  o  lO  a,  H  o  ‘.0 a-’  H  wa,  ‘.DW  H 0  U1  U1H  ‘.0  U)  C 1 U)  CD  u—  0  ri-li  I-’-CD  (DO  Cot’  H M  H  a,  C)  (‘3  t’3t’J  a,a,  .1-.-)  a, H  0  Ok)  H  a,l-j  H  H H  U) 1 U)  CD  —  HCD ri-ti 0  CD  CoC  Cl)  H H  Cl)  H 0  Cl)  T  ‘-3  H  ‘.0  lxj  CD ri-0  03<  CDZ  (‘3 0  H  -J l-  t’)  -‘  w  —_I  F-.)  0  H  J  —_I  U)  CD  0  C,  C)) CD  —  H H  0)  ti  < CD  o  ‘.0  0  H  a, 1.0  0  —.IU1  W3  F’) -J  i—’  0  ui  J0  0  H  t’3J wo-  JH  —  WC U)  CDd  —  ri-i-( 0  F’-CD  (DO  Ci2  i—  w  H  (31  L’S)  H  1.0  H  (311.0  0  cJl.  a,  H  JH  1.0  0-’  0  H  aw  H  —  U)  U)  CD  l--CD (tti 0  CD  Cl)c  Cl)  H  TH  ;  H  I)  Cl)  ‘-3  WH  lxi  ‘.0 wj  CD ri-c  txj  (DZ —  zCl)  0  C)  Lxi  Lxi  ‘-3  H  z‘-3  1:11  H  txj  t  txi  w  Lxi  ‘-3  C —3  108  interval size and overall standard deviation for each interval,  in semitones  (“Overall” row,  in Table 3).  The results shown in Table 3 are plotted graphically in Figure 18.  The intervals under examination,  (a 5th,  minor 3rd)  are characterized by interval sizes of 7,  semitones,  respectively.  4th, 5,  and 3  The solid horizontal lines which  intersect the vertical axes of the plot at 7,  5,  and 3  semitones indicate the target size of intervals of a 5th, and minor 3rd,  and  respectively.  For S-7 and S-ic,  4th,  the magnitude  of the subjective pulse rate intervals obtained by the method of adjustment corresponded closely to both the physical, equal—tempered intervals and to the subjective interval sizes (PSEs)  obtained by the intonation quality judgements in  Experiment III.  While Subject 11 tended to underestimate the  size of the 5th (in both Experiment III and Experiment IV), the sizes of the 4th and the minor 3rd remained relatively well—preserved. It will be recalled that subjects were required to adjust both  (although in separate sessions)  notes of the intervals.  the upper and the lower  In the case of ascending intervals,  the upper note represents the second note of the intervals, and in the case of descending intervals, represents the second note. a melody,  the lower note  In the memory of the subjects for  the first note of any interval always temporally  precedes the second note.  Thus,  it is not surprising that all  3 subjects reported greater difficulty when they were required to adjust the first note of an interval  (i.e., when the second  109  10 —‘9 .......  LU  LU Ll  z6 0 LU  Z4 LU  z 0  LU  v,1 0  5TH  4TH MUSICAL INTERVAL  MINOR 3RD  Figure 18. Interval reconstruction: mean size (in semitones) of ascending intervals of a 5th, 4th, and minor 3rd, as adjusted by 3 subjects. The vertical axis shows the response intervals in semitone steps. The left and right markers for each subject indicate the mean interval size achieved with adjustment of the upper and lower notes of the interval, respectively. The left and right vertical error bars for each subject show ± 1 standard deviation from the mean interval size with the adjustment of the upper and lower notes of the interval, respectively. The bold horizontal lines intersecting the vertical axes at 7, 5, and 3 semitones represent the musically correct intervals for the 5th, 4th, and minor 3rd, respectively.  110  note of the interval was provided as the fixed reference). For example, when S—b  was required to reconstruct an  ascending 4th, using the first three notes of “Away in a Manger”, he reported that this was not a difficult task when he was provided with the first note.  However, when he was  provided with the second stimulus (i.e., the upper two notes of the interval)  as the fixed pulse rate, and was required to  adjust the first  (lower)  note, he complained that this was  more difficult because this required him “to think backwards”. For this interval, the relatively greater difficulty S-b encountered in this condition appeared to be reflected in a larger standard deviation (1.96 sexaitones),  compared to that  which he obtained with adjustment of the upper note of the same interval  (0.81 semitones).  Similarly,  Subject 10  demonstrated slightly greater intertrial variability when he adjusted the upper  (first)  note of the descending minor 3rd in  “Jesus Loves Me, This I Know”  (0.84 and 0.63 semitones for  adjustment of the first and the second notes of the melody, respectively).  This effect, however, was not evident in his  data for the ascending 5th (perhaps because this interval was the last to be tested with S-b), or in the data for the other subjects.  For S—li, the standard deviations were consistently  larger when she adjusted the upper note of the intervals  (mean  1.75 and 1.30 semitones for adjustment of the upper and lower notes, respectively), possibly due to a deterioration of pitch strength for this subject at pulse rates above 100 pps. S-7 and S—b,  For  standard deviations tended to be smaller for the  111  minor 3rd than for the larger intervals for the minor 3rd, In general,  1.36 for the 4th,  (mean .88 semitones  and 1.15 for the 5th).  tended to adjust intervals somewhat  S-7 and S-b  smaller when they adjusted the upper note than when they adjusted the lower note. The percentage of pulse rate adjustments which were within close proximity of the target  (Figure 19)  was greater  for the minor 3rd than for the two larger intervals. interval sizes,  For all  S—il achieved a smaller percentage of close  approximations than the other subjects.  DISCUSSION  These results show that some subjects with cochlear implants are able to reconstruct musical intervals from melodies stored in long-term memory, by adjusting electrical pulse rates.  The ratio properties of the mean pulse rate  adjustments of S-7 and S-b  for all three intervals  corresponded closely to the frequency ratios which characterize musical intervals for musical subjects with normal hearing.  These ratio properties,  in general,  also  corresponded closely to the subjective interval sizes demonstrated in the intonation quality judgements of Experiment III.  Thus,  the results of the intonation quality  experiments and of the production experiments are in general agreement.  112  w 0  z  w C) Ui 0  5TH  4TH  MINOR 3RD  MUSICAL INTERVAL L1SEMITO1E  2SEMITONES  Figure 19. Interval reconstruction: percentage of pulse rate adjustments 1—2 semitones or less from the target, for three intervals. Subjects 7, 10, and 11.  113  While the adjustments of the intervals of a 4th and minor 3rd by S-il also corresponded closely to the target interval sizes,  this subject’s subjective 5th was approximately two  semitones smaller than its conventional size.  For the  interval of a 5th in Experiment III, this same subject yielded a subjective interval size approximately one semitone smaller than the physical interval.  The contraction of the 5th by  this subject could be due to a deterioration of pitch strength above approximately 100-200 pps, 5th and a 4th.  Thus,  or to a confusion between a  it is equally possible that S—li might  have yielded similar results with acoustical stimuli, deafness.  Both the 5th and 4th are highly consonant intervals  which bear a close relationship to each other. for a 5th,  prior to  For example,  an octave downward displacement of its upper note,  or an octave upward displacement of its lower note results in an interval of a 4th.  Comparable inversions of the notes of a  4th result in an interval of a 5th. contexts,  Furthermore,  these two intervals serve as functional equivalents  (as in the “Subject” and “Answer” of a fugue). 5th and 4th are easily confused by untrained, listeners,  in some  Intervals of a normal—hearing  and under some circumstances, even by some  musically trained subjects  (Wapnick, Bourassa,  and Sampson  1982). In the presumed absence of place coding of electrical signals in deaf ears, this pitch or interval information almost certainly has to be dependent upon the timing of interpulse and interspike intervals.  Dobie and Dillier  (1985)  114  have shown,  using electrical pulse trains from which a  percentage of pulses were stochastically omitted (i.e., interpulse intervals were equal to either the pulse period or to multiples thereof), that cochlear implant subjects rank pitches on the basis of interpulse interval information, rather than on the basis of the total number of pulses delivered per unit time.  These data,  like ours,  suggest that  the human brain is able to make perceptual decisions about musical pitch or musical intervals on the basis of interspike interval information. These findings are in agreement with the results of acoustical experiments which have utilized stimuli designed to provide listeners with exclusively or predominantly temporal information.  Thus, musically trained subjects have been shown  to be able to perform musical interval identification and dictation tasks, as well as interval adjustment tasks, with stimuli which consist of closely spaced, unresolvable harmonics or of sinusoidally amplitude—modulated noise and Viemeister 1976, 1991).  (Burns  1981; Houtsina and Smurzynski 1990; Pierce  While the ratio information conveyed by pitches  resulting from such stimuli has been questioned,  and the  pitches reported to be weaker than those of pure or complex tones with resolved low frequency components  (Houtsma 1984),  others have shown that even musically naive subjects are able to utilize this temporal information in open—set or closed—set melody recognition tasks Moore and Rosen 1979).  (Burns and Viemeister 1976,  1981;  115  The difficulty experienced by one presumably relatively musical subject  (S-ll)  in adjusting pulse rate pitches is  consistent with the greater constant error  (i.e.,  between the subjective and target intervals)  difference  and larger  standard deviations shown for this subject in the intonation quality judgements of Experiment III, larger than the minor 3rd.  especially for intervals  This subject also evidenced a  poorer performance on the closed—set melody recognition experiments in Experiment II.  The standard deviation of the  pulse rate adjustments for this subject in Experiment IV, averaged across the three test intervals, was 1.54 semitones, or 37% greater than the 1.11 to 1.14 semitone standard deviations for S-7 and S-b.  Even simple pulse rate pitch  matching during informal preliminary investigations proved to be extraordinarily difficult for this subject,  who appeared to  become increasingly confused during prolonged efforts at effecting subjectively satisfactory adjustments.  The  relatively poorer results for this subject may reflect poor nerve survival,  relative to the other subjects.  Ears deafened  by bacterial meningitis or labyrinthitis have been shown to have lower mean spiral ganglion cell counts than ears deafened by other pathologies  (Nadol and Hsu 1991).  A number of  investigators have demonstrated a positive correlation between neuronal integrity and performance on psychophysical tasks (Pfingst and Sutton 1983).  It is also conceivable that the  poorer results for S—li reflect insult to the central auditory tracts or pitch processing mechanism.  Major neurological  116  sequelae have been reported in 15 to 71 percent of postmeningitic patients  (Vernon 1967).  In general, the standard deviations for the minor 3rd were smaller than those for the two larger intervals, especially for S-7 and S—iC.  Smaller deviations for narrower  pitch intervals have also been reported in acoustical experiments.  Attneave and Olson (1971),  for example,  examined  the ability of normal—hearing, nonmusical subjects to reconstruct the musical intervals of a familiar tune, using pure tones,  and found a smaller amount of variability for a  (equivalent to 4 semitones)  major 3rd  (equivalent to 9 semitones).  than for a major 6th  These authors attributed this  phenomenon to a possible “keynote” effect,  in that the major  3rd in their test melody ended on the keynote melody.  (tonic)  of the  They suggested that intervals incorporating the  keynote might somehow be easier to adjust. Viemeister  (1976,  1981)  Burns and  tested musically trained subjects with  pure tones and a variety of sinusoidally amplitude-modulated noises,  and also reported smaller deviations for major and  minor 3rds than for larger intervals. The “keynote effect” cannot explain the smaller deviations for the minor 3rd in our data,  as the notes of the  minor 3rd in our melodies consisted of the interval between the third and fifth steps and the dominant,  (in musical terminology, the mediant  respectively)  of the diatonic scale.  It is  possible that the greater consistency observed for smaller intervals,  at least in our data, resulted from the strategies  117  used by the subjects in the adjustment task, small pitch intervals.  especially with  Subjects were frequently observed to  bring the reference and the variable pulse rates into close proximity before effecting an optimal adjustment.  While there  was no evidence that subjects were using these near—matches as a platform for a fixed number of increments or decrements,  the  ability to match the reference and variable pulse rates could be expected to result in a substantial reduction in the probability of large errors,  especially for smaller intervals.  It is possible that the normal—hearing subjects in acoustical experiments utilizing the method of adjustment Viemeister 1981; Attneave and Olson 1971)  (Burns and  employed a strategy  similar to that of our subjects. Data of Elliot, Platt,  and Racine  (1987), however,  suggest that even when musically untrained subjects were unable to use pitch matching as a platform for musical interval adjustment,  smaller pitch intervals were still  characterized by smaller degrees of variability than wider In their protocol,  intervals.  identity matches were precluded  by permitting subjects an adjustment range of only one semitone.  These data and ours suggest that for musically  inexperienced subjects,  small intervals may be easier to  adjust than large intervals.  The smaller amount of  variability noted for smaller intervals in Experiment IV is, furthermore,  consistent with the results of Experiment III,  which showed smaller standard deviations for narrow intervals than for wider intervals.  118  No obvious relationship was observed between the self— reported musical history of the subjects prior to deafness and the accuracy or consistency of their pulse rate adjustments. Subject 7 reported little interest in music prior to deafness, but yielded fairly consistent measurements on the adjustment Subject 11,  task.  on the other hand,  reported a considerable  interest in music prior to deafness from meningitis,  and  reported having been able to play tunes on a piano keyboard, with a fair degree of accuracy, without the benefit of printed music.  The pulse rate adjustment task appeared particularly  difficult for this subject,  and this appeared to be reflected  in a larger constant error and a larger degree of variability. Subject 10 indicated a considerable appreciation for music prior to the onset of total deafness,  but had no musical  training and had never played a musical instrument.  This  subject described his strategy during this experiment as “getting the two notes to harmonize”.  In the intonation  quality judgements of Experiment III, this subject had shown consistently smaller standard deviations than the other two subjects. experiment,  In the results of the interval reconstruction there were no consistent differences between the  standard deviations of S-b  and S-7.  It is interesting to compare the postoperative emergence of speech perception abilities in these three implant recipients.  Following 6 weeks of cochlear implant experience,  S-7 and S—b  scored a mean 88.3-90.5% on the auditory-only  speech perception test battery,  consisting of CID Everyday  119  Iowa Sentences,  Sentences,  Sentences in quiet.  arid Bamford—Kowal—Bench (BKB)  In contrast,  cochlear implant experience,  following six weeks of  S—il scored a mean 58%.  However,  this subject subsequently showed considerable improvement,  and  achieved scores of 80-90% within two years of device fitting. While the delayed improvement observed for S—il is common in cochlear implant recipients  (von Wallenberg and Battmer 1991),  the mechanisms underlying these differences between patients are not understood.  These differences may reflect differences  in neuronal survival, processing abilities,  or differences in  plasticity of the central auditory system.  The  neurophysiological limitations of individual patients may certainly be expected to be reflected in difficult (Tyler, Moore,  and Kuk 1989).  While it could be argued that,  from a musical  psychophysical tasks  perspective, the data of Experiment IV lack precise ratio properties,  because of the substantial amount of variability,  it is important to consider the musical inexperience of these subjects addition,  (Elliot,  Platt,  and Racine 1987).  It must,  in  be considered that musically trained subjects also  perform musical interval adjustment tasks with a degree of constant error. practice,  Furthermore, even following training and  some normal—hearing, musically naive subjects have  been shown to remain unable to perform musical interval adjustment tasks, with pure tone stimuli 1971)  (Attneave and Olson  120  It is instructive to compare the standard deviations obtained in Experiment IV with those reported by other investigators, utilizing the method of adjustment with acoustical stimuli and normal-hearing listeners. Viemeister  (1981)  Burns and  reported a standard deviation of  approximately one—third of a semitone for musically trained subjects adjusting musical intervals with pure tone stimuli. In contrast,  adjustment of modulation frequencies  region of 100 Hz)  (in the  of sinusoidally amplitude-modulated 10 kHz  pure tones or noise by the same subjects yielded standard deviations of approximately one semitone. addition to those of Houtsma  (1984),  These findings,  in  suggest that the pitch of  pure tones and complex tones with resolved frequency components may be more salient than the pitch of stimuli which provide predominantly temporal information.  The standard  deviations in our data for Experiment IV were somewhat greater (mean 1.27 seinitones, when averaged across intervals and subjects)  than those reported by Burns and Viemeister  (1981)  for musically trained subjects adjusting modulation frequencies of amplitude—modulated noise. however,  appear to be relatively small,  These differences,  and can certainly be  explained by the musical inexperience of our subjects,  none of  whom had a verbal structure for labelling musical intervals. Furthermore,  it is well—known that the temporal  characteristics of neuronal response patterns to electrical stimulation differ substantially from those generated in the normal ear with acoustic stimulation (van den Honert and  121  Stypulkowski 1987b).  In the normal ear,  information regarding  the stimulus period is preserved over an array of neurons. The responses of these neurons, frequencies,  over a range of characteristic  are staggered in time,  due to the progressive  phase lag along the basilar membrane.  The responses of single  units in a normal ear are phase—locked to the stimulus,  and  these units fire stochastically and independently of each other.  Thus,  even for frequencies greater than 1000 Hz,  interval histograms could be expected to contain some intervals as small as the stimulus period,  even though single It  units are not able to sustain firing at these frequencies. is believed,  by some,  that the central auditory nervous system  analyzes this aggregate output of an array of stochastically independent neurons.  However, with electrical stimulation,  all neurons within the electrical field are known to fire synchronously to every cycle of the stimulating waveform, to repetition rates of 600-900 spikes per second Moxon 1972; Hartmann, Topp,  and Klinke 1984).  up  (Kiang and  In addition,  the precision of phase-locking is known to be greater with electrical than with acoustical stimulation.  These  differences can be assumed to result in relatively unnatural firing patterns with electrical stimulation (Dobie and Dillier 1985),  and may be expected to result in pitch percepts which  are qualitatively different from those elicited with acoustical stimulation of the normal ear.  122  EXPERIMENT V:  MUSICAL TRANSPOSITION  Appreciation of musical pitch,  in the form of melody or In the  harmony, requires appreciation of musical intervals. frequency range over which musical pitch is perceived, musicians  (and laypersons who are not tone deaf)  interpret  tones in equal frequency ratios as equal musical pitch intervals.  Thus,  as long as the frequency ratio between the  tones of a musical interval is preserved,  the interval will be  readily recognized as a transposed musical equivalent, regardless of its location in the musical pitch range.  To  examine the changes in musical pitch resulting from electrical stimulation over a variety of pulse rates, the subjects were required to transpose the intervals used in Experiment IV to lower and higher pulse rates.  METHODS The methodology for this experiment was similar to that of Experiment IV,  in that subjects heard a note with a fixed  pulse rate and were required to adjust a second note with a variable pulse rate.  The variable note was either the upper  or the lower note of a number of musical pitch intervals abstracted from melodies familiar to the subjects. Experiment V, intervals  In  subjects were required to transpose three  (a 5th, a 4th, and a minor 3rd)  to different pulse  123  rate regions, when provided with different pulse rates as reference  (anchor)  notes.  To preclude the use of identity  matches as a basis for a fixed number of increments or decrements, +“  and  “—“  semitones.  the changes in pulse rate effected by pressing the keys was randomized between 1.0,  0.5,  and 0.25  Step size was calculated by the formula  increment or decrement stepsize  where n equals 1.0,  0.5,  =  /12 2 n  or 0.25.  Subjects were given a minimum of 10 practice trials at a standard low pulse rate.  The reference and target pulse rates  of the standard were 105/125 pps for the minor 3rd, for the 4th,  and 93/140 pps for the 5th.  99/132 pps  It will be noted  that while the reference and target pulse rates differed for each test interval,  the 3 intervals nevertheless shared a  common geometric mean pulse rate intervals).  (114 pps,  for the standard or control  It should also be noted that the  reference and targets for the control pulse rate mean of 114 pps)  (geometric  were either in close proximity or identical  to those used in Experiment IV.  In other words, while the  subjects received practice at reconstructing the target intervals at the control pulse rate, in transposition.  they received no practice  Since the sessions for Experiment IV  preceded those for Experiment V by 1—2 weeks, the practice items in Experiment V served to ensure that the subjects were using the correct musical interval for the transposition task. During the practice items at the control pulse rate,  feedback  124  regarding the correct interval size was provided when adjustments deviated from the target by more than two chromatic steps.  Subject 11 was unable to produce consistent  adjustments for the interval of a 5th at the control pulse rate,  and did not participate in the remainder of the  experiment. Following this re—familiarization,  subjects were required  to transpose each interval to both higher and lower pulse rates,  over approximately a two and one—half octave range.  Subjects were tested at six geometric mean pulse rates: 114  (the standard),  by one—half octave  162,  231,  326,  and 466 pps,  (a tritone or 6 semitones).  81,  each separated These mean  pulse rates were used to define the reference and target pulse rates for each of the three intervals.  For example, when the  geometric mean of the 3 test intervals was 231 pps,  the  reference and target pulse rates were 212/252 pps for the minor 3rd,  200/268 pps for the 4th,  and 188/284 pps for the  5th. The transposition task required subjects to reconstruct the intervals at the standard pulse rate practice trials),  and to transpose the intervals to one pulse  rate region below the standard, standard. proper,  (as during the  and 4 regions above the  The subjects were informed that,  for the experiment  the pulse rate of the fixed anchor note would be  varied from trial to trial,  and that the transposition task  might be compared to having the same interval sung or played by a voice or an instrument with a different pitch range.  At  125  each reference pulse rate, of the upper note,  20 trials involved the adjustment  and 20 trials the adjustment of the lower  note of the intervals.  No limit was imposed on the amount of  time or the number of pulse rate changes required by the subjects to complete a trial.  No feedback was provided during  this portion of the experiment.  Each test session involved  the adjustment of either the upper or lower note of one test interval,  and lasted approximately 2-3 hours,  with 15-minute  rest periods between each 10-20 trials. The presentation order of reference pulse rates was randomized from trial to trial, with the restriction that no pulse rate served consecutively more than once as the anchor. The initial pulse rate value of the variable note was randomly selected by the computer, with maxima and minima from an octave above to an octave below the target pulse rate.  A  range restriction of the variable note was necessitated at the lowest,  and for one subject,  rates.  The lower pulse rate limit of the computer program was  pps.  Thus,  at the highest reference pulse  for example, when testing for an ascending 5th  with an upper note reference pulse rate of 99 pps, limits of the variable note were 54 pps,  the lower  rather than 33 pps  (an octave below the target pulse rate of 66 pps). Observation of the subjects by the experimenter suggested that this range restriction did not significantly affect the adjustments of the subjects, portion of the range.  as they tended to avoid the lower  At higher pulse rates,  for S-10 was set at 900 pps,  the upper limit  to preclude the utilization of  126  idiosyncratic attributes of the sounds anecdotally reported by this subject  (such as “double notes”), which could potentially  be used to identify a region of high pulse rates.  No such  restriction was placed on the upper limits for S-7. limit permitted by the software was 1339 pps.  The upper  All other  procedural details were identical to those of Experiment IV. The treatment of the data was comparable to that of Experiment IV,  and consisted of calculation of the mean pulse  rate values for the adjustment of the upper and lower note of each interval at each reference pulse rate.  This was  accomplished by converting individual pulse rate values to logarithms,  and then computing a mean logarithm for the  adjustment of the upper and lower notes of each interval at each reference pulse rate.  The antilogs of the mean  logarithms then provided the mean pulse rate values. interval size of each adjustment,  The  in semitones, was computed  from the ratios between the adjusted and the reference pulse rates by the formula  n1 0 2 f/f 2 l or  n= 12  *  ) 0 log(.f/.f  /  log 2  0 represent the upper and lower notes of the where f and f  interval,  respectively, and n represents the number of  semitones in the interval.  These interval size conversions  were then used to calculate the mean interval sizes and standard deviations.  The computations for the adjustment of  127  the upper note were performed separately from those for the lower note,  in order to examine potential differences between  these two sets of data.  The data for the adjustment of the  upper and lower notes of each interval in each pulse rate region were subsequently combined to yield an overall mean interval size and an overall standard deviation.  These  interval sizes were then compared to those of musical intervals heard acoustically, and plotted as a function of the geometric means of the reference and target pulse rates. For the data obtained with adjustment of the upper lower)  note of the interval,  (or  the standard deviations reflect  the intertrial consistency of the adjustments. deviations from the overall mean interval size  The standard (i.e., the  combined data for the adjustment of the upper and lower note of the interval), were affected not only by intertrial consistency, but also by the agreement between the two sets of The accuracy of the pulse rate adjustments was further  data.  examined by calculating the percentage of adjustments which fell within reasonable proximity of the target rate  (2  semitones or less from the target). Most of the melodies used in this experiment were the same as those used in Experiment IV.  For both subjects, the  5th was abstracted from “Twinkle, Twinkle,  Little Star”.  The  interval of a 4th was taken from the first 3 or 4 notes of “Away In A Manger” or “0 Christmas Tree”.  The minor 3rd for  S-7 consisted of the first B notes of the chorus of “Jingle Bells”.  It should be noted that all of the above intervals  128  were ascending intervals  (i.e.,  the progression from the first  to the second note of the intervals was associated with an increase in pitch).  Subject 10 did not know any suitable  melodies incorporating an ascending minor 3rd,  and for this  reason, was tested with a descending minor 3rd, from the first three notes of “Jesus Loves Me,  abstracted This I Know”.  In view of the large differences in the results shown by Subject 10 for the ascending 4th vis-a-vis the descending this subject was further tested with a descending  minor 3rd, 4th,  abstracted from the first three notes of “0 Come, All Ye  Faithful”.  RESULTS  The mean pulse rate adjustments and standard deviations obtained by the subjects at each reference pulse rate,  and the  corresponding interval sizes, relative to the reference, shown in Tables 4—6. pulse rate,  These tables detail,  the mean of the adjustments  interval sizes in musical terms the adjustment of the upper (“Lower” rows). parentheses,  and the mean  (“Semitones” column)  The standard deviations  in semitones.  for each reference  (in pps)  (“Upper” rows)  for both  and the lower notes  (S.D.)  In the Tables,  are  are shown in  the reference and  target pulse rates are shown by the “Ref/Target” values. the upper  (lower)  note was adjusted,  served as the reference. show the combined data  the lower  (upper)  note  The “Overall” rows in Tables 4—6  (mean interval size and standard  When  129  deviations,  in semitones)  for the adjustment of the upper and  lower notes of the intervals. The data of Tables 4-6 are shown graphically in Figures 20 to 23.  Because each interval in a given pulse rate region  had different reference and target pulse rates but shared a common geometric mean with the other intervals,  the  performance of each subject was plotted as a function of the geometric mean of the reference and target pulse rates of the intervals.  In Figures 20 to 23, perfect performance,  such as  would be approximated closely by musically trained subjects with normal hearing, when listening to musical tones, would be represented by horizontal straight lines intersecting the vertical axes of the plots at interval sizes of 7, semitones,  representing the 5th,  respectively.  5,  and 3  4th and minor 3rd,  The standard deviations obtained with the  adjustment of the upper note of the intervals are plotted in Figures 20 to 23 as the upper vertical error bars,  and those  obtained with the adjustment of the lower note as the lower error bars. Clearly,  the transposition behaviour of the two subjects  was very different.  Subject 7 transposed the intervals on a  logarithmic frequency scale,  in which equal musical pitch  intervals were represented by approximately equal ratios of pulse rates  (Table 4; Figures 20 and 21).  While small local  deviations from the “correct” interval sizes were evident for all three intervals,  both the relative sizes and the ratio  properties of all three intervals remained well—preserved.  130  0  In In  N  ‘.0 Cl  El  C)  ‘(N  It) -1  0’ In  U) 0  U) U)  -4  ,-l  -  (N  o  In  ‘-4  ‘—  (IN N . U)N N In  I.)  U)  N ‘.0  .  U) Cl  In  0i  0  U)  0 i’ • N  U) r-N (I NO CI) ‘.0(1 Cl 0 0 CI) .  or No  (N 1’  . (N  -1 (N  in (N  (1(1  ,-;  -1  N ‘.0(N (IL( U) Cl  OIn ‘.0 (N  (NO . 1N N Cl  N 0  Cl  Oi  Cl  CN  r  Lfl0  CI’  0•.  00•’  00  •Ca’  C’)  (N  U) (N  01 (N  ‘.0  ‘-I  ‘-4  r  0’U)  In  U)U)  Clfl Lnclr(I.. Or U)CN O•i.(I (N  0(1  In  (I ‘.0  O(1 U) (N  (N  Cl  In  Nfl (N  .  Lfl  Cl  Iir  CN .  In  0 fri  CN Lfl 0 0L. . ‘‘.0 U) 0 ‘.0 Cl (N  O’ 0O . . (IN In CN  In Cl  N Cl  (N N  (N  0 ‘.0  (1 ‘.0  0’ Lf)  0’ r1  Cl  Cl  U)  ,-  (N  (N  ‘-  -  -  r1  c  (N U) (N  It)’j’  rN O’.0 . ON U)  If) N • ‘.0  N(N N . Clc3’ 0 In 0 (N (N  CNU) .-1O’i . (NCl .4 (N  I.’,  (N U)In If) Cl0 (N  0  ClU)  .  .  ClLfl  U) U) ‘-I  ri U) c1::  ‘.0 (N  N U) (N  N ‘.0 (N  0  (N  CI) (N  U)  (N(N 00’  r  . ‘  Cl  0 (N  (N  ‘.0 ‘.0  U)  Cl -4  -I In  ‘Ca’  ‘  Cl  (N 0’  r4 ‘.0  r-1  (N  (N  1  ,4  r4  O  d 0  U) Cl0 0 U)0 -t U)N (N 0 -I  Clr-1 i.0  in N  N ‘.o-t U) 0 . . U)L()  o’.o (N(N  U)  ‘.0 ‘.ON N Nr-I  (NO  0’  CIa)  ‘Ca’  U)  Cl  t  r4  -1Cl ‘Ca’ -I  Cl  0  Nfl N -I  If) N  0 U)  El  N (Y)  Cl U)  El 0  H  El  H U)  -4U) (N  N  ‘Ca’ -I  0  ‘.0  .  0’.0  In  U) ‘-4  N N  -t  0 Cl  (N (N  N (N  0’  ‘  U)  0) in  0  ‘-4  r  ,1  1  d  0 d  (IN 0i Cl0 Cl ‘-I NN . Cl ‘-4 0”  (l(N U)(N  0’ CI)  ‘.0 Cl  U, ‘-t (N 00  CI’.0 (0  ‘  Lfl’.0  r-4N o  N  InLf 0  In  Cl (N  Lf)(N 0 ‘-I  (N  .-I 0’  Cl  (N ‘.0  Cl N  Cl  In (N  ‘  0  N 0  0  (N  -I  ,—t  1  ,—t  N0  ‘.0  (N  ‘.00  O(N  ‘.0 N  U) (NO U) -t’.0  ‘  ’ ‘-4(N 0 El  (Na)  ‘  U)(N  ‘  In  rCl  (Nfl  Cl  O N  0 U)  El  0  11.1  U)  InO 0’-1 . . (NIn Cl ,-1  In 0 ‘-I Cl  ‘.0  U)  0’  d  El  U) I-,  0’  El  f.Z4’.O fri  0  (NU)  . . U)’.0 0’  ,-40 OLn N  . ‘.0  0’.0 0 fri  frI  N  Cl  ‘.0  N  N  ‘-4  icx:  0 0  frI  a)  El  >Ca  r  fri  —.  Q4)  El4-i G)  ••—  0  0  44i  G)-P4 PIG) U)  Q4J a)  111  Ci) P4CQ PIG) —::  P4 P4W 44-)  W•r4 00)  U)  ‘-4 — ca -i  a) > 0  fr.  El4-I  P40)  P4W  P4W PIG)  ‘-  ‘-  0 -i.p  0  0)—I  0)—I  PIG)  00)  U)  U)  HG) r4 — Ca ti 0)  > 0  940)  940)  940) 940)  —c  ‘—t::  t44J  4J  O 0)—I 4-i P4 HG) 940) U)  (D-.-I  Ca  0  0  GE OW i-lU)  —i r1 Ca 4  0) > 0  131  12 11 10  z  w N -J  <5  3 2 I 81  114  162 231 328 GEOMETRIC MEAN OF REF/TARGET  466  114  162 231 328 GEOMETRIC MEAN OF REF/TARGET  466  12 11 10  z Ll LLI  N -J  3 2  I 81  Method of adjustment: interval Figure 20. Intervals of a 5th and minor transposition, Subject 7. Upper and lower error bars show standard 3rd. and lower deviations for the adjustment of the upper Refer to text for notes of the interval, respectively. further explanation.  132  12 11 10 Iii  4TH 28 LU in  7 z N6  :±f a —3 2 1 81  Figure 21. Subject 7.  162 231 328 114 GEOMETRIC MEAN OF REF/TARGET  466  interval transposition, Method of adjustment: Refer to text for explanation. Interval of a 4th.  133  In contrast, the results for the ascending 5th and 4th of S—lO  (Table 5;  Figure 22)  showed a compression of interval For example,  size at pulse rates above those of the standard.  the subjective 5th, which is normally equivalent to 7 semitones, was adjusted to a mean physical interval of 2.5 to 3.5 semitones, at pulse rates above the standard.  These  intervals correspond to approximately a minor 3rd,  for normal—  hearing musical subjects.  This tendency to compress large  ascending intervals at higher pulse rates was even greater than that observed for the same subject in Experiment III Figure 17).  At pulse rates above the standard,  preserve even the rank order of interval sizes. words,  at some pulse rates,  than 5ths.  S-b  (cf.  failed to  In other  4ths were adjusted to be larger  This apparent compression of interval size was  less consistent and less marked when the subject adjusted the lower note of the intervals than when he adjusted the upper note.  At low pulse rates, however,  S—b  transposed  approximately on a logarithmic scale, with roughly equal pulse rate ratios for equal musical intervals.  The interval size  compression did not appear to be electrode—specific,  as  virtually identical results for the ascending 5th and 4th were obtained on electrodes situated in the middle of the array, additional informal experiments. Subject 10 did not know any melodies incorporating a suitable ascending minor 3rd. minor 3rd was substituted. descending minor 3rd,  For this reason,  a descending  The results showed that for the  this subject,  like S-7 for ascending  in  134  U) lD -  cc’1H LflNU)  It).. aH  0 H  OO  ‘.D N  0  c  H  C)H (‘)C’ .  C\1 ‘  .  Qc’) It)  Lfl  Lfl  Q  N  N  O -  U)’DD CU)H Lfl . .  OH  C1  O C)N  0 0  .  OC’  CN  O  El  C.)  ri •  .,  Co  L() C)  H C’.)  C’ N  H  U)  H  C’.)  Co  0  H  C’.  C’.)  H  H  C)  OU)C’4  C)Q  .—  0 CNLflLfl  0ND .  •  .  -1  U)a El  .  Z  0  •  .  CN’.O  ‘  •  N CN  .DC’q  ri  cOCo OD  ..  -  U)Lt)0 H. • ‘.OC’4 cLfl c)H H  H  El  H U)  0  4 :l  •  Cl)  fl  g:  C,)  •  O  LflLC)  N C’.)  U)C’,  CoC’.) •  •  o’  r-i  NC’.)D U)OO H • • ‘.OC’) ON ‘H H  0 ‘.DIt)  c N  C’.)Cl  C’,  .-i  El  OoO  OLfl  •  .  co  Z 0  NC’IH  C’,  N It) .  !  c it) H  (‘,  C’.) H  H C’)  1’,  C’,  0  ‘.0 C’,  0  C’.)  H  H  H  H  H  H  N C’.)  0) CO  c’)H C’) 0 C’)  •.  •  .-.)N 0)  ‘.0  H.. Lfl  —  H  C’.)  El OLC)Lfl C’) N Ii) H.. Lfl’.0 C’)C’) 0)H  > El  z H  r.i  0  ‘  H  H  H  H  H  C’,OH  ‘.DH  ou  H ci’ ‘.0  •  .  H  O)HC’) O)C•  CoO  l  0  ‘.D’.OCO 0 H’. H  ‘.D’.D ‘.0  El  it)  ;• C’)  C’)  Ci)  C’)  0i.-I  C’) • H  •  C’.)  ‘3’  U) ID  rx4  W4W 4W i-i 4’0 >ci  E-i44-l (1)-d fri 4 Elq-P  ra  i  Eloac  N  O’.DN  Nit)  0 N 0 H  ‘.0  4JW CI) 4 WP  1%)  Cl) 4O)  G)  —a 0  44J  a’—i  0)4w  E 00)  HIDCl)  U)  PO)  H H (d 4  (1) > 0  4’—  0  0)—I fri E--14 ZWP4C) HIDU)  W 4 P  1) •— 0  4)-)  W-I E 00) -iU)  H H (  Ii) > 0  135  12 11 10 UI  z UI  u  7 z LU  81  12  114  162 231 328 GEOMETRIC MEAN OF REF/TARGET  466  -  11 10 UI  z  9 4Th  0  I  8  LU  z 7 UI  6 5 4 3 2 81  114  162 231 328 GEOMETRIC MEAN OF REF/TARGET  466  Figure 22. Method of adjustment: interval transposition, Subject 10. Ascending intervals of a 5th and a 4th. Refer to text for explanation.  136  intervals,  transposed on a logarithmic scale.  This suggested  that the apparent pitch compression of the intervals might be due either to interval size (i.e.,  in that perhaps smaller  intervals were easier to transpose than larger intervals), or to the directional properties of the intervals ascending versus descending intervals).  Thus,  (i.e., a further  assessment of this apparent pitch compression was undertaken The  by requiring this subject to transpose a descending 4th. experimental paradigm, reference and target pulse rates,  and  the treatment of the data were identical to those used with the ascending 4th. 6; Figure 23) 4th  The results for the descending 4th (Table  contrasted sharply with those for the ascending  (Figure 22), and showed an essentially logarithmic  transposition, with nearly constant pulse rate ratios throughout the range of pulse rates tested. The adjustment task for S-b,  at high pulse rates, was  complicated by the frequent perception of multiple pitches. Above 500 pps, notes”,  S-ic frequently observed hearing “double  consisting of both high-pitched (“like a pig  squealing”)  and low—pitched  (“like a cow mooing”)  components,  and became confused as to which of the pitches to use for tuning the interval.  Both S-7 and S-b  complained of an  inconsistency of pitches at high pulse rates, identical pairs of pulse trains  so that when two  (i.e., exemplifying identical  ratios) were presented within a few seconds of each other, subjects reported hearing two very different musical intervals.  137  Cl  0  ClOc’J  Cl  Cl  0  c’40 OCl  a)  OCl OOC  If)..  CN Cl-l OLt)  El I-)  •  a)  Q  t)  • Cl)  a)  Cl N  Cl C’  a)C\ OD  CN  CN Cl  Cl  r1  Cl)  a)N  It)  U  N  Lfl •  It).  •  It)  O  C’4  I.))  a)0  Cl  CU)  —%  —  —  —  Cl  0  CN  0  a) •  a) •  .-I •  N  :i.  0  — .-I -4  .—  oO)It)  a)  NU)LK) Cl a)Cl  icE:  00  1K)  Ca)  LX)  . rlLf) N  a)Cl  El  C’4  H  fl  a)C’1 Lfl0O) Cl a)Cl rN  Ca) N  .-4 N  CCl 0)  Cl  OCl Cl  C4  •  Cl  0  Cl a)  .  . 0  H 0 C) a) 0 (N  N1.00) occi • . 1.D  Cl a) • . 1.0  01K)  0  Oc’4 (N  C’4  1.0  0)  N N  CX) a)  0  0  0  00) Lfl00’l • •  N Lfl  Cl N  c’4C’4  a)CN c-i (N  C’1  0  . .-4  It)  Cl  c’i  .  C’4LK) (N  1.0  — fri  N  a)  0  0  N  0-4 0  1.0Lfl NLflO c-I • • 1.0Cl a)N  1.00  a)a)Cl  oH  . . Q ON  El H  a)  1.0(N  •  .  l  I—I  Cl)  0 • Cl)  .-I  Cl  0 1.0  c-I  .-I  0  (NN NIt)  — N 0) •  (N 1.0 •  0  0  .-.  El  —  Cl rI  0) 1.0  CN c-I  c-I  0  c-I  fl(N (Nc-IN c-I • •  tn  0) 0  OCl  a)(N 0  Cl  Cl  0)  (N  Cl  1.0  N  0  ‘-1  0Cl  01.0  0) 0)  Lfl  0  (N  0)  a)  Cl 0) •  0  ‘—  El  zH ..  (NOIt) Clc-I(N c-I • • Lfl 0)Cl O),-i  I4)(N O)Cl • •  0  0) Cd  0 •  z  (N  0) N •  Ci  Lf)Cl  0  c-I  0  c-I  Cl Cl  El  a)  I-) CI  ClO(N  Oa)  0)0  LU  0O  QCN 0)  NO)  a)  It) If)  Cl  rLl  o  El  CI  o El  —  04-3m  rzOjP4Co  P4W  P4W  0  0  El4  QCd  — CO pCO  ICdE-43-I4-) W-.-I r’  q-4P4E frWP4W fl0)  3.44-) W—4  CO HGJQ4(O  P4W  .-I -4 Cd 3  El34  >  HWP4G)  E  W  a)  0  OW  cd  0  E-l44.)  0  W.-4  t4IP4  P4CO  P40) -  r1  0  .-I  3.44) 0)—I  E  Cd 14 0)  a)  0  OW  138  12 11 V..  10  9 z w VI  7 z  4Th  w UI  z 3 2 81  114 162 231 328 GEOMETRIC MEAN OF REF/TARGET  466  12 11 UI  10  MINOR 3RD  Hft  z 3 2 81  162 231 114 328 GEOMETRIC MEAN OF REF/TARGET  466  Method of adjustment: interval transposition, Figure 23. Descending intervals of a 4th and a minor 3rd. Subject 10. Refer to text for explanation.  139  The results at the control pulse rate 114 pps) IV.  (geometric mean of  compared favourably with those obtained in Experiment  Subject 7 adjusted the size of the 5th and 4th to be (by  somewhat larger in Experiment V than in Experiment IV  approximately 1.7 semitones), whereas he adjusted the minor 3rds to be somewhat smaller than those in Experiment IV. the adjustments of S—b  the control pulse rate,  similar to those obtained in Experiment IV.  At  were also  For this subject,  the mean of the adjustments for all three intervals in Experiment V at the control pulse rate differed by only 0.23 semitones from that in Experiment IV, exceeded 0.47 semitones.  and no differences  It should be remembered that the  adjustments of Experiment IV were obtained without feedback, and the close correspondence between these two sets of results for S-b  more closely approximates the replicability which  would be expected of musically trained subjects adjusting musical tones.  The standard deviations obtained at the  control pulse rate for both subjects in Experiment V were also similar to those obtained in Experiment IV. Some systematic differences in interval size were observed which appeared to be related to whether the subject adjusted the upper or the lower note of the intervals 24).  At the lowest reference pulse rates,  (Figure  both subjects  tended to create larger intervals when adjusting the upper note of the intervals, resulting in a slight overestimation of interval size.  This trend was similar for ascending and  descending intervals.  These discrepancies in interval size  140  3 2 uji  z  0 uJ  -1  LI  z  -2  z LU  -3  LU LI  -5 -6  8- ASC. 5TH  --  ASC. 4TH  --  ASC.MINOR 3  3 2  v1 LU  z 0  i-0 LU  u -1  z  UI ‘-I  z LU LU U LI  -  -5 -6 114 162 231 GEOMETRIC MEAN OF REF/TARGET -B- ASC. 5TH  --  A5C. 4TH  -G- DESC. 4TH  -a--  DESC. MINOR 3RD  Figure 24. Method of adjustment: mean interval size with adjustment of the lower note minus mean interval size with adjustment of the upper note, for Subjects 7 and 10, for a variety of intervals. Positive values signify larger intervals when the subjects adjusted the upper note of the intervals. Negative values signify larger intervals when subjects adjusted the lower note of the intervals. Because each interval had different reference and target pulse rates but a common geometric mean pulse rate, results were plotted as a function of the geometric mean of the target and reference pulse rates.  141  may have resulted from attempts by the subjects to effect a more musical sound quality.  For example, when adjusting the  lower note at very low pulse rates,  subjects tended to avoid  rates much below 100 pps, which were described by both subjects as “rough”,  “intermittent”,  “like a pig grunting”.  “like a motorbike” or  This may have resulted in an  underestimation of interval size at very low pulse rates, when the lower note was adjusted, and an overestimation when the upper note was adjusted.  At higher reference pulse rates,  both subjects tended to create larger intervals when they adjusted the lower note. larger for S-b, intervals.  The magnitude of this effect was  especially with wider ascending pitch  These discrepancies may have resulted from a  tendency to avoid higher pulse rates, which both subjects described as “unpleasant”, and “shrill”. The variability in the adjustment data of each subject was examined by computing the standard deviation, semitones,  in  from each subject’s mean pulse rate setting for  each interval, at each reference pulse rate.  Recall that the  standard deviations obtained with the adjustment of the upper note of the intervals are plotted in Figures 20 to 23 as the upper vertical error bars,  and those obtained with the  adjustment of the lower note as the lower error bars.  These  standard deviations were typically less than 2 semitones.  For  S-7, the standard deviations for the 5th showed a marked increase at the highest pulse rates tested.  The standard  deviations obtained with the adjustment of the upper notes of  142  the intervals did not vary systematically from those obtained In  with the adjustment of the lower notes of the intervals.  variability tended to decrease at the higher pulse  contrast,  The  rates for the ascending interval adjustments of S-lO. fair replicability of adjustments for some intervals minor 3rd for both subjects; 4th for S-7), rates, was unexpected,  (e.g.  even at high pulse  in view of frequent comments by both  subjects regarding the difficulty of the adjustment task at high pulse rates. Variability also tended to increase with interval size For S-7, the standard deviation,  (Tables 4-6).  lowest pulse rates,  percentage of target interval size the 5th,  4th,  1.48  averaged 1.87 semitones for the 5th,  When measured as a  and .95 for the minor 3rd.  for the 4th,  at the 4  (7,  5,  and minor 3rd, respectively)  deviations represented 27% of a 5th,  and 3 semitones for these standard  30% of a 4th,  and 32% of  Thus the amount of variability appeared to  the minor 3rd.  represent approximately a constant proportion of the interval size.  For S—b,  there appeared to be no relationship between  interval size and magnitude of the standard deviation.  It  should also be noted that smaller standard deviations were not necessarily associated with a greater proximity to the target pulse rates. in fact,  Some of the most consistent adjustments of S—b,  occurred at higher pulse rates, where mean interval  size showed the greatest deviation from the target pulse rate and the results showed the poorest preservation of interval rank size.  Thus,  our data suggest that measures of  143  variability may,  at least in some circumstances, not provide  an accurate index of pitch salience. The proximity of individual adjustments to the target pulse rate was also examined by calculating, intervals,  for the ascending  the percentage of trials which yielded responses  within a reasonable proximity (2 semitones or less) targets.  This analysis  (Figure 25)  of the  further confirmed that the  adjustments of S-7 in all but the highest pulse rate regions fell within a narrow range of the target,  and that accuracy  was greater for the minor 3rd than for the 4th and 5th.  Only  for the 5th was there a trend towards decreased accuracy at higher pulse rates.  For S—b, the compression of larger  ascending intervals at higher pulse rates was reflected in a marked decrease in the percentage of adjustments within close proximity of the target.  For the minor 3rd, performance was  comparable for both subjects.  DISCUSSION  Musical intervals are ratio—specified. musicians,  Thus,  for  and under some circumstances for nonmusicians  (Attneave and Olson 1971), within a musical context, range over which musical pitches are perceived,  in the  acoustical  tones in equal frequency ratios produce approximately equal musical pitch intervals.  Subject 7 transposed the pitch  intervals to different pulse rates with a reasonable degree of accuracy,  and did so in a manner similar to that observed when  144  ASCENDING 5TH  ‘.1  a-  ASCENDING 4TH 100  80  60  40  20  MINOR 3RD  100  80  60  40  20  74/88  105/125 148/176 212/iSa 298/354 PULSE RATES OF ANCHOR AND TARGET I SEMITONE  4251505  2 SEMITONES  Figure 25. Interval transposition experiment: Subjects 7 and 10. Percentage of pulse rate adjustments 1—2 semitones or less from the target, for three intervals, over a range of reference and target pulse rates.  145  normal—hearing, nonmusical subjects are required to transpose a well—known melody into different frequency ranges and Olson 1971).  Thus,  for this subject,  (Attneave  equal musical pitch  intervals were characterized by equal ratios of pulse rates. This appears to be clear evidence of pitch percepts which permit ratio—governed musical interval recognition.  Pitch  percepts can be ranked and scaled ordinally on a high—to—low continuum,  but this does not necessarily imply that they vary  with the stimulus in such a manner that stimulus ratios can be adjusted with accuracy to the target intervals.  In other  words, while musical interval recognition or production implies the ability of subjects to make ordinal pitch judgements, the reverse is not necessarily true (Houtsma 1984).  Subject 10 also transposed on a ratio scale, but did  so consistently only for descending intervals.  For ascending  intervals at high pulse rates, this subject appeared to be able to make ordinal pitch judgements,  in that the ordinal  properties of the intervals he adjusted were generally correct (i.e.,  ascending intervals were generally adjusted so that the  pulse rate of the upper note was higher than that of the lower note of the interval).  However, at high pulse rates, he  appeared to be unable to adjust pulse rates to the target interval values when the intervals were large and ascending. Perception of small pulse rate ratios as large subjective intervals, S—b  or compression of interval size, as observed with  (for ascending intervals), could be interpreted to  suggest a rapid increase in pitch with increases in pulse  146  such as that anecdotally reported by Shannon (1983).  rate,  However, the data for descending intervals do not support this notion,  and suggest that the apparent increase in pitch may be  due to other factors such as, timbre  (e.g.,  for example, marked changes in  an increase in shrillness,  sharpness,  unpleasantness), which may covary with pitch,  or  as pulse rate is  Changes in timbre or other attributes of a sound have  varied.  been shown to complicate the pitch adjustment process, especially with wide, Viemeister 1981). theory,  ascending intervals  (Burns and  Certainly, from the perspective of music  differential processing of ascending and descending  intervals,  at identical frequencies, would appear unlikely  (Deutsch 1969). Compression of musical interval size has also been observed in normal—hearing musical subjects. Olson (1971),  Attneave and  for example, reported that the two musical  subjects in their first experiment transposed simple 2—note pure tone synthetic melodies on a logarithmic scale over the first 5 octaves.  However, when the upper frequencies exceeded  5 kHz, both subjects showed a consistent compression of some of the larger intervals, to the extent that these intervals were adjusted to less than half their original size.  Smaller  intervals that were wholly below 5 kHz were not significantly affected.  Their data for ascending and descending intervals  were pooled, precluding assessment of effects of interval direction.  The authors hypothesized that the subjects may  have become confused when one tone of a pair was within the  147  musical pitch range,  and the other above it.  These authors  noted that this compressive aberration recovered partially in the octave from approximately 6 to 12 kHz, when both tones lacked musical quality. The standard deviations observed in our data for S—7 and S—b  (descending intervals)  appeared,  on average,  larger than  the deviations reported in the second experiment of Attneave and Olson  (1971),  in which normal—hearing nonmusical subjects  adjusted pure tones in the transposition of a familiar, direct comparison of the  However,  overlearned melody.  variability in our data with theirs is difficult,  as these  investigators reported only the mean deviations from the median frequency adjustment,  and provided no information  regarding the statistical distribution of their data. (1971)  mean deviations reported by Attneave and Olson approximately 0.73 semitones for the major 6th, semitones for the major 3rd, Hz to 4186 Hz. 7,  In comparison,  and 0.45  the standard deviations for S averaged 1.87 semitones for  1.48 semitones for the 4th,  the minor 3rd.  and 0.95 semitones for  The variability in the data for S-b,  descending intervals,  averaged  in the frequency range from 92.5  up to approximately 300 pps,  the 5th,  The  with  was in the order of 1 to 1.2 semitones.  The differences between our results and those of acoustical experiments should not be surprising, number of factors,  and could be due to a  including musical aptitude and musical  experience of our subjects prior to deafness,  or the accuracy  of their auditory memory for interval sizes following  148  prolonged auditory deprivation.  Differences between neuronal  response patterns to electrical and acoustical stimulation may also be expected to be associated with qualitative differences between pitch percepts in normal—hearing subjects and electrically stimulated subjects. The standard deviations obtained for our electrically stimulated subjects were also greater than those obtained by Burns and Viemeister  (1976,  1981) with musically trained  subjects and sinusoidally amplitude—modulated noise, low modulation frequencies.  even at  These authors reported that,  while their subjects performed well on a musical interval identification test when the intervals were separated by a minor 3rd (i.e., when the response set consisted of a minor 3rd,  a tritone,  a major 6th and an octave), they scored much  lower when the intervals in the response set were separated by only a semitone major 3rd).  (i.e.,  semitone, major 2nd, minor 3rd,  Burns and Viemeister (1976)  and a  concluded that  musical interval identification with amplitude-modulated noise, which provides predominantly or exclusively temporal information, was accurate to within approximately one semitone.  Comparable deviations,  in the order of one  semitone, were subsequently obtained by the same authors, using the method of adjustment, with highly trained musicians (Burns and Viemeister 1981).  The most obvious potential  explanation for the larger standard deviations in our data is the difference in musical experience of the subjects. 10 may have had a musical aptitude, but had received no  Subject  149  musical training,  and S—7 reported no particular interest in  music prior to deafness.  Additional explanations for these  differences could be sought in either the quality of the sounds heard by the electrically stimulated subjects, the accuracy of their memory for interval size,  or in  following  prolonged auditory deprivation. The trend towards increased variability of the data at higher pulse rates has some parallels in the results of both acoustical and physiological experiments. (1971)  Attneave and Olson  showed a sharp increase in the mean deviation from the  correct frequency adjustments when acoustical frequencies Burns and Viemeister  exceeded 5000 Hz.  (1976)  reported that  musically trained subjects listening to amplitude-modulated noise at low frequencies  (with a first—note modulation  frequency of 84 Hz), were able to identify musical intervals separated by minor 3rds, with an accuracy of 84%.  While the  upper limits of these temporally mediated pitches for some subjects were reported to be as high as 800 to 1000 Hz,  marked  performance decrements were generally observed for modulation rates above 300 Hz. Physiological data suggest that,  for both acoustical and  electrical stimulation, the upper limit of the range over which musical interval recognition is accurate corresponds approximately to the upper limit at which the stimulus periods are accurately preserved in the temporal patterning of nerve fiber responses.  Thus, with acoustical stimuli,  5000 Hz is  approximately the upper limit for the phase-locking of  150  auditory neurons to the stimulus frequency  (Rose et al.  Data from electrically stimulated auditory neurons  1967).  have shown cycle—for—cycle firing patterns, up to rates of 600—900 spikes per second (Kiang and Moxon 1972; Hartmann, Topp,  and Klinke 1984).  At higher pulse rates,  the temporal  characteristics of the neuronal responses to electrical stimuli are known to become increasingly dominated by interactions between the refractory period of the neurons and the amount of charge delivered during the excitatory portion of the stimulus cycle representation,  (Parkins 1989).  at higher pulse rates,  The increasing of interspike intervals  related to the refractory status of the neurons rather than to the stimulus period,  or multiples thereof, may account for the  weakening of pitch at higher pulse rates. The upper limits of pitch perception with electrical stimulation have been shown to be highly variable between While some studies have reported rate—based pitch  subjects.  discrimination to be restricted to frequencies below 400-500 Hz  (Sachs 1983;  limits.  Simmons 1983),  others have shown higher  Tong and Clark (1985),  for example,  reported that 2  out of 6 implanted listeners accurately labelled pulse rates up to at least 600-1000 pps, while two other listeners saturated at 200-400 pps.  Furthermore, the upper limit of  pitch labelling ability was not found to correlate with the musical history of the subjects.  Others have reported  frequency discrimination in implanted listeners up to 1000 Hz (Spillmann,  Dillier,  and Guentensperger 1982; Hochinair-Desoyer  151  et al.  1983),  or even 2000 Hz  (Bilger 1977).  While it is not  possible to assess the adequacy of the loudness equalization procedures used in these experiments,  our results are  consistent with data that show ordinal ranking of pitch to be possible,  at least in some implanted subjects, up to at least  600 to 1000 pps.  The increased amount of variability in our  transposition data at pulse rates above approximately 300-400 pps suggest that the ratio relations of the stimulus pulse rates became increasingly difficult to extract.  The results  also suggest that judgements of stimulus ratios were possible in only a portion of the range of pulse rates over which ordinal pitch judgements were possible.  152  GENERAL DISCUSSION  While physiological evidence has shown that the discharges of nerve fibers in the peripheral and lower central portions of the auditory system are highly synchronized to acoustical frequencies below 4-5 kHz  (Rose et al.  1967; Kiang  and Moxon 1972; Javel 1980; Greenberg and Rhode 1990),  there  remains uncertainty whether the human brain actually utilizes this temporal information in the analysis of pitch Mott 1988).  Thus,  (Javel and  at least for some types of stimuli which  yield musical pitch sensations,  recordings from single  auditory nerve fibers have shown that the temporal discharge patterns contain sufficient information to account for many pitch—related phenomena. cat,  Evans  (1978)  reported that,  in the  interspike interval histograms obtained in response to  inharmonic stimuli, which in humans are known to evoke pitch shifts, were distinguishable from those obtained with their harmonic counterparts.  In other words,  the interspike  intervals measured were consistent with the period of the pitches reported by humans. data showing,  Javel  (1980,  1988)  has presented  in the interspike interval histograms of single  cochlear nerve fibers of the chinchilla, a representation of intervals corresponding not only to the period of the perceived pitch of both harmonic and inharmonic stimuli, also,  in the predicted proportions, to other temporal  but  153  intervals present in the fine structure of the waveform. These data suggest that the discharge patterns of auditory neurons may contain sufficient information for the analysis of pitch on a temporal basis. Greenberg (1980) (1987)  and Greenberg, Marsh,  Brown,  and Smith  have presented electrophysiological evidence,  scalp—recorded frequency—following—responses (FFR) subjects,  for a temporal processing of pitch.  in the  of human  The spectral  analyses of the FFR recordings obtained in response to a range of harmonic stimuli, with and without fundamental frequency component,  as well as of inharmonic stimuli were found to be  similar to FFR5 generated in response to pure tones equal in frequency to the perceived pitch.  These physiological data  suggest that the pitch of complex tones may be encoded in the temporal discharge patterns of neurons not only at the level of the cochlear nerve,  but also at higher levels within the  auditory brainstem. Psychophysical data from acoustical experiments support the conclusion that,  under certain experimental conditions,  temporal information alone can convey pitch,  even after  appropriate precautions have been taken to eliminate the possible role of distortion products and switching transients. Stimuli consisting solely of high,  unresolvable harmonics have  been found to evoke residue pitches corresponding to the absent fundamental frequency Ritsma 1962; Moore 1973,  (Schouten 1940; de Boer 1956;  1977; Hoekstra and Ritsma 1977;  Houtsma and Smurzyns]ci 1990; Pierce 1991).  Stimuli consisting  154  of amplitude—modulated white noise have also been shown to convey pitch information (Miller and Taylor 1948; Harris 1963;  Small 1955;  Pollack 1969; Patterson and Johnson-Davies 1977)  even under conditions of bandpass filtering and band—reject masking.  Further evidence for temporal mechanisms in pitch  perception can be found in time separation and phase—shift phenomena.  Small and McClellan (1963),  for example,  demonstrated that when a pulse train is added to a delayed replica of itself,  a pitch is heard which corresponds to the  reciprocal of the time delay  (t)  between the two pulse trains,  even though there is no energy in the frequency spectrum at l/t.  Cramer and Huggins  (1958)  showed that dichotic  presentation of white noise of which a small frequency region presented to one ear had been phase—shifted,  resulted in the  perception of a faint pitch corresponding to the frequency of the phase transition. Few studies have addressed musical salience of temporally based pitches.  Burns and Viemeister  (1976)  demonstrated that  the pitches evoked by sinusoidally amplitude-modulated noise were sufficiently salient to enable musically naive subjects to recognize simple melodies.  They also showed that musically  trained subjects were able to identify musical intervals, when the constituent “notes” corresponded to low modulation frequencies.  Recognition with amplitude—modulated noise,  however, was found to be consistently inferior to that achieved with pure tones.  The weakness of the pitches  associated with such stimuli has been hypothesized to result  155  from random fluctuations in the input waveform of the noise. These fluctuations,  in turn,  could be expected to result in  irregularities in interspike intervals, greater than those obtained with tonal stimuli Similar conclusions,  (Moore and Glasberg 1986).  confirming that pitches which presumably  arise solely from the temporal aspects of the waveform are sufficiently salient to convey musical interval information have been reported by others  (Houtsma and Smurzynski 1990;  It appears generally agreed, however,  Pierce 1991).  that  these pitches are weaker than those resulting from tonal stimulation  (Burns and Viemeister 1976,  1981; Houtsma 1984;  Houtsma and Smurzynski 1990). However, with acoustical experiments, impossible to eliminate, unequivocally,  it is difficult or  the contributions of  spectral or place mechanisms in the analysis of pitch. been argued,  for example,  It has  that with amplitude—modulated noise,  short—term spectral fluctuations related to the modulation rate may contribute to pitch perception (Pierce, Cheetham 1977).  Lipes,  and  While these alterations in the statistical  properties of the noise are thought to be small and of limited significance in demonstrations of temporally based pitch, particularly with sinusoidal modulation (Moore and Glasberg 1986),  these possibilities introduce elements of uncertainty  regarding direct utilization of temporal information.  Further  doubts arise from the potential contributions of distortion products generated within the ear.  Thus,  spectral peaks not  156  present in the stimulus may be generated by cochlear nonlinearities  (Horst,  Javel,  and Farley 1990).  In electrically stimulated cochleas,  there is believed to  be no mechanical, place—related frequency analysis of the These ears, therefore,  stimulus waveform.  represent unique  opportunities for the study of temporal mechanisms in audition.  Electrical pulse trains in deaf cochleas are known  to precipitate highly synchronized temporal spike discharge patterns in the target neural population Klinke 1984; Javel et al. Stypulkowski 1987b).  (Hartmann,  Topp,  and  1987; van den Honert and  While many psychophysical studies have  documented basic relationships between electrical frequency or pulse rate and pitch et al. 1983;  1979,  (Eddington et al.  1981; Tong et al.  1978a,  l978b;  Simmons  1982; Hochmair-Desoyer et al.  Shannon 1983; Pfingst 1985; Tong and Clark 1983,  Townshend et al.  1987;  Shallop et al.  1990),  1985;  these  temporally based pitch percepts have not generally been assessed within a meaningful purpose of our experiments,  (i.e., musical)  context.  For the  pitch was narrowly defined as that  attribute of auditory sensation which conveys musical interval information.  Such a definition of pitch is considered to be  both conservative and realistic  (Burns and Viemeister 1976,  1981; Houtsma 1984). Our results support the findings of acoustical studies which have demonstrated temporally based pitches,  and suggest  that pulsatile electrical stimulation of totally deaf ears may,  at least in some subjects,  result in pitches sufficiently  157  strong to support musical interval perception. that,  It is expected  in view of the well-documented relationships between the  temporal characteristics of the electrical stimulus and the temporal characteristics of the rieuronal responses in such ears  (Hartmann, Topp,  and Klinke 1984; Javel et al.  1987; van  den Honert and Stypulkowski 1987b), that these pitch effects must be based on an analysis of interspike interval information.  Additional evidence for the ability of implanted  subjects to utilize interspike intervals in the analysis of pitch has been obtained from experiments utilizing stochastic pulse trains.  Dobie and Dillier (1985)  showed that the  ability of deaf subjects to discriminate pitches remained intact when the probability of pulse delivery at each prescribed time in the pulse train remained greater than 0.5. These data suggest that the auditory nervous system is capable of making perceptual decisions about pitch on the basis of interspike intervals.  While the possibility of excitation of  neurons by means of electrophonic vibrations,  such as might be  hypothesized to result from electrical stimulation of intact outer hair cells,  or by means of direct depolarization of  residual inner hair cells cannot be entirely ruled out, these mechanisms appear to be highly unlikely in ears with no residual acoustic sensitivity (van den Honert and Stypulkowski 1984,  1987b) These conclusions are not intended to imply that temporal  information is the sole mediator of musical pitch or musical interval recognition, or even that the rate—based subjective  158  musical intervals reported in this paper are immediately During some of our  obvious to all implanted listeners. informal pilot experiments,  some listeners commented that  there was little or no noticeable change in pitch as pulse Others commented that a slow apical—to—basal  rate was varied. electrode sweep  (at one electrode per second, using a constant  pulse rate of 125 pps)  produced a stronger, more noticeable  rise in pitch than a slow pulse rate sweep on a single electrode  (one pulse rate per second, using pulse rates which  increased from 54 pps to 1096 pps, Furthermore,  by a factor of 1.22).  informal pilot studies demonstrated that subjects  appeared to be able to make intonation quality judgements also on the basis of variation in the place of stimulation by switching from one electrode to another, pulse rate).  All subjects,  (i.e.,  at a constant  including those who had  considerable experience listening to pulse rate melodies on basal electrodes,  agreed that melodies sounded more “musical”  and “pleasant” on apical electrodes.  This suggests that a  congruence between place and rate information may be important in achieving a musical sound quality  (Evans 1978).  In Nucleus  implant recipients with complete electrode insertions,  the  locations of the most apical and basal electrodes have been estimated to correspond, respectively, to the 600 and 8000 Hz regions of the normal cochlea Dowell,  Brown,  Clark,  (Greenwood 1961,  1990; Blamey,  and Seligman 1987).  Experiment I showed that many familiar tunes were readily recognized and identified by most subjects,  especially when  159  these tunes were played on a single intracochlear electrode, and with intact rhythmical patterns.  at low pulse rates,  This  contrasts with the poor tune recognition results reported for These  Nucleus subjects by Gfeller and Lansing (1991). subjects, however, were tested in the soundfield,  and  stimulation was delivered by the body—worn processor, which performed a feature extraction upon the incoming acoustic signals  (Koch et al.  1990).  This processing strategy  extracts from the acoustic signal the spectral peaks with the highest amplitude,  and quasi-simultaneously activates  electrodes selected on the basis of a place code, at a pulse rate equal to the fundamental frequency.  It is conceivable  that such a processing strategy could result in a confounding of pitch information based on temporal features and pitch information based on place of stimulation.  Our results  suggest that the application of a systematic series of pulse rates to a single intracochlear electrode may yield more robust rate pitches. Experiment II demonstrated that systematic variation of the electrical pulse rate resulted in pitches sufficiently strong to support tune recognition, rhythmical information.  even in the absence of  While variation of pulse rates  appeared to permit pure melody (i.e., musical interval sequence)  recognition, the contribution of melodic contour  remained undetermined. rates for all subjects.  Performance was best at low pulse Although subjects scored  significantly above chance at high pulse rates,  they reported  160  that the melodies “sounded like music” only at the lowest pulse rates. furthermore,  The results of an ancillary informal experiment, suggested that at least one of the subjects  processed the melodies in a musical way only at low pulse rates.  This subject appeared to remain oblivious to the  presence of an unfamiliar tune in the stimulus set until this tune was played at a low pulse rate. possibility that,  This suggests the  at least at high pulse rates,  have been attending to the melodic contour,  subjects may  or applying  complex, nonmusical cognitive strategies, by listening for systematic changes in nonpitch attributes. Large individual differences between the subjects in the ability to utilize temporal information,  such as those  reported here, are not unique to electrically stimulated subjects.  Large intersubject differences in performance on  acoustical pitch perception tasks are common, normal-hearing subjects  even with  (Burns and Viemeister 1976,  Similarly, Tong and Clark (1985)  1981).  showed large differences in  the ability of implanted patients to perform pulse rate identification tasks. cutoff rates, perceived,  These authors concluded that the upper  beyond which no change in pitch could be  did not bear any relation to the degree of previous  musical training.  For S-il, the task appeared to be extremely  difficult, even at low pulse rates.  This subject appeared to  require more time per trial than the other two subjects,  in  all the experiments, and evidenced a significantly greater performance decrement with increasing pulse rate.  It is  161  possible that this subject had poor nerve survival relative to the other two subjects, meningitis  as a result of deafness secondary to  (Nadol and Hsu 1991).  It is of interest to  speculate whether there are any relationships between the ability to utilize temporal information, those reported here,  as in tasks such as  and speech perception results.  Longitudinal comparison of speech perception scores showed that S—7 and S-b  achieved high scores on open-set speech  materials within 6 weeks of being fitted with their processors, whereas S—li required an extended learning period to reach an asymptotic,  although comparable score.  A number  of reports in the literature have supported the possibility of a correlation between temporal measures such as gap detection thresholds and the performance on speech perception tests (Tyler, Moore,  and Kuk 1989; Cazals,  Montandon 1991; Blarney et al.  Pelizzone, Kasper,  and  1992).  The results of Experiment III indicated that musical interval information was available in the memory of the implanted listeners,  and that the ratio relationships  appropriate for acoustical musical tones were at least grossly correct for electrical repetition rates.  This means that the  neural responses to different pulse rates permit both ordinal ranking of pitches as well as ratio information. rates,  At low pulse  the intonation quality judgements demonstrated that all  subjects preferred interval sizes which consisted of pulse rates in the same ratios which characterize musical intervals heard acoustically.  At higher pulse rates,  subjects tended,  162  to varying degrees, to compress the size of large intervals. It is difficult to determine,  on the basis of these results,  whether this apparent compression of interval size was a manifestation of a rapid increase in pitch, anecdotally reported by Shannon  (1983).  such as that  It is also  conceivable that this phenomenon was attributable to marked changes in timbre,  or to other attributes of the sound,  which  might serve to “exaggerate” the subjective size of the physical interval.  Compression of interval size at high  acoustic frequencies has been observed in normal—hearing musical subjects performing a musical interval adjustment task,  by Attneave and Olson  (1971),  tone frequencies exceeded 5000 Hz.  when one of the two pure This compression was much  less marked when both stimulus frequencies were above 5000 Hz. The results of Experiment III also showed that, regardless of the cochlear location of the stimulating electrode and  (presumably)  the responding neural population,  equal ratios of pulse rates appeared to yield approximately equal musical pitch intervals, at least for the interval and repetition rates examined.  Psychophysical and physiological  data support the notion that electrical stimulation via closely spaced bipolar electrode pairs in the scala tympani can result in the excitation of relatively restricted groups of neurons l987a). survival,  (Shannon 1983; van den Honert and Stypulkowski  The spread of excitation,  in patients with good nerve  has been estimated at 2—4 mm,  listening level  (Black,  Clark,  at a comfortable  and Patrick 1981; Merzenich  163  1983; Tong and Clark 1986).  Data regarding spatial  localization of current within implanted cochleas have been extensively corroborated by anecdotal clinical observations regarding gradations of pitchlike effects which are related to the place of electrical stimulation (Muller 1983).  These  effects are generally in accordance with the tonotopic organization of the cochlea (Tong and Clark 1985). Even though equal pulse rate ratios applied to different electrodes appeared to yield approximately equal subjective musical intervals,  subjects reported the pitch changes to be  more difficult to hear and to sound less musical on basal than on apical electrodes.  In spite of the anecdotal observations  of the subjects regarding the greater difficulty of the intonation quality labeling task with stimulation on basal electrodes, there were no consistent differences in standard between these electrode locations.  deviations,  These findings  are in accord with results of acoustical experiments, which suggest that stimulation of a given region of the basilar membrane may give rise to different pitches, depending on the temporal patterning nerve impulses Silverman, 1964).  (Schouten 1940; Davis,  and McAuliffe 1951; Ritsma 1962; Ritsma and Engel  These phenomena have been extensively documented in  the psychoacoustical literature. The difficulties encountered in assigning pitches to stimuli which consist of conflicting temporal and place information,  such as those reported here with basal  electrodes, have also been reported in studies with normal—  164  For example, Davis et al.  hearing listeners.  (1951)  demonstrated that when subjects were required to match the pitch of a pure tone to that of 2 kHz rectangularly gated pulses, presented at rates of 90 to 150 times per second and bandpassed through a 2 kHz filter, the matches of some subjects corresponded to the center frequency of the filter (i.e.,  2 kHz), while the matches of others corresponded to the  repetition rate.  Similarly, Burns and Viemeister (1981)  demonstrated relatively poorer performance with a sinusoidally amplitude-modulated 10 kHz acoustic sine wave than with a variety of broadband and bandpass amplitude—modulated noises, on tasks of melody identification, dictation, interval adjustment. listeners.  and musical  These differences were large for some  It is possible that the strength of temporally  based pitches declines in proportion to the mismatch between place and timing information.  The difficulty of discerning  pitch changes on basal electrodes, as anecdotally observed by our subjects, may explain the findings of Shallop et al. (1990), who reported proportionately smaller pitch changes with variation of electrical pulse rates on basal than on apical electrodes.  Marked qualitative differences in the  sound evoked by electrodes in different cochlear regions may also explain the findings of Eddington et al.  (1978a), who  reported that melodies which were identifiable when played on some intracochlear electrodes were not identifiable on others. Our limited data do not support the hypothesis of Eddington et al.  (l978a)  that different electrical pulse rate or frequency  165  ratios might be required to produce recognizable tunes in different regions of the cochlea. The interval adjustment task of Experiment IV showed that at least two of the three subjects were able to adjust low electrical pulse rates in the reconstruction of common musical intervals, when these intervals were abstracted from familiar tunes.  The ratio properties of these intervals closely  approximated those of the corresponding acoustical musical intervals. The fact that the precision with which our subjects performed the musical interval adjustment task did not correlate with the musical history of the subjects is not entirely surprising.  Clinical studies have amply documented  large individual differences in the speech perception abilities of patients with cochlear implants, unrelated to individual linguistic skills.  Thus,  individual differences in  musical pitch perception, unrelated to musical ability, also be anticipated.  could  Furthermore, musical ability or aptitude  is difficult to define and measure, and may fail to correlate with musical achievement, example,  interest,  or experience.  For  tonal memory has been reported to be possessed by  different subjects in widely varying degrees, which are not necessarily related to length of musical experience Dyson 1982).  (Shuter—  Musical achievement and interest are believed to  relate in a complex fashion to genetic differences between individuals in their capacities for building up appropriate neural and mental schemata, and the stimulation offered within  166  the environment.  The latter,  in turn,  is strongly influenced  by cultural conditions and individual leisure pursuits (Shuter—Dyson 1982).  Conversely,  it must be remembered that  some individuals who do possess a level of musical achievement (Table 1)  have neither musical interest nor aptitude.  The results of the interval transposition task in Experiment V were particularly interesting.  It must be  recognized that some normal—hearing, musically untrained subjects have been shown to be unable to perform this type of task with musical tones  (Attneave and Olson 1971).  Subject 7  transposed all three intervals on an essentially logarithmic scale, where equal musical intervals were represented by equal These results were comparable to those  ratios of pulse rates.  of the second experiment of Attneave and Olson  (1971), whose  nonmusician subjects were required to reconstruct a familiar melody using pure tones.  Our results suggest that low  electrical pulse rate pitches, when judged in a musical context, possess ratio properties similar to those of acoustical musical tones. in our data, Olson  The apparently larger variability  compared to that of the subjects of Attneave and  (1971), may originate from a variety of sources,  including differences in the musical ability of the subjects, the accuracy of their memory for melodic pitch intervals,  the  difficulty of the transposition task, the effects of prolonged auditory deprivation,  and the quality of the sounds perceived  with pulsatile electrical stimulation.  167  In the transposition behaviour of Subject 10,  the  directional properties of the intervals appeared to be critical.  Thus,  ascending intervals were adjusted differently  from descending intervals of the same magnitude,  even when  reference and target pulse rates of the ascending and descending intervals were identical.  Ascending intervals were  adjusted on an essentially logarithmic scale for low pulse rates.  At higher pulse rates,  became severely compressed,  larger ascending intervals  and interval sizes  undifferentiated, especially when the subject adjusted the upper note of the intervals.  In marked contrast, descending  intervals were adjusted on an essentially logarithmic scale, with fairly constant pulse rate ratios over the range of pulse rates tested  (similar to S—7).  These marked discrepancies  between the results for ascending and descending intervals are difficult to explain.  It is possible that marked changes in  attributes such as shrillness or unpleasantness, which may have accompanied changes in pulse rate, may have obscured the pitch percepts and complicated selectively the adjustment of ascending pitch intervals,  and that these qualitative changes  were less obtrusive with descending pitch intervals,  or when  the subject adjusted the lower note of ascending pitch intervals. While our experiments did not specifically address the limits of the existence region of electrical pulse rate pitch, significant performance decrements were observed at pulse rates above 200-800 pps,  in Experiments II,  III and V.  168  Similar upper limits for the processing of temporal information have been demonstrated in acoustical experiments (Burns and Viemeister 1976; Javel and Mott 1988)  and in other  psychophysical experiments with electrically stimulated subjects  (Shannon 1983; Tong and Clark 1985;  Shannon 1992).  On a physiological level, the ability of individual neurons to respond with temporally patterned firing at high electric pulse rates is limited by the relative refractory period, which has been reported to begin at 300 sec following spike onset, Topp,  and to extend to at least 5 msec  (Moxon 1971; Hartmann,  Stypulkowski and van den Honert 1984).  and Klinke 1984;  Single fiber recordings have demonstrated strongly unimodal interval histograms at electrical frequencies up to 200 Hz, indicating a firing on every cycle of the stimulus, at levels well above threshold Stypulkowski l987b).  especially  (van den Honert and  In these studies,  neural responses at  frequencies above 200—500 Hz were characterized by an increasing representation of interspike intervals at multiples of the stimulus period. Thus,  at the relatively high pulse rates used in some of  our experiments,  it is unlikely that individual nerve fibers  were responding on a cycle—for—cycle basis to the electric stimulus.  It is more likely that at these rates,  there was an  increasing representation of interspike intervals at multiples of the stimulus period.  Psychophysical studies involving  stochastic electrical pulse trains have shown that frequency discrimination can remain essentially unimpaired, even when  169  every period of the test frequency is not represented in the stimulus  (Dobie and Dillier 1985).  our data at high pulse rates,  This is consistent with  suggesting that the central  auditory nervous system remains capable of extracting interpulse intervals, even when it is highly unlikely that every stimulus cycle results in a neural response,  in a given  It is possible that the increasing  single fiber.  representation of a range of interspike intervals at multiples of the stimulus period, pulse rates,  such as might be expected at high  results in a weakening of the pitch percept.  Pitóh extraction at high pulse rates could also be based on a measurement of multiples of the interpulse intervals, volleying mechanism (Wever 1949) stimulated neurons.  or on a  between electrically  This would allow temporal processes to  extend to pulse rates higher than the limit imposed by the refractoriness of individual neurons.  Little is known about  individual differences in the spatio—temporal response patterns in a population of electrically stimulated neurons. Parkins  (1989)  has pointed out that the gradation of stimulus  intensity within electrical fields may result in differences in the temporal patterning of responses of neurons situated near the center of the field than for those at the periphery, and that the effects of a range of interspike intervals on pitch perception, particularly with electrical stimulation,  is  not known. In conclusion,  it appears that variation in pulse rate on  single intracochlear electrodes can, at least in some cochlear  170  implant recipients, result in pitch changes which are sufficiently salient to support musical interval perception and tune recognition.  In the absence of spectral analysis in  such ears, these results are almost certainly due to the temporal firing patterns of residual auditory neurons.  171  REFERENCES  American National Standards Institute: Acoustical Terminology, (ANSI S3.20-1973) New York. Attneave, F. and R.K. Olson. 1971. Pitch as a medium: a new approach to psychophysical scaling. Am. J. Psychol. 84: 147—166. Bilger, R.C. 1977. Psychoacoustic evaluation of present prostheses. Ann. Otol. Rhinol. Laryngol. 86 (Suppl.38), 92—140. Black, R.C., G.M. Clark, and J.F. Patrick. 1981. Current distribution measurements within the human cochlea. IEEE Trans. Biomed. Eng. 28: 721-725. Blarney, P.J., R.C. Dowell, A.M. Brown, G.M. Clark, and P.M. Seligman. 1987. Vowel and consonant recognition of cochlear implant patients using formant—estimating speech processors. J. Acoust. Soc. Am. 82: 48—57. Blarney, P.J., R.C. Dowell, G.M. Clark, and P.M. Seligman. 1987. Acoustic parameters measured by a formant— estimating speech processor for a multiple—channel cochlear implant. J. Acoust. Soc. Am. 82: 38—47. Blarney, P.J., B.C. Pyman, B.M. Clark, R.C. Dowell, M. Gordon, A.M. Brown, and R.D. Hollow. 1992. Factors predicting postoperative sentence scores in postlinguistically deaf adult cochlear implant patients. Ann. Otol. Rhinol. Laryngol. 101: 342—348. Burns, E.M., and N.F. Viemeister. 1976. J. Acoust. Soc. Am. 60: 863-869.  Nonspectral pitch.  1981. Burns, E.M., and N.F. Viemeister. Played-again SAN: Further observations on the pitch of amplitude-modulated noise. J. Acoust. Soc. Am. 70: 1655—1660. Burns, E.M., and W.D. Ward. 1978. Categorical perceptionphenomenon or epiphenomenon: Evidence from experiments in the perception of melodic musical intervals. J. Acoust. Soc. Am. 63: 456—467.  172  Cazals, Y., M. Pelizzone, A. Rasper, and P. Montandon. 1991. Indication of a relation between speech perception and temporal resolution for cochlear implantees. Ann. Otol. Rhinol. Laryngol. 100: 893—895. 1978. Multiple intracochlear electrodes for Chouard, C.H. rehabilitation in total deafness. Otolaryngologic Clinics of North America 11: 217-233. Clark, G.M., R. Black, D.J. Dewhurst, IC. Forster, J.F. Patrick, and Y.C. Tong. 1977. A multiple-electrode hearing prosthesis for cochlear implantation in deaf patients. Med. Progr. Technol. 5: 127-140. Clopton, B.M., F.A. Spelman, I. Glass, B.E. Pfingst, J.M. Miller, P.D. Lawrence, and D.P. Dean. 1983. Neural encoding of electrical signals. In Cochlear Prostheses: An International Symposium, ed. C.W. Parkins and J.W. Anderson. Ann. N.Y. Acad. Sci. 405: 146-158. Cramer, E.M., and W.H. Huggins. 1958. through binaural interaction. J. 413—417.  Creation of pitch Acoust. Soc. Am. 30:  Cross, I., P. Howell, and R. West. 1985. Structural relationships in the perception of musical pitch. Musical Structure and Cognition, ed. I. Cross, P. London: Academic Press Inc. R. West, 121-142.  In Howell,  Davis, H., T. Morgan, J.E. Hawkins, R. Galambos, and F.W. 1950. Temporary deafness following exposure to Smith. loud tones and noise. Acta. Otolaryng. (Stockh). (Suppl.8): 1—57. Davis, H., S.R. Silverman, and D.R. McAuliffe. 1951. observations on pitch and frequency. J. Acoust. 23: 40—42. de Boer, E. Thesis, Deutsch, 76:  1956. On the Residue in Hearing. University of Amsterdam.  D. 1969. 300—307.  Deutsch, D. 1972. recognition.  Music recognition.  Some Soc. Am.  Academic  Psychological Review  Octave generalization and tune Perception & Psychophysics 11: 411—412.  173  1983. Dillier, N., T. Spillmann, and J. Guentensperger. Computerized testing of signal-encoding strategies with Ann. N.Y. Acad. Sci. 405: 360round-window implants. 369. Some aspects of temporal 1985. Dobie, R.A., and N. Dillier. coding for single-channel electrical stimulation of the cochlea. Hearing Res. 18: 41-55. Scale and Contour: Two components of a 1978. Dowling, W.J. Psychological Review 85: theory of memory for melodies. 341—354. The importance of 1981. Dowling, W.J., and J.C. Bartlett. interval information in long—term memory for melodies. Psychomusicology 30-49. Contour, 1971. Dowling, W.J., and D.S. Fujitani. and pitch recognition in memory for melodies. Soc. Am. 49: 524—531.  interval, J. Acoust.  Eddington, D.K., W.H. Dobelle, D.W. Brackmann, M.G. Auditory l978a. Mladejovsky, and J.L. Parkin. prosthesis research with multiple channel intracochlear Ann. Otol. Rhinol. Laryrigol. stimulation in man. 87(Suppl.53): 5—39. Eddington, D.K., W.H. Dobelle, D.E. Brackmann, M.G. Place and 1978b. Mladejovsky, and J. Parkin. periodicity pitch by stimulation of multiple scala Trans. Am. Soc. tympani electrodes in deaf volunteers. Artif. Intern. Organs 24: 1-5. Speech recognition in 1985, Eddington, D.K., and J.L. Orth. a deaf subject with a portable, multichannel cochlear In Cochlear Implants, ed. R.A. Schindler implant system. and M.M. Merzenich, 277-282. New York: Raven Press. 1987. Adjustment of Elliot, J., J.R. Platt, and R.J. Racine. successive and simultaneous intervals by musically Perception & experienced and inexperienced subjects. Psychophysics 42: 594—598. Place and time coding of frequency in the 1978. Evans, E.F. peripheral auditory system: some physiological pros and Audiology 17: 369-420. cons. Flanagan, J.L., and N. periodic pulses.  On the pitch of 1960. Guttman. J. Acoust. Soc. Am. 32: 1308-1319.  The physical criterion for determining 1928. Fletcher, H. Phys. Rev. 23: 427-437. the pitch of a musical tone.  174  Gfeller, K., and C.R. Lansing. 1991. Melodic, rhythmic, tiinbral perception of adult cochlear implant users. Speech Hear. Res. 34: 916—920.  and J.  Glass, I. 1983. Tuning characteristics of cochlear nucleus units in response to electrical stimulation of the cochlea. Hearing Res. 12: 223—237. 1984. Phase—locked responses of cochlear nucleus Glass, I. units to electrical stimulation through a cochlear implant. Exp. Brain Res. 55: 386-390. Goldstein, J.L. 1978. Mechanisms of signal analysis and pattern perception in periodicity pitch. Audiology 17: 421—445. Greenberg, S. 1980. Neural temporal coding of pitch and timbre. UCLA Working Papers in Phonetics (Number 52), Los Angeles CA. Greenberg, S., C.D. Geisler, and L. Deng. 1986. Frequency selectivity of single cochlear—nerve fibers based on the temporal response pattern to two—tone signals. J. Acoust. Soc. Am. 79: 1010—1019. Greenberg, S., J.T. Marsh, W.S. Brown, and J.C. Smith. 1987. Neural temporal coding of low pitch. I: Human frequency— following responses to complex tones. Hearing Res. 25: 91—114. Greenberg, S., and W.S. Rhode. 1990. Periodicity coding in cochlear nerve and ventral cochlear nucleus. In Auditory Processing of Complex Sounds, ed. W.A. Yost and C.S. Watson, 225-236. Hillside, N.J.: Lawrence Erlbaum. Greenwood, D.D. 1961. Critical bandwidth and the frequency coordinates of the basilar membrane. J. Acoust. Soc. Am. 33: 1344—1356. 1990. A cochlear frequency—position function Greenwood, D.D. for several species——29 years later. J. Acoust. Soc. Am. 87: 2592—2605. Harris, G.G. noise.  1963. Periodicity perception by using gated J. Acoust. Soc. Am. 35: 1229—1233.  Hartmann, R., G. Topp, and R. Klinke. 1984. Discharge patterns of cat primary auditory fibers with electrical stimulation of the cochlea. Hearing Res. 13: 47—62. 1967. Hind, J.E., D.J. Anderson, J.F. Brugge, and J.E. Rose. Coding of information pertaining to paired low-frequency tones in single auditory nerve fibers of the squirrel monkey. J. Neurophysiol. 30: 794-816.  175  Hochmair—Desoyer, I.J., E.S. Hochmair, K. Burian, and H.K. Stigibrunner. Percepts from the Vienna cochlear 1983. prosthesis. In Cochlear Prostheses: An International Symposium, ed. C.W. Parkins and J.W. Anderson. Ann. N.Y. Acad. Sci. 405: 295—306. Hoekstra, A., and R.J. Ritsma. 1977. Perceptive hearing loss and frequency selectivity. In Psychophysics and Physiology of Hearing, ed. E.F. Evans and JP. Wilson, 263-271. London, New York: Academic Press. Horst, J.W., E. Javel, and G.R. Farley. 1990. coding of spectral fine structure in the auditory nerve. II: Leveldependent nonlinear responses. J. Acoust. Soc. Am. 88: 2656—2681. Houtsma, A.J.M. 1984. Pitch salience of various complex sounds. Music Perception 1: 296-307. Houtsma, A.J.M., and J. Smurzynski. 1990. Pitch identification and discrimination for complex tones with many harmonics. J. Acoust. Soc. Am. 87: 304-310. 1980. Javel, E. Coding of AN tones in the chinchilla auditory nerve: Implications for the pitch of complex tones. J. Acoust. Soc. Am. 68: 133-146. Javel, E., J.A. McGee, J.W. Horst, and G.R. Farley. 1988. Temporal mechanisms in auditory stimulus coding. In Auditory Function: Neurobiological Bases of Hearing, ed. G.M. Edelman, W.E. Gall, and W.M. Cowan, 515-558. New York: John Wiley and Sons. Javel, E., and J.B. Mott. 1988. Physiological and psychophysical correlates of temporal processes in Hearing Res. 34: 275-294. hearing. Javel, E., Y.C. Tong, R.K. Shepherd, and G.M. Clark. 1987. Responses of cat auditory nerve fibers to biphasic electrical current pulses. Ann. Otol. Rhinol. Laryngol. 96 (Suppl.128): 26—30. Jones, M.R., L. Summerell, and E. Marshburn. 1987. Recognizing melodies: A dynamic interpretation. Psychol. 39A: 89—121.  J.  Exp.  Kiang, N.Y.S., and E.C. Moxon. 1972. Physiological considerations in artificial stimulation of the inner Ann. Otol. Rhinol. Laryngol. 81: 714-731. ear. Koch,  D.B., P. Seligman, C. Daly, and L. Whitford. 1990. A multipeak feature extraction coding strategy for a multichannel cochlear implant. Hear. Instrum. 41: 28-32.  176  Krumhansl, C.L., and E.J. Kessler. 1982. Tracing the dynamic changes in perceived tonal organization in a spatial representation of musical keys. Psychological Review 89: 334—368. Krumhansl, C.L., and R.N. Shepard. 1979. Quantification of the hierarchy of tonal functions within a diatonic context. J. Exp. Psychol. 5: 579-594. Langner, C. 1983. Evidence for neuronal periodicity detection in the auditory system of the guinea fowl: implications for pitch analysis in the time domain. Brain Res. 52: 333—355.  Exp.  Langner, G., and C.E. Schreiner. 1988. Periodicity coding in the inferior colliculus of the cat: I. Neuronal mechanisms. J. Neurophysiol. 60: 1799-1822. Licklider, JC.R. 1954. “Periodicity” pitch and “place” pitch. J. Acoust. Soc. Am. 26: 945. Loeb,  G.E., M.W. White, and W.M. Jenkins. 1983. Biophysical considerations in electrical stimulation of the auditory nervous system. In Cochlear Prostheses: An International Symposium, ed. C.W. Parkins and J.W. Anderson. Ann. N.Y. Acad. Sci. 405: 123—136.  Massaro, D.W., H.J. Kailman, and J.L. Kelly. 1980. The role of tone height, melodic contour, and tone chroma in melody recognition. J. Exp. Psychol.: Human Learning and Memory 6: 77-90. Merzenich, N.M. 1983. coding of sound in a cochlear prosthesis: some theoretical and practical considerations. In Cochlear Prostheses: An International Symposium, ed. c.w. Parkins and J.W. Anderson. Ann. N.Y. Acad. Sci. 405: 502-508. Merzenich, N.M., R.P. Michelson, C.R. Pettit, R.A. Schindler, and M. Reid. 1973. Neural encoding of sound sensation evoked by electrical stimulation of the acoustic nerve. Ann. Otol. Rhinol. Laryngol. 82: 486-504. Miller, J.R., and E.C. carterette. 1975. Perceptual space for musical structures. J. Acoust. Soc. Am. 58: 711-720. Miller, G.A., and W.G. Taylor. repeated bursts of noise. 182.  1948. The perception of J. Acoust. Soc. Am. 20: 171-  177  Mladejovsky, M.G., D.K. Eddington, W.H. Dobelle, and D.E. 1975. Artificial hearing for the deaf by Brackmann. cochlear stimulation: pitch modulation and some parametric thresholds. Trans. Amer. Soc. Artif. Intern. Organs 21: 1-7. 1987. Monahan, C.B., R.A. Kendall, and E.C. Carterette. The effect of melodic and temporal contour on recognition Perception & Psychophysics 41: memory for pitch change. 576—600. 1973. Some experiments relating to the Moore, B.C.J. perception of complex tones. Q. J. Exp. Psychol. 25: 451—475. Effects of relative phase of the 1977. Moore, B.C.J. components on the pitch of three—component complex tones. In Psychophysics and Physiology of Hearing, ed. E.F. London, New York: Evans and J.P. Wilson, 349-362. Academic Press. 1986. The role of frequency Moore, B.C.J, and B.R. Glasberg. selectivity in the perception of loudness, pitch and time. In Frequency Selectivity in Hearing, ed. B.C.J. London: Academic Moore and R.D. Patterson, 251-308. Press Inc. 1979. Tune recognition with Moore, B.C.J., and S.M. Rosen. reduced pitch and interval information. Q. J. Exp. Psychol. 31: 229—240. 1965. Electrical stimulation of the cat’s Moxon, E.C. cochlea: a study of discharge rates in single auditory Masterts Thesis, MIT, Cambridge, MA. nerve fibers. 1971. Neural and mechanical responses to Moxon, E.C. Doctoral electric stimulation of the cat’s inner ear. dissertation, MIT, Cambridge, MA. Comparison of percepts found with 1983. Muller, C.G. Ann. N.Y. Acad. Sci. 405: cochlear implant devices. 420.  412-  Histopathologic correlation 1991. Nadol, J.B., and W. Hsu. of spiral ganglion cell count and new bone formation in the cochlea following meningogenic labyrinthitis and deafness. Ann. Otol. Rhinol. Laryngol. 100: 712-716. 1986. Phase-locking in the Palmer, A.R., and I.J. Russell. cochlear nerve of the guinea—pig and its relation to the Hearing Res. 24: receptor potential of inner hair cells. 1—15.  178  Parkins, C.W. 1989. Temporal response patterns of auditory nerve fibers to electrical stimulation in deafened squirrel monkeys. Hearing Res. 41: 137-168. Patterson, R.D., and D. Johnson—Davies. 1977. Detection of a change in the pitch of AN noise. In Psychophysics and Physiology of Hearing, ed. E.F. Evans and J.P. Wilson, 363-372. London, New York: Academic Press. Pfingst, B.E. 1984. operating ranges and intensity psychophysics for cochlear implants. Arch. Otolaryngol. 110: 140—144. Pfingst, B.E. Psychophysical data from cochlear 1985. implants: relevance to strategies for rehabilitation. Seminars in Hearing 6: 7-21. Pfingst, B.E. 1988. comparisons of psychophysical and neurophysiological studies of cochlear implants. Hearing Res. 34: 243—252. Pfingst, B.E., D.R. DeHaan, and L.A. Holloway. 1991. Stimulus features affecting psychophysical detection thresholds for electrical stimulation of the cochlea. Phase duration and stimulus duration. J. Acoust. Soc. Am. 90, Pt.1.: 1857—1866.  I:  Pfingst, B.E., J.A. Donaldson, J.N. Miller, and F.A. Spelman. 1979. Psychophysical evaluation of cochlear prostheses in a monkey model. Ann. Otol. 88: 613-625. Pfingst, B.E., and D. Sutton. 1983. Relation of cochlear implant function to histopathology in monkeys. In Cochlear Prostheses: An International Symposium, ed. C.W. Parkins and J.W. Anderson. Ann. N.Y. Acad. Sci. 405: 224—239. Pfingst, B.E., D. Sutton, J.M. Miller, and B.A. Bohne. 1981. Cochlear implants: Relation of psychophysical measures to histological findings. Acta. Otolaryngol. 92:1. Pierce, J.R. Acoust.  1991. Periodicity and pitch perception. J. Soc. Am. 90: 1889—1893.  Pierce, J.R., R. Lipes, and c. cheetham. 1977. Uncertainty concerning the direct use of time information in hearing: place cues in white spectra stimuli. J. Acoust. Soc. Am. 61: 1609—1621. Pijl,  1992. S. and F. Noel. The Nucleus multichannel cochlear implant: comparison of scala tympani vs. scala vestibuli electrode placement in a single patient. Otolaryngol. Head Neck Surg. 107: 472-474.  179  Plomp, R. York:  1976. Aspects of Tone Sensation. Academic Press.  Pollack, I. noise 238(L).  -  London and New  1969. Periodicity pitch for interrupted white fact or artifact. J. Acoust. Soc. Am. 45: 237-  1978. Musical Acoustics. Risset, J-C. In Handbook of Perception Vol.4, ed. Carterette, 521-564. New York: Academic Press Inc. Ritsma, R.J. 1962. Existence region of the tonal residue: J. Acoust. Soc. Am. 34: 1224—1229.  I.  Ritsma, R.J. 1967. Frequencies dominant in the perception of pitch of complex sounds. J. Acoust. Soc. Am. 42:191-199. Ritsma, R.J., and F.L. Engel. 1964. Pitch of frequencymodulated signals. J. Acoust. Soc. Am. 36: 1637-1644. Rose,  J.E., J.L. Brugge, D.J. Anderson, and J. Hind. 1967. Phase—locked response to low—frequency tones in single auditory nerve fibers of the squirrel monkey. J. Neurophysiol. 30: 769—793.  Sachs, M.B. 1983. Round table discussion. In Cochlear Prostheses: An International Symposium, ed. C.W. Parkins and J.W. Anderson. Ann. N.Y. Acad. Sci. 405: 510-511. Sachs, M.B., and M.I. Miller. 1985. Pitch coding in the auditory nerve: possible mechanisms of pitch sensation with cochlear implants. In Cochlear Implants, ed. R.A. Schindler and N.M. Merzenich, 185-194. New York: Raven Press. Sachs, M.B., and E.D. Young. 1980. Effects of nonlinearities on speech encoding in the auditory nerve. J. Acoust. Soc. Am. 68: 858—875. Schouten, J.F. 1938. The perception of subjective tones. Proc. Kon. Ned. Acad. Wetensch. 41: 1086—1093. Schouten, J.F. hearing.  1940. The residue and the mechanism of Proc. ICon. Ned. Acad. Wetensch. 43: 991-999.  Schreiner, C.E., and C. Langner. 1988. the inferior colliculus of the cat: organization. J. Neurophysiol. 60:  Periodicity coding in II. Topographical 1823-1840.  Schubert, E.D. 1983. Use of the pitch response in assessing cochlear implants. Abstracts of the Sixth Midwinter Research Meeting, Association for Research in Otolaryngology, 127.  180  Shallop, J.K., A.L. Beiter, D.W. Goin, and R.E. Mischke. 1990. Electrically evoked auditory brainstem responses (EABR) and middle latency responses (EMLR) obtained from patients with the Nucleus Multichannel Cochlear Implant. Ear Hear 11: 5-15. Shannon, R.V. 1983. Multichannel electrical stimulation of the auditory nerve in man. I. Basic psychophysics. Hearing Res. 11: 157—189. Shannon. R.V. 1989. Detection of gaps in sinusoids and pulse trains by patients with cochlear implants. J. Acoust. Soc. Am. 85: 2587—2592. Shannon, R.V. 1992. Temporal modulation transfer functions in patients with cochlear implants. J. Acoust. Soc. Am. 91: 2156—2164. Shannon, R.V., D.D. Adams, R.L. Ferrel, R.L. Palumbo, and M. Grandgenett. 1990. A computer interface for psychophysical and speech research with the Nucleus cochlear implant. J. Acoust. Soc. Am. 87: 905-907. Shuter-Dyson, R. 1982. Musical ability. In The Psychology of Music, ed. D. Deutsch, 391-412. New York, London: Academic Press. Siegel, J.A., and W. Siegel. 1977. Absolute identification of notes and intervals by musicians. Perception & Psychophysics 21: 143—152. Simmons, F.B. 1966. Electrical stimulation of the auditory nerve in man. Arch. Otolaryngol. 84: 24-76. Simmons, F.B. 1983. Percepts from modiolar (eighth nerve) stimulation. In Cochlear Prostheses: An International Symposium, ed. C.W. Parkins and J.W. Anderson. Ann. N.Y. Acad. Sci. 405: 259—263. Simmons, F.B., R.G. Mathews, M.G. Walker, and R.L. White. 1979. A functioning multichannel auditory nerve stimulator. Acta. Otolaryngol. 87: 170-175. Simmons, F.B., M.G. Walker, R.L. White, and R.G. Mathews. 1981. Pitch correlates of direct auditory nerve electrical stimulation. Ann. Otol. Rhinol. Laryngol. 90(Suppl 82): 15—18. Sloboda, J.A., and D.H.H. Parker. 1985. Immediate recall of melodies. In Musical Structure and Cognition, ed. I. Cross, P. Howell, and R. West, 143-167. London: Academic Press Inc.  181 Small, A.M. 1955. Some parameters influencing the pitch of amplitude modulated signals. J. Acoust. Soc. Am. 27: 751—760. Small, A.M., and M.E. McClellan. 1963. Pitch associated with time delay between two pulse trains. J. Acoust. Soc. Am. 35: 1246—1255. Spillmann, T., N. Dillier, and J. Guentensperger. 1982. Electrical stimulation of hearing by implanted cochlear electrodes in humans. Appi. Neurophysiol. 45: 32-37. Stevens, S.S., and J. Volkmann. 1940. The relation of pitch to frequency: a revised scale. Am. J. Psychol. 53: 329353. Stypulkowski, P.H., and C. van den Honert. 1984. Physiological properties of the electrically stimulated auditory nerve. I. compound action potential recordings. Hearing Res. 14: 205—223. Sullivan, W.E., and M. Konishi. 1986. Neural map of interaural phase difference in the owl’s brainstem. Proc. Nati. Acad. Sci. USA 83: 8400-8404. Takahashi, T., A. Moiseff, and M. Konishi. 1984. Time and intensity cues are processed independently in the auditory system of the owl. J. Neurosci. 4: 1781-1786. Tong,  Y.C., R.C. Black, G.M. Clark, I.C. Forster, J.B. Millar, B.J. O’Loughlin, and J.F. Patrick. 1979. A preliminary report on a multiple—channel cochlear implant operation. J. Laryngol. Otol. 93: 679—695.  Tong,  Y.C., P.J. Blarney, R.C. Dowell, and G.M. Clark. 1983. Psychophysical studies evaluating the feasibility of a speech processing strategy for a multiple—channel cochlear implant. J. Acoust. Soc. Am. 74: 73-80.  Tong,  Y.C., and G.M. Clark. 1983. Percepts from scala tympani stimulation. In Cochlear Prostheses: An International Symposium, ed. C.W. Parkins and J.W. Anderson. Ann. N.Y. Acad. Sci. 405: 264—267.  Tong,  Y.C., and G.M. Clark. 1985. Absolute identification of electric pulse rates and electrode positions by cochlear implant patients. J. Acoust. Soc. Am. 77: 1881—1888.  Tong,  Y.C., and G.M. Clark. 1986. Loudness summation, masking, and temporal interaction for sensations produced by electric stimulation of two sites in the human cochlea. J. Acoustic. Soc. Am. 79: 1958—1966.  182  Tong,  Y.C., G.M. Clark, P.J. Blamey, P.A. Busby, and R.C. Dowell. 1982. Psychophysical studies for two multiplechannel cochlear implant patients. J. Acoust. Soc. Am. 71: 153—160.  Tong,  Y.C., J.B. Millar, G.M. Clark, L.F. Martin, P.A. Busby, and J.F. Patrick. 1980. Psychophysical and speech perception studies on two multiple channel cochlear implant patients. J. Laryngol. Otol. 94: 1241-1256.  Townshend, B., N. Cotter, D. Van Compernolle, and R.L. White. 1987. Pitch perception by cochlear implant subjects. J. Acoust. Soc. Am. 82: 106-115. Tyler, R.S., B.C.J. Moore, F.K. Kuk. 1989. Performance of some of the better cochlear-implant patients. J. Speech Hear. Res. 32: 887—911. van den Honert, C., and P.H. Stypulkowski. 1984. Physiological properties of the electrically stimulated auditory nerve. II. Single fiber recordings. Hearing Res. 14: 225—243. van den Honert, C., and P.H. Stypulkowski. 1987a. Single fiber mapping of spatial excitation patterns in the electrically stimulated auditory nerve. Hearing Res. 29: 195—206. van den Honert, C., and P.H. Stypulkowski. 1987b. Temporal response patterns of single auditory nerve fibers elicited by periodic electrical stimuli. Hearing Res. 29: 207—222. Vernon, M. 1967. Meningitis and deafness: the problem, its physical, audiological psychological and educational manifestations in deaf children. Laryngoscope 77: 18561874. von Wallenberg, E.L., and R.D. Battmer. 1991. Comparative speech recognition results in eight subjects using two different coding strategies with the Nucleus 22 channel British J. Audiology 25: 371-380. cochlear implant. Wapnick, J., G. Bourassa, and J. Sampson. 1982. The perception of tonal intervals in isolation and in melodic context. Psychomusicology 2: 21-37. Ward, W.D. 1970. Musical Perception. In Foundations of Modern Auditory Theory, ed. J.V. Tobias, 405-447. New York: Academic Press. Warren, R.M., and J.A. Bashford. 1988. Broadband repetition pitch: spectral dominance or pitch averaging? J. Acoust. Soc. Am. 84: 2058—2062.  183  Watkins, A.J., and M.C. Dyson. 1985. On the perceptual organization of tone sequences and melodies. In Musical Structure and Cognition, ed. I. Cross, P. Howell, and R. West, 71-119. London: Academic Press Inc. Wever,  E.G.  1949.  Theory of Hearing.  New York,  Dover.  Wever, E.G., and C.W. Bray. 1930. Action currents in the auditory nerve in response to acoustical stimulation. Proc. Nat. Acad. Sci. (U.S.) 16: 344-50. Wing,  H. 1948. Tests of musical ability and appreciation. Brit. J. Psychol. Monogr. (Suppl.27).  Woodworth, R.S. 1938. Experimental Psychology. Henry Holt and Company.  New York:  184  APPENDIX 1  OPEN-SET TUNES*  1.  TWINKLE TWINKLE LITTLE STAR (14) ccggaagffeeddc 22222242222224  2.  JINGLE BELLS  3.  LONDON BRIDGE IS FALLING DOWN (13) gagfefgdefefg 3122224224224  4.  0 CANADA  (11) eeeeeeegcde 22422422224  (10) eggcdefgad 4316222224  5.  0 COME ALL YE FAITHFUL (12) f f c f g c a g a 2422442222  a g 44  (14) f g f d f g f d C C a bb bb f 31263126426426  6.  SILENT NIGHT  7.  MARY HAD A LITTLE LAMB (13) edcdeeedddegg 3122224224224  8.  HAPPY BIRTHDAY  9.  GOOD KING WENCESLAS (13) fffgffcdcdeff 2222224222244  (12) ccdcfeccdcgf 112224112224  (14) cfefagfgaffaCD 23122312231224  10.  AULD LANG SYNE  11.  CLEMENTINE  12.  HARK THE HERALD ANGELS SING (15) a g a c f f e f a a g C C C 223122222231224  (15) fffcaaaffaCCbbag 14 1122112211311  185  Appendix 1  -  Continued  13.  DECK THE HALLS  14.  FRERE JACQUES  15.  0 SUZANNA  (17 agfgafgabbgag f ef Cb 31222222111131224 (14) cdeccdecefgefg 22222222224224  (14) ceggagecdeedcd 22231223122224  16.  OLD MACDONALD HAD A FARM (12) fffcddcaaqgf 222222422224  17.  HOME ON THE RANGE  (11)  c c f g a f e d bb bb bb 222241122  2  4  (14) ffgafagcffgafe 22222222222242  18.  YANKEE DOODLE  19.  STAR SPANGLED BANNER (12) g e C e g C E D C e f# g 4 11222411222  20.  GOD SAVE THE QUEEN (16k ddec def# f#gf#edeac#d 2223122223122224  21.  BICYCLE BUILT FOR TWO (10) Cafcdefdfc 3333111314  22.  AWAY IN A MANGER (13) c f f g a f f a 222112211  C C D 2224  (14) geedeggaaCaagg 22222242222224  23.  JESUS LOVES ME  24.  WE WISH YOU A MERRY CHRISTMAS (16) cffgfedddggagfec 2211112222111122  25.  0 WHEN THE SAINTS (16) cefgcefgcefgeced 2226222622244446  186  Appendix 1  -  Continued  26.  WALTZING MATILDA (20) gggge CCCbagggagggf ed 21122211222112112112  27.  GOD REST YOU MERRY GENTLEMEN (14) ddaagfedcdefga 22222222222224  28.  ON TOP OF OLD SMOKEY (11) ccegCaafgag 22226422226  29.  FOR HE’S A JOLLY GOOD FELLOW (16) ceeedefeedddcdec 2422226424222264  30.  ON THE FIRST DAY OF CHRISTMAS (13) c c c f f f e f g a bb g a 24 22422422222  *Numbers preceding the tunes correspond to those of the Numbers in parentheses represent horizontal axis in Figure 7. Pulse rate and note duration segment. tune in notes of number the assignments: c = c= d = d#= e =  100 pps 106 112 119 126  1=250msec 2 = 500 msec  f = 133 pps f#= 141 g = 150 g#= 159 a = 168  a#= b = C D = E =  3=75omsec 4 = 1000 msec  6=l500msec  178 pps 189 200 224 252  187  APPENDIX 2 PULSE RATES FOR 7-NOTE MELODIES;  BASE PULSE RATE 100 PPS. LARGEST INT INTERVAL EXTENT**  MEL.* RANGE (pps) NOTES (pps) 1l 133 150 150 150 G G G F E  M2nd  3  100 126 150 150 168 A E G G C  150 126 E G  M3rd  9  100—168  100 100 150 150 168 C G G A C  168 A  5th  9  4  119—150  iig iig 133 F E E  150 iig 150 133 G F G E  M3rd  5  5  112—150  133 F  iig 133 E F  M2nd  5  6  112—150  150 150 150 112 D C G G  126 126 112 E D E  4th  5  7  112—150  150 126 126 112 D E G E  126 150 150 G G E  m3rd  5  8  106—159  141 141 141 15 141 141 10 F F C F F  4th  5  1  119—150  150 133 F G  2  100—168  3  150 133 F G  iig 112 E D  *MELODIES: 1. Mary Had a Little Lamb 2. 0 Suzanna 3. Twinkle, Twinkle, Little Star 4. Yankee Doodle  **  5. 6. 7. 8.  150 G  London Bridge old MacDonald Jesus Loves Me Good King Wenceslas  Range in semitones between highest and lowest notes of the Largest interval in melody (M = Major; melodies. m = Minor)  

Cite

Citation Scheme:

        

Citations by CSL (citeproc-js)

Usage Statistics

Share

Embed

Customize your widget with the following options, then copy and paste the code below into the HTML of your page to embed this item in your website.
                        
                            <div id="ubcOpenCollectionsWidgetDisplay">
                            <script id="ubcOpenCollectionsWidget"
                            src="{[{embed.src}]}"
                            data-item="{[{embed.item}]}"
                            data-collection="{[{embed.collection}]}"
                            data-metadata="{[{embed.showMetadata}]}"
                            data-width="{[{embed.width}]}"
                            async >
                            </script>
                            </div>
                        
                    
IIIF logo Our image viewer uses the IIIF 2.0 standard. To load this item in other compatible viewers, use this url:
http://iiif.library.ubc.ca/presentation/dsp.831.1-0099227/manifest

Comment

Related Items