UBC Theses and Dissertations

UBC Theses Logo

UBC Theses and Dissertations

Bitemark overlays: an analysis of effectiveness Pretty, Ian A. 2000-12-31

You don't seem to have a PDF reader installed, try download the pdf

Item Metadata

Download

Media
ubc_2000-0138.pdf [ 12.2MB ]
Metadata
JSON: 1.0089349.json
JSON-LD: 1.0089349+ld.json
RDF/XML (Pretty): 1.0089349.xml
RDF/JSON: 1.0089349+rdf.json
Turtle: 1.0089349+rdf-turtle.txt
N-Triples: 1.0089349+rdf-ntriples.txt
Original Record: 1.0089349 +original-record.json
Full Text
1.0089349.txt
Citation
1.0089349.ris

Full Text

BITEMARK OVERLAYS -AN ANALYSIS OF EFFECTIVENESS by Iain A. Pretty B.D.S., The University of Newcastle Upon Tyne, 1998 A thesis submitted in partial fulfilment of the requirements for the degree of MASTER OF SCIENCE in THE FACULTY OF GRADUATE STUDIES (Department of Oral Biological and Medical Sciences; Faculty of Dentistry) We accept this thesis as conforming to the required standard The University of British Columbia February 2000 © lain A. Pretty, 2000 UBC Special Collections - Thesis Authorisation Form http://www.library.ubc.ca/spcoll/thesauth.html In presenting this thesis in partial fulfilment of the requirements for an advanced degree at the University of British Columbia, I agree that the Library shall make it freely available for reference and study. I further agree that permission for extensive copying of this thesis for scholarly purposes may be granted by the head of my department or by his or her representatives. It is understood that copying or publication of this thesis for financial gain shall not be allowed without my written permission. Department of QcsJ^Vl--The University of British Columbia Vancouver, Canada 1 of 1 2/29/00 5:25 PM Abstract It is not unusual to see dentists testifying in Court. Such professionals assist criminal proceedings by identifying the victims of crime and by analysing bitemarks with the hope of determining the biter. Contemporary legal history is littered with cases where it has been possible to match a bite injury on a victim to the person suspected of the crime. In many of the cases, this type of evidence is often crucial to the successful outcome of the trial. Bitemark evidence has been almost universally accepted in the Courts, but the fundamental validity and scientific basis for its use is frequently challenged. Rapid advances in forensic science, particularly within the field of DNA evidence, have caused concern to the judicial system. Recent rulings, such as those of Daubert and Kumho in the United States, have placed a greater emphasis on the validity and reliability of opinion testimony based upon supposed scientific principles. Judges have stated that witnesses must be able to identify published works that define operational parameters of any tests or procedures that form the basis of scientific conclusions. Such works do not exist within the field of bitemark analysis. As the most commonly employed analytical technique in bite injury assessment, this study defines quantifiable variables for transparent overlays. A series of 10 simulated, postmortem bites were created on pigskin and, with accompanying overlays, assembled into cases. Using two separate studies with four examiner groups the study defined values of intra- and inter-examiner reliability, accuracy, sensitivity, specificity, and error rates for transparent overlays. Methods and statistical treatments from medical decision-making and diagnostic test evaluation were employed. Forced decision models and receiver operating characteristic analyses were utilised. The results determined that transparent overlays were effective in the determination of biters. The sensitivity and specificity values were consistent with those of other dental tests, although due to a paucity of equivalent studies it has been impossible to rate the values within a forensic context. The relatively low values of inter-examiner reliability were thought to reflect the nature of both bitemarks as physical evidence and the variability ii of examiner thresholds. It was concluded that the weak inter-examiner reliability values explain the divergence of odontologists' opinions regarding bitemark identifications often stated in Court. The positive and negative predictive values suggest that bitemarks may be more effective at excluding individuals than including them. The effect of training and experience was found to have little effect on the application of overlays within this study. The work concludes that further research is required within the field of bitemark analysis so that the results of the current study can be placed into context. This work represents a significant first step in establishing the scientific basis for this aspect of forensic dentistry. iii TABLE OF CONTENTS Abstract !! List of Tables vn List of Figures . List of Abbreviations Quotation x!^ m Acknowledgementsxiv Dedication Funding . Preface CHAPTER 1 INTRODUCTION SECTION 1 - BITEMARKS IN FORENSIC DENTISTRY 1.1.1 Forensic Dentistry 3 1.1.2 The History of Forensic Dentistry 4 1.1.3 Human Bitemarks as Forensic Evidence 6 1.1.3.1 Appearance 6 1.1.3.2 Anatomical location 9 1.1.3.3 Forensic significance 11 1.1.3.4 Bitemarks as physical evidence 11.1.3.5 Bitemarks as biological evidence 5 1.1.3.6 Bitemarks as psychological evidence .16 1.1.4 Collection of Bitemark Evidence 11.1.4.1 Victim evidence 17 1.1.4.2 Suspect evidence 21 •1.1.5 Analysis of Bitemark Evidence 24 1.1.5.1 Production of Transparent OverlaysSECTION 2 - THE SCIENCE AND JUSTICE OF BITEMARKS 1.2.1. The Scientific Basis of Bitemarks - A Literature Review 29 1.2.1.1 Introduction .....30 1.2.1.2 Analysis of the articles 31.2.1.3 Accuracy of bitemarks on human skin .32 1.2.1.4 The uniqueness of the human dentition 35 1.2.1.5 Analytical techniques ....44 1.2.2. The Judicial View of Bitemarks - A Legal Review 49 1.2.2.1 Introduction 41.2.2.2 The use of scientific evidence in the U.S. legal system 50 1.2.2.3 The admission of bitemarks as evidence in the U.S. legal system. 53 1.2.3.4 Case assessment of bitemark admissibility 56 iv 1.2.3.5 Witness prejudiced or other witness-related objections 1.2.3.6 Summary of bitemark testimony admission 63 65 SECTION 3 - STATEMENT OF PROBLEM 1.3.1 Summary 68 1.3.2 Statement of Problem 69 CHAPTER 2 ASSESSMENT METHODS OF DIAGNOSTIC EFFECTIVENESS 2.1 The assessment of diagnostic tests 71 2.1.1 Quantifying the effectiveness of diagnostic tests 72.1.2 The use of forced decision models in the assessment of bitemark overlays 78 2.2 The use of ROC in the assessment of diagnostic accuracy. 79 2.2.1 Area under the curve .82 2.2.2 Applications of ROC in forensic bitemark analysis 82.3 Summary 84 2.4 Application of ROC and forced decision models to bitemark overlay comparisons 85 CHAPTER 3 METHODS AND MATERIALS 3.1 Number and nature of examiner groups, examiners and cases 87 3.2 The simulation of bitemarks for study 88 3.2.1 Selection of dentitions and determination of biters 83.2.2 Production of occlusal views and overlays 9 3.2.3 Creation of bitemarks on animal model 91 3.3 Study One 94 3.4 Study Two 5 CHAPTER 4 RESULTS SECTION 1 4.1 Study One 99 4.1.2 Pilot data 100 4.1.3 Intra-examiner reliability. 104.1.4 Summary statistics from Study One 101 SECTION 2 4.2 Study Two 103 4.2.1 Results from ABFOs 104 v 4.2.2 Results from ASFOs 107 4.2.3 Results from GDPs 9 4.2.4 Comparisons between the three examiner groups __ Ill 4.2.5 Comparison of ABFO groups from Study One and Study Two 112 CHAPTER 5 DISCUSSION 5.1 Experimental design 116 5.1.2 Selection of examiners5.1.3 Number of simulated cases 118 5.1.4 Use of simulated cases 9 5.1.5 Selection of dentitions 121 5.1.6 Choices of conclusion level 2 5.1.7 Materials provided to the examiners 123 5.1.8 Summary 126 5.2 Findings5.2.1 Intra-examiner reliability. 125.2.2 Inter-examiner reliability 9 5.2.3 Effectiveness 132 5.3 Impact on discipline 14CHAPTER 6 CONCLUSIONS 6.1 Experimental design 159 6.2 Overlay reliability.6.3 Overlay effectiveness 160 REFERENCES References ; 149 Legal References 204 APPENDICES Appendix A - Articles examined during literature review 160 Appendix B - Analysed U.S. Appellate cases 176 Appendix C - Sample case materials 21Appendix D - Data 214 Study One 5 Study Two 223 vi LIST OF TABLES 1 Northwestern's classification of bitemarks 7 2 Anatomical distribution of 148 bites classified by sex of victim 10 3 Common types of physical evidence 12 4 Distribution of males, females and races in Rawson's study 41 5 Various overlay fabrication techniques ranked according to accuracy 46 6 Frye and Daubert - a comparison 53 7 Distribution of the Appellate bitemark cases examined by U.S. State 54 8 Decisions of two forensic dentists examining 29 bitemark suspects 73 9 Kappa values and strength of agreement 74 10 Five levels of conclusion recommended by the ABFO 80 11 Areas underneath ROC curves of different observer groups 84 12 Distribution of bites among the ten simulated cases 88 13 Materials for producing overlays 89 14 Materials for producing bitemarks on pigskin 91 15 Distribution of biters and non-biters among the ten simulated case 92 16 Summary of kappa values from Study One 101 17 Results from ROC analysis of ABFOs 106 18 Cross-wise kappa comparison between ABFOs 1019 Results from ROC analysis of ASFO members 108 20 Cross-wise kappa comparison between ASFO members 109 21 Results from ROC analysis of GDPs 110 22 Cross-wise kappa comparison between GDPs 111 23 Summary of means from the FDM and cross-wise kappa analyses 112 24 Summary of ROC analysis 1125 p values for inter-group comparisons 114 26 Comparisons between ABFO results from Study One and Study Two 114 27 Studies examining intra-examiner agreement of dental diagnostic tests 127 28 Studies examining inter-examiner agreement of dental diagnostic tests 130 vii 29 Examples of sensitivity and specificity from dental diagnostic tests 134 30 ABFO Conclusion levels with pooled group results 139 31 ROC results from the ABFO workshop 140 TABLES IN APPENDICES Al Bitemark literature review including citation analysis 161 B1 State-by-State summary of indexed cases 176 B2 Common objections to bitemark evidence from 103 cases 177 B3 Bitemark cases in the appellate Courts 178 vm LIST OF FIGURES 1 Dr. Amoedo (1863- 1945) 5 2 a) Example of a bite with a high degree of forensic significance 8 b) Bite demonstrating clear marks caused by the maxillary teeth 8 c) Postmortem bite 8 3 An injury caused by a knife mimics a bitemark 9 4 A footprint from a crime scene showing a Nike® logo 13 5 A simulated human bitemark on pigskin 14 6 A comparison of a tool mark with a suspect screwdriver 17 The ABFO No. 2 Scale 18 8 The use of an orthopaedic bandage to stabilise an impression of a bite* 19 9 Recovery of a ligature mark from the neck of a homicide victim* 20 10 Collection of evidence from the victim of a bitemark* 21 11 DNA collection from a suspect by buccal swab and lanced finger* 22 12 Taking dental impressions from a suspect* 23 13 Common categories of publication in the bitemark literature 30 14 An example of skin distortion from DeVore's study 33 15 Bitemark patterns produced by twins 38 16 Variation in pattern obtained by using the same dentition 39 17 The number of possible positions for each tooth identified by Rawson 42 18 Overlays from same dental cast using variety of techniques 45 19 Images that Nam's computer program correctly identified as a 'T' 47 20 2x2 contingency table of data presented in Table 8 73 21 2x2 contingency table comparing diagnostic test to gold standard 75 22 The probability distributions of the results of a perfect diagnostic test 76 23 Diabetic and non-diabetic patient distribution using FBS levels 77 24 The use of an ROC curve to graphically represent threshold values 81 25 ROC curve from six groups of observers 83 26 Occlusal view of one of dental models used in the study 89 ix 27 Summary of overlay production 90 28 Bitemark overlay showing 12 anterior teeth 91 29 Colour and black-and-white photographs of a simulated bitemark 93 30 Summary of production of simulated bites 931 Mandibular wax bites from Suspect A and B, Case 1 94 32 Completed answer sheet from Study One 95 33 Completed answer sheet from Study Two 6 34 Example of data from Study One 99 35 Example of data from Study Two 103 36 Distribution of Don't Knows among ABFOs 105 37 Distribution of Don't Knows among ASFOs 107 3 8 Distribution of Don't Knows among GDPs 110 39 Summary of Don't Knows for each examining group 113 40 A comparison of postmortem (left) and antemortem (right) bitemarks* 120 41 Differing occlusal tooth height represented on scanned model 125 42 ROC curve from ABFO Workshop data 140 These Figures are from the personal collection of Dr. David Sweet, and used with his kind permission. I am grateful for the opportunity to reproduce them in this thesis. LIST OF ABBREVIATIONS ABFO American Board of Forensic Odontology ASFO American Society of Forensic Odontology BM Bitemark DABFO Diplomate of the American Board of Forensic Odontology DK Don't know FN False negative FNR False negative rates FP False positive FPR False positive rate GDP General Dental Practitioner NPV Negative predictive value PPP Positive predicative value PREV Prevalence PTP Post test probability ROC Receiver operator characteristics TN True negative TP True positive A suspect may lie through his teeth, though the teeth themselves cannot lie. JFurness, 1981 Acknowledgements There are many people to thank for their time, effort, support, and help during my time here in Canada. First, to my parents, who have always supported me financially, academically, and emotionally and who continue to be my inspiration by their personal dedication to education and career focus. Secondly, to my friends and colleagues in the UK, whom I have missed profoundly, and who supported my decision to leave the homeland, Drs. Gillian Adams, Christopher Davies and Paul Keniry, Catherine Fell and my brother, Robert. A special mention to David Webb and Louise Maggi—to David for his help and assistance with academic endeavours and to Louise for allowing him to spend his time on such projects. Thanks also to my dental school tutor, Dr. Peter Heasman, and to Gary and Fiona Stacey. I must now turn to my new friends I have made here in Canada, Ms. Juliana Kim, Dr. Dean Hildebrand, Ms. Linda Chin, Dr. Bruce Marshall, Dr. Teri-Lee Norfolk, Mr. Christopher Lannon, and Ms. Amber Butler. Their support and encouragement has been a constant help and I have been privileged to know them. Sincerest thanks to my committee members for their guidance and support, Drs. J. Douglas Waterfield, Gerardo Maupome, and Edward Putnins. Also thanks to Dr. Pierre Dow, for his assistance and friendship and to all the examiners who participated in the study. Finally, I wish to thank Dr. David Sweet, my supervisor, mentor, and friend who introduced me not only to forensic odontology, but also to the world of academia. His devotion to forensic science is truly inspirational. The purpose of this work is to advance the science of odontology, and yet without dedicated scientists like David this ideal will never be achieved. I feel privileged to have been David's first graduate student, and this work represents a true collaboration. From a summer in 1997 to a Masters in 2000,1 shall always be in his debt. xiii Dedication I dedicate this work to my Grandfather: Mr. John Walker 1920-1997 Sources of funding The author gratefully acknowledges financial support from the following groups: Research Scholarship Forensic Science Society Research Grant Award American Board of Forensic Odontology Research Grant Award American Society of Forensic Odontology xv Preface As we enter a new millennium, society is faced with fresh challenges in every conceivable area. Despite the leaps in modern technology, medical breakthroughs and geographical changes that the last century brought, crime still runs through all aspects of our lives. Violent and heinous activities occur everyday, shattering the lives of victims, their friends, and families. Often, little can be done to repair such damage; however, the apprehension of the perpetrator and subsequent conviction is essential to maintain law and order. In a small, but significant way, dentistry plays a role in this process, through the speciality of forensic odontology. By identifying the victims of crime and disaster through dental records and assessing bite injuries, dentists assist those involved in criminal investigations. Always part of a bigger team, such personnel are dedicated to the principles of all scientists involved in forensic casework: the rights of the dead and those who survive them. xvi CHAPTER ONE INTRODUCTION SECTION 1 BITEMARKS IN FORENSIC DENTISTRY 1.1.1 FORENSIC DENTISTRY Forensic dentistry, forensic odontology and legal dentistry (the terms are synonymous) can be described as the overlap between the specialist fields of dentistry and the law. A more limited definition can be adapted from that usually quoted for forensic science in general: "Forensic dentistry is the application of dental knowledge to those criminal and civil laws that are enforced by police agencies in a criminal justice system" [1]. The applications of forensic dentistry are expanding constantly, mirroring advances made in other forensic disciplines. In addition to the identification of whole or fragmented bodies, forensic dentists may be asked to assist in determining age, race, occupation, previous dental history and socio-economic status of found human remains [2]. Information can be sought regarding bitemarks in living and deceased victims and many dentists will become involved in the medico-legal aspects of the discipline [2,3]. Forensic dentists will typically devote most of their time to assisting investigative agencies to identify found human remains. Identifications are achieved by the comparison of antemortem and postmortem dental records and using the unique features visible on dental radiographs, including both those resulting from dental treatment and those that occur naturally [4]. Recently, scientists have extracted deoxyribonucleic acid (DNA) from teeth to identify individuals [5]. Dental identifications are frequently employed in a mass disaster situation where the number of deceased is usually high and identifications confounded by body fragmentation and extensive perimortem damage [6]. Forensic dentists are usually familiar with the presentation of evidence in Court and thus are often asked to render opinions in medico-legal cases. Such cases typically involve a civil action brought by a patient against a dentist regarding the quality of care provided. Forensic dentists may testify as to the nature of the treatment, and whether or not the case is justified. Another responsibility of the discipline is the detection and documentation of physical and sexual abuse, whether it is child, spousal, or elder. In many instances of abuse, oro-facial injuries are present and it is the duty of dentists to recognise and report 3 such findings. Forensic dentists are often involved in the education of general dentists about abuse through lectures, workshops, and the generalist dental literature. Although they represent a smaller proportion of most forensic dentists' caseload, bitemarks present the most challenging aspect of the discipline. Bitemarks can be found on a victim, suspect, or inanimate object at the crime scene. The identification of such patterned injuries and subsequent comparison to a suspect's teeth may reveal important links between the suspect, the victim, and the scene [7]. Bitemark analysis represents one of the most contentious issues in forensic dentistry and is discussed in detail in Section 2 of this chapter. While the demands on the forensic dentist are both wide ranging and particular, the certification of forensic dentists varies. Many dentists are called upon on an ad hoc basis and may have no further education other than their basic dental training. Others pursue board certification from the American Board of Forensic Odontology (ABFO). A Diplomate of this organisation (DABFO) has been involved with a minimum number of forensic cases, including identifications and bitemarks, and has passed a comprehensive examination. In the United Kingdom, a one-year university diploma course is offered leading to an award of the Dip.F.Od. Around the world dentists are now able to pursue postgraduate research in forensic dentistry or related speciality leading to degrees at the Masters or doctoral levels. 1.1.2 THE HISTORY OF FORENSIC DENTISTRY The history of forensic odontology is extensive and can be traced back to medieval times when wax seals on important documents would be imprinted by the teeth of the sender to prove authenticity, so called indentured documents. History that is more contemporary dates to the late 19th Century when the first significant use of forensic dentistry occurred in Paris, France. On the 4th May, 1897, 126 people met their death during a fire at the Bazar de la Charite. The source of the fire was found to be an explosion in a gas lamp being used in a cinematograph [8]. Visual examination was used to determine the 4 identity of the victims but, due to the condition of many of the bodies, 30 individuals remained unidentified. Following a suggestion by an important diplomat, dentists were called to assist in the identification of the remainder of the bodies. This was completed with great success, and was described by dentist Dr. Oscar Amoedo, regarded by many as the father of forensic dentistry [8]. Although not directly involved in identification of the victims of the Bazar de la Charite fire, Dr Amoedo addressed all those who were. His doctoral thesis "L'Art Dentaire en Medecine Legale" was published in 1898. See Figure 1. FIGURE 1 Dr. Amoedo (1863 - 1945) From Hill, 1984 Forensic medicine has been taught in Canada since 1845. In 1920, a treatise on forensic medicine was published by Dr. Wilfred Derome [8]. Within this volume, references to bitemarks and dental identifications can be found. The 17th September, 1949 heralded the first major use of dental identification in a Canadian mass disaster when the S.S. Noronic caught fire. One hundred and eighteen bodies were identified by dental records with a team of over 40 dentists assigned to the identification process. It is disheartening to note that many of the complaints made by these dentists, namely of poor practitioner records, are still voiced by their contemporary colleagues [9,10]. 5 In 1970, the Canadian Society of Forensic Dentistry was founded and this was incorporated into the Canadian Society of Forensic Science in 1972. The introduction of dental records into the Canadian Police Information Centre (CPIC) was a major advance achieved by the Society [8]. The contemporary history of bitemarks is thought to have started with Sorup. In 1924, Sorup used transparent paper, upon which biting edges of a suspect's dentition were rendered, to compare with life-size photographs of a bitemark [11]. The earliest bitemark case in documented U.S. law is thought to be that of Ohio v. Robinson, in 1870. Charged of murdering his mistress, Ansil Robinson was acquitted despite the fact that evidence matching his teeth to a bitemark on the victim's arm was presented [12]. 1.1.3 HUMAN BITEMARKS AS FORENSIC EVIDENCE Human bitemarks occur in the most violent of crimes tried in the criminal Courts. Bites have been found in cases of homicide, attempted homicide, sexual assault, assault, and child abuse [13]. Bites have been described as occurring on both the victim and the suspect: teeth are used as a weapon by the aggressor and in self-defence by the victim [14]. It should also be noted that bites may be self-inflicted either in a deliberate attempt to make a false claim or involuntarily e.g., hand pressed against mouth to stifle a scream) [15,16]. It is thought that self-inflicted biting may be an emotional response to trauma or a type of counter irritation to alleviate pain [16]. 1.1.3.1 Appearance Bite injuries can be found at almost every anatomical location, although some sites are more common than others are. The appearance of a bitemark is dependent upon a number of different variables, such as anatomical location (fat deposition, underlying hard tissue, skin thickness, elasticity, and vascularity), number of teeth contacting the skin, amount of force, direction and type of biting action, the biter's occlusion and oral health, and whether the victim was alive when the bite was inflicted [17]. In living 6 victims, the effect of healing will alter the appearance of a bitemark over time. Postmortem bites lack the classical erythema and contusions found with vital bites. Bites can also be found on foodstuffs and less frequently on a variety of other materials such as chewing gum and paper towels [18-23]. A classification of bitemark injuries related to severity and appearance was developed at Northwestern Dental School, Chicago, IL and is shown in Table 1. It is important to note that this classification has not been officially adopted but is commonly used. Examples of typical bitemark presentations are shown in Figure 2. In Figure 2a, the arrow indicates a gap in the bitemark. This could be because the tooth is not present or it did not mark the skin because the tooth is loose or displaced from the arch. The arrows in Figure 2b illustrate points at which the anatomy of the lingual surfaces of the teeth is represented on the skin. Figure 2c demonstrates the appearance of a postmortem bite. Erythema and bruising are not seen in perimortem or antemortem bites. This is illustrated in Figures 2a and 2b. TABLE 1 Northwestern's classification of bitemarks CLASSIFICATION DESCRIPTION Class I Erythema Class II Contusions or bruises Class III Abrasions Class IV Lacerations Class V Avulsion Bites usually appear as oval or circular contusions, bruises, or abrasions. Sometimes indentations, lacerations, or avulsions made by specific teeth are seen on the skin surface. In most cases bites show markings from the upper and/or lower six anterior teeth. In some instances, bites have been identified with molar teeth represented on the injury. A double-arched pattern is a common presentation of human bites [24]. Despite the described presentations in terms of location, appearance, and severity, there are some 7 basic features of bites that can be used to identify them [24]. The initial identification of an injury as a bitemark is a pre-requisite to the proper handling of the evidence. FIGURE 2a An example of a bite with a high degree of forensic significance From Whittaker, 1989 • i ft 50 40 FIGURE 2b Bite demonstrating clear marks caused by the maxillary anterior teeth From Whittaker, 1989 4 * FIGURE 3c A postmortem bite Bites can be created in a number of ways. They can be the result of direct contact from the teeth, by the tissue being pressed against the teeth by the tongue, or by a scraping action [2], Bites can occur singly, but are often present at multiple sites or multiple bites at a single location [13,25]. Bitemarks are therefore complex injuries and their recognition and interpretation of forensic significance relies upon a thorough understanding of the mechanisms involved [2]. Not all marks on skin are caused by human bites. Many injuries can replicate the classical semi-circular appearance of a bite [26]. Cardiac defibrillators and electrocardiogram monitors removed after a patient has died in an emergency room can leave bruises resembling the characteristics of bites [27]. Figure 3 illustrates a knife wound that appears similar to a bite injury [2]. Definitive methods of determining a bite injury involve saliva swabbing to test for the presence of salivary amylase or DNA [28,29]. Animal bites such as dog bites, may also be found on individuals, and can be of forensic value [30,31]. FIGURE 3 An injury caused by a knife mimics a bitemark From Whittaker, 1989 1.1.3.2 Anatomical location Several studies have analysed the location of bitemarks on victims [13,32,33] and recently on suspects [25]. The studies vary in the most common bite location, although the majority of cases would suggest that females are most frequently bitten on the breasts 9 and males on the extremities. Table 2 shows the distribution of 148 bites found in 101 U.S. cases. The results of the studies demonstrate that human bitemarks can be found at almost every anatomical location, although there is clearly a bias toward certain areas. The crime type, age, and sex of the victim influence the likely anatomical location of a bite injury. TABLE 2 Anatomical distribution of 148 bites classified by sex of victim LOCATION FEMALES (%) MALES ( %) PERPETRATORS VICTIMS Abdomen 2.8 0.0 0.0 Arms 13.0 36.4 27.2 Back 7.5 0.0 9.1 Breast 40.0 0.0 0.0 Face or head 6.6 0.0 0.0 Foot 0.0 0.0 0.0 Genitals 6.6 0.0 0.0 Hands or Fingers 3.8 18.2 0.0 Legs 7.5 0.0 0.0 Neck 6.6 0.0 0.0 Shoulder 0.0 0.0 9.1 Thigh 5.6 0.0 0.0 100 100 Results show that females are four times more likely to be bitten than males, and the bites are concentrated on the breasts, arms, and legs in descending order of frequency. Female children may suffer a multitude of bites to many body locations, but primarily to the face, legs, and arms. Males are most frequently bitten on the arms, back, and hands. Significant proportions of male bitemark victims are themselves the perpetrators of the violent act. It is common to find more than one bitemark on a victim, often in a different anatomical location from the first [13,25]. 10 1.1.3.3 Forensic significance Bite injuries can be a pivotal aspect of the criminal investigation since they can establish that a suspect was in violent contact with the victim [7,21,34,35]. A bite on an abused child can indicate that other injuries may not be accidental. Bites can also provide evidence that a suspect was present at a particular crime [23]. In order to ensure that this type of evidence is retained, it is important for odontologists to inform investigators about the proper recognition and preservation of bitemark evidence [36]. It is the role of forensic odontologists to confirm that a particular injury is indeed a bitemark, to collect the required evidence from both the victim and the suspect, and to analyse the bite in light of the collected evidence. Good practice encourages odontologists to present their results in a written report, adhering to strict guidelines relating to wording and levels of conclusion. As described above, bitemarks have a varied appearance and hence their forensic significance or level of discriminatory ability varies. Those bites that are at the extremes of the Northwestern Classification typically offer little in the way of forensic significance. Very mild, diffuse bruising can be problematic to interpret, while an avulsive injury leaves little impression of the teeth used to produce the bite. Bites represent the shape and form of the biter's teeth; some people's teeth have many individualising features, while others do not. The central tenant of bitemark analysis is that each person has a unique dental arrangement and that these unique features are sufficiently replicated in a bitemark to identify an individual to the exclusion of all others. 1.1.3.4 Bitemarks as physical evidence Physical evidence is widespread at most crime scenes. Table 3 illustrates some of the most common types of physical evidence that can be found at crime scenes. This type of 11 evidence can yield significant information about the nature and circumstances of a crime. Bitemarks and toolmarks are described as impression evidence in Saferstein's classification [1]. Many of the terms used in the discipline of toolmark or firearms examinations can be applied to bitemarks. TABLE 3 Common types of physical evidence EVIDENCE TYPE LIKELY SOURCES Blood, Semen, and Saliva This includes blood or semen dried on fabrics or other objects and cigarette butts that may contain saliva Documents Any item demonstrating handwriting or typing. Related items include paper, ink, indented writings, and burned or charred documents Drugs Any substance seized in violation of laws regulating sale, manufacture, distribution, and use of drugs Explosives Any device containing an explosive charge or any items that have been affected by such a charge including those suspected of containing explosive residue Fibres Any natural or synthetic fibre whose transfer may be useful in establishing a relationship between objects and persons Fingerprints All prints, both latent and visible Firearms and Ammunition Any firearm, as well as discharged or intact ammunition Glass Any glass particle or fragment that may have been transferred to a person or object Hair Any animal or human hair that could link a person with a crime Impressions This includes tire markings, shoe prints, depressions in soft soils, and all other forms of tracks. Bitemarks in skin or foodstuffs. Toolmarks Organs and Physiological Fluids Body organs and fluids are submitted for toxicology to detect the possible existence of drugs and poisons Paint Any paint, liquid or dried, that may have been transferred Plastic Bags Commonly found in drug situations; may be matched to others in the possession of a suspect Powder Residues Any item suspected of containing firearm residues Serial Numbers All stolen property submitted to a laboratory for the restoration of erased identification numbers Soil and Materials All items containing soil or minerals that could link an individual to a particular geographical location Vehicle Lights Examination of vehicle headlights and tail-lights is normally conducted to determine whether a light was on or off at the time of impact Wood and Other Vegetative Matter Any fragments of sawdust, wood, or vegetative matter discovered on clothing or shoes that could link a person or object to a crime The examination of physical evidence by a forensic scientist is usually undertaken to identify its origin, and this is also true of bitemarks. The analysis regimen for bitemarks is broadly split into two main components. Firstly, metric analysis that involves the measurement of specific traits and features. Secondly, the comparison of the configuration and pattern of the bite injury to that of the suspect's teeth. This comparison 12 of features is often referred to as pattern association [7]. Specific terms are used to describe the features or characteristics of patterned injuries [37]. Three main classifications of characteristic exist: gross, class and individual [37]. Gross characteristics are those that identify the general origin of the object. Gross characteristics determine that, for example, a pry mark on a brass striker plate was made by a screwdriver and not a tire-iron. A semi-circular injury with a central area of ecchymosis and surrounded by small areas of incision or bruising demonstrates the gross characteristics of a bitemark [1,7,38,39]. Class characteristics can be defined as properties of evidence that can only be associated with a group and never with a single source [1]. An example of this is shown in Figure 4. The footprint has a clear impression of a manufacturer's logo. This class characteristic indicates that Nike® rather than Reebok® manufactured the shoe, but it does not permit identification of a specific Nike® shoe [40]. FIGURE 4 A footprint from a crime scene showing a Nike® logo in the centre of the print Class characteristics result from design features and are determined before manufacture. As with other items, class characteristics of bitemarks are those measurable features that indicate a restricted source or origin [7,41]. Sweet describes dental class characteristics as "... the number and shapes of individual teeth and the familiar arched arrangement of From Safer stein, 1998 13 the teeth in the upper and lower jaws". Using measurements it can be determined if a bitemark has been created by a child or an adult, i.e. defining a group but not an individual. Figure 5 illustrates class characteristics using a simulated human bitemark: the number 1 indicates the upper dental arch, and number 2 indicates the lower dental arch. Several distinct teeth are shown, allowing identification of incisors, canines, and premolars. FIGURE 5 A simulated human bitemark on pigskin •i i|iui|Hii|ii MMi niiHi ii-l|H „" i|M'Mmwimin Individual characteristics, often called accidental characteristics, can be defined as properties of evidence that can be linked to a common source with an extremely high level of certainty [1]. Individual characteristics are created by the use, misuse, or abuse of a tool or are caused by unique irregularities that occur during its manufacture [7,42,43]. Figure 6 shows individual characteristics on a screwdriver. Note how the presence of irregularities on the edge of the tool helps to individualise the tool to the mark. FIGURE 6 A comparison of a tool mark with a suspect screwdriver From Safer stein, 1998 14 Individualising characteristics on teeth can be divided into two main categories: developmental and acquired. Developmental features that can be considered unique include prominent marginal ridges, additional cusps, talon cusps, macro- or micro-dontia and genetic abnormalities of tooth form. Acquired characteristics include restorations, fractures, occlusal adjustments, and occlusal wear [17,24,34,44]. These characteristics provide the odontologist with the necessary detail to enable a single person to be identified as the biter. It should be remembered that some dentitions are likely to be highly unique exhibiting numerous individual characteristics while others, possibly in younger suspects, may offer fewer individualising features. Of importance in the final analysis is the replication of these individual features in the bitemark to an extent that they can be compared to a suspect's teeth. The inherent problems of physical bitemark analysis are discussed in depth later in this Chapter. 1.1.3.5 Bitemarks as biological evidence In an attempt to address some of the limitations of physical bitemark analysis, researchers turned to biological evidence. The potential for human bitemarks to yield biological evidence has been known for many years [28]. Initially this evidence was limited to the blood typing of saliva stains using ABO antigen groups [45]. Later, Sweet found that saliva deposited by a biter could be collected, using a double-swab technique, and would yield DNA for forensic analysis [29]. Indeed, it has been possible to analyse DNA from bites on victims who have been subjected to extreme environmental conditions [46]. The advent of the polymerase-chain-reaction (PCR) technique has ensured that the use of DNA will play an increasingly crucial role in the investigation of bite injuries. DNA analysis avoids many of the pitfalls associated with physical bitemark comparisons, but it does not represent a forensic panacea. Contamination, degradation, expense, and environmental assaults [47-49] all restrict the use of DNA analysis. However, DNA 15 analysis represents the most scientific, and defensible, method of bitemark analysis currently available to the forensic investigator. The advent of salivary DNA analysis raises an important question - why investigate physical analysis when more discriminatory techniques are available? Despite the clear advantage of salivary evidence, the use of the technique has been limited, normally by lack of experience or knowledge, and efforts are required to promote further this method within the field. Physical bitemark evidence will always play an important part in criminal investigations. The relative simplicity of physical comparisons compared to the seemingly esoteric nature of DNA is easily explained to juries. DNA can sanitise an attack, while the use of physical bitemark evidence can effectively demonstrate to a jury the violent and heinous nature of a crime. Physical evidence is, and is likely to remain a crucial part of bitemark evidence. 1.1.3.6 Bitemarks as psychological evidence The two classical sources of evidence, namely physical and biological, have been described above. Recent advances in criminal profiling have suggested that a third source of evidence may be elucidated from bitemarks, that of the psychological profile of the biter [50]. Research in this area is limited to three articles, with further work required to determine the value and validity of this source of evidence [51-53]. Current theories suggest that psychological techniques, such as Personal Construct Theory, may be applied to this problem [53,54]. It is worrying to note that despite the dearth of validated studies in this area, psychological evidence has been presented in Court. See Section 2 for examples of Court use of psychological evidence. 1.1.4 COLLECTION OF BITEMARK EVIDENCE Obtaining the correct evidence in a proper manner is crucial to bitemark investigations. The legal system demands that evidence should be collected in a particular way and the effectiveness of any subsequent analysis is dependent on the availability of high quality 16 materials. The collection of evidence is initiated following the identification of an injury as a bite. Due to much criticism of forensic science disciplines from the Courts concerning the lack of standards for evidence collection, the ABFO appointed a committee in 1981 to develop guidelines for bitemarks [55]. From February 18-20*, 1984 the bitemark workshop in Anaheim, CA, decided upon the evidence collection guidelines that would be recommended to the profession. These original guidelines have been modified and refined to reflect advances in the discipline. It is important to note that these guidelines are not standards, and hence there is no compulsion for individuals to follow them. The adoption of rigid standards was thought to be counterproductive to the development of new techniques and thus the guidelines remain as recommendations only. However, odontologists will have to explain any reason for departure from such recommendations and may open themselves to criticism in Court, particularly if a recommended item of evidence is not collected. 1.1.4.1 Victim evidence It is important to ensure that suitable permission has been granted before the collection of evidence from deceased or unconscious victims. Living victims may give informed consent, unless they are also implicated as a suspect in the crime in which case an authorised seizure warrant is indicated. The process of evidence collection begins with extensive photography of the bite injury. Photographs should ideally be taken in both colour and black-and-white, with and without ruler scales. Close-up photographs of injuries should be supplemented by orientation images. The scale preferred for bitemark photography is the ABFO No. 2 scale, Figure 7 [56]. The ABFO scale is an L-shaped spatial reference allowing accurate scaling in both vertical and horizontal directions. The circles on each corner permit correction of errors created by oblique camera angles. The 18% grey areas allow accurate colour matching during laboratory processing of the film. Bitemark photographs 17 should be exposed with and without this scale to prevent accusations of 'hiding' important areas of the injury under the ruler. FIGURE 7 The ABFO No. 2 Scale It is beneficial to take serial photographs of the injury over a number of days in living and deceased victims as the bite can alter in appearance over time [36]. These changes may benefit later comparative analyses. Following photographs, the bite injury should be swabbed for saliva. In the past, salivary evidence was used to establish, via the presence of salivary amylase, whether or not an injury was a bitemark. Increasingly sensitive techniques allowed the analysis of ABH blood groups from saliva but DNA typing has now replaced this [28,29,46]. Saliva should be collected using the Double Swab Technique [29]. This simple technique involves the use of two sterile swabs. The first swab is moistened with sterile distilled water and rubbed gently over the bite injury. A second, dry swab is used to collect the moisture from the skin deposited by the initial swab. Both swabs are dried and stored in appropriate containers. It is thought that the initial swab re-hydrates dried salivary cellular material that is subsequently collected by the second swab [29]. Following swabbing, surface impressions should be taken of the bitemark whenever it appears that this will yield useful information. In practice it is wise to take impressions 18 of all but the most decomposed injuries to ensure that established protocols are followed [24]. It is usual to use dental impression materials that have a high degree of accuracy, such as those used for recording the details of crown and bridge preparations (silicones, typically vinyl polysiloxanes (VPS)). The ability of these materials to resist distortion and permit the pouring of several casts from a single impression makes them well suited to this application [24]. Any materials employed should be approved by the American Dental Association (ADA) and used according to manufacturers' instructions. The surface impression should be supported using a rigid backing material to prevent distortion. This is usually achieved by placement of a thermoplastic orthopaedic bandage called Hexcelite® on to the surface of the impression material [24]. Other materials used for this purpose include Hygon-acrylic, plaster, or VPS putty. The bandage should be pre-moulded to the curvature of the skin by soaking in warm water and then gently adapting it to the skin's surface before impression taking, see Figure 8. The impression should be cast as soon as possible using a hard stone, again using an ADA approved material. This technique has also been used to recovery other patterned injuries from skin. Figure 9 illustrates the recovery of a ligature mark from a neck. FIGURE 8 Use of an orthopaedic bandage to stabilise an impression of a bite injury 19 FIGURE 9 Recovery of a ligature mark from the neck of a homicide victim Although recommended by the ABFO, the collection of tissue samples from deceased victims is a debated issue [55]. Many odontologists believe that the removal of skin bearing a bite injury is an essential part of evidence collection protocol, and using transillumination such evidence can provide useful detail that might otherwise be unobtainable. Others find the process distasteful or argue that little additional detail can be acquired. The ABFO recommends that tissue be removed whenever it may provide useful information. Many articles have been written describing effective means of removing the skin [57], stabilising it [58], and how to examine it after removal [59]. The validity of using removed skin in comparative bitemark analyses has not been established. When examining injuries using transillumination, only class characteristics can be identified [60]. Class characteristics can be used to exclude an individual but are insufficient to identify positively a suspect. Issues of distortion and preservation have yet to be addressed [60]. It is normal practice to take an impression of the victim's teeth to ensure that the bite was not self-inflicted unless the anatomical location indicates that this would be impossible. Figure 10 summarises the collection of bitemark evidence from victims. Written notes should be taken indicating the time that each procedure was performed. Photographs are 20 normally taken during all these procedures as a means of documentation in addition to those films used as evidence proper. FIGURE 10 Collection of evidence from the victim of a bitemark 1.1.4.2 Suspect evidence collection Before collecting evidence from a suspect, it is important to ensure that all of the necessary legal documents have been acquired. This may include warrants, Court orders, or informed consent from the suspect. It should be noted that some jurisdictions restrict the collection of certain items of evidence, e.g. dental impressions. Some warrants will permit the use of reasonable force to acquire the evidence while others do not. The collection of DNA evidence may have to be performed by a physician even if the procedure is limited to a buccal swab [60]. 21 Upon meeting the suspect, the first evidence that should be collected is a brief dental history detailing any dental treatment carried out following the alleged date of the bite injury. This dental interview may also allow an opportunity to establish rapport with the suspect and make subsequent examination easier. Following the dental history, extensive photography should be undertaken. Colour images of the suspect's facial profile, maximal opening, lateral, occlusal, and anterior views of teeth and close-ups of any unusual dental features should be taken. It is necessary to incorporate a scale in these photographs [55]. An extraoral examination should be performed to identify any soft or hard tissue factors likely to disrupt the biting process. The temperomandibular joints should be carefully examined and the muscles of mastication palpated. The extraoral examination should be followed by an intraoral examination [36]. All teeth should be carefully charted, with any recent restorative or surgical treatments noted. Periodontal condition, and in particular tooth mobility, should be noted. Damage to the anterior teeth, such as fractures or notches should be carefully noted and supplemented with additional photographs. If DNA swabs were taken of a bite injury and the warrant permits it, a DNA sample for the suspect should be recovered [55]. Commonly this is achieved by either a buccal swab or a blood sample from a lanced finger, which is placed on to an appropriate medium (e.g., FT A paper). See Figure 11. FIGURE 11 DNA collection from a suspect by buccal swab and lanced finger 22 Dental impressions of maxillary and mandibular teeth should be taken using an ADA approved material. The ABFO recommends at least two good quality impressions be taken of both arches [24]. A recent survey of odontologists found that alginate and a variety of VPS materials were used most commonly [61]. If the suspect wears full or partial prostheses, impressions should be taken with these both in situ and when removed. The impressions should be poured as soon as possible using an ADA approved stone material. It is crucial to follow the manufacturer's instructions at all times. The original model produced should be kept in pristine condition and used only as a Court exhibit. The second model produced should be used in subsequent comparative analysis [55]. A bite registration, such as with dental wax, should be produced and photographed. These wax bites can be used to assist in the articulation of casts or can be poured in stone to create a cast of the incisal edges. If the bite was found in a foodstuff, the ABFO recommends that the suspect bite into a similar foodstuff to create a sample bitemark. All materials should be retained until all legal proceedings, including appeals, are concluded. Figure 12 shows a suspect undergoing a dental impression. FIGURE 12 Taking dental impressions of a suspect 23 1.1.5 ANALYSIS OF BITEMARK EVIDENCE Following appropriate collection of materials from both the victim and suspect, the forensic odontologist must then analyse the evidence to see if there is a link between the two. One of the criticisms of bitemark analysis is the multitude of techniques that are used. A review of the literature finds numerous technical papers describing the use of scanning electron microscopy (SEM), tone-line photography, reflex microscopes, digital techniques, test bites (in wax, Styrofoam, etc.), bacterial products from saliva and many others. Achieving standards and consensus is difficult when so many different methods are employed, with fierce defence of such techniques by some and vigorous opposition from others. The most commonly used technique for physical bitemark comparisons is that of transparent overlays, first described in 1960 [62]. The basic premise of the overlay technique is that the incisal edges of the suspect's anterior teeth are accurately represented on a transparent acetate sheet. This sheet is then placed over a 1:1 photograph of the bite injury and a pattern association is performed. 1.1.5.1 Production of transparent overlays There are five common methods for the production of overlays for bitemark analysis: computer-based methods [63,64], hand-traced from the original stone dental casts [62], hand-traced from wax impressions, hand-traced from photocopied images [65], and radiographic techniques [66]. A recent study determined that computer generated overlays are the most accurate method of representing the biting edges of teeth in bitemark analysis [67]. Hand-Traced Overlays Several methods of producing overlays by hand tracing have been described. One of the simplest is to place an acetate sheet over the biting surfaces of the suspect's dental cast. 24 Securing the sheet with moderate finger pressure, a fine-tipped felt pen is used to outline each of the anterior teeth. This process is highly subjective and is also quite challenging as the acetate sheet has a tendency to slip during the tracing process. To address this problem, many individuals place a sheet of clear glass over the dental cast and lay the acetate over this, securing the sheet and allowing easier tracing. The method is very economical and proponents claim that it is effective [68]. Another method of producing hand-drawn overlays utilises wax and addresses the issue of acetate stability during tracing. In this system, the maxillary and mandibular study casts are pressed into a single wafer of dental wax. A representation of the incisal edges is produced. The acetate sheet is then simply rested upon the wax wafer and the outlines are traced. The acetate sheet is easily stabilised. This method is still highly subjective [66]. Critics state that it is not the biting surfaces of the teeth that are recorded, but rather the perimeter of the teeth at a level determined by the depth to which the cast penetrates into the wax [67]. The use of a photocopier to assist in the overlay process was first described by Dailey [65]. In this system, a photocopier is adjusted and calibrated to ensure life-sized reproduction. It is used to record an image of the study casts on to white paper. The xerographic image is placed on a light box with the image facing downwards. Using a fine-tipped pen, the biting edges of the teeth are traced on to the reverse of the page bearing the image. The sheet is then placed on to the photocopier's platen with the tracing of the teeth against the glass. A sheet of transparent acetate is loaded into the photocopier and the tracing is rendered on to the acetate. The process is somewhat time consuming. The calibration of photocopiers to ensure 100% reproduction is not always accurate across the whole area of the image. As this technique requires two photocopies, any errors will be multiplied. The subjective nature of hand tracing is also an inherent problem with this method. 25 Radiographic Technique Two main techniques utilise radiography to create an overlay. The first technique employs dental wax in which the biting edges of the teeth have been recorded. Amalgam powder or barium sulphate is mixed with alcohol to produce a slurry that is carefully placed into each of the wax hollows formed by the teeth. The alcohol content of the slurry evaporates in approximately five minutes leaving the wax impression coated with amalgam or barium. A radiographic film is then placed beneath the wax and exposed. Following radiographic processing, the teeth are represented on the film as radiopacities. Using photographic or computer techniques the image is inverted, which converts the teeth to black areas on a clear background [66]. This technique also results in an overlay that depicts the perimeter of the teeth, rather than the true biting surfaces. The second radiographic technique is described by Sweet. While slightly more complex, it does provide a true representation of the biting surfaces [69]. The method uses toneline photography and radiographic images of metal models of the suspect's teeth to produce the overlay. The initial step is to produce a metal cast of the suspect's teeth, using either a second impression of the suspect's mouth or a VPS duplicate impression taken from the study casts. A fusible metal is cast into the anterior region of the dentition. Following cooling of the metal, Type IV ADA-approved dental stone is poured into the impression, covering the fusible metal and casting the remainder of the teeth [69]. Acrylic is poured on to the mould and placed into a pressure cooker. Following acrylic polymerisation, the stone base is ground away until the cervical portion of the metal teeth is visible. A radiograph is exposed. The model is continually ground with radiographs exposed at 1-mm increments. The radiograph selected for the production of the overlay is that which contains the incisal edges of the teeth. Using toneline photography, the radiograph is converted to an image resembling a pen-and-ink drawing [69]. Although this technique allows a representation of the incisal edges of the teeth to be made (rather than the perimeter) its use in case work has been limited thus far [69]. Computer Generated Transparent Overlays 26 The use of a computer to create transparent overlays was described by Sweet [64]. The equipment required is relatively inexpensive, and many odontologists will have such a system at home. The technique was developed to reduce operator bias and subjectivity during overlay production. The technique is easy to follow for those with a moderate level of computer experience and is economical in both time and materials following the initial purchase of computer equipment. This work has used the computer technique for the production of overlays. The technique is described in detail in Chapter 3, Materials and Methods. It is worth stating that while the technique does reduce the level of operator involvement in the process of overlay production, it is still not a completely objective process. The ideal process of simply inserting a rendition of a suspect's cast into a computer and pressing a key for a transparency still alludes us. However, it can be stated that at the present time this technique presents the most accurate, and hence defensible, method of transparency fabrication. 27 SECTION 2 THE SCIENCE AND JUSTICE OF BITEMARKS 28 1.2.1 THE SCIENTIFIC BASIS OF BITEMARKS - A LITERATURE REVIEW The purpose of a literature review is to select and describe those articles that best represent the development and current status of a discipline. In forensic dentistry and bitemarks in particular, the volume, type, and quality of the available literature limit us. The publication habits of forensic scientists in general and the professional status of forensic dentists (most are general practitioners in private practice) defines the nature of the peer-reviewed materials. The analysis performed in this review concentrated on areas of contention within the field of bitemark analysis, and revealed a lack of valid evidence to support many of the assumptions made by forensic dentists during bitemark comparisons. The new level of judicial scrutiny of such scientific evidence is likely to emphasise this lack of knowledge upon which bitemark analysis relies. MedLine was utilised to locate articles pertaining to the forensic use of bitemark evidence. Searches were performed on the entire database, 1960 - 1999. In total 1508 articles were found that contained the keyword "Forensic Dentistry"; 120 English language papers within this group were related to bitemarks. Each of these papers was located and summarised, including a Science Citation Index value. The details entered in a Table. See Appendix A. For comparison purposes, it should be noted that within the 1960 - 1999 MedLine database there are 1457 articles related to Forensic DNA, 60 related to Forensic Entomology, and 3538 related to Periodontology. It is useful to point out that the vast majority of forensic dental literature relates to the use of dentistry in determining the identity of found human remains, so-called 'dental identifications'. The effectiveness of dental identifications is well established and rarely questioned in Court. The unique features of the human dentition used in personal identification cases are legally and scientifically well accepted. These features should not be confused with the individual characteristics of teeth used in bitemark identification cases. The debate over this issue is discussed in depth later in this Section. The practice of bitemark analysis is highly contentious and yet the issues associated with the technique do not appear to have initiated a desire among the community to deal with these topics in 29 the literature. The relative lack of material may reflect the inherent difficulties i bitemark research or time available to general dental practitioners for research. 1.2.1.2 Analysis of the articles Due to the relatively small number of papers found, it was possible to identify trends within the entire search. Approximately 20 papers have been published on the subject each year and the mid-eighties were the most productive period. The type of publication is an important aspect to consider when evaluating the core research and scientific basis for bitemarks. Figure 13 shows the distribution of the categories of papers among those identified in this research. It is interesting to note that case reports were responsible for 28% of the total literature while empirical research from well designed experimentation contributed only 15%. FIGURE 13 Common categories of publications in the bitemark literature. Values from 1960- 1999 30 Forensic dentists, lawyers, and others involved in the criminal justice system are concerned by this lack of a sound scientific basis for bitemarks. The lack of empirical evidence for a variety of assumptions made by forensic odontologists is frequently alluded to in the Courts (see next section). However, the forensic dental community continues to rely upon those few articles that have addressed the scientific issues. The plethora of case reports, reviews, and anecdotal commentaries cannot address this deficit. Many techniques have been developed to assess the value or quality of scientific articles. One such measure is the citation value. The citation value indicates the number of times that a particular article has been cited in another paper. The argument is that other authors will cite well-respected and valid papers; thus, the quality of the paper can be quantified. Citation analysis has many critics who state that the judgement of papers based upon a single value is flawed. A simple variable, such as the number of researchers in a particular field, will have a significant impact on the citation value. Others state that reviews often considered being of poorer scientific value than empirical research are often highly cited. This argument also extends to papers that describe laboratory techniques that are subsequently used by researchers in their own work. Use of the citation value in this thesis is judged valid, as the subject papers are from the same discipline. The citation value has been used to assess those papers that are regarded as significant within the field. Any reader of the forensic dental literature will be able to identify three or four articles that are constantly quoted. However, just how valid are these works? For those papers that have been cited since publication, the mean citation value is 8.1, although only 53 of the 120 papers examined had been cited at all. It is also important to recognise that self-citation can also contribute to the citation value. The frequency of reviews on the subject of bitemark analysis is also responsible for many of the citations. 31 Rothwell's 1995 review breathed life into many papers that had never been cited, or had not been cited for several years [70]. The popularity of forensic reviews in the general dental literature guarantees that such papers will continue to be cited giving an impression of continued scientific growth that belies the actual publication practices of the discipline. Rather than summarise the entire body of the literature, those papers that address the main areas of contention within bitemark analysis will be presented. Contentious areas in bitemark evidence are well accepted as the: a) accuracy of the bitemark itself, b) uniqueness of the human dentition, and c) analytical techniques [70]. Each of these is discussed below. 1.2.1.3 Accuracy of bitemarks on human skin The use of dental impression materials to record the features of the teeth and surrounding hard and soft tissues is well known and accepted [71]. Such materials are used in general and specialist dental practice daily to produce study casts and prostheses. A large volume of biomaterials literature and the anecdotal evidence of treatment success have established the accuracy of such materials. The considerable variation of bitemark presentations on human skin brings the accuracy of skin as a registration material into doubt [70]. While many studies have examined the accuracy of bitemarks on other substrates, such as cheese [72,73], apples [74], sandwiches [75,76], and soap [21], this review is restricted to human skin. This represents both the most debated area of substrate accuracy and the most commonly bitten material [70]. Skin is a poor registration material [77] since it is highly variable in terms of anatomical location, underlying musculature, or fat, curvature, and looseness or adherence to underlying tissues [78]. Skin is highly visco-elastic, which allows stretching to occur during either the biting process or when evidence is collected. In 1971, DeVore issued a preliminary report describing studies performed on the variability of bitemarks found on skin [79]. The experiment involved the inking of human skin (living volunteers) using a 32 stamp with two concentrically placed circles with intersecting lines. The resultant mark on the skin resembled a target. Following the placement of the stamp, the inked area was photographed with a scale in place. The subject was then instructed to change positions e.g., from lying to sitting or to move an extremity from extended to flexed. The geometric pattern of the stamp was then re-photographed and the resultant prints compared to assess the degree of distortion. Following the analysis of the photographs it was found that in all cases there was an expansion or shrinkage of the stamp, with a maximum linear expansion of 60% at one location [79]. The design of the stamp permitted the investigators to examine the distortion in both size and direction. DeVore concluded that, due to the level of distortion found, photographic images of a bitemark in comparative analysis should be used only if the exact position of the body can be replicated. The placement of a body in such a position is usually impossible, as the exact position of the body during an attack is rarely known. Figure 14 demonstrates a result from this experiment. The inked stamp has been placed on the subject's arm and the target photographed with the arm flexed and then extended. There was a linear superio-inferior expansion of 60% between the two images. Although difficult to see in this figure, the scale is located just superior to the inked mark. DeVore stated that further research to investigate the effect of postmortem changes on skin distortion were required. 33 In 1974, researchers from the Bioengineering Unit of the University of Strathclyde examined the features of the biting process likely to impact upon the appearance of bitemarks on human skin [80]. They described the differing characteristics of skin from a variety of anatomical locations; e.g., Langer's Lines represent directional differences in the degree of extensibility of skin. Like DeVore, they emphasised the importance of body location during biting as the directional variations or tension lines will alter with movement. The report also described distortion that can occur in skin after biting. The oedematous response of skin to trauma is likely to stiffen the area, thus rendering it more stable. However, the subsequent resorption of this fluid will cause a large amount of distortion [80]. They concluded that the changes in bitemark appearance are likely to be greater as the injury grows older. This was found equally applicable to both living and dead victims [80]. The article concluded that forensic odontologists where "still ignorant... of the conditions during normal biting... considerable research is required [to address this]" It was not until 1984 that this unresolved issue was re-visited, with an examination of the morphology of breast tissue. Rawson and Brooks published a paper in which they classified breast morphology to assist with the determination of distortion of bitemarks in this location [81]. It is interesting to note that Rawson states: "The nature of skin and its underlying structures are still of some concern and will probably be a major source of research interest during the next decade." The paper reviewed the literature regarding breast morphology but did little to explain how this could affect bitemark analysis. Following this paper, no other articles considered the topic from a physiological or anatomical viewpoint. Instead, attention was turned to the photographic treatment of distortion, and 1984 saw the publication of Krauss' article on photographic techniques [82]. Subsequent articles were published, including those on the development and use of appropriate photographic scales, yet no further work was performed on the quantification of bitemark distortion on human skin [56]. The discipline seemed more interested in dealing with the distortion, during evidence collection rather than considering the nature and degree of distortion that may 34 have occurred at the time the injury was produced. Rawson's expectation for further research has not been realised. There are probably several reasons that the issue of skin distortion has not been addressed further. Such research is expensive, involves human subjects and ethical reviews, and may require skills not normally held by forensic dentists. These difficulties are compounded by the difficulty of obtaining research funding from traditional sources (MRC, NSERC, NIH, etc.) for forensic research. From this review of the literature, it is possible to state that the issue of skin distortion in bitemark analysis has not been fully addressed and the cautions issued by DeVore, and others, should still be heeded today. 1.2.1.4 The uniqueness of the human dentition Bitemark analysis is based on two postulates: a) the dental characteristics of anterior teeth involved in biting are unique amongst individuals, and b) this asserted uniqueness is transferred and recorded in the injury [83]. A distinction must be drawn from the ability of a forensic dentist to identify an individual from their dentition by using radiographs and dental records and the science of bitemark analysis. Dental identification, as opposed to bitemark identification, utilises the number, shape, type, and placement of dental restorations, root canal therapies, unusual pathoses, root morphology, trabecular bone pattern, and sinus morphology [84]. The debate over the uniqueness of human teeth is probably one of the most fierce in current forensic dental discourse. Many forensic dentists, appellants, and lawyers have questioned this fact and demand to know from testifying experts the relative frequency of dental features identified in bitemarks. An examination of the literature divulges the scientific evidence for this commonly held belief. Before this examination, it is pertinent to separate the dental uniqueness used in dental identifications from the uniqueness of human bitemarks. Dental identifications use dental records and radiographs in a systematic and well-validated method that has little to do with the features examined during a bitemark analysis. There is little question that the identification of an individual 35 based on their dental records is a sound, scientific, and reliable method of identification [84,85]. The first article to consider the statistical nature of dental uniqueness was published by MacFarlane and Sutherland in 1974 [86]. The authors began by differentiating between "positive" and "negative" features of the dentition. A positive feature was described as the presence of a tooth with a certain rotation or other individualising feature. A negative feature was the absence of a tooth. This study concentrated on the positive features that occurred on the anterior teeth (canine to canine, maxillary and mandibular). Patients were selected from an outpatient clinic and in total 200 study casts (maxillary and mandibular) were produced. The authors only studied the dental casts, not bitemarks that would have been produced by such casts. The investigators noted the number and shape of each tooth, the presence of any incisal restoration, relationship of teeth to arch form, and tooth rotation (four categories). The study did not examine the presence or absence of spacing between teeth. The assessments of each cast were entirely subjective. Disappointingly, the authors elected not to publish a table of results. Rather they presented images of casts and calculated, using their data, the frequency of the traits shown. The authors noted that certain characteristics were not inter-related and thus the products of their incidences could be used to indicate an overall frequency. However, certain features, such as mesio-palatal rotation of the upper central incisors were inter-related with a significance of p<0.001. The authors stated that mesio-palatal rotation of the maxillary central incisors should therefore be taken as a single feature. This demonstrated that the true frequency of such features was almost four times greater than the frequency when the rotations were considered as individual variables. In an example, MacFarlane concluded that a particular dentition would only be seen in eight people in 100,000 of the population with natural teeth. This figure was introduced in an U.S. trial to much debate (State v. Garrison1). The authors concluded that they had 1 Case No. 09 Appendix B. State v. Garrison, 120 Ariz. 255, 585 P.2d 563, 1978 36 not confirmed the individuality of the human anterior teeth, nor had they considered the impact or representation of any of the features examined on a bitemark in human skin. The highly subjective examination of the casts by multiple examiners and lack of tabulated results make this study weak, especially in light of the increased scientific scrutiny required by recent Court rulings. However, a large (200) sample was used of a defined population and efforts were made to ensure that this sampling was randomised. The next paper to address the issue of individuality of human teeth was published in 1982. It is frequently cited by authors as conclusive evidence for dental individuality [87]. The premise of the paper was to examine the dentitions of five pairs of monozygotic (identical) twins and, should individualisation among the pairs be established, to extrapolate this finding to the general population. The twins were selected from another, unrelated study—the authors state that no selection based on dentition was performed. None of the subjects had crowns or removable prostheses. All teeth were determined to be healthy and representative of young adults in their early 20's. Each twin underwent a complete oral examination including alginate impressions. The impressions were immediately cast in plaster and subsequently epoxy-resin replicas of the anterior teeth were made and used to create test bites in a variety of materials, including plaster of Paris and silicone impression materials. The test bites were then treated by the wax radiographic technique for overlay production and the resultant radiographs were analysed by computer. A large number of measurements were carried out by the investigators who carefully noted asymmetries in each of the anterior teeth, angulations of test bites, and the depth of the test bites. Although the article stated that efforts were taken to standardise the production of these test bites, there was no discussion of how this was obtained. One crucial aspect would have been the amount of pressure applied to the epoxy replicas when creating the test bite. Many of the individual features claimed by the authors were dependent on the depth of penetration of the test bite into the substrate, and therefore a 37 standardisation of this pressure would have been necessary. The substrate, plaster of Paris, has very dissimilar properties to that of human skin. Figure 15 illustrates the computer comparison of each of the five pairs of twins. Note how many similarities exist between the outlines and that many of the differences shown could be explained by the depth of substrate penetration (and hence increased width of tooth outline) by the replicas. The authors noted that the teeth did not meet at the same horizontal plane at the incisal edges in each twin. This is described as an individualising difference between the twins rather than as an artefact of experimental variation. Figure 16 shows the difference in test bite outline produced from the same dentition applied to wax at a variety of different pressures. FIGURE 15 Computer printouts of the bitemark patterns produced by each of the twins After Sognnaes 1982 38 FIGURE 16 Variation in bitemark pattern that can be obtained by using the same dentition applied to dental wax using increasing pressure Even if it was to be accepted that the variation caused by inconsistent pressure application is negligible, the selection of substrate is questionable. Are investigators interested in the representation of uniqueness in plaster of Paris or human skin? Should a study that determined morphological human dental uniqueness in wax or plaster be extrapolated to fulfil a legally sound statement that a bitemark on skin is unique? With the current interest in the proper application of the scientific method, this would be unlikely to meet the legal burden. Sognnaes concluded that, in terms of dental arch form and individual tooth position, even identical twins are not dentally identical. As previously mentioned the effect of different wear-and-tear rates, exposure to environmental factors, dental treatments, and disease experience among such individuals will obviously cause differences over time. The twin study, despite the described problems, is one of two papers frequently cited as resounding evidence for the uniqueness of the human dentition. The other is Rawson's 1984 article 'Statistical Evidence for the Individuality of the Human Dentition' [88]. Rawson, an author on the twin study, in co-authorship with another dentist, two dental students, and a statistician wrote arguably the most cited and well-known bitemark paper describing an empirical experiment. In an attempt to prove finally the uniqueness of the anterior segment of human teeth, Rawson examined 397 bites and applied a statistical 39 probability theory to the results. The significance of this paper warrants the comprehensive assessment of its validity that follows. Twelve hundred wax bites were obtained from forensic odontologists in various geographic locations in the United States. Each bite was made on a custom wax wafer 1-mm thick supported by a 1-mm hard cardboard backer. The subjects were instructed to bite to the maximum depth of 1-mm. This design removed the variation of incisal penetration found in the twin study. A calibrated 1 -cm scale was also impressed upon the wax. From the 1200 samples received, 384 bites were selected although this was later increased to 397. It is unclear as to how this selection was made. There is no indication that this was a randomised sample. In order to adhere to strict scientific methods, the bites should have been selected in a random fashion to prevent any selector or observer bias being introduced. Rawson stated that the screening process involved an assessment of the clarity and accuracy of the marks as well as the completeness of an accompanying questionnaire. Another aspect of the study that is unclear is at what point the sampling was performed. Was it before or after the radiographic treatment of the bites? The bitemark indentations were filled with zinc powder and then radiographed using a technique designed to minimise any enlargement. Following the exposure of one side of the wax the zinc was removed and the procedure repeated for the other side. A study described earlier determined that the radiographic process for overlay production was relatively accurate [67] but it found that hand-traced overlays were less accurate and generally unsuitable for use. Rawson's study used a combination of both techniques thus increasing the chance of errors considerably. In this study, the radiographic overlays were enlarged three times and then hand-traced on to gridded computer paper. The article stated that the resolution of bitemark examinations should be within ±1 mm of the centre point of a tooth and ±5 degrees of rotation. Results of the study of Sweet suggested that this resolution might be difficult to obtain using the hand tracing method [67]. 40 Following the selection of the bites, the population sample was described as shown in Table 4. A comparison of these figures to U.S. census data found that the population sampled in the study was a reasonable measure of the U.S. population, although African Americans were under-represented and Orientals slightly over-represented. TABLE 4 Distribution of males, females, and races in Rawson's study ETHNICITY NUMBER PERCENT Male 222 55.9 Female 175 44.1 White 301 75.8 Black 28 7.1 Oriental 33 8.3 Hispanic 24 6.1 Other 11 2.7 After Rawson, 1984 Following the tracing of the biting edges, several elements of tooth position were assessed. A centre point for each tooth was determined and the x and y co-ordinates noted. The angulation of each of the teeth was measured and all the data were entered into a computer for analysis. It was determined that the minimum number of positions that a tooth can occupy is 150 and the greatest 239.9. These figures were determined by multiplying the number of positions of x by y and by the angles observed. The occurrence of fractions of positions (i.e. 239.9) is a reflection of this multiplication. Rawson elected to use 150 as the number of possible positions for each tooth as this represented a conservative sample. Using this premise, the article then stated that the probability of finding two sets of dentition with all six teeth in the same position was 1.4xl013. With an assumed world population of 4 billion (4xl09) Rawson stated that a match at five teeth on a bitemark would be sufficient evidence to positively identify an individual as the biter to the exclusion of all others. 41 One concern with this use of the product rule to multiply individual probabilities to establish an overall likelihood is that of independence of the variables. The article assumed that the position of each of the teeth was entirely independent of the position of any others. However, the independence of these features was not established by this or any other study. This has been shown incorrect; e.g., the dependence of mesio-palatal rotation described by MacFarlane [86]. It is likely that every tooth position influences another—intra-quadrant, intra-arch and between opposing arches. This lack of independence renders Rawson's certainties of individualisation invalid. Rawson's results also showed a possible sampling error, as evidenced by the data sets regarding possible tooth position for each unit. Intuitively it should be anticipated that the left and right quadrants should represent a mirror image of each other in terms of possible tooth centre positions. This was not the case. The upper right lateral incisor was reported to have 239.9 possible locations while the upper left lateral incisor had 161.5 locations. This asymmetry of tooth positions is shown in Figure 17. FIGURE 17 The number of possible positions for each tooth identified by Rawson 173.9 239.9 \ 132.7 ^ ^ V 166.9 / 161.6 '* -i*' _ _^ 150.0 107.2 — Vv ji 204.0 / 116.0 - 'Ai. 103.6 \ 1504 163.6 After Rawson, 1984 It can be argued that this paper, without the statistical treatment, confirms the anecdotal evidence of almost any practising dentist that the human dentition is unique. It can be 42 stated that, with an extremely high resolution of measurement, such as in this article, the minutia of the dentition can be described and proven unique. I would argue that this is the wrong question to ask. It is the rendition of these asserted unique features on human skin that is the unknown quantity. Rawson alluded to this point within his article: "... [the question is] whether there is a representation of that uniqueness in the mark found on the skin or other inanimate objects". [88] Rawson has proven what his article claims, although perhaps not to the mathematical certainty expressed. The article determined that the dentition is unique; however, when this paper is cited, authors often extend this conclusion to incorporate the uniqueness of bitemarks. The question of bitemark uniqueness remains unanswered. I believe that the problem can be approached more successfully from another perspective, that of bitemark analysis. By examining the ability of forensic dentists to identify correctly biters from their bitemarks, the issues of bitemark uniqueness can be answered. If it is apparent that odontologists have a great deal of difficulty in correctly identifying bitemarks, the question of uniqueness will become moot. 43 1.2.1.5 Analytical techniques An essential component of the determination of the validity of bitemark analysis is that the techniques used in the physical comparison between suspect dentition and physical injury have been assessed and found valid. One of the fundamental problems with this task is the wide variety of techniques that have been described in the literature. Techniques using confocal, reflex and scanning electron microscopes, complex computer systems, typing of oral bacteria, special light sources, fingerprint dusting powder and overlays have all been reported [19,90-93]. It is a widely held belief that while methods that are more esoteric exist, the dominant technique for comparison of exemplars is transparent overlays. The lack of direction from the ABFO complicates this matter. This group has reported advice and guidance on many aspects of bitemarks and yet one of the most pivotal questions, i.e. what is the best comparison technique to use, has not been addressed [55,94]. Should a Court wish to review the literature to ensure that a testifying expert is using generally accepted techniques they would find the task daunting and ultimately unrevealing. It is difficult, for many dentists to gain access to a microbiological laboratory or SEM in order to employ some of the suggested comparison techniques. Indeed, many of the 'minority' techniques are reported once in the literature and are never cited or utilised again. Such techniques are often criticised by the practising odontologists as nothing more than the products of 'ivory tower' thinking. Transparent overlays utilise materials found in any dental office. The vast majority of forensic dentists use techniques that utilise materials that are inexpensive and easily obtainable, hence the popularity of overlays. As described in Section 1 there are numerous techniques for the fabrication of transparent overlays. The only article that has assessed the accuracy of such overlays is that of Sweet and Bowers in 1998 [67]. This paper, described earlier in Section 1, compared five 44 common techniques of producing transparent overlays. Of all the techniques, an examination of case reports and experiments reveals that the xerographic and radiographic techniques are the most popular. Figure 18 illustrates the appearance of overlays produced from the techniques examined by Sweet's article. Note the variation between the overlays and consider the resolution that Rawson used in the "Uniqueness" article. Would these overlays, produced from the same study cast, result in different values of centre point and angulation as examined using Rawson's criteria? Sweet and Bowers used 30 randomly selected study casts to examine the accuracy of overlays produced from each of the five techniques concerning tooth rotation and surface area. The computer-generated overlays were the gold standard. See Table 5. From these results, it can be seen that the computer technique represents the most accurate fabrication method with respect to representation of rotation and area of the biting edge. The authors of the paper concluded that the fabrication methods that utilised the subjective process of hand tracing should not be used in favour of techniques that are more objective. The use of computer generated techniques was advised over any other method. FIGURE 18 Overlays produced from the same dental cast using a variety of different techniques Note, not to scale. 1 = computer-generated 2 = hand-traced from study cast 3 = hand-traced from wax bite 4 = xerographic method 5 = radiographic method After Sweet and Bowers, 1998 1 * 2 01 0 0 A Q 3 4 • « R 5 45 TABLE 5 Various overlay fabrication techniques ranked according to accuracy RANK AREA ROTATION 1 Computer-based Computer-based 2 Radiopaque wax Xerographic 3 Hand-traced from wax Hand-traced from wax 4 Hand4raced from study casts Hand-traced from study casts 5 Xerographic Radiopaque wax After Sweet and Bowers, 1998 The study described is an example of an experiment quantifying the techniques that are used in bitemark analysis in order to ensure that the most effective systems are employed. While the paper determined the accuracy of the overlays, it did not address the application of these overlays to the successful identification of a biter. Sweet and Bowers recommended methods in bitemark analysis that are more objective. Unfortunately, even if an objective technique is used, the subsequent comparison of this to a photographic reproduction of a bitemark is largely a subjective process [67]. 46 Attempts have been made to carry out the entire analysis within a computer system. One of the most recent articles describing this work used a specially designed computer and software to perform complex image analyses requiring no interaction with the operator [95]. The entire system was objective and required the odontologist to merely scan the suspect's dentition and the bite injury into the computer. The use of representative correlation co-efficients was proposed to identify the most likely biter. Despite the promising nature of the project, when it was applied to a real bitemark case the incorrect biter (based upon a Court decision) was implicated by the system. Figure 19 illustrates some of the patterns that were recognised by the system. In his discussion of these results, Naru states that the skin may simply not record the dentition accurately enough to enable analysis. The pathological record of the bite on skin is subject to many variables, such as distortion and colour changes that confound computer systems [95]. Naru recommended that further work would be required to modify the algorithms to contend with these variations. FIGURE 19 Images that Nam's computer program correctly identified as the letter 'T' After Naru, 1998 Currently, the best practice for physical bitemark comparisons should be regarded as a life-sized computer-generated overlay that is carefully compared with a scaled 1:1 photograph of the injury. The use of multiple photographic images and the careful collection of the evidence from both victim and suspect should control distortion. The need for a completely objective bitemark analysis system is recognised, although the 47 problems of variability of presentation of the injuries may render this ideal difficult to accomplish. In her excellent legal review, Zarkowski states that bitemark analysis has never progressed through the rigorous scientific examination that is common to other sciences to determine its accuracy or reliability [96]. This section of the thesis has highlighted the lack of hard scientific evidence to support the assumptions made by forensic dentists when analysing bitemarks. Major areas of contention have been discussed but there is still no consensus of opinion or definitive research concerning these, especially in relation to the dental/bitemark uniqueness issue. Nonetheless, the use of computer generated overlays in conjunction with techniques to control distortion is the most objective process available at present. The inherent difficulties of bitemark research, coupled with the professional status of forensic dental practitioners, means that advances to objectify fully bitemark analysis may be slow. There is a perceived acceptability in the level of scientific support for bitemark conclusions. Yet, with the new level of judicial interest in the validity of forensic evidence, it is likely that odontologists will have to revisit many of these issues. 48 1.2.2 THE JUDICIAL VIEW OF BITEMARKS - A LEGAL REVIEW When examining most traditional sciences a thorough review of the relevant primary literature is usually sufficient to provide the investigator with a sound insight into the discipline. Forensic science differs in this regard, as it is presented in two main arenas: the peer-reviewed forensic journals and the Courts of Law where testimony is proffered. Because of this duality of scientific assessment, no review of bitemarks would be complete without an examination of the legal aspects. American Appellate law was used to review the legal position. Appellate cases were chosen as lower Court proceedings are rarely published unless new law is being established. The on-line legal database Lexis® was used for this research [97]. American case law has been chosen due to the volume of material available. The identification of trends within the fourteen published Commonwealth cases proved impossible due to the small number of appeals available. A recent study showed that 42% of bitemark cases handled by forensic dentists resulted in a Court appearance [32]. The acceptance of bitemark evidence into the Court system and the qualification of forensic dentists as experts are essential to the continued development of the discipline. It is also essential that forensic dentists ensure that their testimony in Court strengthens the discipline rather than sets negative precedents. In 1978 Hale, wrote a paper entitled "The Admissibility of Bite Mark Evidence", which was published in the Southern Californian Law Review [83]. This extensively cited article concluded that the admissibility of bitemark evidence should be barred until forensic odontologists produced policies for the analysis of bitemarks. This influential article was partly responsible for the creation of the ABFO's working committees on bitemark standards, initiated to satisfy the recommendations of Hale and others [55]. This section analyses the U.S. Appellate literature to see if such a conclusion should still be reached today. Areas of investigation centre on the admissibility of bitemark evidence and the acceptance of forensic odontologists as expert witnesses. 49 Appendix B contains a Table (B3) detailing the cases examined in this review. Case numbers in this text relate to this Table. The Freestyle™ search engine on the Lexis/Nexis database using the "Mega" library of US Appeals identified cases. The search terms used were "Bite", "Mark", and "Odontologist". Following examination of the cases, it was found that the admission of bitemark evidence within the U.S. legal system is commonplace. Cases were identified where bitemark evidence was proven unreliable or unfairly prejudicial to the defendant. These were normally related to mistakes made by the expert rather than the bitemark evidence itself. 1.2.2.2 The use of scientific evidence in the U.S. legal system Frye The perceived inability of the trier-of-fact (judge or jury) to assess adequately scientific expert testimony against past experience led to the development of rules governing the admissibility of such evidence [83]. For almost three-quarters of a century the Frye test, from Frye v. United States ruled as the standard governing the admissibility of scientific testimony in the U.S. Courts. In 1923, the Court of Appeals for the District of Columbia, when examining polygraph evidence formulated the Frye standard for determining whether testimony proffered as scientific should be admitted into evidence [98]. Frye states that the "scientific principle or discovery from which [a] discovery is made must be sufficiently established to have gained general acceptance in the particular field to which it belongs". While this ruling pertained only to lower Courts within the D.C. Circuit it was eventually adopted by all of the Federal Circuits and by most of the individual states. It is useful to note that the Frye Court prefixed this comment by stating that it can be difficult to determine when a principle or discovery ceases to be experimental and becomes demonstrable. Following the original judgement it was found that the general acceptance rule does not ensure that the technique that an individual expert employs is valid. For example, the use 2 Frye v. United States, 293 F. 1013 (App. D.C. 1923) 50 of bitemarks to identify a biter may be generally accepted but the use of hand-traced overlays to achieve this goal may not be. In order to address the discrepancy a number of states adopted an enhanced Frye test, one in which the technique employed by a particular expert must also be "generally accepted". This test became known as Frye-2 [98]. Despite the clarification presented by Frye-2 a further area was covered in Frye-3. This test examines if the technique was used correctly by the expert; i.e. did the odontologist use the correct materials and methods to create the overlay. Many jurisdictions have decided that Frye-3 should be applied by the trier-of-fact in determining the weight, rather than the admissibility, of the evidence offered. This decision is frequently applied in bitemark cases. Once in use, Courts and commentators found the general acceptance test to have significant limitations [99]. Frye-1 is often described as overly conservative as it imposes a waiting period, a "cultural lag", that scientific principles and techniques must endure before gaining legal acceptance [99]. Indeed, the fundamental principle of general acceptance, upon which Frye is based, can be difficult to establish. The most cohesive and rigorous sciences rarely enjoy full consensus and dissenting voices can be interpreted as lack of acceptance. To further this, good scientific practises encourage a sceptical approach to discoveries and this can further confuse the general acceptance rule. It is also worthwhile mentioning that individual fields have varying standards. Many fields have long traditions of vigorous testing, re-testing, and scientific debate while others lack such traditions and thus would accept a new technique with relatively less scrutiny [99]. The Federal Rules of Evidence In 1975, the Federal Rules of Evidence (FRE) were introduced. The sections pertaining to scientific evidence are found in Article VII of the FRE (Opinions and Expert Testimony). Rule 702 pertains most directly to the admission of expert evidence and 51 states that expert witnesses may testify in the form of an opinion if it assists the trier-of-fact to understand technical or other specialised evidence. Following the introduction of FRE, there were many years of debate as to whether 702 had superseded Frye in governing the admission of scientific evidence. Many observers believed that there was little conflict between the two tests. FRE 702 became characterised as a "relevancy test", and formed the basis of the Daubert standard. Despite the limitations of Frye, it remains in use by many jurisdictions today [100]. The Daubert Standard On the 28th of June, 1993, the Supreme Court of the United States handed down a new standard for determining the admissibility of scientific evidence in the Federal Courts, the Daubert standard3. Federal Courts use the Daubert ruling exclusively, and local Courts are increasingly using the decision. Daubert states that there are four non-exclusive factors that Courts should consider when examining expert testimony: a) testability, b) error rate, c) peer review and publication, and d) general acceptance [99]. Because of these four tests, the Daubert ruling is often described as a validity measure for scientific evidence. There has been much discussion regarding the Daubert standard and over 100 articles have been written describing its impact. The issue of whether or not FRE 702 supersedes Frye was answered in the affirmative by Daubert [100]. The Court found nothing in 702 that establishes general acceptance as an absolute prerequisite to admissibility. The Court summarised by saying that since the FRE have a liberal application, the rigid general acceptance approach of Frye is at odds with the approach taken in FRE 702 [101]. Another of the major issues raised is that judges are now responsible for determining the admissibility of evidence4 and concern has been voiced that judges will become "amateur scientists" [100]. 3 Daubert v. Merrell Dow Pharmaceuticals, Inc., 113 S.Ct 2786, 61 U.S.L.W. 4805 (1993) 4 What the Court advocates in Daubert is that trial judges must be ultimately responsible for the scientific accuracy and validity of evidence presented in their Courts. 52 Commentators have argued that Daubert will lead to confusion. They cite the fact that one State Court has concluded that Daubert makes it more difficult to introduce scientific evidence5, which is at odds with the stated intention of loosening the rigid requirements extolled by Frye. Bohan and Heels describe the major differences between Frye and Daubert. See Table 6. TABLE 6 Frye and Daubert - a comparison ADMISSIBILITY TESTS FOR TESTIMONY THAT IS OSTENSIBLY SCIENTIFIC FRYE DAUBERT The "scientific principle or discovery" on which the testimony is based must be sufficiently established to have gained general acceptance in the particular field in which it belongs [T]he reasoning or methodology underlying [the] testimony [must be] scientifically valid WHAT DOES THE TEST INVOLVE? A pre-trial ruling on whether the basic principle on which the testimony is ultimately based has been generally accepted within the relevant community A preliminary ruling based on FRE 104(a) on whether the theory or technique is scientifically valid WHAT FACTORS ARE INVOLVED? Though there may be some dispute as to what compromises the relevant technical community, general acceptance within that community is the beginning and end of the inquiry. No examination is made of whether the community is correct in accepting or rejecting the principle or discovery Indicia of scientific validity to be examined include: 1) widespread acceptance 2) peer review 3) publication 4) testing 5) rates of error 6) existence of standards After Bohan and Heels 1995 1.2.2.3 The admission of bitemarks as evidence in the U.S. legal system Bitemark identification, as it is most commonly referred to in legal terms, has been virtually unanimously admitted by the Courts. Indeed, most U.S. jurisdictions have allowed such testimony. Table 7 provides a state-by-state summary of bitemark cases used in this review. A more comprehensive table providing case numbers can be found in Appendix B. 5 State v. Foret, 628 So. 2d 1116 (LA 1993) 53 TABLE 7 Distribution of the Appellate bitemark cases examined by U.S. State STATE NO. OF CASES STATE NO. OF CASES Alabama 2 Nevada 3 Arizona 2 New Jersey 1 Arkansas 3 New York 5 California 5 North Carolina 4 Connecticut 2 Ohio 3 Florida 5 Oklahoma 4 Georgia 2 Oregon 3 Illinois 13 Pennsylvania 2 Indiana 2 Rhode Island 1 Kansas 2 South Carolina 1 Louisiana 2 Tennessee 1 Massachusetts 1 Texas 7 Michigan 1 Vermont 1 Military Cases 2 Virginia 2 Minnesota 1 Washington 4 Mississippi 7 West Virginia 1 Missouri 5 Wisconsin 1 The Historical Basis for the Admissibility of Bitemark Evidence While Doyle v. State6 represented the first bitemark case in modern U.S. legal history, it did not examine the scientific basis for the admissibility of the evidence. People v. Marx1 is generally regarded as the landmark case for bitemark evidence. However, it is interesting to note that cases that are more recent have cited Doyle as the basis for rejecting arguments for unproven reliability and acceptance [96]. The Marx case involved the murder of an elderly woman who sustained a bitemark on her nose that, following exhumation of the body, was examined by four forensic odontologists, three of which presented for the prosecution. The case is well described in the Journal of Forensic Sciences [102]. All three witnesses for the prosecution testified that the defendant caused the bite and an attempt was made to demonstrate the significance of Marx's highly unusual dentition. At appeal, the defence stated that the 6 Case No. 01 Appendix B. Doyle v. State, 159 Tex.Crim. 310, 263 S.W.2d 779 (1954) 7 Case No. 03 Appendix B. People v. Marx, 54 Cal.App.3d 100, 126 Cal.Rptr. 350 (1975) 54 techniques and skills utilised were not generally accepted and should have failed the Frye test. The appeals Court stated that they considered the use of bitemarks as novel, although the techniques employed were not i.e., photographs, models, and radiographs. The Court went further by stating that unlike some other forensic disciplines, "... the Court did not have to sacrifice its independence and common sense in evaluating it". This was saying that the jury could perform their own analyses by examining the methods that the forensic dentists had used, and they did not require the expert to explain the nuances of the techniques to them. The evidence was, in essence, self-explanatory. The Court contrasted this to polygraph evidence stating that the trier-of-fact had to rely entirely on the testimony of the polygrapher with only "marks on paper" to verify the claims being made. The result of this reliance on the expert would lead to the jury sacrificing its independence in deference to the expert. It is interesting to note that an expert is called precisely for their knowledge and to aid the jury [99]. Indeed many forensic dentists may be unsettled with the thought that once the physical exemplars are collected, no further expertise is required! This case initiated the premise that bitemarks should be admitted although the weight of such evidence should be carefully examined by the trier-of-fact. The Marx Court also commented on the experts' enthusiasm to develop or extend forensic dentistry into the area of bitemark identification. Before Marx forensic odontological work had largely been limited to the identification of found remains by dental records. An Indiana Court8 also compared bitemark identification with polygraph techniques finding that bitemark comparison was simply the examination of items of physical evidence to see if they were reciprocal. The methods of achieving this comparison, while complex, were determined to be accurate. As a concise statement of the current status of bitemark admissibility the following, written in 1981, serves well: Case No. 06 Appendix B. Niehaus v. State, Ind. 655, 359 N.E.2d 513 (1977) 55 "The reliability of bite mark evidence as a means of identification is sufficiently established in the scientific community to make such evidence admissible in a criminal case, without separately establishing scientific reliability in each case, but subject to the establishment by foundation evidence of the authenticity of the materials used and propriety of the procedure followed in the particular case and to cross-examination intended to test the reliability of the conclusion reached in that case." 9 1.2.3.4 Case assessment of bitemark admissibility Following an examination of the admission issues for bitemarks it is possible to isolate several important trends pertaining to bitemark admissibility from the 103 cases examined. Table B2 in Appendix B illustrates these trends. Each of the areas is briefly discussed below. Bitemark evidence not sufficiently reliable or accepted This argument is frequently used by defence teams attempting to bar the admission of incriminating bitemark evidence and, despite many years of uninterrupted bitemark admission, was used as recently as 199710. One of the pervasive reasons for refusing appeals on this basis is that since a science has been accepted as reliable under one Frye hearing then general acceptance has been established. Judge Cox11 stated that bitemarks have been so overwhelmingly accepted by the Courts that a proponent need not establish the principle of general acceptance on each occasion. The case of State v. Hodgson12 is significant as it was the first appeal case to examine bitemark evidence in the light of the Daubert ruling. Convicted of two counts of first-degree murder, Hodgson appealed the admissibility of the odontological evidence linking a bitemark on his arm to one of the decedents. Arguing that bitemark evidence was not generally accepted he claimed that the science did not meet the requirements of Frye. The Court disagreed with Hodgson stating that Daubert and FRE 702 had superseded Frye and that they were satisfied that bitemark evidence by an accepted expert was 9 Case No. 17 Appendix B. People v. Middleton, 54 N.Y.2d 42, 429 N.E.2d 581 (1981) 10 Case No. 98 Appendix B. Howards. State, 697 So. 2d 415 11 U.S. v. Gibson, 24 M.J. 246 CMA 56 neither novel nor an emerging science and thus was admitted correctly. Following Marx and Hodgson no bitemark evidence has been refused admission due to arguments regarding Frye, FRE, or Daubert. Arguments regarding the uniqueness of the human dentition In the previous section, the importance of the uniqueness of the human dentition was discussed. Several appellants have raised this point as an argument against the admission of bitemark evidence. In State v. Garrison13, the appellant argued that the testimony of the forensic dentist, who stated that the probability of the bitemarks not being made by Garrison was eight in one million, was unreliable and flawed. When questioned regarding the validity of the stated probability the witness testified that the figure had been arrived at following consultation of several leading textbooks and journal articles. The majority opinion in this case stated that experts quoting from books or articles fell under the hearsay exception for learned treatises, and thus, the point of appeal was overturned. It is interesting, however, to examine the dissenting opinion in this appeal. Justices Gordon and Cameron noted that the witness had neither performed any of his own mathematical calculations nor was he aware of any of the formulae used to derive the quoted figures. The expert's ignorance of the statistical weighting that should be given to each variable used in the equation and his inability to replicate the findings in Court were serious shortcomings of his testimony. The Justices carried out a literature search and were unable to locate the articles or formulae to which the witness alluded. The dissenting opinion continued by describing the inherent difficulties of determining the uniqueness of the human dentition and in particular the hazards of applying the product rule. Gordon and Cameron concluded that witnesses who offered statements representing direct quotes from books or similar materials should only be permitted to do so if the referenced sources were available to the Court and opposing council. 12 Case No. 87 Appendix B. State v. Hodgson, 512 N.W.2d 95 13 Case No. 09 Appendix B. State v. Garrison, 120 Ariz. 255, 585 P.2d 563 57 Constitutional arguments. Improper seizure of exemplars The Fifth Amendment, that forms the basis of most constitutional appeals against bitemark evidence, states that no person shall be compelled to be a witness against them self. A case example of the Fifth Amendment in a bitemark appeal can be found in State v. Sapsford14, an appeal against a conviction of three counts of rape and one count each of attempted aggravated murder and felonious sexual penetration. Sapsford claimed that he was compelled to submit to dental impressions that resulted in the production of exemplars making him the source of incriminating evidence. Using this argument, he claimed that such compulsion was in violation of his Fifth Amendment privilege against self-incrimination. Examining this point the Court overturned his claim by stating that the Fifth Amendment privilege extended only to communicative or testimonial acts and not to the taking of dental exemplars. In this manner, dental impressions did not differ from the taking of fingerprints, photographs, or blood. In an attempt to use the protective shield of self-incrimination to overturn the admission of bitemark evidence, Asherman15 stated that the Connecticut State Constitution offered further protection than the Fifth Amendment. Claiming that the use of the word "evidence" rather than "witness" in the State Constitution extended the protection to non-testimonial evidence, Asherman appealed his conviction. The Judges disagreed and found the nature, spirit, and principle of the two statements were the same. They noted that some jurisdictions had widened the meaning of such clauses by finding that evidence that required the defendant to perform an affirmative act should be excluded16. This wider interpretation would allow dental impressions and fingerprint samples but would not allow handwriting or speech samples. Photographs of bitemark evidence inflammatory Photographs play a crucial role in both the analysis and subsequent Court presentation of bitemark injuries. It is usually essential to the expert witnesses' testimony that such 14 Case No. 27 Appendix B. State v. Sapsford, 22 Ohio App.3d 1 15 Case No. 33 Appendix B. State v. Asherman, 193 Conn. 695; 478 A.2d 227 58 photographs are available for demonstration to the jury. Defendants frequently object to 1 7 the display of such images in Court. State v. Kendrick offers a typical example of such an appeal. During the original trial against Kendrick, the dental expert presented testimony regarding a bitemark that involved over 180 exhibits, including numerous photographs. Kendrick argued that several of the photographs, including those of his mouth, should not have been admitted, as they were unnecessarily gruesome. The Court examined the photographs of the victims (including shots depicting the bitemarks) and found that they were indeed gruesome, but not overly so. They accurately depicted the horrific nature of the two victims' last moments and so were determined to be probative. The Court stated 18 that violent crimes could not be explained to the jury in a "lily-white" manner . Kendrick particularly objected to the photographs of his mouth that were taken using cheek retractors on the grounds that it made him look "vampirish". The Court stated that the photographs were essential aids to the often complex testimony of the forensic odontologists. The reasons for the use of the cheek retractors were carefully explained and thus the Judges concluded that the photographs were correctly admitted in the original trial. Inaccuracy of techniques and errors in bitemark protocol Defendants in Court can question the accuracy of the techniques involved in the analysis of bitemark injuries. A representative case is that of State v. Peoples19 in which Peoples, on appeal, challenged the accuracy of the exhibits and models used by the forensic dentist in arriving at his conclusions. Peoples' concerns were centred on the enlargement of a series of photographs and the production of plaster models of his teeth. The Court carefully assessed the exhibits and the techniques used to produce them and found no error in the original trial to admit them into evidence. The Court stated that any doubts regarding the accuracy of the exhibits should be applied to the weight of the evidence and Hansen v. Owens, 619 P.2d 315 Case No. 52 Appendix B. State v. Kendrick, 47 Wash. App. 620; 736 P.2d 1079 State v. Adams, 76 Wn.2d 650, 656,458 P.2d 558 Case No. 13 Appendix B. State v. Peoples, 227 Kan. 127, 60S P.2d 135 59 not dictate its admissibility. It should be noted that forensic dentists must be prepared to defend the accuracy of their exhibits and be able to describe how they check and validate such materials. A case example of a technique being questioned by a defendant can be found in People v. 20 Holmes . The expert used a plaster cast of Holmes' teeth to make an imprint in a sheet of Styrofoam from which hand-traced overlays were produced. The accuracy of this technique was questioned in light of the availability of more precise methods. The odontologist was asked to repeat the analysis using a radiographic technique and the original verdict was affirmed. There are many techniques and methods employed by those who examine bitemarks and such experts should expect to have their protocols questioned by opposing counsel. The use of controversial or novel systems is likely to make such inquiries more probable. Banks v. State where the single item of physical evidence linking Banks to the crime scene was a bitemark in a sandwich highlights a more serious example of protocol error. Following his analysis of the bitemark, the prosecution's dental expert threw the sandwich away fearing that it would become susceptible to mould and hence be useless. The destruction of this evidence denied Banks the opportunity to obtain his own expert who could examine the bitemark and rebut the prosecution's expert. The Court agreed that this error had caused an unfair disadvantage to the defence and that the bitemark evidence should not have been admitted. Due to the pivotal nature of the evidence, the verdict was reversed. The use of previous bitemarks or evidence of previous biting behaviour Examples exist of historical bitemarks being used to compare to contemporary injuries allegedly caused by the same defendant. An example of this can be found in State v. Smith22. The prosecution in this case used two techniques to identify the biter. The first used a plaster cast of the suspect's teeth to compare to a scaled photograph of the injury. 20 Case No. 80 Appendix B. State v. Holmes, 234 111. App. 3d 931, 601 N.E. 2d 985 21 Case No. 97 Appendix B. Calvin Banks v. State, 725 So. 2d 711 60 The second and contested method used a photograph-to-photograph comparison. The prosecution presented a black-and-white photograph of a bite injury allegedly made by Smith in 1977 on the nose of a murder victim. Smith had confessed to this crime and thus the prosecution argued that it was reasonable to assume that Smith made the bitemark. The expert then compared the historical bite with the bitemark on the latest victim and found them similar. The defence strongly objected to this technique stating that the method was not well accepted. The Court, however, disagreed and the original verdict was affirmed. The premise that if an individual has bitten before then they will be likely to bite again has been offered into evidence by State prosecutors and tenaciously objected to by defence teams. United States v. Martin23 represents an example of such a prosecutorial technique. The prosecution offered testimony that at times of stress the defendant bit or chewed items, such as toothbrushes or pencils. A bitemark was found on the neck of Martin's murdered wife and the prosecution stated that because of the aforementioned biting behaviour the injury was likely to have been caused by Martin. Upon appeal, Martin claimed that this evidence was wrongly admitted; the linking of biting objects to biting his wife was nonsense. The Court found that the evidence had been admitted in error. Had the expert established a link between the biting of objects and a propensity to bite humans it may have been marginally admissible. However, despite agreeing with Martin's point of appeal the Court determined that the evidence did not have a substantial prejudicial effect on the outcome of the trial and the original verdict was upheld. Defence requesting prosecution's testimony or funds for own witness The issue of the State withholding evidence from the defence is taken seriously by Courts and timely, accurate disclosure is well grounded in the doctrine of the U.S. legal system. The disclosure rules insist that a defendant be entitled to all results or reports of physical or mental examinations and of scientific tests or experiments conducted concerning a particular case. The rules also state that, subject to an appropriate protection order, all Case No. 30 Appendix B. State v. Smith, 63 N.Y.2d 41, 468 N.E.2d 879 Case No. 22 Appendix B. United States v. Martin, 13 M.J. 66 (CMA) 61 tangible objects that were used in the execution of such tests should also be released to the accused. State v. Adams24 demonstrates an example of disclosure issues within the context of a bitemark case. Adams claimed that the Court had erred in failing to dismiss the case when it became apparent that the State had not disclosed the existence of a scientific report concerning an alleged bitemark of the victim or the existence of a cast impression of the injury. The prosecution was instructed to disclose fully the materials, but when this was done the cast of the impression was not included. The Appellate Court stated that there had been a deliberate misinterpretation of the disclosure rules and this had resulted in gross error in favour of the State. This non-compliance was compounded by the ultimate failure of complete disclosure despite specific instructions from the Court to do so. The implications of the lack of disclosure were significant, as Adams was unable to secure an independent forensic dentist who could have offered an alternative opinion. Following several other points of appeal the original verdict was overturned, which was a very serious consequence to the State's actions. The issue of Courts providing funding for accused individuals to secure expert witnesses is heavily debated. Washington v. State25 provides an example of this issue in relation to a forensic dental expert. Washington's appeal against the death sentence was based upon many grounds but in particular he claimed that the Court erred when it denied him funds to obtain a forensic dentist to refute the prosecution's witness. In examining the original trial, the Appeals Court found that the bitemark evidence had a "high impact" upon the trials and the Court's refusal to grant funds for a defence expert was an irreversible error. The verdict was overturned and the case was remanded for a new trial. This example is often contradicted by other jurisdictions that believe that it is not the State's responsibility to provide the defendant with numerous experts to testify on their behalf. Case No. 34 Appendix B. State v. Maurice Adams, 481 A.2d 718 Case No. 74 Appendix B. Washington v. State, 836 P.2d 673; Okla. Crim. App. 62 1.2.3.5 Witness prejudiced or other witness-related objections This section examines the objection to bitemark testimony based on expert witness issues. The cases represent instances where the witness has been accepted as an expert and offered an opinion during the trial. It is a recognised defence strategy to suggest to the jury that the witness is less credible, and thus reduce the weight afforded to their testimony. Before examining some of the case-related issues, it is worthwhile to examine some aspects of what it is to be an expert witness. A legal definition of an expert witness is "one who possesses extraordinary knowledge concerning a subject which was obtained from experience or by careful study" [103]. A more general view is that experts are persons with special knowledge, skill, experience, training, and/or education that goes beyond the normal experience of ordinary members of the public. Some Courts have stated that experts can be qualified if they are, without other qualification, merely "helpful" to the jury. It is argued that nearly anyone could provide expert testimony in some form or another. If the brakes fail on your new vehicle, a brake specialist from a local automotive shop would be able to inform the Court of the processes behind the failure. In this field, they would be providing expert testimony. If you receive dental treatment that is of poor quality, a general practitioner with many years experience would be an excellent witness to choose. When examining bitemark evidence, however, the selection of a forensic dentist can be problematic. Dentists in all jurisdictions have an obligation to pursue continuing education throughout their professional careers. This can result in a plethora of diplomas, additional degrees, and memberships in organisations. Lawyers, and others, who employ dental expert witnesses, need to be able to interpret what may appear to be an extensive curriculum vitae and extract the salient features. With regard to bitemark evidence an expert will either have to have a) been board certified by the ABFO, b) completed a research degree followed by extensive casework experience, or c) extensive experience in the discipline and provide documentary evidence. The selection of Court specialists should be limited 63 to those individuals represented by these groups. The use of non-dental personnel to testify concerning bitemarks is fraught with danger. This was shown by the Appeal case Oft • * State v. Adams , where the use of a physician's testimony concerning a bite injury was ruled inadmissible. What is a Bitemark Expert Witness? Dental experts testifying regarding bitemarks use their knowledge of: a) dental materials, b) associated instruments, and c) the dynamic interaction between teeth and objects to assist the Court [7]. Dentists also bring knowledge of the masticatory system to Court in order to explain the biting process. The odontologist's testimony is the culmination of extensive research and preparation. The results of such preparation, combined with direct observations, examinations, and so forth represents the foundation of all expert testimony [104]. One of the most serious allegations that can be brought against a witness is that of perjury27. In Bromley v. State2* the defence alleged at appeal that the State's witness had lied during cross-examination. At Bromley's original trial, the dentist was asked if he had consulted with any other expert during his analysis of the evidence and the formulation of his conclusions. He responded that he had not. It was later proved that he had infact consulted with the defence witness in the case. The Court found that the testimony given by the expert, although false, was harmless to the appellant, and did not warrant an assignment of error. Another example where the integrity of the witness was questioned is Brewer v. State , a capital case in which the appellant had been convicted of the rape and murder of a three-year-old child. In this case, the appellant claimed that the forensic dentist's testimony should not have been permitted, as previous testimony by the witness in another trial had 26 Supra note 30 27 "A person to whom a lawful oath or affirmation has been administered commits the offence of perjury when, in a judicial proceeding, he knowingly and wilfully makes a false statement material to the issue or point in question" 28 Case No. 62 Appendix B. Bromley v. State, 380 S.E.2d 694 (Ga. 1989) 29 Case No. 102 Appendix B. Brewer v. State, 725 So.2d 106 (1997) 64 been deemed inadmissible30. Brewer also stated that the witness had been less than forthright concerning his qualifications in previous testimony31 [105]. The Court examined the issues and found that one of the previous trials did not involve dental testimony and the second was about membership of a professional organisation that the witness had properly explained. The Court stated: "... the record evidence shows that Dr. ... possessed the knowledge, skill, experience, training, and education necessary to qualify as an expert in forensic odontology. The problems in Maxwell and Keko went to the weight and credibility to be assigned to his testimony by the jury—not his qualifications." The appeal Court found no assignments of error and affirmed the trial Court ruling. Other cases examined the issues of prejudice of experts and prosecution witnesses working in teams, none of which was found to have any merit. However, the cases do illustrate that the behaviours and actions of forensic odontologists are open to negative interpretation. Therefore, care should be taken to demonstrate no impropriety, lest it be brought in front of a public Court. Forensic odontologists must subscribe to rigorous and comprehensive standards of practice to ensure fair and equitable treatment for all parties concerned [106]. 1.2.3.6 Summary of bitemark testimony admission It can be stated in summary that bitemark evidence has been generally accepted within the forensic field, and the admission of such evidence on this principle is correct. It is important to note, however, that the degree of acceptance of bitemark evidence does vary widely in the field with many odontologists sceptical about the conclusions that can be drawn from such analyses [107]. Despite such reservations, bitemark evidence has been, almost without exception, admitted post Marx. The trends analysed previously describe attempts by defence lawyers to highlight the weaknesses inherent in bitemark analysis. It is important for testifying odontologists to be aware of such issues and strategies, and be prepared to address them if required. The role of the prosecutor in such circumstances is 30 Maxwell v. State of Mississippi 65 also pivotal in assisting and preparing the State's witnesses to deal with any objections or defence tactics. State of Louisiana v. Keko 66 SECTION 3 SUMMARY AND STATEMENT OF PROBLEM 67 1.3.1 SUMMARY In the previous sections, the basic premise of bitemarks in criminal investigation has been described. Accompanying this has been a review of the scientific basis for such a premise. It is possible to conclude that the core articles upon which the discipline relies are not satisfactory and do not address the contentious issues surrounding the probative use of bitemarks. Many odontologists believe that bitemark analysis should be limited, or even excluded, until sound scientific evidence can be presented to support its continued use [107]. Forensic science as a whole in the U.S. has been placed under a microscope. The rulings of Daubert and Kumho explicitly state that a judicial examination of the scientific premise of testimony must be conducted before the acceptance of any proffered testimony. It is likely that this increased scrutiny will spread to other legal arenas, including Commonwealth law. The Daubert Court specifically stated that for any forensic method the accuracy and reliability of the technique must be clearly stated and that experts should be able to quote error rates. There are no such data for bitemark analysis. The field of bitemark analysis requires the determination of these criteria to ensure that the evidence may continue to be presented in Court, or its use is halted due to its unreliable nature. In a recent guest editorial in the ASFO newsletter, senior forensic dentist John Clement stated that forensic dentistry was "hopelessly exposed in [regard to bitemarks]" [108]. Recent testimony by forensic dentists in Court (State v. Oterro, State v. Amolsh) that the perpetrator of a bite was positively identified have proven, by DNA analysis, to be false. In Oterro, the bitemark evidence was crucial to the conviction and following the exculpatory DNA analysis, Oterro was released, and the verdict overturned. A civil suit against the forensic dentist in this case is pending. Against this background it is not difficult to see why Dr. Clement states that "...in our whole repertoire it is only the bitemark cases that have given rise to notoriety" [108]. 68 1.3.2 STATEMENT OF PROBLEM In order to address the deficiencies described during the previous sections, this work aims to establish the reliability, accuracy, validity, specificity, and sensitivity of bitemark physical comparisons performed using computer generated bitemark overlays. The methodology will include an assessment of these variables using a range of odontologists with varying levels of experiences. Using the principles of diagnostic test evaluation and Receiver Operating Characteristics will allow us to quantify these aspects of bitemark analysis and examine the validity of continued use of bitemarks to identify perpetrators of crime. These quantifiable features will satisfy the known error rate requirements of Daubert. The analysis of such features is central to the verification of bitemark analysis in forensic science. 69 CHAPTER TWO ASSESSMENT METHODS OF DIAGNOSTIC EFFECTIVENESS 2.1 THE ASSESSMENT OF DIAGNOSTIC TESTS The purpose of this study is to determine error rates and other quantifiable features of bitemark overlay use. In order to design a study to obtain meaningful data, the field of diagnostic assessment provides appropriate methods. The use of tests in medicine and dentistry to aid diagnosis is well established. In the era of evidence-based clinical practice, an increased interest in the effectiveness of diagnostic tests has developed and techniques to assess these have been re-deployed. The results of these investigations are challenging many of the widely held beliefs regarding the use of well-respected tests. This work will consider the use of bitemark overlays as a diagnostic test, with the bitemark as the disease and the suspect either been diagnosed with the disease (is the biter) or without the disease (is excluded as the biter). The application of these principles to bitemark comparisons is explained in further detail below. 2.1.1 Quantifying the effectiveness of diagnostic tests Any clinician using a diagnostic test, whether it is physical, laboratory, or screening needs to understand how effective the test is, and thus is able to give appropriate weight to the result [109]. To assess the quality of a diagnostic test it is necessary to obtain values for validity, reliability, sensitivity, specificity, and the positive and negative predictive values. Laypersons and experts testifying in Court frequently use these terms, yet often they are used incorrectly. It is worthwhile, therefore, to define each of these terms. Validity The validity of a diagnostic test is the extent to which it measures what it claims to measure. In order to quantify validity, the proportion of all test results that are correct (based upon comparison with an accepted or gold standard) is used. It is often stated that validity is synonymous with accuracy. The accuracy of a measurement is the degree to 71 which it is immune from systematic error or bias and thus it is not synonymous with validity [109]. Reliability Reliability is equivalent to repeatability or reproducibility [109]. A reliable test is one that gives the same result within accepted ranges during repeated measurements of the same variable. Reliability is linked to the precision of a test i.e., the degree of random variation that occurs when measuring a constant value. A reliable test is consistent, stable, and dependable. Two main areas of reliability are usually assessed when determining the effectiveness of a test: inter- and intra-examiner reliability. Intra-examiner reliability compares the application of a test by the same examiner on several occasions using the same case materials [110]. Inter-examiner reliability is determined by how different observers classify individual cases into the same category on a measurement scale [111]. Examples in dentistry exist that have examined the reliability of periodontal examinations, determination of orthodontic treatment need, and the assessment of teeth for restorative treatment [112]. Forensic examples of inter-examiner reliability are limited, but a Swedish group assessed this variable among examiners determining the age of individuals based on dental development [113]. Considerable variation between examiners was discovered and the authors concluded that the assessment of age by odontological means might be flawed. No study was found that identified intra- or inter-examiner reliability of bitemark physical comparisons. The determination of diagnostic agreement, either between an examiner and a gold standard or between different examiners, is concerned with the similarities of categorical or nominal measurements. For example, suppose two forensic dentists were provided with 29 bitemark cases and 29 suspects. They were asked to state if they believed that the suspect did or did not cause the bite in each case. Table 8 and Figure 20 illustrate the results from this example. Data for this example was taken from a study involving diagnostic decisions by physicians described by Dunn [114]. 72 TABLE 8 The decisions of two forensic dentists examining 29 bitemark suspects in 29 bitemark cases SUSPECT DENTIST A DENTIST B AGREEMENT SUSPECT DENTIST A DENTIST B AGREEMENT 1 No Yes No 15 No Yes No 2 No No Yes 16 No Yes No 3 No No Yes 17 No Yes No 4 Yes Yes Yes 18 No Yes No 5 No No Yes 19 No No Yes 6 Yes Yes Yes 20 No No Yes 7 No No Yes 21 Yes Yes Yes 8 No No Yes 22 Yes Yes Yes 9 No No Yes 23 No No Yes 10 Yes Yes Yes 24 No No Yes 11 No Yes No 25 Yes Yes Yes 12 Yes Yes Yes 26 No Yes No 13 No No Yes 27 No Yes No 14 Yes Yes Yes 28 Yes Yes Yes 29 Yes Yes Yes Number of yes ratings: Dentist A = 12, Dentist B = 19 Number of agreements: 22 FIGURE 20 A 2x2 contingency table of data presented in Table 8 Dentist B Dentist A No Yes Total No 10(34.5) 7(24.1) 17(58.8) Yes 0 (0.0) 12(41.4) 12(41.4) Total 10 (34.5) 19 (65.5) 26 A total of 22 agreements (75.9%) between the two forensic dentists were observed. The forensic dentists agreed that 10 suspects were not responsible for a bitemark and 12 did bite the victim. Dentist A indicated seven suspects as biters. Dentist B indicated no suspects as biters. The probability of Dentist A and Dentist B determining that a suspect is a biter was 65.5% and 41.4%, respectively. However, to what extent did the two forensic scientists agree in their assessments? A simple index of agreement would be the proportion of agreements between the two examiners, 22/29 = 0.76, or 76% agreement. However, this measure ignores the 73 agreement between the two dentists that would have occurred purely by chance. In order to correct for this chance agreement Cohen's Kappa is used. While it is theoretically possible to achieve a negative value for kappa, the values normally fall between zero (no agreement above chance) to one (perfect agreement). Landis and Koch suggested a range of kappa values to express certain strengths of agreement [115] shown in Table 9. These categories are purely arbitrary but are well accepted as reasonable means of determining agreement among examiners [114]. In this example where kappa = 0.54 it is possible to say that the two forensic dentists in the example had a moderate level of agreement regarding bitemark suspects. TABLE 9 Kappa values and strength of agreement KAPPA STRENGTH OF AGREEMENT 0.00-0.01 Poor 0.01-0.20 Slight 0.21-0.40 Fair 0.41-0.60 Moderate 0.61-0.80 Substantial 0.81-1.00 Almost perfect From Landis and Koch, 1977 Sensitivity and Specificity Sensitivity and specificity are measures of a test's validity i.e., its ability to correctly identify those individuals with and without the questioned disease, or a test's ability to identify correctly those responsible or not responsible for creating a bite injury. In a diagnostic situation, there can be two outcomes, either: a) the person has the disease or b) the person does not have the disease [109]. When the results of an examined test are compared to a gold standard, there can be four possibilities: 1) True Positive (TP): the test results indicate that an individual has the disease and this is confirmed by the gold standard 74 2) False Positive (FP): the test result indicates that an individual has the disease but this is not confirmed by the gold standard that finds the individual is disease free 3) False Negative (FN): the test result indicates that an individual is not suffering from the disease but the gold standard indicates that the disease is present 4) True Negative (TN): both the test result and the gold standard agree that the individual does not have the examined disease. The gold standard can be either an established test (e.g. radiographs for caries), or a confirmatory standard (e.g. histology sections for caries). Figure 21 illustrates the application of these principles in a 2 x 2 contingency table that permits a clearer view of the system. These tables are commonly used to present the results of such comparisons. A 2x2 contingency table illustrating the outcomes of a comparison between a diagnostic test and a gold standard Gold Standard Positive Negative Total Test Result Positive True positive (TP) False positive (FP) TP + FP Negative False negative (FN) True negative (TN) FN + TN Total TP + FN FP + TN FN + TN + FP + TP The sensitivity of a test is its ability to detect correctly people who have the disease, i.e. the percentage of diseased people who are correctly diagnosed. A test that is 100% sensitive will identify every diseased individual; an insensitive test will lead to missed diagnoses. A sensitive test results in very few false negative results. Diagnostic tests that have a high degree of sensitivity are used in situations where the consequences of a false negative result are serious. An example of this is the screening of donated blood for HIV. 75 Such highly sensitive tests are used for screening or ruling out disease; if the result of a highly sensitive test is negative, it allows the disease to be ruled out with confidence. The specificity of a diagnostic test is the percentage of disease-free individuals who are diagnosed correctly. A test that is always negative for healthy individuals will have a specificity of 100%. A highly specific test produces few false positive results. Tests that exhibit a high degree of specificity are used in situations where the consequences of a false positive diagnosis are serious. Examples include where a positive diagnosis leads to the initiation of complex and painful surgery, such as where it may cause an individual to make irreversible life decisions (Alzheimer's). Other examples include where a diagnosis could result in someone being stigmatised with an incorrect label (schizophrenia) [109]. Very specific tests are used for confirming the existence of a disease. If a highly specific test is positive, the disease is almost certainly present. With many diagnostic tests, sensitivity and specificity are inversely related: an increase in one will cause a reduction in the other [116]. Figure 22 shows a diagrammatic representation of a diagnostic test that has a specificity and sensitivity of 100%. The test results of the diseased and non-diseased subjects show no overlap, and so the threshold level for a diagnosis is between these distributions. If the test result is higher than the threshold then the test is positive, if below then the test is negative. All the diseased and non-diseased patients have been classified [117]. FIGURE 22 The probability distributions of the results of a perfect diagnostic test After van Erkel, 1998 Number of patients Threshold value negative test result positive test result Test result 76 Figure 23 demonstrates a more realistic situation, one in which the patients' test results overlap each other rather than forming two separate groups. In this example, the use of fasting blood sugar (FBS) level to diagnose diabetes is shown. It is apparent that the use of this measure to diagnose diabetes will require the imposition of a cut-off or threshold point that will determine the sensitivity and specificity of the test. If we use the cut-off point of 100 mg/100 ml FBS the test would be 100% sensitive, i.e. correctly identifying all the patients with diabetes. However, this threshold choice would cause a reduction in specificity producing a large number of false positive results. This demonstrates the inverse relationship between sensitivity and specificity. If the threshold point was moved to the right along the x-axis to 140 mg/100 ml FBS the specificity would be 100% but the test would become highly insensitive [109]. This would result in a large number of patients being incorrectly diagnosed as not having diabetes. FIGURE 23 Diabetic and non-diabetic patient distribution using fasting blood sugar levels Fasting Wood sugar, mg/100 ml After Glaser, 1995 From this example, it is clear that a test can only be 100% sensitive and specific when there is no overlap between the normal and diseased populations. This is a rare circumstance. When this happens the presence of disease is often so obvious that no diagnostic testing is required [109]. 77 Predictive Values With the quantification of the sensitivity of a test, it is possible to determine the ability of that test to establish if a patient has a disease. Once the specificity is known, it is possible to assess the ability of the test to discover that a patient does not have the disease. These values are not of great value to the clinician who wants to know: "I have a positive result for this patient. How likely is it that they actually have the disease?" Alternatively, "I have a negative result for this patient. How likely is it that they are healthy?" In order to answer these questions the predictive values of the test must be determined. The predictive values are derived easily from the contingency table described earlier. The positive predictive value (PPV) indicates the likelihood of the patient actually having the disease following a positive diagnosis [111]. The negative predictive value (NPV) determines the likelihood of the patient not having the disease should a negative test result be obtained. It is important to realise that the values of sensitivity and specificity depend only on the characteristics of the test itself. However, the PPV and NPV will vary according to the prevalence of the disease. Predictive values cannot be quoted, therefore, without prior knowledge of the prevalence of the disease. PPV and NPV are not qualities of the test itself; rather they are a function of the characteristics of the test and the environment in which it is being used. 2.1.2 The use of forced decision models in the assessment of bitemark overlays It is important to realise that the diagnostic tests described above are based upon a 'forced decision model'. The clinician is forced to indicate whether the disease is present or absent. The threshold is set to maximise either the sensitivity or specificity depending on the nature of the disease and the relative importance of minimising false positives or false negatives. Is the use of a forced decision model in bitemark analysis valid? In bitemark analysis, the conclusion is usually expressed from a range of certainties to indicate the strength of the comparison that has been reached. However, juries, police officers, and others often simply extrapolate these findings as either positive or negative. In addition, 78 as forensic dentists claim that they can use overlays to detect the recorded unique features of the dentition on human skin, is it unreasonable to ask examiners to indicate simply whether or not a suspect is responsible for the bite? Another advantage of the forced decision model is the simplicity of the statistical analysis to determine the values explained previously. For these reasons, this study will use, in part, a forced decision model to assess the effectiveness of bitemark overlays. The application of this model to the current study is described in Chapter 3. What of the use of a range of certainties? Is there a statistical technique that enables the use of a more realistic approach to the bitemark comparison but still yields statistical values enabling the assessment of effectiveness? Such a technique exists in receiver operator characteristics (ROC). This is methodology adapted from radar signal detection and used widely in the assessment of complex, medical diagnoses [118,119]. 2.2 THE USE OF ROC IN THE ASSESSMENT OF DIAGNOSTIC ACCURACY Sensitivity and specificity describe the results of a test in a dichotomous way; either a test result is positive or negative [117]. This can be replicated in clinical practice. For example, should we extract this tooth or not? Should we place this restoration or not? However, many clinical tests are not dichotomous, such as probing of periodontal pockets or assessment of radiographs for caries where a range of features are examined to produce a certainty regarding the presence or absence of disease. This is also the case in bitemark analysis where the conclusions of the physical comparison are expressed using a range of certainties. The ABFO describe five levels of certainty that should be used when expressing a conclusion from a physical comparison of a bitemark to a suspect's teeth [55]. These conclusions are shown in Table 10. 79 TABLE 10 The five levels of conclusion recommended by the ABFO LEVEL DESCRIPTION 5. Inconclusive Insufficient evidence to make any statement 4. Excluded Not the biter 3. Possible Could be, may or may not be 2. Probably More likely than not 1. Reasonable medical certainty No reasonable or practical possibility that someone else did it. After ABFO, 1995 The use of ROC analyses has increased rapidly over the past fifteen years. It was initiated following the publication of Swets and Pickett's landmark textbook [119,120,121]. Many of the early applications of ROC were in the field of radiology where subjective results are recorded on a rating scale. Many of the papers describing the application of ROC in biomedical applications result from the efforts of radiologists. The expansion of ROC into the evaluation of diagnostic and prognostic tests that yield numerical results indicates the acceptance of its use and validity. ROC analysis is a graphical representation of the reciprocal relationship between sensitivity and specificity calculated from all possible threshold values. ROC analysis can be performed for tests that provide either continuous data or rating-scale data. The graphical nature of ROC is shown in Figure 24. Each of the threshold values shown in (b) correspond to an operating point on the ROC curve (a). When a high threshold is used (1) all patients are determined to be non-diseased resulting in both a true-positive fraction (TPF) of zero, and a false-positive fraction of zero i.e., a high specificity (100%). This relates to the operating point in the lower left-hand corner of the ROC curve. Using a very low threshold (5), it can be seen that the TPF and FPF are both 1 with a specificity of 0% and an operating point on the upper right-hand corner of the curve. This means that all the patients are determined to be diseased. The other threshold values represent intermediate points of specificity and sensitivity. 80 FIGURE 24 The use of an ROC curve to graphically represent threshold values A ROC curve represents the relationship of sensitivity and specificity (and hence a tool to determine these values) when a clinician is allowed to indicate a degree of uncertainty in their decision-making not afforded when making dichotomous decisions [122]. The method is equivalent to repeatedly asking clinicians to make simple, dichotomous decisions while holding different treatment attitudes or thresholds. For example, dentists asked to assess caries in patients who: a) will return in six months and b) will not attend for two years [123]. The discriminative ability of a test is defined by the distribution of diseased and non-diseased patients. The overlap of these groups determines the shape and position of the ROC curve. A straight line from the lower-left corner to the upper-right corner (shown red in Figure 24) describes a test in which the diseased and non-diseased distributions overlap completely and the TPF and FPF are equal at any threshold. This test has no discriminative value and is worthless. A perfect test has no overlap between the distributions of diseased and healthy, and would result in the straight line shown green in Figure 24. 81 2.2.1 Area under the curve The area under the ROC curve (AUC) is a measure of the diagnostic accuracy of a test and is frequently used to permit comparisons between tests or observers [122,124]. Using statistical software the AUC can easily be computed and tested for significant differences using z-scores (univariate) [125]. ROC curves can be generated for each observer in a study, the AUC can be calculated, and the results can be compared. It is also possible to pool data from observers and produce a single ROC. Using different groups of examiners, these AUCs can then be compared to identify differences between groups, typically using a paired /-test. Authors have stated that pooling results to create ROC's can be misleading as it ignores the effect of case sample variation [126]. This issue has been addressed by ensuring that each examiner assesses the same cases. In the example illustrated in Figure 24, the AUC for the test that yields no discriminative value, represented by the red diagonal line, has an area of 0.5 or 50%. It is no better than a random allocation of positive and negative results. The perfect ROC line, represented in green, has an AUC of 1.0 or 100%. The results from studies fall within these two ranges. The closer an AUC is to 100% or 1.0, the more accurate the diagnostic test. 2.2.2 Applications of ROC in forensic bitemark analysis The use of decision-making principles within forensic science, and forensic dentistry in particular, has been limited. However, one article exists that applies ROC analysis to human bitemarks. Whittaker used ROC to compare the abilities of experts and non experts to differentiate between adult and child bitemarks [127]. The study used 50 colour photographs of human bitemarks that were issued to 109 observers. The observers were divided into six distinct groups: senior forensic dentists, junior forensic dentists, general dental practitioners, final-year dental students, police 82 officers, and social workers. Each observer was asked, using a six-point rating scale, to indicate if an adult or a child had made the bitemarks. Examiners subjectively assessed the photographs and if the case exhibited sufficient characteristics of an adult or child's bite, then the mark was labelled at a level of certainty reflecting that observer's threshold. Whittaker plotted ROC curve with separate curves for each of the groups. The AUC for each of the six groups was calculated for comparison purposes. These results are shown in Figure 25 and Table 11. FIGURE 25 ROC curve from six groups of observers 0 0.1 0.2 0.3 0 4 0.5 0.9 07 0.0 09 I FiliaPoiitlnRalt • • a- • • Senior Fenrafe DanMt - -A - Jiriat Formic Dwtka - *- • Gtrml Otftt PndBmn -•-DrtrSrudwn - * — SeeMWortm POMM Offlsn After Whittaker, 1998 83 TABLE 11 Areas under ROC curves of different observer groups classifying bitemark as of child or adult origin OBSERVER GROUP AREA UNDER ROC CURVE STANDARD ERROR ± Senior forensic expert 0.693 0.0248 Junior forensic expert 0.680 0.0206 General practitioner 0.618 0.0262 Student dentist 0.690 0.0157 Police officer 0.618 0.0171 Social worker 0.634 0.0295 After Whittaker, 1998 Whittaker's study found that senior and junior forensic dentists and final-year dental students were best able to differentiate between the adult and child bitemarks. Whittaker did not use the different levels of certainty to establish individual specificities and sensitivities for each examiner group at the various threshold levels. One interesting aspect of Whittaker's study is the decision to introduce a 'don't know' level within his rating scale. Initially it could be considered incorrect to include a rating that does not indicate that a decision has been made, and hence the ability to represent a determined specificity and sensitivity. However, Swaving discussed in detail the statistical nuances of ROC and methods to implement the technique correctly, and determined that the inclusion of a 'don't know' level within ROC was legitimate [126]. 2.3 SUMMARY In this work, ROC analysis will be used in conjunction with a forced decision model to ensure authenticity of the bitemark comparison conclusions. ROC analysis will enable the determination of sensitivity and specificity of bitemark overlay comparisons at all ranges of conclusions. It will also enable a comparison of a variety of training levels on the efficacy of the observers to determine the biter by using the AUC. The combination 84 of both techniques will enable a thorough statistical treatment of the issues surrounding the effectiveness of bitemark overlays. 2.4 APPLICATION OF ROC AND FORCED DECISION MODELS TO BITEMARK OVERLAY COMPARISONS In order to establish the effectiveness and validity of bitemark overlays and to satisfy the requirements of the increased judicial interest in bitemark comparisons, assessment of the following variables was required: • The specificity and sensitivity of computer generated overlays • The positive and negative predictive values of the technique • Reliability measured by both intra and inter-examiner comparisons • Accuracy • The effect of levels of certainty upon the specificity and sensitivity • The AUC of both individual examiners and discrete groups of examiners • Kappa (chance corrected) agreements between examiners and the gold-standard and between groups of examiners • The effect of training and experience upon the effectiveness of bitemark overlay comparisons Two studies were designed to determine these values. Both studies presented examiners with photographs of bitemarks and computer generated overlays. The examiners were asked to assess if the suspects represented by the overlays were responsible for the bites. The first study examined the intra-examiner reliability of bitemark overlays and acted as a pilot study to test the materials using a forced decision model. The second study was used to determine the remaining variables described above and used a combination of a forced decision model and ROC. The design of the studies and the method of implementation are described in Chapter 3. 85 CHAPTER THREE METHODS AND MATERIALS 86 3.1 NUMBER AND NATURE OF EXAMINER GROUPS, EXAMINERS, AND CASES In order to address the impact of training and experience on bitemark overlay use, the following groups of examiners were identified: • Diplomates of the American Board of Forensic Odontology (ABFO) • Members of the American Society of Forensic Odontology (ASFO) • General Dental Practitioners (GDP) The ABFO group was represented by two, separate groups. The first ABFO group was used in Study One to provide data for intra-examiner reliability. The second ABFO group was involved in Study Two determine the inter-examiner reliability. These groups represented those examiners with the highest level of training and experience. Members of the ASFO were recruited who were practising dentists with an interest in forensic dentistry and had been involved in at least one bitemark case or had attended a training course on the subject. General dental practitioners were selected from a forensic dental study group concerned with mass disaster preparation. They had no practical bitemark experience other than attending three bitemark lectures. Ten cases were presented to each of 10 examiners. It was anticipated that this would take the examiners approximately 3-4 hours. This was the maximum time commitment considered likely to facilitate the recruitment of participants in the study. Each bitemark case was allocated two suspects resulting in a total of 20 decisions for each examiner and 200 decisions for each group. In total, this represented 40 examiners and 800 bitemark decisions (two ABFO groups, one ASFO group, and one GDP group) 87 3.2 THE SIMULATION OF BITEMARKS FOR STUDY 3.2.1 Selection of dentitions and determination of biters It was decided to create in situ postmortem bites on pigskin since this is widely accepted as an accurate analogue of human skin [128]. The use of animal skin analogues to produce simulated bitemarks is well established within the forensic dental field [129]. Previous studies have used pigskin (postmortem), dog skin (antemortem), and lambskin (postmortem) [94,129,130]. Dental casts were selected to ensure that the bitemarks produced represented a range of difficulty. In total 22 sets of high quality dental casts (upper and lower) were selected. Each of the ten bitemark cases had two casts associated with it, one biter and one non-biter. The biter in each case was determined randomly. In order to create cases where neither suspect was the biter, two cases had three dental casts associated with them. Models were labelled 'Suspect A' and 'Suspect B' for each of the ten cases; the unseen biters were labelled 'Suspect C. See Table 12. TABLE 12 Distribution of biters among the ten simulated cases CASE NUMBER SUSPECT A SUSPECT B SUSPECT C 1 Biter Non-Biter 2 Non-Biter Biter 3 Non-Biter Non-Biter Biter 4 Non-Biter Non-Biter Biter 5 Non-Biter Biter 6 Biter Non-Biter 7 Biter Non-Biter 8 Non-Biter Biter 9 Biter Non-Biter 10 Non-Biter Biter 88 3.2.2 Production of occlusal views and overlays TABLE 13 Materials for production of overlays ITEM MODEL MANUFACTURER LOCATION Scanner HP ScanJet 4c Hewlett Packard Co. Palo Alto, CA Scanning software HP DeskScan Hewlett Packard Co. Palo Alto, CA Scale ABFO No. 2 Lightning Powder Co., Inc. Salem, OR Computer PowerMac G3 Apple Computer Inc. Cupertino, CA Imaging software Photoshop v5.0.2 Adobe Systems Inc. Mountain View, CA Laser printer LaserWriter 4/600PS Apple Computer Inc. Cupertino, CA Transparency film Catalogue no. 9055 3M Visual Systems Division Austin, TX Sweet describes the most accurate form of overlay production currently available, and this method was employed [67]. The technique is described briefly here. Each dental cast was placed on to the glass platen of a flatbed scanner with an ABFO No. 2 scale and scanned into a computer using appropriate scanning software. The image was stored on the hard drive. Figure 26 illustrates the result of this procedure. Note that the scale acts as a left-right laterality indicator (L-shape indicates the suspect's left). FIGURE 26 Occlusal view of one of dental models used in the study 89 The image was opened in Photoshop and the following settings were made: Brightness 125, Contrast 118. A new layer was created and named 'Overlay'. Using the Magic Wand tool (Tolerance 18), the biting edges of the anterior teeth of interest were selected. Then the selected areas were copied into the 'Overlay' layer and a 2-pixel outline was produced around the perimeter of the biting edges using the Stroke command. The Clear command was used to produce a transparent centre within the outlined edges. The ABFO scale was then copied into the new layer and the background layer was discarded. Using the Text tool the case number and suspect identifier was added. The overlay images were printed on to transparency film using a laser printer set to 600 dpi. The ABFO scale was used to verify life-sized reproduction. The process is summarised in Figure 27. FIGURE 27 Summary of overlay production Selection of dental casts I Occlusal surfaces scanned into Photoshop with scale at left Incisal edges of anterior teeth selected with Magic Wand I Edges copied into new layer named 'Overlay' I Centre area of teeth cleared with Clear I Scale copied into Overlay layer, original image discarded I Identifying text added I Printed on to transparency film using laser printer This technique was used to produce overlays of Suspect A and Suspect B for each case. Note that overlays were not produced for Suspect C in Cases 3 and 4. Figure 28 shows the finished overlay (not life-sized, not transparent) that was produced using the dental 90 cast shown in Figure 26. Appendix C contains an example of a life-sized, transparent overlay. FIGURE 28 Bitemark overlay showing twelve anterior teeth 3.2.3 Creation of bitemarks on animal model TABLE 14 Materials for producing bitemarks on pigskin ITEM MODEL MANUFACTURER LOCATION Colour camera 801S Nikon Canada Inc. Toronto, ON Black-and-white camera 601 Nikon Canada Inc. Toronto, ON Flash SB-27 Speedlight Nikon Canada Inc. Toronto, ON Colour film Kodak Gold, ISO 200 Eastman Kodak Company Rochester, NY Black-and-white film SFX, ISO 200 Ilford Limited. Cheshire, England Inkjet printer 740i Epson Canada Inc. Toronto, ON Photographic paper Premium Photo Paper, Glossy, 7.5 Hewlett Packard Co. Palo Alto, CA Baseplate wax Pink, No. 10 Kerr Dental Inc. New York, NY Two piglets (7-8 weeks old), freshly slaughtered, weighing approximately 15 kilograms each were obtained from a local abattoir. Anatomical locations were selected on each 91 piglet that represented areas of minimal skin curvature and distortion. The lower abdomen and ears were found to be ideal locations. The dental casts from each randomly selected biter were clamped to the skin for a total of 10 minutes to create a bitemark. Following the release of the clamp the bitemark was subjectively examined to ensure that sufficient detail was recorded. Table 15 illustrates the selections made. Note that in Cases 3 and 4 neither suspect is responsible for the bite. This Table represents the gold standard of known biters and non-biters. TABLE 15 Distribution of biters and non-biters among the ten simulated cases CASE NUMBER SUSPECT A SUSPECT B 1 Positive Excluded 2 Excluded Positive 3 Excluded Excluded 4 Excluded Excluded 5 Excluded Positive 6 Positive Excluded 7 Positive Excluded 8 Excluded Positive 9 Positive Excluded 10 Excluded Positive The injury was photographed following the ABFO guidelines [55]. One colour and one black-and-white photograph were exposed with the ABFO No. 2 scale in place. An off camera flash was held at an oblique angle to the bitemark to ensure that the depth of the injury was emphasised. The films were developed and the best reproduction of each bitemark was selected. These were sent to specialist photographic laboratories for printing at life-size. Subsequently, the photographs were scanned into a computer and stored in JPEG format at 1440 dpi. These images were printed on an inkjet printer at 1440 dpi using special photographic paper. Prints were made for each examiner. An example of a photographic series is shown (not to scale) in Figure 29. The process is summarised in Figure 30. Appendix C contains an example of a photographic series distributed to the examiners. 92 FIGURE 29 Colour and black-and-white photographs of a simulated bitemark on pigskin i % \ HP' 4>> 1 '*t .1..,. ; Nr*-i • mmmm. ' 1 MM c III TIM 1 FIGURE 30 Summary of production of simulated bites Selection of dental casts I Clamp casts to pigskin for 10 minutes i Examine bitemark, repeat at alternate site if necessary I Colour, black-and-white photographs with oblique flash and scale I Develop film, print images at life-size (1:1) I Scan at 1440 dpi I Label with text, print on photographic paper Following production of the bitemarks each cast was used to create a wax bite by softening wax in warm water and then gently pressing the dental casts into the material 93 producing an impression of both maxillary and mandibular arches. Each wax bite was labelled with a case number and suspect letter. FIGURE 31 Mandibular wax bites from Suspect A and B, Case 1 3.3 STUDY ONE An anonymous pilot group consisting of ten Diplomates of the ABFO was selected. Each participant received 10 simulated bitemark cases containing one colour and one black-and-white photograph of the bite, two computer-generated overlays labelled Suspect A and Suspect B, one wax bite from each suspect, occlusal views of the suspects' dentition, instructions, and an answer sheet. The examiners were asked to determine whether each suspect was the biter or not for the appropriate case. The examiners were asked to either indicate 'positive' for a biter or 'excluded' for a non-biter. No other option was included. An example of a completed answer sheet is shown in Figure 32. Ten Diplomates returned answer sheets for the first assessment (100%); however, only seven returned the study materials. As three Diplomates retained the materials, the second assessment carried out three months later involved only seven of the Diplomates. These Diplomates were sent the same materials and asked to carry out the exercise as before to assess intra-examiner reliability. 94 FIGURE 32 Completed answer sheet from Study One Case Number Suspect A Suspect B Positive Exclusion 2 VP&H *J3L £*AUI>D£ Susnficr <A ft 3 OME Ptleujsfe. a^i 5 OAse-f« CAsfc 7 c.**e. * T The results from Study One were entered into tables and treated statistically. Each of the examiners responses were compared between the two different assessments and kappa was applied to correct for chance. The statistical software package PEPI was used [131]. Results from this study can be found in Appendix D and Chapter 4. 3.4 STUDY TWO A group consisting of ten Diplomates of the ABFO, ten members of the ASFO, and ten General Dental Practitioners was selected. Each participant received ten bitemark cases containing 1 colour and 1 black-and-white photograph of a simulated bitemark, 2 computer generated overlays labelled Suspect A and Suspect B, occlusal views of each suspect's dentition, instructions, and an answer sheet. The instructions and answer sheet were revised from Study One to incorporate the inclusion of the ROC analysis using the five levels of certainty described by the ABFO and the 'Don't know' option within the forced decision model. An example of a completed answer sheet from Study Two is shown in Figure 33. 95 FIGURE 33 Completed answer sheet from Study Two Results of Analysis Using Comparison Overlays - Forced Decision Case Number Suspect A Suspect B Example Positive Exclusion 1 2 3 fr*elu 4 L-*.e U«ic»r\ 5 U>wX» U- to 4< cid* 6 Pot) +iv«-7 rosi+.ve. 8 U.*atA<fW cUxii^e 9 10 gKCliA slon Results of Analysis Using Comparison Overlays - Level of Conclusion Case Number Suspect A Suspect B Example Uvel 1 Level 5 1 Uutl 5 2 Uutl 2-3 4 Uut\ <* 5 UIA( 3 6 Uu*t 1 7 Uvt{ 2_ 8 i-tv*( 2-9 Uvel Z- Ll^m 10 Leuil If U.\M 1 ^-Responses were returned by 30 examiners in Study Two (100%). Results were entered into tables and analysed using the PEPI statistical application [131]. Data were obtained for values of sensitivity, specificity, PPV, NPV, and the ROC values of sensitivity and specificity for each level of conclusion. The AUC for each examiner was determined. Results from this study can be found in Appendix D and are described in Chapter 4. 96 CHAPTER FOUR RESULTS 97 SECTION 1 STUDY ONE 98 4.1 STUDY ONE The results from Study One that deal with intra-examiner agreement are described. In total seven examiners returned correctly completed answer sheets on both occasions (70%). The data from the examiners is organised as shown in Figure 34 and can be found in Appendix D. FIGURE 34 Example of data from Study One Panel 1 CASE NUMBER FIRST EXAMINATION SECOND EXAMINATION SUSPECT A SUSPECT B SUSPECT A SUSPECT B 1 Positive Excluded Positive Excluded 2 Excluded Positive Excluded Positive 3 Excluded Positive Positive Excluded 4 Positive Positive Excluded Positive 5 Excluded Positive Positive Excluded 6 Excluded Positive Positive Excluded 7 Positive Excluded Positive Excluded 8 Excluded Positive Excluded Positive 9 Positive Excluded Positive Excluded 10 Excluded Positive Excluded Positive Panel 2 - Contingency Tables Forced Decision Model - First Examination Forced Decision Model - Second Examination Gold Standard - Known Biter Gold Standard - Known Biter Positive Excluded Total Positive Excluded Total Overlay result Positive 7 4 11 Overlay result Positive 7 3 10 Excluded 1 8 9 Excluded 1 9 10 Total 8 12 20 Total 8 12 20 Panel 3 — First Examination Data Sensitivity % Specificity % PPV NPV PREV Accuracy TD FPR % FNR % PTP+ PTP-% % % % % % 87.5 66.6 63.6 88.9 40 75 20 33.3 12.5 63.4 11.1 PA% PA+% PA-% Kappa SE 75 74 76 0.51 0.214 Panel 4 - Examination Data Sensitivity % Specificity % PPV NPV PREV Accuracy TD FPR % FNR % PTP+ Pip. % % % % % % 87.5 75.0 70 90 40 80 20 25.0 12.5 70 10 PA% PA+% PA-% Kappa SE 80 78 82 0.60 0.219 Panel S - Comparison Between Examination 1 and 2 Exam 1 | | Kappa Results From Intra-Examiner Data Positive Excluded Kappa SE P PA% Exam 2 Positive 7 3 10 0.30 0.222 0.089 65 Excluded 4 6 10 Total 11 9 20 99 Panel 1 of the Figure represents the raw data returned by the examiner and is split into the responses from the first and second examination of the cases. The repeated examinations were three months apart. Panel 2 shows the contingency tables produced from the examiners' responses. The values of TP, FP, TN, and FN are displayed. Panel 3 and Panel 4 provide summary statistics that are derived from the contingency tables. Information provided includes the sensitivity and specificity, positive and negative predictive values (PPV, NPV), prevalence of biters (PREV), accuracy, total number of decisions made (TD), false positive and false negative rates (FPR, FNR), and the post-test probabilities of the suspect being the biter given a positive or negative decision by the examiner (PTP+, PTP-). Panel 5 provides the data for the determination of intra-examiner reliability - the results of the first examination compared to those of the second. Cohen's Kappa was applied and the results of this, its standard error (SE) and the significance (P) of the value are presented. Percentage agreement (PA) is also included. 4.1.2 Pilot data It was found that the case materials provided a good spread of correct and incorrect responses, with the accuracy scores for the examiners ranging from 75 - 95%. No case was found to be answered significantly better or worse than any other (p=0.645). 4.1.3 Intra-examiner reliability The intra-examiner reliability was calculated for all seven Diplomates. The results are summarised in Table 16. The kappa values varied from 0.30 - 1.00, or from fair to almost perfect agreement. Mean kappa was 0.72, indicating substantial agreement. Percent agreement (non-chance corrected) ranged from 65 - 100% with a mean value of 87.2%. 100 TABLE 16 Summary of kappa values from Study One EXAMINER KAPPA S.E. P % AGREEMENT 1 0.30 0.222 0.089 65 2 0.38 0.219 0.041 70 3 1.00 0.224 >0.000 100 4 1.00 0.224 >0.000 100 5 0.52 0.224 0.010 80 6 0.88 0.222 >0.000 95 7 1.00 0.224 >0.000 100 4.1.4 Summary statistics from Study One The mean accuracy for the seven examiners' first and second attempt was 85.7% and 83.5% respectively, with no statistically significant difference between the attempts (p=0.6286). When examining kappa values for comparisons with the gold standard, the initial examination mean was 0.70. This decreased slightly to 0.65 on the second attempt. Both scores rate as substantial agreement and no significance was detected between the attempts (p=0.5568). The mean values for initial sensitivity (79.8%) and specificity (90.0%) were calculated and compared to the mean sensitivity (73.2%) and specificity (89.3%) values from the second examination. No statistical significance was detected between these values (sensitivityp=0.5218, specificityp=0.5792). 101 SECTION 2 STUDY TWO 102 4.2 STUDY TWO The results from Study Two (ROC, forced decision model (FDM) and inter-examiner agreements) are described. In total 30 examiners returned correctly completed answer sheets, ten ABFO Diplomates, ten ASFO members, and ten General Dental Practitioners. The data from the examiners involved in the study can be found in Appendix D. The data presented in Appendix D is organised as shown in Figure 35. FIGURE 35 Example of data from Study Two Panel 1 CASE NUMBER SUSPECT A SUSPECT B FDM LEVEL ROC FDM LEVEL ROC 1 Positive 2 Excluded 5 2 Excluded 5 Don't Know 5 3 Excluded 5 Excluded 5 4 Don't Know 5 Excluded 5 5 Excluded 5 Excluded 5 6 Positive 1 Excluded 5 7 Excluded 5 Positive 2 8 Excluded 3 Positive 2 9 Excluded 3 Excluded 5 10 Excluded 5 Excluded 5 Panel 2 Forced Decision Model ROC Analysis Gold Standard - Known Biter Level Sensitivity % Specificity % Youden's Index Pos. Excluded Total 5 100.0 22.3 0.28 Overlay Positive 3 1 4 4 50.0 83.3 0.33 result Excluded 5 11 16 3 50.0 83.3 0.33 Total 8 12 20 2 37.5 91.7 0.29 1 12.5 100.0 0.13 Panel 3 Sensitivity % Specificity % PPV % NPV % PREV % Accuracy % TD FPR % FNR % PTP+ % PTP-% 37.5 91.7 75 68.7 36 55 18 8.3 62.5 75 31.3 AUC% SIG (50 %) PA% PA+% PA-% Kappa SE BCO 68.2 0.080 70 50 79 0.32 0.199 4 103 Panel 1 of the Figure represents the actual responses returned by the examiner and is split into the FDM decision ('Positive', 'Excluded', or 'Don't Know') and the ROC level determined. Panel 2 shows the results of the statistical treatments that were performed. The values of TP, FP, TN, and FN are displayed for the forced decision model and the individual sensitivity and specificity scores for each level of the ABFO conclusions are listed. Other information provided includes sensitivity and specificity of the FDM results, positive and negative predictive values (PPV, NPV), prevalence of biters among the FDM (PREV), accuracy, total number of decisions made (TD), false positive and false negative rates (FPR, FNR), and the post-test probabilities of the suspect being the biter given a positive or negative indication by the FDM (PTP+, PTP-). Panel 3 gives the area under the curve following ROC analysis and the significance of this result (AUC, SIG). Other variables include the percentage agreement (PA%) between the examiner and the gold standard. This is further divided into the agreement with biters (PA+%) and non-biters (PA-%). Cohen's Kappa (Kappa) is provided with standard error (SE). The best ABFO conclusion threshold determined by ROC, given equal weighting to false positives and false negatives, is provided (BCO). 4.2.1 Results from ABFO Forced decision model (FDM) Ten Diplomates of the ABFO returned correctly completed answer sheets (100%). From a total of 200, 28 (14%) were Don't Knows. However, 24 (12%) of these Don't Knows were attributable to two Examiners (Examiner 2, 16 and Examiner 10, 8). Excluding these examiners the total uncertain decisions is reduced to only 4 (2%). The distribution of the Don't Knows among the simulated cases is shown in Figure 36. Sensitivity was calculated for each examiner and ranged from 28.6 - 100%) with a mean sensitivity of 73.7 ± 22.0%>. Specificity among this group ranged from 54.5 - 100%) with a mean sensitivity of 84.1 ± 14.9%. There was no significant difference between the sensitivity and specificity scores (p=0.2721). 104 Accuracy, determined as percent agreement with the gold standard, ranged from 65.0 -100% with a mean value of 83.17%. Agreement determined by Cohen's Kappa ranged from 0.22 (fair agreement) - 1.00 (almost perfect agreement). The mean Kappa was 0.58 (moderate agreement). The mean false positive rate was 15.9% (FPR), ranging from 0 - 45.5%, and the mean false negative rate was 25.0% (FNR) ranging from 0 - 71.4%. The positive predictive value (PPV) ranged from 55.5 - 100%) with a group mean of 77.7%. The negative predictive value (NPV) ranged from 66.6 - 100% with a group mean of 83.2%). ROC Analysis The mean sensitivity, specificity, and Youden's Index for each of the conclusion levels is shown in Table 17. 105 TABLE 17 Mean results from ROC analysis of ABFOs CONCLUSION LEVEL SENSITIVITY % (SD) SPECIFICITY % (SD) YOUDEN'S INDEX 5 100.0 (±0.0) 0.0 (±0.0) 0.00 4 88.8 (±19.1) 47.7 (±24.0) 0.36 3 81.3 (±22.2) 55.3 (±30.0) 0.40 2 57.5 (±26.5) 94.9 (±11.0) 0.52 1 27.5 (±24.1) 98.3 (±5.2) 0.26 The AUC for the ABFOs ranged from 62.0 - 97.7% (mean 80.5 ± 11.8%). Inter-examiner reliability Using Cohen's Kappa, each of the examiners was paired and compared using a cross wise system based upon their FDM decisions. The results of these kappa calculations are shown in Table 18. From these data it was determined that 10 pairs (22%) had slight agreement, 11 pairs (24%) had fair agreement, 13 pairs (29%) had moderate agreement, 3 (7%) pairs had substantial agreement and 8 pairs (18%) had almost perfect agreement. The mean kappa from the cross-wise analysis was 0.47 ± 0.31 or moderate agreement. TABLE 18 Cross-wise kappa comparison between ABFOs ABFO EXAMINER 1 2 3 4 5 6 7 8 9 10 1 X 0.54 0.03 0.03 0.28 0.19 0.48 0.52 0.02 0.07 2 X 1.00 1.00 1.00 1.00 1.00 1.00 1.00 1.00 3 X 0.39 0.37 0.29 0.20 0.18 0.11 0.33 ABFO EXAMINER 4 X 0.50 0.47 0.38 0.30 0.16 0.50 5 X 0.44 0.53 0.56 0.47 0.53 6 X 0.89 0.61 0.22 0.50 7 X 0.70 0.29 0.35 8 X 0.11 0.50 9 X 0.07 10 X 106 4.2.2 Results from ASFO Forced decision model (FDM) Ten members of the ASFO returned correctly completed answer sheets (100%). From a total of 200 decisions, 18 (9%) were Don't Knows. The distribution of the Don't Knows among the simulated cases is shown in Figure 37. Sensitivity was calculated for each examiner and ranged from 28.6 - 85.7% with a mean sensitivity of 60.9 ± 22.9%. Specificity among this group ranged from 34.6 - 100% with a mean sensitivity of 82.4 ± 19.7%). There was no significant difference between the sensitivity and specificity scores (p=0.0378). Accuracy, determined as percent agreement with the gold standard, ranged from 55.0 - 94.1% with a mean value of 75.8%. Agreement determined by Cohen's Kappa ranged from 0.16 (slight agreement) - 0.88 (almost perfect agreement). The mean kappa was 0.50 (moderate agreement). FIGURE 37 Distribution of Don't Knows among ASFOs i o £3 & 2 (D Si E 3 < CO <CD<DQ<CQ<CD<CQ<CQ cNCMCOCO^-^rWlOCDCOr^r- < CD < CQ co co CR cn < CO o o Suspect The mean false positive rate was 11.9% (FPR), ranging from 0 - 27.3%, and the mean false negative rate was 39.3% (FNR) ranging from 14.3 - 74.4%. The positive predictive value (PPV) ranged from 59.9 - 100%) with a group mean of 79.7%. The negative predictive value (NPV) ranged from 58.4 - 91% with a group mean of 78.1%. 107 ROC Analysis The ROC data reveal two main results: a) the individual sensitivity and specificity of each ABFO threshold, and b) the area under the curve as a measure of effectiveness. The mean sensitivity and specificity for each of the conclusion levels is shown in Table 19. Youden's Index, a measure of agreement using sensitivity and specificity, was also calculated for each of the five conclusion levels. The closer Youden's Index is to 1.0 the greater the level of agreement. TABLE 19 Mean results from ROC analysis of ASFOs CONCLUSION SENSITIVITY SPECIFICITY MEAN YOUDEN'S LEVEL % (SD) % (SD) INDEX 5 100.0 (±0.0) 0.0 (±0.0) 0.00 4 77.5 (±14.1) 68.7 (±14.7) 0.46 3 72.5 (±12.9) 74.4 (±11.2) 0.47 2 53.8 (±17.7) 94.3 (±8.4) 0.48 1 23.8 (±17.1) 98.5 (±4.9) 0.24 The AUC for the ASFO members ranged from 62.5 - 89.6% (mean 81.0 ± 8.8%). Inter-examiner reliability Using Cohen's Kappa, each of the examiners was paired and compared using a cross wise system based upon their FDM decisions. The results of these kappa calculations are shown in Table 20. It was determined that 3 pairs (7%) had poor agreement, 5 pairs (11%>) had slight agreement, 9 pairs (20%) had fair agreement, 16 pairs (36%) had moderate agreement, 11 (24%) pairs had substantial agreement and 1 pair (2%) had almost perfect agreement. The mean kappa from the cross-wise analysis was 0.44 ± 0.22 or moderate agreement. 108 TABLE 20 Cross-wise kappa comparison between ASFOs ASFO EXAMINER 1 2 3 4 5 6 7 8 9 10 1 X 0.22 0.44 0.56 0.44 0.55 0.30 0.19 0.24 0.26 2 X 0.64 0.51 0.74 0.01 0.25 0.41 0.55 0.44 ASFO EXAMINER 3 X 0.71 0.72 0.38 0.48 0.45 0.76 0.76 4 X 0.71 0.13 0.60 0.45 0.62 0.43 5 X 0.43 0.48 0.65 0.63 0.44 6 X 0.27 0.17 0.04 0.27 7 X 0.76 0.20 0.05 8 X 0.12 0.36 9 X 0.89 10 X 4.2.3 Results from General Dental Practitioners (GDP) Forced decision model (FDM) Ten General Dental Practitioners (GDPs) returned correctly completed answer sheets (100%). From a total of 200 decisions, a total of 15 (7.5%) were Don't Knows. The distribution of the Don't Knows among the simulated cases is shown in Figure 38. Sensitivity was calculated for each examiner and ranged from 62.5 - 100% with a mean sensitivity of 80.7 ± 13.5%. Specificity among this group ranged from 50 - 100% with a mean sensitivity of 77.9 ± 15.0%. There was no significant difference between the sensitivity and specificity scores (p=0.6001). Accuracy, determined as percent agreement with the gold standard, ranged from 55.6 - 84.2% with a mean value of 74.7%. Agreement determined by Cohen's Kappa ranged from 0.14 (slight agreement) -0.89 (almost perfect agreement). The mean kappa was 0.56 (moderate agreement). The mean false positive rate was 22.0%» (FPR), ranging from 0 - 50.0%>, and the mean false negative rate was 19.3% (FNR) ranging from 0 - 37.5%. The positive predictive value (PPV) ranged from 46.0 - 100%) with a group mean of 72.7%. The negative predictive value (NPV) ranged from 70.1 - 100% with a group mean of 85.7%. 109 FIGURE 39 Distribution of Don't Knows among GDPs. ROC Analysis The mean sensitivity, specificity, and Youden's Index for each of the conclusion levels is shown in Table 21. TABLE 21 Mean results from ROC analysis of GDPs CONCLUSION SENSITIVITY SPECIFICITY YOUDEN'S INDEX LEVEL % (SD) % (SD) 5 100.0 (±0.0) 0.0 (±0.0) 0.00 4 83.6 (±10.3) 55.9 (±11.3) 0.37 3 76.3 (±10.9) 64.2 (±11.9) 0.37 2 60.0 (±18.4) 93.4 (±5.3) 0.55 1 12.5 (±11.8) 99.2 (±2.3) 0.13 The AUC for the GDPs ranged from 64.1 - 90.6% (mean 80.8 ± 8.0%). Inter-examiner reliability Using Cohen's Kappa, each of the examiners was paired and compared using a cross wise system based upon their FDM decisions. The results of these kappa calculations are shown in Table 22. It was determined that 3 pairs (7%) had poor agreement, 6 pairs (13%>) had slight agreement, 8 pairs (18%) had fair agreement, 17 pairs (38%) had 110 moderate agreement, 10 (22%) pairs had substantial agreement and 1 pair (2%) had almost perfect agreement. The mean kappa from the cross-wise analysis was 0.45 ± 0.23 or moderate agreement. TABLE 22 Cross-wise kappa comparison between GDPs GDP EXAMINER 1 2 3 4 5 6 7 8 9 10 1 X 0.38 0.28 0.47 0.26 0.14 0.30 0.18 0.30 0.30 2 X 0.45 0.73 0.13 0.68 0.78 0.50 0.48 0.78 3 X 0.49 0.03 0.20 0.45 0.30 0.06 0.45 GDP EXAMINER 4 X 0.05 0.38 0.47 0.14 0.12 0.47 5 X 0.43 0.41 0.47 0.49 0.41 6 X 0.55 0.54 0.75 0.55 7 X 0.77 0.74 1.00 8 X 0.75 0.77 9 X 0.75 10 X 4.2.4 Comparisons between the three examiner groups Table 23 shows a comparison of mean values obtained from the FDM study. Table 24 shows similar data from the ROC results of the three groups. Figure 38 demonstrates the distribution of Don't Know responses to the FDM. There was no statistical significance between these distributions when tested with a Student's /-test. Table 25 demonstrates the results of significance tests between the three groups. No significant differences were identified between the three experience groups for kappa, AUC, accuracy, sensitivity, or specificity. A discussion of these findings can be found in Chapter 5. Ill TABLE 23 Summary of means from the FDM and cross-wise kappa analysis ABFO ASFO GDP Don't Knows 14.0% 9.0% 7.5% Mean Sensitivity 73.7% (±22.0) 60.9% (±22.9) 80.7% (±13.5) Mean Specificity 84.1% (±14.9) 82.4% (±19.7) 77.9% (±15.0) Mean Accuracy 83.2% 75.8% 74.7% Mean Kappa (GS) 0.58 0.50 0.56 Mean Kappa (CW) 0.47 0.44 0.45 Mean FPR 15.9% 11.9% 22.0% Mean FNR 25.0% 39.3% 19.3% (GS) kappa comparisons with the gold standard (CW) inter-examiner cross-wise kappa comparisons TABLE 24 Summary from ROC analysis ABFO(%) ASFO (%) GDP (%) Mean AUC 80.5 ± 11.8% 81.0 ±8.8% 80.8 ± 8.0% Mean Sensitivity (Ll) 27.5 23.8 12.5 Mean Sensitivity (L2) 57.5 53.8 60.0 Mean Sensitivity (L3) 81.3 72.5 76.3 Mean Sensitivity (L4) 88.8 77.5 60.0 Mean Sensitivity (L5) 100.0 100.0 100.0 Mean Specificity (Ll) 98.3 98.5 99.2 Mean Specificity (L2) 94.9 94.3 93.4 Mean Specificity (L3) 55.3 74.4 64.2 Mean Specificity (L4) 47.7 68.7 55.9 Mean Specificity (L5) 0.0 0.0 0.0 4.2.5 Comparison of ABFO groups from Study One and Study Two The results of the FDM decisions of Diplomates participating in Study One were compared with those from Study Two. Table 26 summarises these comparisons. No significant differences were found between the examiners in these two groups. The kappa agreement of the Diplomates from Study One falls within the substantial rating of Landis, whereas the kappa from the Study Two falls within the moderate category [115]. 1.12 FIGURE 38 Summary of Don't Know responses for each examining group TABLE 25 p values for inter-group comparisons from one way ANOVA VARIABLE pVALUE Accuracy 0.8406 Kappa 0.5866 Sensitivity 0.2536 Specificity 0.4663 AUC 0.9283 TABLE 26 Comparisons between mean ABFO results from Study One and Study Two STUDY KAPPA SENSITIVITY (%) SPECIFICITY (%) Study One - Exam 1 0.70 79.8 90.9 Study One - Exam 2 0.65 73.2 89.3 Study Two 0.58 73.7 84.1 114 CHAPTER FIVE DISCUSSION 115 5.1 EXPERIMENTAL DESIGN A key feature of modern science is that of scepticism; no longer are scientific principles accepted based on authority or common-sense anecdotal beliefs. This is no truer than in the field of forensic science and has been enforced by legal judgements such as those described in Daubert and Kumho. Claims made should be checked against empirical evidence and the value of that empirical evidence is based upon the way that it has been collected and presented [114]. The purpose of this work has been to establish empirical justification for the use of transparent overlays in the analysis of bitemark injuries and their use in the identification of biters within a forensic context. The increased interest in evidence-based medicine and dentistry has revitalised techniques for the assessment of diagnostic effectiveness. The discipline of medical-decision making has employed them in increasingly novel ways to challenge the basis upon which clinical practise is built. Using these techniques, this work has arrived at quantitative values for the analysis of overlay effectiveness. Before these values are examined, the experimental technique should be discussed so that appropriate weight can be afforded to them. 5.1.2 Selection of examiners Forensic dentistry is a relatively small field, with approximately 100 board-certified odontologists practising in the United States. Canada has only four such individuals. The recruitment of observers from such a small pool was challenging. It was confounded by the fact that the majority of Diplomates were involved in private dental practise and, therefore, had limited time available to participate in research. These considerations were a crucial component in determining the number of examiner participants. A review of studies employing similar methods was carried out to discover the number of examiners commonly used in such investigations: Maupome (15) [132], Verdonschot (10) [133], Kay (20) [123], Chapman (12) [134], Steinbach (4) [135], Getty (16) [136]. It was 116 decided that ten examiners for each observer group was appropriate given previous studies and the number of potential participants available. Determination of the three examiner groups was based upon gaining a range of training and experience. The Diplomates of the ABFO represented the highest level of training and experience. It is important to note that not all individuals that can be described as experts have necessarily challenged the Board's certification examination. However, this group represents a gold standard of forensic experience and is a convenient method of sampling such individuals without the need for questionnaires gauging experience. The selection of the ASFO for the second group of examiners was convenient as membership of this group implied an interest in forensic dentistry. It was thought that this interest would facilitate the recruitment of participants and enable an intermediate level of experience or training to be represented in the study. The General Dental Practitioners in the third group were sourced from a forensic dental study group whose remit is the design and implementation of a dental GO-TEAM for the identification of victims of a mass disaster in British Columbia. The GDPs represented a novice level of experience within the study. This group also has an expressed interest in forensic dentistry and this was thought likely to facilitate recruitment of examiners. This study has utilised three groups of examiners although other investigators have chosen additional participants. Whittaker, in addition to qualified dentists, involved dental students, police officers, and social workers within his ROC study [127]. The inclusion of such groups within the current study was considered, but the value of the resultant data in relation to the expressed purpose of the investigation was not thought to be worthwhile. A feature that could be included in future work is a more defined measurement of experience. A questionnaire could be used to determine an experience value based upon the number of years of forensic practise, number of cases performed, and additional training undertaken. This experience value could then be compared to individual results to assess with greater resolution the impact of training and experience 117 upon the performance of bitemark examiners. In this study, the use of clearly defined examiner groups has allowed this factor to be examined in a gross fashion. 5.1.3 Number of simulated cases A large number of decisions rendered by the examiners will translate into a greater significance of the study, until a data saturation point is reached. At this point, the increase in data no longer results in an equivalent increase in understanding [114]. Several factors limited the number of cases that were used in this investigation. The first is one of expense. The production of the simulated bites and the subsequent production of exemplars was an expensive process. The duplication of photographic images, in particular, limited the number of potential cases. The second is time. While financial considerations are important in the design of any study, the time available to the examiners was also crucial. As previously described the vast majority of the individuals invited to participate in the study are dentists involved in private general practise. It was determined that 10 bitemark cases should take approximately three hours and that this was a reasonable amount of time to request of participants. An increase in cases could have limited the acceptability and uptake of the study by the dentists. A total of 200 decisions from each group were thought to present a sufficient number for this initial study while still ensuring maximum opportunity to recruit examiners. The issue of participant recruitment will be important in future studies. While the pool of General Dental Practitioners is large and relatively easy to recruit from, the limiting factor will always be the availability of forensic dentists of Diplomate status. The very fact that these dentists are actively involved in forensic casework in addition to their regular clinical practise limits their availability for research time. Future studies that require the input of such individuals should be designed to facilitate this group's participation by ensuring that time and other factors are considered. It is also important to motivate the examiners by clearly demonstrating the use of the study to their individual forensic practise. Diplomates are more likely to be involved with studies that have a 118 direct impact upon their work rather than those that appear to be esoteric academic exercises. 5.1.4 The use of simulated cases During initial planning considerable thought was given to the production of the bitemark cases concerning the use of either real or simulated cases. The use of real forensic cases as study material has advantages. Firstly, authenticity - materials used would be similar to those handled by forensic dentists during their casework. Secondly, many examples of bitemarks exist both at the Bureau of Legal Dentistry and in other centres. The collation and duplication of such materials would therefore be a simple matter. Several disadvantages are associated with the use of real cases. The most important of these is that of the gold standard. One of the criteria for assessing the effectiveness of a particular test is to ensure that it is compared against a suitable gold standard. The use of probative case materials requires that the conclusion of the original examining forensic dentist is regarded as such a standard. Due to the lack of published studies, it is impossible to determine how accurate the original conclusion is likely to be. Indeed, it is the purpose of this study to provide such data. In Whittaker's ROC study, forensic casework was used as study material and the final Court verdict was deemed the gold standard. Whittaker concedes that some error may have been introduced by such a determination of truth [127]. The inconsistency of forensic case materials (quality, type, and number of photographs, anatomical location, decomposition, etc.) can also lead to bias. The inability to control critical features of the real bitemarks supplied to examiners is another drawback of this technique. The production of robust and reliable case materials is an essential feature of an investigation such as this. The use of simulated bitemarks enables greater control over the injury. Variables such as anatomical location, the teeth used to create the bite, and the collection of the evidence are easily managed and standardised. The use of simulations also permits a consistent 119 quality of materials to be produced, allowing parity between each of the study cases and removing any potential bias introduced by case variability. However, simulations do have limitations. Significantly, the simulated bites were not on human skin. Due to ethical issues, surrounding the use of animals the bites would have to be produced postmortem. Postmortem bites do not display any of the ecchymosis or bruising patterns that are seen in antemortem or perimortem bite injuries and this could be considered a limitation. However, postmortem bite injuries do record the details of teeth well. A comparison of postmortem and antemortem bites is shown in Figure 40. The use of postmortem simulated bites is a well-accepted technique within forensic dental research [129,137]. FIGURE 40 A comparison of postmortem (left) and antemortem (right) bitemarks The result of using simulated bitemarks ensured that each of the cases was produced in a similar way and hence there were common features among the cases. The lack of variability between the 10 cases removed any potential bias resulting from inconsistencies. The simulated cases were stable and robust. This was proven by the equal distribution of Don't Know responses among the examiners. The cases were produced to represent a range of difficulty to ensure that the quality of the materials favoured a correct response. 120 5.1.5 Selection of dentitions Some studies have used volunteers to bite animal skin either in situ or post-removal [130]. Others have used stone dental models to create the bite injuries [129]. In this study, stone models of human teeth were selected to create the bite injury. This provided the greatest control over the appearance and therefore relative difficulty of the simulated bite cases. The criteria for the selection of the models were purposefully open. In order to ensure meaningful data, the simulated cases must not be so simple that all the examiners would correctly identify the biters in each case. It is also important that they not be so difficult that the examiners would be unable to identify correctly any biters [111]. The premise of case selection in medical decision-making studies is that cases should be borderline with a spread of difficulties within the materials offered [114,138]. The use of dental casts facilitated the production of such cases, ensuring that dentitions were similar enough to present a challenge and yet not too dissimilar to ensure that correct responses on the cases was possible. When addressing issues of prevalence in bitemark cases the situation is somewhat artificial. Only one individual can be responsible for a bite and therefore the prevalence is dictated by the number of non-biting suspects presented to the odontologist for analysis. The use of bitemark 'line-ups' is recommended for forensic casework and is regarded as good practise [35,55]. This involves the inclusion of a number of randomly selected dental casts in addition to those presented to the odontologist. The principle behind this is based upon blind testing. Arguably, the premise of line-ups is at odds with the successful identification of a biter. An increase in the number of dentitions examined will decrease the prevalence. As described in Chapter 2, the predictive values of a test are affected by the prevalence. An example of this would be an HIV test that is administered in a population where the prevalence of the infection is 10 in 10,000 people. With an assumed sensitivity of 90% and specificity of 99%, the positive predictive value (PPV) of this test would be 0.08. This means that there is only a 9%> chance that an individual with a positive result would actually have the disease. However, if the same test were applied to a higher-risk 121 population with a disease prevalence of 10 in 1000 people the PPV would be 47% [109]. The conflict between sound forensic practise and the desire to maximise the efficiency of a test to identify a biter needs to be discussed within the discipline. Future work could test the effect on prevalence within individual cases. For example, in this study each case contained two suspects. Further investigations could employ differing number of suspects, and gauge the effect of this upon the successful identification of the biter. The 1999 ABFO Workshop, whose results are discussed later in this Chapter, used seven suspects with four cases presented. In studies that utilise case materials to assess diagnostic effectiveness it is usual to set prevalence at 50% [134,139,140]. The use of a prevalence of 50% ensures a result most favourable to the examiners. Deception, such as including no biters or all biters is likely to skew the results unfairly and confounds the study by incorporating issues of the mind set of the examiners. Most individuals would not expect to be presented with 10 cases, none of which contained a biter. The 40% prevalence in this study is a reflection of the two cases in which neither Suspect A or Suspect B is the biter, lowering the overall prevalence but increasing the authenticity of the study. It is unlikely that every real case presented to a forensic odontologist will contain the biter and this reduced prevalence is a reflection of this. 5.1.6 Choices of conclusion level One of the changes made to the design of the experiment following the pilot study was the incorporation of a Don't Know option in the forced-decision model. This change was incorporated as several of the pilot examiners felt that they were forced into making decisions that may have been incorrect. The lack of the Don't Know option within the intra-examiner study was valid as the purpose was to compare examiner responses between two separate examinations carried out using the same criteria. The actual conclusions offered, therefore, did not affect this analysis. It is interesting to note that there were no significant differences in the performance of the two groups following the inclusion of the Don't Know option in Study Two. Indeed, performance of the first 122 group, measured by kappa agreement with the gold standard, was slightly higher. This group was not afforded the opportunity of expressing uncertainty in this way. Lack of the Don't Know option within the initial FDM study did, arguably, reflect the nature of the examiners' attitude to bitemark analysis. Those examiners who were more conservative in their approach to bitemark identification would be likely to exclude more suspects while those who were less conservative with the process would be likely to identify a suspect as the biter. The inclusion of the Don't Know option within Study Two enabled this to be seen more clearly. Two examiners (ABFO Examiners 2 and 10) clearly felt that they were unable, or unwilling, to make decisions regarding many of the cases and the number of Don't Know decisions chosen reflected this. There are two obvious explanations for this. The examiners could have felt that the materials supplied to them were insufficient, or their perceived experience in bitemarks may not have been adequate for them to feel confident in rendering verdicts that are more decisive. Questionnaires or structured interviews with these examiners would be required to gauge the reasons behind these decisions more accurately [140]. The conclusion levels used in the ROC analysis were sourced from the ABFO Guidelines for Bitemark Analysis [55]. Five levels were offered to the examiners in this portion of Study Two. The minimum number of conclusion levels to provide sufficient data for a powerful ROC result is also five [126]. The incorporation of a Don't Know option within the conclusions levels is valid and has been used by many other investigators employing this technique [126]. ROC, while a more complex statistical treatment than that used in the FDM, was used to ensure an authentic assessment of the application of bitemark overlays by incorporating the levels of uncertainty used in real casework. 5.1.7 Materials provided to the examiners Several comments were received from the participants of the study in relation to the materials that were provided to them. Concerns were expressed with the use of two 123 photographs and a single overlay to identify a biter. Examiners argued that in order to identify a biter a whole range of additional materials would be required: "... the concept that one picture does not a bitemark make ... [must be realised.]" ABFO Examiner It is true that the ideal bitemark case includes all possible evidence; however, this is not always possible. Indeed, the BOLD lab has received several cases for peer-review that consisted of only one photograph and the models of the suspect's teeth. A recent study that examined the collection of evidence from suspects determined that Diplomates did not always collect all the items recommended by the ABFO [61]. Additionally, this ABFO Examiner has somewhat missed the purpose of the study. The investigation aimed to provide quantitative data for bitemark overlay use, not the whole gamut of evidence types used in bitemark analyses. Had all the materials been included it would have been impossible to: a) control for examiners creating their own overlays or other materials, and b) ascertain which technique enabled or hindered the correct identification of a biter. This was the purpose behind the removal of the wax bites from Study Two, leaving merely the overlays, photographs, and occlusal views of the cast. Further research is required to investigate the role of each of the collected exemplars to determine their worth in the analysis process. Collective results could be pooled, although it is likely that the individual exemplars would have a synergistic effect and increase the overall success of bitemark identification. Speculatively, it may be that certain items of evidence actually confound the bitemark identification process and this would be another area to investigate. This study chose to investigate overlays, as they are the most commonly utilised method of comparing a suspect's teeth to a bite injury [67]. The decision to use computer-generated overlays was based on previous research that determined these to be the most reliable [67]. However, it would be an interesting facet of further research to identify, using a survey, the overlay technique that is actually employed most frequently and utilise this method to see if any differences exist in biter identification effectiveness. 124 Further comment related to some of the inherent problems of overlays follows: "What if one tooth was way out of the plane of occlusion and would not mark. The computer-generated overlays included with the study do not take this into account so even if the general alignment of the teeth fits the bitemark pattern a tooth or teeth that would not mark would basically eliminate the models as a source of the bitemark." ABFO Examiner This argument is actually flawed. When the suspect's dental casts are placed against the scanner's glass platen some teeth come into direct contact and others that are worn or fractured do not contact the glass. When the images are scanned into the computer this difference in distance from the scanner results in a difference of incisal edge brightness. Using the settings described by Sweet, this difference in occlusal height will be represented on the overlay. Figure 41 shows an example of this. Notice that tooth #43 has undergone attrition due to bruxism and is duller in appearance than teeth #31 or #33. However, we cannot claim that overlays are perfect representations of the suspect's teeth. They are not. It has been the purpose of this study to determine how they assist the odontologist. FIGURE 41 Differing occlusal tooth height represented on scanned model The ABFO Examiner quoted above stated that, during his case works, he used a radiographic technique to address this problem. The radiograph produced from the wax bite was then scanned into the computer, inverted and then printed onto transparency film. Interestingly, the difficulty described regarding the occlusal height of teeth may be compounded by the use of wax. Like the twin study, the depth to which the stone cast is 125 pressed into the wax will determine the appearance of the overlay. It is unlikely the odontologists utilising this technique will apply pressure equally to wax on every occasion, nor is it likely that the wax will be of equal softness or thickness on every occasion. The technique is, therefore, rather subjective. 5.1.8 Summary All scientific studies are in some way compromises. In this study, the opportunity to exercise greater control over the appearance of the bites led to a choice of postmortem bites on animal skin. This led to a compromise in the appearance of the bites concerning erythema and other colour details. However, the decision did lead to the production of sound, robust, and common case materials. The decision to include only overlays and scans of the dental casts was justified, as the stated purpose of this study was to determine the use of this technique within the bitemark context. 5.2 FINDINGS Chapter 4 and Appendix D present the raw and treated data that was obtained from the two studies. Following is a discussion of the findings and description of the possible implications upon the use of bitemark overlays in forensic casework. Opportunities for further investigations are described. 5.2.1 Intra-examiner reliability The purpose of Study One was to test the materials and assess, by two measurements, the intra-examiner reliability of bitemark overlays. Two values were used to measure the degree of agreement between the examiners' first and second attempts: kappa and percentage agreement (PA). Before considering these values, it is helpful to examine some kappa and PA values from other dental tests and, thus, enable us to place the values 126 from this study in context. See Table 27. The results from this study have been added to facilitate comparison [141]. TABLE 27 Studies examining intra-examiner agreement of dental diagnostic tests TEST KAPPA % AGREEMENT AUTHOR Periodontics Plaque 0.22 47.5% Clemmer, 1974 Bleeding on probing (BOP) - 64% Janssen, 1986 Lack of BOP - 78% Janssen, 1986 Probing depth - 81.2% Smith, 1970 Dental radiographs Vital/nonvital - 72% Abdel Wahab, 1984 Caries 0.80 - Valachovic, 1986 Periodontal disease 0.79 - Valachovic, 1986 Periapical condition Normal - 81.5% Reit, 1983 Widening ofPDL - 40.2% Reit, 1983 Periapical lucency - 76.2% Reit, 1983 Bitemarks Overlays 0.72 87.2% Current Study After Brunette, 1996 The mean kappa value obtained for the examiners in this study was 0.72, which compares favourably to the data from other studies examining this form of reliability among dental tests. The percent agreement mean was 87.2%, again in line with previous studies from dental research. Unfortunately there are no forensic studies (from any discipline) to compare these results to. The seemingly favourable results for intra-examiner reliability must have several caveats attached. Firstly, there was a relatively large range of results obtained, with the lowest agreement being 0.30 or fair agreement. The highest level obtained was 1.00 or perfect agreement. There is a multitude of reasons for such variation. On a simple note, some examiners could have recalled their answers. Environmental effects, such as the examiners' mood on each day of examination, the time available, and the desire to participate in the study, could all influence the reliability of repeated measurements. 127 Speculatively, it can be stated that some examiners considered the study a test of their abilities and may have afforded it more time than an examiner who merely considered it an interesting exercise. Such variables are difficult to both measure and control. Secondly, the sample size decreased in the second part of Study One. Although 10 examiners completed the first stage of the study, only seven completed the exercise the second time. Two examiners had retained their materials, returning only their answer sheets on the first occasion and so it was decided to exclude them from the second study to prevent bias. The third examiner failed to complete the exercise in time. While only seven participants were involved in the study, this represented 280 bitemark decisions. The values of intra-examiner reliability must be considered with sample size in mind. An increase in examiners would have permitted a clearer view of the trends and potentially reduced the wide standard deviations observed. The issue of participant involvement was discussed earlier in relation to the total number of forensic dentists available for research. This study highlights this problem further. In future studies, exercises may be performed at meetings or other training conferences where a large number of forensic dentists are likely to attend. This will enable an expeditious response to the materials and facilitate the assessment of the likely commitment of the participants to the study. Recruitment of volunteers will represent a significant challenge to researchers in this area. Thirdly, it is possible to say that the ABFO examiners in Study One performed to a level similar to that of their Diplomate colleagues in Study Two. No statistical significance was detected between these two groups in terms of agreement with the gold standard. This is favourable to the intra-examiner reliability scores as it permits a bolder statement regarding the extrapolation of the results to the practising community as a whole. It is possible to say, therefore, that in a simulated bitemark situation involving postmortem injuries, Diplomates of the ABFO achieve a substantial degree of agreement in regard to repeated examinations of the same bitemarks. This level of agreement is in 128 line with those found in other dental disciplines. It is noted that while the average agreement was substantial, variability among the participants was high. The participants within the study performed the task to a level similar to that of other groups of the same experience and training levels. Direct extrapolation of this result to all Diplomates may not be warranted due to the small sample size; however, performance with other groups was similar. Bitemarks are a good example of a measurement that requires observers to make subjective judgements based upon poorly described criteria. The use of intra-examiner agreement is a useful measure of the reliability in such cases [141]. Should a test that does not have a well-described series of stages perform well on repeated examinations, this is a valid method of determining effectiveness. Intra-examiner reliability is an important aspect of bitemark analysis to measure. As described above it is a good indicator of overall reliability of the technique, but it also has real-life relevance. In forensic casework, there is often a large hiatus between the initial examination of the materials and subsequent appearance in Court. It is, of course, usual for witnesses to review materials before testimony and the substantial agreement found within this study indicated that such repeated examinations are likely to be in line with the initial results. 5.2.2 Inter-examiner reliability One of the purposes of Study Two was to compare ratings of suspects in the sample cases between different examiners, and thus determine the inter-examiner agreement. In this study, a cross-wise kappa technique was used to determine the agreement of every examiner with every other examiner within a particular group. As before, examples from clinical dental practise enable us to place the values obtained within context. See Table 28 [141]. 129 TABLE 28 Studies examining inter-examiner agreement of dental diagnostic tests TEST KAPPA % AGREEMENT AUTHOR Periodontics Plaque 0.22 44% Clemmer, 1974 Probing depth 0.26 69% Smith, 1970 Dental radiographs Vital/nonvital - 43% Abdel Wahab, 1984 Caries 0.73 - Valachovic, 1986 Periodontal disease 0.80 - Valachovic, 1986 Periapical condition Normal - 37% Reit, 1983 Widening ofPDL - 9% Reit, 1983 Periapical lucency 0.27 76.2% Reit, 1983 Bitemark overlays ABFO 0.47 - Current Study ASFO 0.44 - Current Study GDP 0.45 - Current Study After Brunette, 1996 The cross-wise kappa comparisons for each of the three examining groups produced means of 0.47 (ABFO), 0.44 (ASFO), and 0.45 (GDP). All three groups had a mean agreement rated as moderate on the Landis scale [115]. Table 27 shows the highly variable nature of inter-examiner agreement. The values obtained in this study are highly consistent with each other. The values for inter-examiner agreement fall within the normal range found in dental diagnostic tests. Dental bitemark overlays performed better in relation to this variable than all but radiographic tests for caries and periodontal disease. Maupome examined inter-examiner reliability when making restorative decisions. He found that the vast majority (91.4%) of the examiners were in fair to moderate agreement (0.21 - 0.60), in line with the results obtained in this study [132]. During the literature search, one study was found that examined inter-examiner reliability in forensic dentistry [113]. Borrman's study compared the decisions of forensic dentists who were asked to age individuals based upon the radiographic appearance of their teeth. The results indicated a very poor level of examiner agreement and the authors concluded 130 that this method of age determination was likely to be too unreliable for general use. This study extrapolated the inter-examiner reliability to the general reliability of the test. Inter-examiner agreement is another measure of the reliability of a particular test. Should a number of different individuals achieve the same result after applying the test on the same materials this confirms the stability of the test. The stability of bitemark overlays when applied by the three groups to simulated forensic cases appears to be moderate as measured by kappa. The result is similar to those obtained by other dental clinical tests. It is important to remember that the inter-examiner reliability data was based upon decisions made in the FDM. Individuals hold different beliefs on the use of bitemarks to identify biters and will set their thresholds at vastly different levels. When forced to make decisions this threshold cannot be expressed using a range of uncertainties and thus some examiners may exclude a large number of suspects or may respond Don't Know. This is evident within the ABFO group of Study Two. Compare ABFO Examiners 1 and 2 from Appendix D. Speculatively, we can say that these examiners have different thresholds for the identifications of biters and thus there is little opportunity for agreement between their decisions. This argument concerns the separation of the examiner and the test, and is discussed later in the Chapter. It is important to note that the performance of the two Diplomate groups (Study One and Study Two) did not significantly alter when the Don't Know option was incorporated into the FDM. One of the findings from the legal review was the polarisation of odontologists' opinions. Several cases are reported in which prosecution experts testified that the questioned injury was a bitemark and that it could be matched to the defendant to the exclusion of all others. Defence experts questioned if the injury was even a bitemark. The inter-examiner results obtained from this study go someway to quantifying these differences in opinion. Clinical dentists are often criticised for their lack of agreement regarding treatment planning. It is common for the media to dispatch a 'patient' to numerous dentists for a treatment assessment and then describe the vast differences in their clinical decisions. These differences are often explained by the personal threshold for dental treatment that clinicians hold. Older dentists may feel that stained fissures should be 131 treated while those trained under ideals that are more conservative may choose to observe the stains over a period. It would appear that such issues are present in bitemark analysis with the individual beliefs and thresholds influencing the agreement between odontologists. It is certainly true that bitemarks represent a very contentious and debated issue within the discipline [107,108]. An interesting addition to the study would have been a questionnaire that examined the participant's attitude to bitemark analysis in general and overlays in particular. Data from this questionnaire could have been used to group examiners into classifications based upon their willingness to identify a biter from available evidence. The results from these discrete groups could then have been compared with kappa and any differences tested statistically. The strength of the inter-examiner result is higher than that of the intra-examiner result. This is due to the increased number of participants. Another comparison could assess the inter-examiner agreement between odontologists determining the identity of individuals from dental records and those from this study. The determination of identity from dental records is well established and does not attract the controversy that surrounds bitemark examinations [85]. 5.2.3 Effectiveness Before discussing the effectiveness of the overlays, it is important to discuss the issue of operator and test separation so that the results from the FDM and ROC analysis can be placed in the correct context. Originally, it was decided to assess the use of overlays in the determination of biters. To this end, materials supplied to examiners were limited to those that permitted the use of overlays only. The performance of individual examiners and their decision-making processes were thought to be separate entities. As described in the previous section this has not been the case and the application of bitemark overlays has proven to be both case and operator sensitive. This is clearly shown by the comparison of ABFO Examiners 1 and 2, where the thought processes of the examiners 132 are reflected in their willingness to indicate suspects as biters. Despite the objectivity of the overlay production technique, the subsequent application of that technique is highly subjective [11]. In tests where subjectivity is high, the separation of operator and test in assessment of performance is impossible. There is always interplay between the two [141]. With this caveat in mind, the discussion of the performance follows. FDM Performance The forced decision model allowed the use of simple statistical analysis. The use of terms such as false positive and true negative are easily understood and hence the power of this model is in its ease of use and explanation of results. There are, however, drawbacks to the model. Firstly, the protocols described by the ABFO recommend the use of particular levels of conclusion that are not replicated in the dichotomous decisions offered by the FDM. There is a speculative argument, however, that these levels of conclusion are simply extrapolated by Courts and jurors to a positive or a negative judgement. Secondly, as explained above, the FDM is especially prone to 'infection' by the personal threshold of the examiner and the integration of test and examiner is great. The study provided 539 decisions from the FDM (600 minus 61 Don't Knows). The data thought most useful from this study were the values of sensitivity, specificity, accuracy, and kappa agreement with the gold standard. No forensic study describing these values was found within the literature. See Table 29. Sensitivity across the three groups of examiners was not significantly different (ANOVA). The novices (GDP) who also had the smallest standard deviation among the groups (GDP>ABFO>ASFO) achieved the highest sensitivity. The mean sensitivity from the three groups was 71.8%. Specificity was not significantly different across the three groups with the expert group (ABFO) achieving the highest score. In no group was there a significant difference between the sensitivity and specificity scores. The mean specificity was 81.5%. These mean values are in line with the sensitivity and specificity from other dental tests. 133 TABLE 29 Examples of sensitivity and specificity from dental diagnostic tests TEST SENSITIVITY % SPECIFICITY % AUTHOR Caries Clinical examination 13 94 Verdonschot, 1992 Fibre-optics 13 99 Verdonschot, 1992 Radiographs 58 66 Verdonschot, 1992 Fissure discoloration 74 45 Verdonschot, 1992 Electrical resistance 96 71 Verdonschot, 1992 Bite-wing radiographs 73 97 Mileman, 1985 Probe and look 58 94 Mileman, 1985 Periodontics Bleeding on probing 29 88 Lange, 1991 PMN gelatinase 79 88 Teng, 1992 Bone loss 91 96 Jeffcoat, 1992 Beta glucuronidase 89 89 Lamster, 1988 Temperature 83 83 Kung, 1990 Gingival redness 27 67 Haffajee, 1983 Plaque 47 65 Haffajee, 1983 Antibody assay 65 80 Hujoel, 1990 Bitemarks Overlays 71.8 81.5 Current Study After Brunette, 1996 134 The relation of these values to forensic dentistry is fundamental to the study and the comparison with dental tests is inappropriate at this stage. The implications of missed caries due to an insensitive radiographic test are quite different to the inclusion of a biter in a criminal case due to a non-specific test. This study has determined that practising forensic dental experts are able to achieve a sensitivity of 73.8% and a specificity of 84.1%. This would indicate that bitemark overlays are more useful at indicating biters than excluding them if they are used in a FDM. However, as described previously no statistical difference between sensitivity and specificity was detected and the FDM does not permit the use of uncertainty when expressing conclusions. The use of percentage agreement (accuracy) and kappa allowed a different perspective on the data obtained. In simple terms, how often were the examiners correct? Percentage agreement is a simple measure of this and the mean across all three groups was 77.9%. Individually, the Diplomates were the most accurate examiners scoring a group mean of 83.2%. The differences between the groups were, however, small and statistically insignificant. It is interesting to note that two of the Diplomates chose Don't Know responses for more than half of the cases, making up over 85% of the total Don't Knows for this group (Examiners 2 and 10). Both of these participants obtained 100% accuracy. This could indicate that they had very high personal thresholds to identify or exclude biters. Mathematically they have skewed the Diplomate's mean accuracy. With these individuals removed the mean accuracy of the group drops to 78.5%. The results indicate that these examiners are unlikely to render opinions in bitemarks presented to them. However, if they were prepared to reach a conclusion it would be highly accurate. Another explanation is that these examiners felt that the information provided to them was insufficient. Since each case was produced in the same manner and contained the same materials this argument would tend to suggest that they would have made no decisions regarding the suspects. Interviews with each of the examiners to elucidate information regarding personal thresholds and opinions on bitemark analysis would be required to confirm these speculations. 135 A more powerful technique for quantifying agreement with a gold standard is the chance corrected kappa value. The mean for all three groups with this value was 0.54 with the Diplomates scoring the highest kappa at 0.58. With Examiners 2 and 10 removed the mean Diplomate kappa drops to 0.54 placing the GDP kappa score of 0.56 as the highest. Including or excluding these examiners the total kappa score across all three groups falls into the 'moderate agreement' category of Landis' rating scale for kappa values [115]. The values that have been discussed above pertain to values that are associated to the test and the operator alone. Prevalence has no impact upon these ratings of performance. However, the positive and negative predictive values are influenced by the prevalence of the disease within the population to which the test is being applied. Previously, the decision to maintain prevalence at 40% was justified. From the individual results of the examiners, it is possible to observe that the prevalence fluctuated around this value. This is due to the inclusion of the Don't Know option. If an examiner placed a Don't Know upon a suspect then this decision was not included in the analysis and the population decreased by one. If the suspect was also one of the biters determined by the gold standard then the prevalence of the disease also decreased by one. Combinations of these situations meant that in certain examiners the prevalence increased (e.g., GDP Examiner 5, PREV 44%) and in others it decreased (e.g., ASFO Examiner 2, PREV 36%). Due to the unstable nature of the prevalence within the groups, it was determined to be invalid to calculate the mean of these figures and report group values. Individual PPV and NPV were calculated and can be found within Appendix D where the appropriate prevalence is indicated. The impact of prevalence on bitemark suspect line-ups was discussed earlier in this Chapter. However, it is possible to report PPV and NPV for the Diplomates involved in the first study. Denied the option of indicating a Don't Know response, the prevalence of these examiners remained constant throughout the group (40%). The mean PPV was 79.0% and mean NPV was 94.3%. No statistical difference was detected between the NPV and PPV (p=0.0233). Despite the lack of significance, the data would suggest that overlays, operating within a prevalence of 40%, were better at 136 excluding biters than including them. These data were obtained with only 140 decisions (7 examiners, 20 suspects) and further investigation is required to determine if this difference is significant. Anecdotally, many individuals have stated that the exculpatory value of bitemarks far outweighs their ability to incriminate [107,108]. The values described may represent the first empirical evidence to support this view. The FDM facilitated analysis of the examiners' use of overlays. It allowed a comparison of the inter- and intra-examiner reliability. The FDM in Study Two was operator-sensitive in that personal threshold levels were applied. This was demonstrated particularly in the Diplomate group. The use of the Don't Know option in Study Two invalidated the predictive values but tentative results were obtained from the prevalence stable Study One. The incorporation of the Don't Know option did not have a significant affect on the performance of the examiners as measured by the FDM. No significance was detected between the three groups of examiners using any of the determined values. This would suggest that training and experience have little effect on the application of overlays to bitemark identifications. However, caution must be applied when making such assumptions. Detailed questionnaires would be required to identify correctly all the variables surrounding the facets of experience and training to substantiate such a claim. It must also be remembered that this study has utilised simulated postmortem bites and examined only the use of one analysis method using limited evidence materials. ROC Analysis The use of ROC enabled a range of conclusions, including Don't Knows, to be incorporated into the analysis. Because this technique enabled the examiners to express their certainty within the established levels of conclusion, the operator sensitivity issues found in the FDM were minimised. ROC analysis provides a means by which the identification of biters using transparent overlays can be distinguished from the judgement of the operator. This separation is achieved by using the rating scale that is equivalent to varying the examiner's personal threshold while holding the properties of 137 the bitemark constant. The ROC analysis provided three main variables: a) the measurement of total performance or the area under the curve, b) the sensitivity and specificity of individual levels of conclusion, and c) Youden's Index that determined the best cut-off point. ROC is a useful technique to compare the accuracy of tests and observers. The AUC provides an objective parameter of the diagnostic accuracy of a test (the ability to determine biters) that is far superior to comparing single combinations of specificity and sensitivity, as the influence of threshold is eliminated [117,122]. The AUC is a combination and generalisation of the concepts of sensitivity and specificity into a single measure of accuracy [119]. In this study, the AUC values from the three groups were very similar with the ASFO having the value closest to 100% (perfect diagnostic test) [124]. Six hundred decisions made up the AUC analysis in total. The mean AUC for the combined groups was 80.7%. It is difficult to place this value into any context. The value of 50% assumes that a test is non-diagnostic. Thus, bitemark overlays are closer to the perfect diagnostic test than a purely random allocation of biters and non-biters. Whittaker's study determined a mean AUC of 63%) for the determination of whether bites were caused by adults or children. Comparing these results with that of the current study we could argue that the use of overlays in determining biters is more effective than the subjective determination of biter age-group [122]. This is not a particularly useful comparison, and serves only to allow a point of reference. Further research into bitemark identification techniques is required to produce a range of AUC values. Such data will enable a comparison of techniques and move the discipline towards a more evidence-based approach. The ease by which AUC can be calculated and compared promises to allow exciting research possibilities in the future [124]. Studies could be carried out using the same base materials as this study (i.e., bitemark photographs) but include different items of recommended evidence. Examples could include just providing wax bites or injury impressions. Following analysis it would be possible to determine the relative impact (using AUC) of each item of evidence on the determination of biters from bitemarks. 138 In addition to the AUC, we are able to determine the relevant combinations of sensitivity and specificity at varying conclusion levels [118,142,143]. The conclusion levels are shown in Table 30 in conjunction with pooled mean results from the three examiner groups. The results in all three examiner groups support the general theory that sensitivity and specificity are reciprocally related [123]. For example, at Conclusion Level 1, specificity was close to 100% while sensitivity was low at 21.3%. However, at the other end of the scale, Level 5, sensitivity was always high (100%) while specificity was always zero. So when the forensic dentists operated at the very strict Level 1 conclusion they achieved low sensitivity with a high specificity. That is, they operated at a point where they accepted that they would miss some biters but ensured that no non-biters would be included. Recall that specificity is a measure of the ability of a test to detect the absence of a disease. High specificity can only be obtained when the occurrence of false positives is low. It follows therefore that when an examiner issues a conclusion at Level 1 of the ABFO scale there is a high probability that the suspect is indeed the biter. TABLE 30 ABFO Conclusion levels with pooled group results LEVEL SENSITIVITY % SPECIFICITY % 1. Reasonable medical certainty 21.3 98.6 2. Probably 57.1 94.2 3. Possible 76.7 64.6 4. Inconclusive 75.4 57.4 5. Excluded 100.0 0.0 We are fortunate to have data to compare to these results. In 1999, the ABFO held a bitemark workshop in Orlando, FL. Diplomates completed four mock bitemark cases and give their conclusions using an expanded version of the ABFO rating scale (seven conclusions). Actual forensic cases were employed. Evidence included photographs of the injury and a set of seven dental casts of suspects. The gold standard was taken to be the decision of the forensic dentist who originally examined the case. The results of this 139 workshop were handled by two separate centres in North America, one of which was the BOLD lab. Both centres chose, independently, to use ROC to analyse the data. The results of this are shown in Table 31. TABLE 31 ROC results from the ABFO workshop RATING SENSITIVITY % SPECIFICITY % 1. Reasonable medical certainty 19.5 99.9 2. Probable 53.1 98.2 3. Possible 80.5 88.7 4. Improbable 84.4 72.3 5. Incompatible 96.1 2.5 6. Inconclusive 99.2 0.8 7. Non-diagnostic 100.0 0.0 Statistical comparisons of these data showed that there was no significant difference between the results obtained from the ABFO workshop and the current study (p=0.273). Visual examination of the results shows the similarities in the two studies. Interestingly the AUC from the workshop results (86%) is very similar to that obtained from the current investigation (80.5%). It should be noted that a relatively small increase in overall accuracy of 5.5% was achieved when Diplomates received more evidence, i.e. study models. Figure 42 illustrates the ROC curve produced by the results shown in Table 30. FIGURE 42 ROC curve from ABFO Workshop data 0.0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.0 1.0 1-Speclflcity 140 Examining the individual sensitivities and specificities at each conclusion level can be confusing. ROC enables us to determine the optimal cut-off point that maximises the sensitivity and specificity. Computer software allows the user to define the relative importance of false-positives and false-negatives, and modify the cut-off point to reflect this threshold importance [131]. Youden's Index is a measure of the value of any one threshold level (combination of sensitivity and specificity) and assumes that there is equal weighting for FP and FN. The closer Youden's Index (YI) is to 1.00 the more effective that level of conclusion. The best cut-off point for the all three groups was determined to be Level 2, although the ASFO group had a YI score of 0.48 for Level 2, 0.47 for Level 3 and 0.46 for Level 4. Differences between YI were more pronounced in the other two groups. It is interesting to consider how much weight should be appropriated to FP or FN. Is it more important to ensure that a biter does not go unidentified at the expense of including non-biters? The legal system dictates that evidence should be looked at in a light most favourable to the witness. The premise of "innocent before proven guilty" would tend to suggest that FP should be minimised, even at the expense of a reciprocal increase in FN. If consensus is achieved, individual examiners will be able to study ROC data of their own decisions and decide their personal cut-off point to maximise FP or FN. Summary The ROC analysis has shown the level of diagnostic discrimination afforded by transparent overlays compared to a random allocation of biter and non-biter to suspects. The technique has enabled, to a large extent, the separation of operator bias from the investigation. It must be stated that while the use of thresholds permit a less tainted view of overlay effectiveness, individual operators are still likely to hold differing views on the application of each threshold level. The study has enabled a cut-off point to be determined (Conclusion Level 2). This cut-off can later be altered to reflect the relative importance of FP or FN results if needed. Forensic dentists must realise that no diagnostic system is perfect, and bitemark analysis is no different. Therefore, they must 141 be encouraged to consider the values and repercussions they associate with errors and modify their decisions upon these. ROC data for individuals will facilitate this. Qualitative research is required in the future to investigate the delineations of forensic dentists' thresholds and their views upon error rate [140]. 5.3 IMPACT ON DISCIPLINE The current methods of analysing human forensic bitemark evidence have been shown to depend upon the subjective assessment of pattern associations. Using teeth represented on transparent overlays, odontologists with varying degrees of skill, aptitude and experience compare suspect's teeth with photographs of bitemark injuries. The ability for human skin to record and hold impression evidence is far from ideal and is prone to distortion due to anatomical location, skin configuration, texture, and foundation. The inability to place the body in a similar position to that during the bite compounds these factors. The subjectivity of the technique is recognised as a significant problem in both the forensic literature and the legal opinions of the judiciary. Forensic odontologists strive for definitive results and wish to express their conclusions with a high degree of moral certainty. Such individuals look to groups such as the ABFO for guidance in this regard. The guidance has been limited. To fill this hiatus of knowledge this study has determined that transparent overlays are useful, valid, accurate, and reliable within the ranges and limitations described. The impact of training and experience had no detectable affect on the effectiveness of overlays to indicate biters. Satisfying the requirements of the judiciary, odontologists, and the increasing interest of the media, this study provides empirical support for the use of computer generated overlays. 142 A first in forensic dentistry, this study has examined the scientific basis for bitemark comparisons. The significance of this study will be realised in Courts both in the U.S. and Commonwealth theatres. While the overall effectiveness of overlays has been established, the variation in individual performance is of concern. This variation is of particular importance to those odontologists testifying in Courts who must be aware of their own values of accuracy and reliability. The implications of experts who find themselves performing poorly with bitemark analysis testifying in Court has very serious implications for the accused, the discipline and society. This work has provided tools by which odontologists can assess their personal abilities and modify their thresholds to ensure that their conclusions are reasonable and stable. Further avenues of investigation have been described and the future for forensic dental research promises to be exciting and revealing. Without this research the discipline is doomed to be added to the junk-science category, joining the polygraph and psychic investigators. 143 CHAPTER SIX CONCLUSIONS 144 6.1 EXPERIMENTAL DESIGN • The use of postmortem, simulated bites to assess the ability of forensic dentists to identify biters is valid. • The distribution of correct and incorrect decisions demonstrates the appropriateness of pigskin as a substrate upon which to place simulated bites. • Prevalence of biters in studies of this kind should be as close to 50% as possible. • Forced decision models enable simple, well-recognised statistics to be applied to the decisions rendered, but are operator sensitive. • The incorporation of Don't Know responses within forced decision models does not influence the performance of examiners. • ROC analysis provides an opportunity to increase the authenticity of decision studies by permitting the incorporation of levels of uncertainty while not imposing false thresholds upon examiners. • The recruitment of study participants in studies examining decision-making in forensic dentistry will be limited by the availability of Diplomates of the American Board of Forensic Dentistry. Future studies should explore methods of facilitating the participation of this essential group of forensic professionals. • The number of cases should be based upon the time requirements to complete the examination and time available to examiners 6.2 BITEMARK OVERLAY RELIABILITY • Both measures of intra-examiner reliability (kappa and percentage agreement) demonstrated a substantial level of agreement between first and second examinations of the case materials. • This result indicates the stability of decisions made by examiners and supports the contention that bitemark overlays are reliable. 145 • The inter-examiner reliability of bitemarks can be regarded as moderate and this may explain the polarisation of bitemark opinions within the field. • Further work is required to establish the nature of forensic dentists' attitudes to bitemarks in order to place this result into context. 6.3 BITEMARK OVERLAY EFFECTIVENESS • Overlay effectiveness, as measured by the forced decision model, was good with high values of specificity and sensitivity obtained. • It can be stated that, in a forced decision situation, with stable prevalence, bitemark overlays may be more effective at excluding than including individuals. Further work is required to substantiate this result. • Using chance corrected statistics, the effectiveness of bitemark overlays can be regarded as moderate, when applied to postmortem, simulated injuries on pigskin. • The AUC values obtained show that bitemark overlays are effective. Further data are required to enable this finding to be placed into context. • The inclusion of additional evidence material (such as in the 1999 ABFO Workshop) to supplement transparent overlays results in only a small increase in performance. • The ABFO conclusion level 2 represents the best cut-off point (i.e., biter if above the threshold and exclusion if below) to maximise specificity and sensitivity, if equal weight is afforded to both false positives and false negatives. • In this study, no significant differences were detected between the three examiner groups, suggesting that training and experience in forensic casework does not effect the success of overlays in determining biters. • The continued use of computer generated overlays in bitemark analysis appears to be justified, although further work is required to investigate the effect of examiner factors. 146 The results of this study are in broad agreement with other investigations of bitemark identification effectiveness. This work has satisfied the requirements of Daubert in relation to determining error rates and other quantifiable features of overlay use. 147 REFERENCES 148 1. Saferstein R, Criminalistics: an introduction to forensic science. 6th ed. 1998, Upper Saddle River; NJ: Prentice-Hall. 2. Whittaker DK and MacDonald DG, A Colour Atlas of Forensic Dentistry. 1989, Ipswich: Wolfe. 3. Brannon RB and Kessler HP, Problems in mass-disaster dental identification: A retrospective review. Journal of Forensic Sciences, 1999. 44(l):123-7. 4. Silverstein HA, Comparison of antemortem and postmortem findings., in Manual of Forensic Odontology, Bowers CM and Bell G, Editors. 1997, Manticore: Ontario. 5. Sweet D, Hildebrand D, and Phillips D, Identification of a skeleton using DNA from teeth and a PAP smear. Journal of Forensic Sciences, 1999. 44(3):630-3. 6. Clark DH, The British experience in mass disaster dental identification. United Kingdom disasters. A historical review. Acta Medical Legal Society, 1990. 40(3):159-65. 7. Sweet D, Identification of stains of human saliva using forensic DNA analysis. Doctoral Thesis. Department of Forensic Medicine. 1995, University of Granada: Spain. 8. Hill IR, Keiser-Nielson S, Vernylen Y, Free E, De Valck E, and Tormans E, Forensic odontology: its scope and history. Privately Published 1984. 9. White EA, Dental identification in the Noronic disaster. Journal of the Canadian Dental Association, 1952. 18(1):3-18. 10. Pretty IA, Talwar S, and Sweet D, The medico-legal importance of dental records. BDALaunchpad, 1999. 6(3):38-45. 11. Strom F, Investigations of bitemarks. Journal of Dental Research, 1963. 42(1):312. 12. Pierce LJ, Strickland D, and Smith ES, The case of Ohio vs. Robinson: an 1870 bitemark case. American Journal Forensic Medicine and Pathology, 1990. ll(6):171-7. 13. Vale GL and Noguchi TT, Anatomical distribution of human bite marks in a series of 67 cases. Journal of Forensic Sciences, 1983. 28(l):61-9. 14. Furness JA, A general review of bite mark evidence. American Journal of Forensic Medicine and Pathology, 1981.2:49-52. 15. Anderson WR and Hudson RP, Self-inflicted bite marks in battered child syndrome. Forensic Science, 1976. 7(l):71-4. 149 16. Warnick AJ, Biedrzycki L, and Russanow G, Not all bites are associated with abuse, sexual activities, or homicides: a case study of a self-inflicted bite mark. 1987. Journal of Forensic Sciences, 1981. 3(4):12-6. 17. Vale GL, Dentistry, bite marks and the investigation of crime. Journal of the Californian Dental Association, 1996. 25(5):29-34. 18. Layton JJ, Identification from a bite mark in cheese. Journal of Forensic Science Society, 1966. 6(2):76-80. 19. Solheim T and Leidal TI, Scanning electron microscopy in the investigation of bite marks in foodstuffs. Forensic Science, 1975. 6(3):205-15. 20. Webster G, A suggested classification of bite marks in foodstuffs in forensic dental analysis. Forensic Science International, 1982. 20(l):45-52. 21. Corbett ME and Spence D, A forensic investigation of teeth marks in soap. British Dental Journal, 1984. 157(8):270-1. 22. Ligthelm AJ and van Niekerk PJ, Comparative review of bitemark cases from Pretoria, South Africa. Journal of Forensic Odontostomatology, 1994. 12(2):23-9. 23. Sweet D and Hildebrand DP, Saliva from cheese bite yields DNA profile of burglar: a case report. International Journal of Legal Medicine, 1999. 112(3):201-3. 24. Sweet D, Human Bitemarks: Examination, recovery, and analysis., in Manual of Forensic Odontology, Bowers CM and Bell G., Editors. 1995, Manticore: Ontario. 25. Pretty IA and Sweet D, Anatomical location of bitemarks and associated findings in 101 cases from the United States. Journal of Forensic Sciences, 1999. 45(4):In Press. 26. Sperry K and Campbell H, An elliptical incised wound of the breast misinterpreted as a bite injury. Journal Forensic Sciences, 1990. 35(2): 1226-35. 27. Grey TC, Defibrillator injury suggesting bitemark. American Journal of Forensic Medicine and Pathology, 1989. 10:144-5. 28. Clift A and Lamont CM, Saliva in forensic odontology. Journal of the Forensic Science Society, 1974. 14(3):241-5. 29. Sweet D, et al, An improved method to recover saliva from human skin: The double swab technique. Journal of Forensic Sciences, 1997. 42(2):320-2. 30. Glass RT, Jordan FB and Andrews EE, Multiple animal bite wounds: a case report. Journal of Forensic Sciences, 1975. 20(2):305-14. 150 31. Rollins CE and Spencer DE, A fatality and the American mountain lion: bite mark analysis and profile of the offending lion. Journal of Forensic Sciences, 1995. 40(3):486-9. 32. Atkinson SA, A qualitative and quantitative survey of forensic odontologists in England and Wales, 1994. Medicine, Science and the Zaw, 1998. 38(1):34-41. 33. Harvey W, Dental identification and forensic odontology. 1976, London: Henry Kimpton. 34. Irons F, Steuterman MC and Brinkhous W, Two bitemarks on assailant. Primary link to homicide conviction. American Journal of Forensic Medicine and Pathology, 1983. 4(2):177-80. 35. Glass RT, Andrews EE and Jones JD, Bitemark evidence: a case report using accepted and new techniques. Journal of Forensic Sciences, 1980. 25(3):638-45. 36. LeRoy HA and Sweet D, Take a bite out of crime - Ask a forensic odontologist for assistance. Royal Canadian Mounted Police Gazette, 1993. 55(3): 1-8. 37. Bonte W, Aspects of identification of tool marks in stab injuries. Archive Kriminologie, 1972. 149(3):77-96. 38. West MH and Barsley RE, Uniqueness of wound patterns. ASFO Newsletter, 1990. 4: 6-7. 39. Nickell J and Fischer JF, Crime Science: Methods of forensic detection. 1998, New York: Kimpton. 40. Davis RJ, An intelligence approach to footwear marks and toolmarks. Journal of the Forensic Science Society, 1981. 21(3):183-93. 41. DeForest PR, Gaensslen RE and Lee HC, Forensic Science - An introduction to criminalistics. 1983. New York: Prentice. 42. Davis JE, An introduction to tool marks, firearms, and the striagraph. 1958, Springfield: Charles C. Thomas. 43. Bonte W, Tool marks in bones and cartilage. Journal of Forensic Sciences, 1975. 20(2):315-25. 44. Rawson RD, et al, Statistical evidence for the individuality of the human dentition. Journal of Forensic Sciences, 1984. 29(l):245-53. 45. Wang B, et al, Measurement of ABH blood group substances in human saliva by immunoassay using artificial antigens as standard substances. Japanese Journal of Legal Medicine, 1996. 50(2):43-9. 151 46. Sweet D and Shutler GG, Analysis of salivary DNA evidence from a bitemark on a body submerged in water. Journal of Forensic Sciences, 2000. 44(5): 1069-72 47. Schwartz TR, et al, Characterization of deoxyribonucleic acid (DNA) obtained from teeth subjected to various environmental conditions. Journal of Forensic Sciences, 1991. 36(4):979-90. 48. McNally L, et al, Evaluation of deoxyribonucleic acid (DNA) isolated from human bloodstains exposed to ultraviolet light, heat, humidity, and soil contamination. Journal of Forensic Sciences, 1989. 34(5): 1059-69. 49. Toledano T, et al, An assessment of DNA contamination risks in New York City Medical Examiner facilities. Journal of Forensic Sciences, 1997. 42(4):721-4. 50. Turvey B, Criminal Profiling; An Introduction to Behavioural Evidence Analysis. 1999, London: Academic Press. 51. Walter RA, An examination of the psychological aspects of bite marks. American Journal of Forensic Medicine and Pathology, 1984. 5(l):25-9. 52. Walter RA, Anger biting - the hidden impulse. American Journal of Forensic Medicine and Pathology, 1985. 6(3):219-21. 53. Webb DA, Pretty IA and Sweet D, Psychological aspects of bitemarks. Human Aggression and Violent Behaviour, 2000. In Review. 54. Fromm M. Difficulties of asking people what their constructs are. in Inaugural conference of the EPCA. 1992. York, England. 55. American Board of Forensic Odontology, Inc, Guidelines for bitemark analysis. Journal of the American Dental Association, 1986. 112:383-6. 56. Hyzer WG and Krauss T, The bitemark standard reference scale - ABFO No. 2. Journal of Forensic Sciences, 1988. 33(2):498-506. 57. Dorion RBJ, Preservation and fixation of skin for ulterior scientific evaluation and courtroom presentation. Journal of the Canadian Dental Association, 1984. 50(2): 129-30. 58. Sweet D and Bastien RB, Use of an ABS plastic ring as a matrix in the recovery of bitemark evidence. Journal of Forensic Sciences, 1991. 36(5):1565-71. 59. Dorion RBJ, Transillumination in bitemark evidence. Journal of Forensic Sciences, 1987. 32(3):690-7. 60. Sweet D and Pretty I A, Human bitemarks - a review. British Dental Journal, 2000. Submitted. 152 61. Pretty IA and Sweet D, The adherence of forensic odontologists to the ABFO guidelines for suspect evidence collection. Journal of Forensic Sciences, 2000. Unpublished Data. 62. Fearnhead RW, Facilities for forensic odontology. Medicine Science and the Law, 1960. 1:273-7. 63. Nam AS and Dykes E, The use of a digital imaging technique to aid bitemark analysis. Science & Justice, 1996. 36(l):47-50. 64. Sweet D, Parhar M and Wood RE, Computer-based production of bitemark comparison overlays. Journal of Forensic Sciences, 1998. 43(5): 1050-5. 65. Dailey JC, A practical technique for the fabrication of transparent bitemark overlays. Journal of Forensic Sciences, 1991. 36(2):565-70. 66. Cottone JA and Standish SM, Outline of forensic dentistry. 1982, Chicago: Yearbook Medical Publishers. 67. Sweet D and Bowers CM, Accuracy of bite mark overlays: a comparison of five common methods to produce exemplars from a suspect's dentition. Journal of Forensic Sciences, 1998. 43(2):362-7. 68. Luntz L and Luntz P, Handbook for dental identification. 1973, Philadelphia: Lippincott. 69. Sweet D, Radiographic techniques in bite mark analysis, in Forensic Radiology, Brogdon GB, Editor. 1998, Boca Ranton: CRC Press 70. Rothwell B, Bitemarks in forensic dentistry: a review of legal and scientific issues. Journal of the American Dental Association, 1995. 126(2):223-32. 71. McCabe JF, Applied dental materials. 7th ed. 1995, London: Blackwell Scientific. 72. Atsu SS, Gokdemir K, Kedici PS, and Ikyaz YY, Bitemarks in forensic odontology. Journal of Forensic Odontostomatology. 1998. 16(2):30-4. 73. Stoddart TJ, Bitemarks in perishable substances. A method of producing permanent models. British Dental Journal, 1973. 135(6):85-7. 74. Marshall W, Bitemarks in apples - forensic aspects. Criminology 1974. 9(32):21-34. 75. Simon A, Jordan H and Pforte K, Successful identification of a bitemark in a sandwich. International Journal of Forensic Dentistry, 1974. 2:17-22. 76. Kerr NW, Apple bitemark identification of a suspect. International Journal of Forensic Dentistry, 1977. 4:20-23. 153 77. Sopher I, Forensic Dentistry. 1976, Springfield, Illinois: Charles C Thomas. 78. Breathnach AS, The Herman Beerman Lecture: Embryology of human skin, a review of ultrastructural studies. Journal of Investigative Dermatology, 1971. 57(3):133-73. 79. DeVore DT, Bitemarks for identification - A preliminary report. Medicine, Science and the Law, 1971. ll(3):144-5. 80. Barbanel JC and Evans JH, Bitemarks in skin - mechanical factors. Journal of Forensic Sciences, 1974. 14(3):235-8. 81. Rawson RD and Brooks S, Classification of human breast morphology important to bitemark investigation. American Journal of Forensic Medicine and Pathology, 1984. 5(l):19-24. 82. Krauss TC, Photographic techniques of concern in metric bitemark analysis. Journal of Forensic Sciences, 1984. 29(2):633-8. 83. Hale A, The admissibility of bite mark evidence. Southern Californian Law Review, 1978. 51(3):309-34. 84. Sweet D and DiZinno JA, Personal identification through dental evidence - Tooth fragments to DNA. California Dental Association Journal, 1996. 24(5):35-42. 85. Pretty IA and Sweet D, The determination of human identity from teeth. British Dental Journal, 2000. In Review. 86. MacFarlane TW and Sutherland DA, Statistical problems in human identification. Journal of the Forensic Science Society, 1974. 14(2):47-52. 87. Sognnaes RF, Rawson RD, Gratt BM and Nguyen NB, Computer comparison of bitemark patterns in identical twins. Journal of the American Dental Association, 1982. 105(3):449-51. 88. Rawson RD, Kinard G, Johnson J and Yfantis A, Statistical evidence for the individuality of the human dentition. Journal of Forensic Sciences, 1984. 29(l):245-53. 89. Ligthelm C and van Niekerk PJ, The identification of bitemarks using the reflex microscope. Journal of Forensic Odontostomatology, 1987. 5(1): 1-8. 90. Rao VJ, Dusting and lifting the bite print: a new technique. Journal of Forensic Sciences, 1984. 29(l):355-8. 91. Elliot TR, Rodgers AH, Haverkamp JR and Groothuis D, Analytical pyrolysis of streptococci on human skin as an aid to identification in bitemark investigation. Journal of Forensic Odontostomatology, 1984. 4(2): 12-7 154 92. Nambiar P, Bridges TE and Brown KA, Quantitative forensic evaluation of bitemark with the aid of a shape analysis computer program: Part 2; "SOP" and bitemarks in skin and foodstuffs. Journal of Forensic Odontostomatology, 1995. 13(2): p. 26-32. 93. Nambiar P, Bridges TE and Brown KA, Quantitative forensic evaluation of bitemark with the aid of a shape analysis computer program: Part 1; The development of "SOP" and the similarity index. Journal of Forensic Odontostomatology, 1995. 13(2): p. 18-25. 94. Rawson RD, Vale GL, Sperber ND, Herschaft EE and Yfantis A, Reliability of the scoring system of the ABFO for human bitemarks. Journal of Forensic Sciences, 1986. 31(4): 1235-60. 95. Nam AS, Digital image cross-correlation technique for bitemark investigations. Science & Justice, 1997. 37(l):251-8. 96. Zarkowski P, Bitemark evidence: its worth in the eyes of the expert. Journal of Law and Ethics in Dentistry, 1988. l(l):47-57. 97. Lexis-Nexis, On line database, www.lexis.com, 98. Bohan T and Heels EJ, The case against Daubert: The new scientific "Standard" and the standards of several States. Journal of Forensic Sciences, 1995. 40(6): 1030-44. 99. Faigman DL, Kaye DH, Saks MJ and Sanders J, Modem Scientific Evidence: The Law and Science of Expert Testimony. The Legal Standard for The Admissibility of Scientific Evidence. 1997, St. Paul, Minn.: West Publishing Co. 100. Meaney JR, From Frye to Daubert: Is a pattern unfolding?. Jurimetrics, 1995. 35:91. 101. Wagner GN and Williams LD, Issues Regarding Scientific Testing, in Forensic Dentistry, Mertz CA, Stimson PG, Editors. 1997, New York: CRC Press. 102. Vale GL, et al, Unusual three-dimensional bite mark evidence in a homicide case. Journal of Forensic Sciences, 1976. 21(3):642-52. 103. Woolridge ED, Glossary of Legal Terminology. Dental Clinics Of North America, 1977. 21(1): 181-93. 104. Meckler M, The dentist as an expert witness. Dental Clinics Of North America, 1982. 26(2):383-8. 105. Hansen M, Out of the blue. American Bar Association Journal, 1996. 80:50-5. 155 106. Sweet D, Career paths in forensic science for dentists. American Society of Forensic Odontology News, 1999. 18(1):1, 18. 107. Bowers CM, A statement why court opinions on bitemark analysis should be limited. American Board of Forensic Odontology News, 1996. 4:5. 108. Clement J, Guest Editorial. American Society of Forensic Odontology News, 2000. 18(3):3,12. 109. Glaser AN, High-yield bio-statistics. 1995, Baltimore: Williams and Wilkins. 110. Yerushalmy J, Reliability of chest radiography in the diagnosis of pulmonary lesions. American Journal of Surgery, 1955. 89:231-40. 111. Everitt BS, Statistical methods for medical investigators. 1989, London: Edward Arnold. 112. Eaton KA, Rimini FM, Zak E, Brookman D and Newman HN, The achievement and maintenance of inter-examiner consistency in the assessment of plaque and gingivitis during a multi-centre study based in general dental practices. Journal of Clinical Periodontology, 1997. 24(3): 183-8. 113. Borrman H, Solheim T, Magnusson B, Kvaal SI and Stene-Johansen W, Inter-examiner variation in the assessment of age-related factors in teeth. International Journal of Legal Medicine, 1995. 107(4):183-6. 114. Dunn G and Everitt B, Clinical bio-statistics - an introduction to evidence-based medicine. 1995, London: Edward Arnold. 115. Landis JR and Koch G, The measurement of observer agreement for categorical data. Biometrics, 1977. 33:159-74. 116. Smith AF, Diagnostic value of serum-creatine-kinase in coronary-care unit. Lancet, 1967. 2(7508): 178-82. 117. van Erkel AR, Receiver operator characteristic (ROC) analysis: Basic principles and applications in radiology. European Journal of Radiology, 1998. 27:88-94. 118. Metz CE, ROC methodology in radiological imaging. Investigative Radiology, 1986.21:720-33. 119. Swets JA and Pickett RM, Evaluation of diagnostic systems: methods from signal detection theory. 1982, New York: Academic Press. 120. Begg CB, Advances is statistical methodology for diagnostic medicine in the 1980's. Statistics in Medicine, 1991. 10:1887-95. 156 121. Goddard MJ, Receiver operator characteristic (ROC) curves and non-normal data: an empirical study. Statistics in Medicine, 1990. 9:325-37. 122. Hanley JA and McNeil BJ, The meaning and use of the area under a Receiver Operating Curve (ROC). Radiology, 1982. 143:29-39. 123. Kay EJ and Knill-Jones R, Variation in restorative treatment decisions: application of ROC analysis. Community Dentistry and Oral Epidemiology, 1992. 20:113-7. 124. Hanley JA and McNeil BJ, A method of comparing receiver operating characteristic curves derived from the same cases. Radiology, 1983. 148:839-43. 125. Metz CE, Statistical significance tests for binomial ROC curves. Journal of Mathematical Psychology, 1980. 22:218-43. 126. Swaving M, Ottes FP and Steerneman T, Statistical comparison of ROC curves from multiple readers. Medical Decision Making, 1996. 16:143-52. 127. Whittaker DK and Evans L, A comparison of the ability of experts and non experts to differentiate between adult and child human bitemarks using receiver operating characteristics (ROC) analysis. Forensic Science International, 1998. 92:11-20. 128. Zhang Z and Monteiro-Riviere NA, Comparison of integrins in human skin, pig skin, and perfused skin: an in vitro skin toxicology model. Journal of Applied Toxicology, 1997. 17(4):247-53. 129. Whittaker DK, Some laboratory studies on the accuracy of bitemark comparisons. International Dental Journal, 1975. 25(3):166-71. 130. Ligthelm AJ and de Wet FA, Registration of bitemarks: a preliminary report. Journal of Forensic Odontostomatology, 1983. l(l):19-26. 131. Gahlinger PM and Abramson J, Computer Programs for Epidemiologists, 1998, London: Brixton Books and Software 132. Maupome G, A comparison of senior dental students and normative standards with regard to caries assessment and treatment decisions to restore occlusal surfaces of permanent teeth. Journal of Prosthetic Dentistry, 1998. 79(5):596-603. 133. Verdonschot EH, Wenzel A and Bronkorst D, Applicability of receiver operating characteristic (ROC) analysis on discrete caries depth ratings. Community Dentistry and Oral Epidemiology, 1993. 21:269-72. 157 134. Chapman DM, Using Receiver Operating Characteristics (ROC) analysis to establish the previous experience threshold for critical procedure competency. Academic Medicine, 1996. 71(10):S7-9. 135. Steinbach WR and Richter K, Multiple classification and receiver operating characteristic (ROC) analysis. Medical Decision Making, 1987. 7:234-7. 136. Getty DJ, Pickett R, D'Orsi CJ, and Swets JA., Enhanced interpretation of diagnostic images. Investigative Radiology, 1988. 24:234-45. 137. Rawson RD, Bell A, Kinard BS and Kinard JG, Radiographic interpretation of contrast-media-enhanced bitemarks. Journal of Forensic Sciences, 1979. 24(4):898-901. 138. Greene HL, Johnson W and Maricic MJ, Decision making in medicine. 1993, New York: Mosby. 139. Maupome G, An analysis of clinical decision-making in restorative dentistry, Doctoral Thesis. Department of Community Dental Health. 1991, The London Hospital Medical College: London. 140. Boyatzis R, Transforming qualitative information. 1998, London: Sage. 141. Brunette D, Critical thinking. 1996, London: Quintessence Books. 142. Metz CE, Some practical issues of experimental design and data analysis in radiological ROC studies. Investigative Radiology, 1989. 24:234-45. 143. Rockette HE, Gur D and Metz CE, The use of continuous and discrete judgements in receiver operating characteristic studies of diagnostic imaging techniques. Investigative Radiology 1992. 27:169-72. 158 APPENDIX A BITEMARK LITERATURE REVIEW 159 APPENDIX A This appendix contains a Table (Table Al) that outlines the papers identified during the literature search. The review of the literature contained within Chapter concentrated on those articles that addressed the most contentious issues within bitemark science. In this appendix articles that pertain to bitemarks evidence of any type are presented. A brief synopsis and the citation value for the paper is given. A statement of the papers' current importance is given, where appropriate. The papers described in the Table were identified using MedLine and the search criteria "BITEMARK", OR "BITE MARK". From the returned list of articles a further search using the criteria "FORENSIC" was applied to produce the final list. Searches were restricted to the English language. A key can be found following Table Al that defines the abbreviations used within the text. 160 td H o z "9 OS ^ ft. o z > OS lis »«S c- &. H < w ~ (a ^ OS Bi o X ill II •S 3 ogi i B 8 8 =3 f o •s B ^ H o 2 -5 M -5 5 — B | 1 S|5 I 84 «-< W > r-8 1-5 3 •5 Q S e : Q O s O 53 ^ a. s a C —3 aj 8 g£ u c •Mi 8§ Si CO .0 Jj a J2 .s •a u e E o u u • s JS .0 <Ji 1% "O to 4> "~ 5 ts 3 o = • E •c •* pa — 0 J= M 1 •§ .1 '5 a O 1-. « 00 u ra jl "S "O .5 S £ 2 .S g a 2 c 8.* a a. u. > S2 W s 5 s •a c "3 § O W ° u u d 6 & 5 |3g u ^ O o Vi . <" b o <* 3 -K 2 U &Ji ^ 3 ra ^ £.i-g a. W O o S « ro tu S5 •— J H 3 u xi ."8 1 o 73 El , U - — !"5 § T3 C s u a CO — ra = ° § —1 tu -o ,_; a. , u a « I a*! C 4> 3 DO u cjo vn 0 ^ u co 1 8 2 —> C/2 <S J 5 3'% X PQ tt. J on 813 eH '0 c « fi^ S S " tl e *3 o c P e3 3 » O M c ^ a P « a. a 5 ^ * 1 ^ IS 0 I* 8 51 •g.s .•fa . "op E S Jj a sf'i CS u vo o 00 J? -a 2 a ?o c -5 S i§3 S "5 S-o .ts b S Jj = « e S cu t a O e< « cj •a -a II f 1 tt. o —. Vi sO •a u o *^ , g o S O c M O «2 ° 3 £ g > o2 . u W 3 1*2 J 2 -E H 9 cu BTJMXI OS O 'I? I E I? T3 Tf 22 O ^ 5 ai & 1 u ° 60 E * c * s o 'i o 5 s <= -3 5 JS c 'S> » S EJ'i s s p s §".2 Bb-g « o 51 p e 2 a s o p ° E •2 £ S 8 8-o Q t .S •§> > ro u u J CO S § P ro * If, lis. — -p ^ 2 5P I <§ a a* 2 s s 8 JJ c S o g C . .TS u ro G •§ < J 42 O. 5 S .3 J 00 i cu 1 S£ J 3 K c 3 W5 W 'ofl-gu a If-s a .5-g.fi S s, c u £ . « 11' "5 "5 ^ wa , 11a g. o H S -E •o c J2» S J3 "S 3 o 3 "j - > E g 2 oo o • 2 f. .S £ -s ti o •si E .1 CQ o. • - 3 2 = •c « " I •8 .§ u •3. 3 Q. .2" S" 2 c B O 1 g , 00 «J •S g 3^ -a " 2 "i-a 1. ^3 -a ,<2g g rs ^ s 1 c 0 ti H < 3 .2 o e o o. u "S -° .S e o o <U ON Q ^ 00 — S3 O Jn •a § Q tu c — »> -o o .2 2 Z JJ o, •§ op s in 3 ill .a -o s 5, o O- • s % E 83 ro O is JO c 'C 3 ^ fc JJ r§ •° _ P E S ou o S 2 .11 C/5 O •a £ Q •E "S •a ~ s g II S ^ CA O 1- iJ g.J= oJ E <s £ g O <U 5 4> o 0 S 3 8-8 5 I1 12 C/3 V!^ a* —^ 22 o s o W Is s CN s tO C s « a a on o <U c u V- .3 co t. s g < ^ fe Q •a « c <u .-^ c "Si *« o 5 = £ -^- s o 5 • * o ! B !•& , 3 ! rS a ID S Z U " c u u XI c C "O u 0 s J3 -a t> 3 3 » 0\ O y £ •o u-i a to" 1 = 1^ '5 »f c S » « .3 CM CQ. <U *0 <y X> « 60 3 u J 19 3 Xi -O X> 60 ; 3 S3 fe S -° s.2 a-1 g 3 -S 2 § ' ^ ii U U — tS JJ u . ... O W a xi te i s & a c « s 8 .§ .s •§ I .1 <L> O IA' 'C ii o = u 3 >>.!2 i—] CO 3 ^ 2s o 3 a % 3 a a E u u eg O (4_, u ° a. a o o u u to o. a a 60 O a 'a 3 l« g §• 'C to _ Si? is cd bA u ~™ u n : o o. u .a . > a •o u's; fe «.i 8 g •8 a 2 & 3 -S o-.E — O D 6 cd <*-> e ^ Q e3 X> is oi is ! • I . i S5 .2 T3 in H g x> -3 -a Cjj§ 3 Q. S u I I •a eo .2 c S '•3 J .2 U 3 E ~ u 3 •S S « cd R 3 O °cB .§1 §-E u ti n ^ S •-»> S ^ a -s a 1 Q o. 3 5 i? &. cd CN E <3 fc> s •S " .2, S •2 "! -E* 3 o 3 a u td .5 t» .S -o ° 11 SJ a o u o on OJ cj o » u u o SPo. * S-S Em B> O b HU — o O o « -a 6 60 H <3 •o £ 3 a o .2 S -2 *^ Ir! S3 ro o .3 tM u O Xj I'S cd X) to tfl U M a a S E Q xi I u c E „ - 'C cd • C vt 3 u o !3 2 .S £ OJ fe b x; to o *3 ej xj tw -a tt ° -a < SS • to _ to .2 -5 <u a p- -tt (j . S .S . fi 3 is If to U c/1 c 3 « E t3 3 2 <u « > 6 6 JJ •as* — ^ I E cd O 5 I XI " o > H 2 * ° 31 2 ro 8 o -a g'M E •§ tU O = •1 8 .§ •a o 5 c .1 &'i xj E op H xi « j* O > po.S t^ *o '51^ o a 3 E .2 3 X 15 s i u o u II 11 I J on 12 •s 3 .2 2. 3 cy to £ Q oo on CO ^ C CN 0 c Ov O CN on <N 5 on <N >-, on B. OO i-> on -d-o °f O CN ba S c^ 3 3 u E ^ ,p 3 £ o w < ^ E E g b it u, tt. CJ ^ o <g I D J o •-> § CO •ii ca w 0-§ 60 _ S o 3 Q C3 « O S Q ^ ^ s ^ oi OS c ll fl .5 00 lit O CJ ?5 3 • Sa >, S3 if c L. 3 u S « O- 3 S r-11* _ D. 2 1-1. ^ * g > -g £ tn S <fc - x a -a IS S « s •5 s •o 2 u ™ JI: CJ VI vi a » a E « .S2 M fl I -S a .S E "SJ 6J1'- -H £ U 3 c t ^ ~ S " C < O rj JS o u . 2 o •*=> u S H 3 a a 1 £ JS 2 c O. ~ O- CJ Q.<5> g T3 . cd O D. CJ CJ C/> £ M g op o .2 £ 3 j= S .3 a o -a S o -1 8 . J W w 12 U E .£ £ -2 .S S .SP r^W tn S > SJ E £ .5 (cj M C l-S g _ -a -S2 <2 3 *0 B 00 CJ E SJ M o .ESS I -5 u SP <5 £ - o c o ~ to — CJ •O 3 i3 .3 vi ji c cj 3 ra 3 g § « Ul fl) ¥ 3 " ~ > cn g — CL. C "* c © O cu 1^ 2* if a 111 o JJ U1J5.2 cj eo .5 — J= fl K g -tt E ra ra a cj .E CO t! — " a a a « = o o E g til 3 A -5 W r*. 00 •2 c i o w -o •a a =2 .3 2 Jo 2 s3^ T I Sr S3 "8=8 ?! a 5 "8 ,5 S w .2 E «"> B 73 3 c8 -3 * I S 'E1 E B ^ —„ 00 S3 5 vi -C .ti ra cj S fl E°.l/S * •E T3 £ a I g 1 O o £ > 3 •S 3 II ra -a 00 5 °P-o .S u <-> E CJ r3 CJ 3 E V) O JJ *° ^ *a S-* O fl 'B »-ESC a E. a ^•s a OUr tn fl w .SJ g. B I S| a > tS 73 » 'Sil cj > -a Vt Vi 3 ^ -a u E A •S CO £ S S 3 ""i3 8 S §S a •d c 3 •= 2 o. O o u 2 o, O U U 111 S ^1 IE^T s 1 .2 > cn 'S , a g-^ 2 u a 1 '£ CQ fl I* jg E o §-.s tn §1 E I E j3 § 0 U B u .2 00 Is B 00 3 3 B O CJ Vi *-i cn o CD o l-s o o •§ i? u S E. 3 E w cn ra o .2 u c/i E ^ Si cn W cn 1 -a 1 B fl 8-S .s J2 e ^ >> e •c B vi JS E •S o O -J3 o ra £ fc 0 8 cn _Q •^o "a «i •I I & & 00 g. s.s <?• E • u< O a ta g u 'S .a BO CJ O..E g 'C G O 0 S 22 •° E •2l B w , CJ cn Odontol d; 34(1):1 rensn r^-<N Odontol d; 34(1):1 F01 rn Odontol d; 34(1):1 IntJ Dent Acta Scan 11 at e o Um N ^ O m 11 „ =•= E ^J O 00 XX o CJ vi •a -a •a -p E .3 o ra Q J J O 2 Q cj ens ton o CO UOI t: icat COU ipp c me i que; o to CO chni cnbi tal te Des den ?3 1 vo .2 •* CO c 2 2;. p u. ^ c °J £1 Q 3 c c cj •|<* JJ rn p rn 3 : ej i Q a o 3 -S w w tu" s| a > W C ed o <U § I CD .*i Q CQ o o .2 -55 •a g *o ci | E E eg CJ :t2 • S "3 cj SU u co 3 o B S3 =» 8 >\ ts E CJ s u w 853 *3 ^ * o o OS O w u O CJ J E-•a S & 'i •9.8 tu rt 1-s _ CJ ca cj 3 3 CO CJ 3 -a 3 D CJ .2 CN CO Tt 3 VO w o a w, CN CJ CO O 5 .5 S3 •° J3 3 2 CJ 3 , CJ co $5 5 "° S CJ s 5 -a , CJ CJ a cj tj JO D & CO * 3 ca ed S2 ^§ a E o 3 E T3 CJ 2 "3 2 o "w 2 0 o. X M S .2 o 1 & 3 cj w ca liJJ Cw '•5 ts o O-J op •o — Is OS o CJ CO . CJ *S ^cS • "O CJ 3 co cj |3 8 [III B s a 5 .2 £ :» isl 3 a •s 8 I? i CJ E O h3 2 § I co w a S 3 CO CO CJ CO Q 13 s _r (U 3 3 al 2? fl Ii li tU CD ° E E O 3 3 O H H Sis « CO a fe CJ S3 J= a H B. I.E CO CJ tS CO O 3 O. Q s 3 W O C/5 cj -£2 co sas O ci " E o CJ IB cm 3 JD 1 ^ 5 Su Bis o CO 3 a CJ u CJ a a 3- & 3 m cu — P Tt 3 o a <n ,° •* 3 *i cj £ a Ji 3 1 ° co .2 ° cj iS "8 -g a 'i 111 s 5 E E > a • o .5 8 § | a § 11 u o CA to C tU t/3 *S CU (2 ^ o u 3 C U > 3 g-s E > s 12 cu c3 •a g •a CJ CJ o cj co SJ E tj BO CJ a b-S co O. tu ' CJ op Sd 3 ' I si < -3 ^ S c •a e 1 a ca IS-8' •3 ^ — cu " o o S S = OS Ov CJ Tt /-V fe 3 <N 2> 1/3 Tl °^ 3 r? < Tt < Is CQ OS OS cj cj ?~> O. S3 »1 e3 <u O w ll « 8 •S tw 3 O cj to J3 3 5 O •* Xi .S o <- 2 CJ Tt co1* Sj o. > 60 ° 3 to •a « .1 B tj J3 •3 o " o 1"° 3 co sj 60 CO | 1 •S- 3 o ca I'S -o - CJ o E ' J° 6o a •5 'C w S •il ls-« w > •O CO l> o- > 2 w2u cj u. So E .a u 12 . C tw to CJ o w " • u 'o» 3 co e 3 CN . - o i " wn l»5 CN to Ov 3 Tt w CN M * § 8 S3 S •a EA 2 u CJ JB to XJ 3 r u . to to ^ u 2 3 ?5u^ UJ H C to € E g-l .3 tu •li &3 & 1 O CO 13 "8 OS E CJ O Vi oi co Ov 3 60 s £ tw ij' w, CN O^ "J a ^ 3 T3 co *o3 on OS CD ca CU ^ 4> < -3 S "11 60.2 ° a ' .2 a •s & s 2 O ft. . a-a' DO U I £ E 2 a £ * § t: 8 a> „_ _ oj •=> ' S3 g CJ E U 2 n c 3 7 to CO •-- i •a oo eg o •g -a a a E B-C 4> g l*Q cn « S S E -S I S 6. ODTJ fa o c 05 a C >> " oi .2 E •§» s c o.S O .CJ -«-» " ta o « . •= j: .ti < -O O.JS Z O a! U O _' 1/1 5t > 5 c r to <*> ; s ^ 1? o < 1 .— cn <M C O S CN 1— O 0 a w -r-r2' 3 < JS cj "g CN : cu — ! 2 CN •c s JJ TJ J3 1 § ? : i£<2 ! S &a 18 2 ! . 22 i a «'E i ; u O ' *•% % . J B OJ . « 3 O..S c3 to E c a £ 09 o. <3SS u h w 28.S CO "* ti "O 2 u S 3 3 JS 3 a to a -3 to •-•3 °^ " .2 pa rs a . •2 2 1 E a S 3 3 .tt O co -C JS A-cd CJ •« a O 3 ™ P JS cd a £ •°lS E A .2 « > «-CJ CJ a -a O E CJ Q CJ JS C/J J CJ CT) c/3 in CJ —< Ctf J2 ™ CJ 3 Z CU <*T. 1^ c/5 •a o CQ , -a a E •c cj a CJ •a; JO cn C -n >- cu al IS 3 S co ca •a -sj 2 CO s a, 5 -a to » •- 00 cd • 1 § CJ 00 tS .5 • JS fc. cj si 2 g •a >. — o 00 > •al 3 co S J t M a a .s £ CJ JS .sis S F cn a -o •ale |a| lo 00-s u 3 S J3 ^ 2 13 *=! CL C U E a a §^-a S°l J -ti o D js o O W u 2 E S a g £1 <u ^ CU cn 0D O < .E O CN cj m •* OJ —• o »-5 1 °- 5S g •O 3 73 CJ CO u tt -S CJ JS 3 0 00 p" "U 3; c« o. 3 a 3 E a- 3 b o JS ill I cj ill >1.E j S -a i £ ? cS CO cu ^ o tn ^ ^ ' C — "cn ill •a §• s •ScS.i-g.2§ 3 >- 3 a-1 D. g - "O c — S 8 o CJ C •a 3 OJ o c > a 3 u 2& a .2 o- tt OJ —1 3 g 3 OJ U o. E a CJ CJ •- SP £ 2 "S 2 f H 1 g» •a 3. O CJ 'to ca C cd E JJ oi I' Sa • .S ts < cn S o T3 -J ; CN Ill Sis CJ 51 a. U .5 » a 0 3 r3 cd cd +3 : C £ *0 o 3 in si cfl CJ a H "5 oji u o H 3 " s& 52 fa B o w O .2£ o cu *^ •o :3 * -a -s I - a T3 eg <u ci u a.° ^ g c o O u c E is | JD O I-a sg •o g a u •S 5 8 "H <L> O <u o •S 2 .S 3 a fe 'js tx fa ~ o Ii t- ed eU 73 5 8 O 3 B a e w 8 w tt ,•0 3 » •§ g l3i O vi 2 +J ^ a. e3 a t/j . — -*-* = -0 ° 2> CO cn 5 .o .a-g JO o se 8 9-oi x •is ° o oo •a >^ U fN S 03 2 j2°?| So S Q 3 a CO 3 , to to b "-a 2 H o. a O-J3 E * 8£ la 35 8" "1* 3 c o co w c |« I a S °-J2 I'si 2 S S co • — .tt hs > jS I a E s. P <2 Q 00 g E O <N 2 2 T3 a £ (O U •a cj e ° 5 <= 2 a 5 £ a CO tN 3^ CO >^ 00 rs E 2 « Q S •§ a if ,= s'E S 2 ' 2 o r2 'C 1-E .2 S3 II < is o C-U. oo •-. <N •a o > z o JS I " -2 g ~ oo a "o E •i S ~ 3 •s •S S S.S2 S -2 ^ Q 2 „ 03, « o.oo| » o > O E .E I 2 ai CQ >3 O tn .22 -O T3 T3 u C a c .III tn .Sal ? 8.s u CJS 3 P. fa O J2 2 S S s go* 3 "a cr C « u 2 i u E .a<S S - > ffl o -s h CO U U k. &• >> a C3 to O. d.ao 1-8 ^> .a-s •3 JS 1 cj S 5 a s E CJ o S t: Q. < -s -aJ .. o to o •a <! a Cl a. CN I o —. 1 ^ ^ , *5 >n a 9 w CN 3 u 00 Q ^ oa —1 UJ SQ O Vi o3 75 JS n a ca • ! 0 -I 2 e I c/i 3 o.2 I •: i3<fi ' > to S cn ^2 • cn O 2 £ a c 00 §1 «a S tu cd «73 £P S h ra • to .2 == " a 3 fa -„ _ o O CO 3 o. 21 cd < C/3 11 „ CL cn x U 3 2 2 Cu ^ tu ti c B § a.I to 72 co >.-a gp S s ts j?a £ 1 3> ,5,'a 1 73 > g 73 to JS "a J-00 ci 1-3 o w tu o ^3 O cd 1— III i E .3 ; tto to £.2 g S a s2 c a 3 CO JS j~ S £P>n: ,0 -E CN : 73 CO tt •tt 00 P . fa *o "eg 3 B a 2 C-o >o UH CN <; .BP fa2 ci < S - " a H ss q j= ca pa ci H S3 CJ . Q. ** JJ 11 8 I 3 8 Ox: t« x: — i_ JC v) D. :3 o. CA CJ ' P. c/5 U 3 u <B 3 CJ ^ 3 X) £ '3 « M S « So? ca c tn co 2 S3 I -s 2 8 xi O XI s cj o CO w - „ g .a 2 3 _ * § -o § -a -a S • 2 U s icl •§-s leg .s i U to is I o « ^ CO "r- tyi to tV 0) o p j-w ed ^3 III1 « cj g a o ca tu 5 « 1 5 *& -S •2 J= 2-Ei &o ca s .a e. 3 •= Z 2 ._ CJ § 2 ca JO c o o . -cn CJ cj 2 c B..2 a 1 CJ X £ o J3 O. O ll tU •—) — OJ CJ w x .fi 2 +5 " —J cj c •a Z, & 2 JB •-cj .a g » _ xj cj *p *c3 2 S ca u. o fc. Q. CA a a -/ 2S v 1 O .S £•? t, <u ed tu g « u • •p -S tt ' t ^ CJ . &g S E • u cn *3 i B -a a i Is i ills ' E <o -S : Il S j-S-3 S ^ ° C u ~ 3 O « £2 a 3 °ij |l 5 3 3 OO o o -a g S5 S 3 O. cj 3 -Is! § - "3 TOW.. ca o o •2 o d |8 .1 2 2 o s .2 -a ! ty_i 15 TJ 3 *i 11. 1> _CJ .5 11 u § 8 S3 xj g g S u • l« 8 5. IH X l. Sj2 0 O 3 > •3 2 ° 1 ga. E s -o x; is 2 c a 2'=5 X) P- o 73 CJ Q . S S^cS: Oil II ii 3 w S < 2 S3 E : CA ° S 1 3-Xi E 1 1 cj ca cj e XI S 3 s; ^ S3 ^ S CJ X) 2 'C cs-e ss CJ .S •a 3 If |-Iff u o o r J: u .2^ •i. s ed* to w 3 to O 3 ^ B I i 0 j3 1 c 1 tu .S ed OD Of) o 3 S 3 "o 3 fJ CJ s ° -111 o .o .3 oj i n. E CO VI CJ CJ 8 X> a) Q S ^ H o 98 T3 • 5 3 S 2 a O u H O OS w tj > S o, •OS s I 3 s s -a .2 iw s a i its » J -2 cj I rc ° E & 3 1 CJ Si 3 • C ^ o a. = o .2" "3 J S •§.2 ca 2 1.2 JS 2 ft. 3 •a 3 wi CJ s5 cu ed E S ctt ca ! 1^ E <-• • « g • «- S ' o .g a. 3 11 o o. CJ o a ca cj -~ aj >i ° -5 e 3 _ « O 3 > •a •— oj ca oo = •3 3 CJ M .3 V, 3 -ft. i_ CJ IJI 3 CJ Q 2 o CJ u •S ."2 ,S S >, JM •§•1 a 2 2 JS x: o <£" g o .2 2 2 O 3 fc- cj cj « Q ..a o op •*> cA 5 <N tt. os CN « lA cfl Tj-3 CN 2 £ Sac CN CJ Ov cj o .2 vi c/l CN 3 m 2 ^ o r_v tt. os crt so 3 CN cj — fc" ^ s tt, & ° O g Os M 'ob 6b « Ills 11 £12 2 3 ca o oi oo O CJ - 00 O O < I IE—i UJ ci £ .a* O PJ o 2 co 3 -a m so .. in Cm J3-2 ~ ' 3 3 Q 3 £2 Q 2 on • -a " : s i 00 I 3 if I cj 3 "3 a 5 3 ca u E cr •b S " S a S •» T3 .5 3-3 8 c J§ • 3 .2 5 u t3 »>* f l-s ca.se 2 ci O o o a K o JJ > ca H 8 5 so Vi oi S: Ti 3 C\ O tn UL, ^ _£ •a a E JJ I •» o. ! « E ! S 8 : S 2 j!l £ S E . 3 0.-H .« 2 « — a*| 3 1 E S33 • s §^ 3 « .2 3 2 3 ^2 — v3 S-S 8 SI2 sis' c"§ cl ,u *j s las ' c § c u i: u 252 8 J= •3 CjS _ I o r-C/l 00 S1? 5 £ co e 2 ^ c til §§.= ° u 8 U >> 3 2 Ci O C_> O •a ° 1 8 < E S J1: — 73 o c3 1° 2 "5 o E >, c. -c 3 9 8 5 - V 00 •S ci s1 •8< a » E - E • 5 *^ 'l E •«-> (L> tn +.» s -s S-t! cj -S 2 o c J= JJ E fa a u o i» » •3 "SI C3 — 00 • - 00 s £ 5 O S ca ci > | 1 < ^ u * n a. « H K c s o cu .5 C C +-; ^ a> J— -S co 5 cn <u C fS 3 ^3 8 g?§c2 •5 E S o * _ i_ o a ca o<8 .2 •3 « as 8 oojf c E ^3 SP c o o ° '-3 S> O CO V CO 'B la's I "8 - IS §•3-1 &,2 „ •3 c w cu a "P A i3 CO C/5 52 ^ fa 73 r- w cj cd C cn •* > u u c s * S I g £.2 S CJ3 o o r- fa O. co co CJ o 0> o CO CO L. *3 CJ H UJ o C OJ L- fa^ o S. § o a • J3 O ca ca Q. cj fa Ci > txi X >-.s a ? JS s J= 13 a «3 -S 'A c 2 w -N cn >*-s cu Q T3 3 0DXJ « ?3 cu u tn D cu e ^- & 8 <u -g tu E E • 00 ^ c ca a J3 U E co -O CJ S 3 .1= "WW cu o S g a a s Cu ID tu * tS •« s SJJ.I & •SJ O co 00 ca cj co .op o 3 .E to p 8 o .E &. *o 8 M o JJ a S..2 Sal cu ™ n. H CQ BcSi5 O UJ o 2 o 5. •a .5 o E OX) oj • — - si a-a J n s s a JS a a "a I? ,.2?9 |!S | g 1 a|fi o ° « r: « ea §.§ &a cd S CJ S .13 •o a « ca c .ti « I -o •3 JS -o o 1. a 8 0 .2 J2 S e3 eci cd Si's 3 " 3 S3 a 5 j? g J Z fl U 4 , 5 O-B's -a I  > * 3 > CQ Ci *3 u & a ed *5 8S» J3 3 r2 S3 •« c3 £ •-3 3 eo 5 <X> g-•8'8 I 3ll « E {SI M O in •a 3 fa w ill a .-H fa» — CO cfa S .E IS .2, CL, CO E E E •& co cd 3 3 8 x j2 co cd co CO* •- 2 •5 -° 0 a o.« •S >E 3 CK fa fa co >^ co O •_ C CO CO 3 € >•£ s h ~ o •-*I*'S •3 Cl.' u, co 5 £ 2 3 1 a s ° ^5-g S • L_ CO J2 c o 2 co 'jo a •o -3 'C cd CO C si O ra 0 r~-co ci co OS 8 co ^ o r^i, - 2 °fa •a i 00 _ 6 CQ fa '— o = > CO > co a a co 00 «_» co I 3 & ^ c3 CO E g a LU J3 •a c3 CO "H >» 3 (fl D <y> S o & 8 I £ s « ' ? § -o c3 5 : 2 o1 DO O C OJ CU 3 S3 oi 3 JS . '5b °->. 0 00 .ti -° .E ~ 1 8.1 8 a S Cd _ tt CO CO a g-§ fa o CO CO oa sT fa1? •o E •a 10 a e 3 -CO JD I 8 o -a B .S to CU C -= « cd i-o ,0 S.-2 co U PS ?5 " w 1. J 60 g •i s §1 CO ' °"3 < O i. eo •2 £ CO CJ r" CO fa CJ O T3 cfi * 0 s Si •si CJ — co o. <u & j- cd , CO O (U rCU 3 cu o a 2 2 CQ a <! 5 I CJ o on in o CJ. u. _ O CJ 3 ^ cd •a 1 g E 1 B a * a co ^ s "> 2 3 js c * 0 •a o & "g 2 o.J3 -a S u « S m «ca 3 S* j-a| § s c 2 g * 1 £ op a E 3 JS CJ k. fa eg fafa u fa ci o „ E o co 3 t» S ' .2 « J2 g g> a •S E » o s |'Q 8 s > ci •o •a 3 3 a >' g I l.s: CJ JO 1 ^ 4> cd CJ " 3 e3 cj S3 5 co yd •O B a a 111 co cd u l=§ 3 ^ ill Ci O CO cj 6 .E E a if. . 2 60 CJ • r- O ^ 3 = s 8 0 cd cj ^3 fa cd o 2 a T: ^ 2 Ci O r-o 5 ja 6 co cr S3 S •3 -2 fa CJ u 3 a 2 "p 2 J= = s Si JJ CO s 2 8-2 0..2 .2 u !0 > ji> 3 cd ^ 2 a •o P i?S 1 co » E 3 fa CJ CJ cj -3 cd cj a o •o Cf. 3 a, o I •§ 8 2 a S 2 « • o g. CJ g 3 O S 'O a s 1 ss S-g.2 g. 111 2 si OJ o .« s a 8 jl § 5 3 2 . -o L-o y s a .3-2 .3 111 "O B OJ a I -° E a g> O 3 § E o e Q, <U -S E SI-u o S S3 2 2 TJ JS -s w ^ C S3 -° "O 5 « a •a * 8" g *~ <L> o if S3 If 3 -3 ° 0 O.IB lit = s it? B B T: •— CJ OJ •£••0 ° 2 ° S" cd oj 3 • 1 3 I c2 ^ "2 o a S3 c/l U ° H «.s S 3 "8 a 3 E •o 2 2 JS •a 1-3 E 55 CO (ii o *i u III 2 2 C 3- »-2 2 •a E -'5 1° Crt w B J2 2 •** CJ cn CJ L3 i' tu S3 &3 •3 JS O "° * a c3 c/f B v 0) H 3 3 •o-.S" = CO 3 33 P IB 8 ' .S 3 3-3^ .5 S a S I a I § 2 5 Ji "v\ 1.5 o So ar => — •~ w 13 "! objs S 2 u >SE . 3 O o 3 OJ ... OS S3 $ > > £ .5 S OJ V? OJ a 3 « o M- g •a s u S 3 B 3 OJ S. il -j a. a .2 S 3 E .52 8 ! £ & a •E o a • 3 cd o CJ < 2 o E b J2 3 JS S So.*' •5 « -a I*5 & ^ .5 -a S3 >• I £ 'L, TJ •a. ! 3 M ta o L 2 3 < 3 . •3 La o U CJ CJ L-L. D, O vi *-> cj 3 g CL- S ° s " S3 o E D. tu 3 .22 tu ,JS ft 3 |S Js ^ " "S 2 u td 73 in. S OU E .3 m -o j2 ^, 3 S3 -o g g i o ra o '1 'g 5 2 a.-§ 2 £ -4 „ s > O UJ CJ o 2 rn cj & CJ H W X U OS H V "a . E -a cu .ti co O CO 4-> tO CO CO I _ E fj 2 S E VI CJ JS o ^ s § I °! a B o E OJ oj H ja cd a .3 HJ VI 3 3 •C Cu 3- O .2 oo £ B 8 "a> u- cd O E U. .E ii 2 oj § =r S CJ 6 .ti 2 3 I CU •a a J2 -S cu ti| CQ 5 Xi o w cu y ex 1 S3 g .2 CJ VI Cw 3 O JJ 2 2 ccf r c •s 3 E E o o CJ cb cSJ. vi 2^ = 3 !£, OH •a CN 12 = 3 Q ,-ifS CO CN .2 >A co ON 3 —i OJ L-i o C-m o o PL 2 o o O 1^ " l o o O Ov o cn is pa II 2 Q IS CQ UL J2 j a w 2? * S'S E u co s$ 2; ° S B a JS 3 o- J3: CL. 5-L. (11 1-1 > CO 2 oo .3 "a « £ ed 1-2 a ;°a - I I'S o Q oo B § S .2 -a l-g.-a's 11 * » e <u to a-•g .1 g-TJ CJ *W « CJ a a II 4> Vi •s IJS >,-3 2 ca o- c !?f TJ _ 3 ' 3 O • 1) w t> SI'S 5 E « -I Pi Sob J= g 3 " u 5 -3 £ ,? > Cw w » C o s fc S" cj o a 5 ^ OJ fc O o }S S c£ § 10 co -e co 3 « ill eg §1 S "§ .2 3 ,2<2 to "O .s s .11 S S3 3 i 3 CJ CO cj S OJ TJ * § • R c3 CJ JQ •E > V. u tS JJ« fc CJ _ w o. o oo S 3 Z S S „ to JJ CJ M 3 J3 3 M SI LO c2 5 CJ CJ 3 S cj -fc-CJ CJ CJ 3 E5 i'? ca TJ •w a « a TJ O. 2 E CU O £ ° S3 " 3 « * S-2o 'Sf I" CO 3 *o cU 2 S3 fS Sf.S <u c3 B|1 o o u £ -S I s ™ Sj C ci 13 1 2-g is « +•» Cfl « •- w > CD o 1 s & 2 oj 4J S g-a U cd 3 ti £ > 2 2 OJ *a J= 3 SD o E B •a ? cw co o co w c2J M s .!§ O. to "5 Tl tJJ CO to 3 O cd tU "O c ~ O -C cu ° & g co '43 S3 E b "8 g-a a = i, co CJ cj in 3 73 ON CT e 2 '3 8 S •§ S2 fe 2 £ <S <w H cd o •a •a <*> a » 5. 2 > fc tw ci CJ o ca c/3 > 2 S—'tS 3 CJ S Si g-wT E O — jD I l-l S 3--| Q § i) 3 g ^N CW > o fc co EE la H 11 cw1 -g TJ 8 ° e " 1 | iS 2 'fc ,o o E g- 2 2. UJ CU *3 "t3 LL] O 2 o ,1 — to LL) LO o - 3. cj u a '5? '-a S II-H^ op E ca cj 11^ -= 2 t. If -_ OJ •S S3 * bb 8 ,._ 'ob _o ca '•s.i §1 Vi 3 i | 5^ :a S3 ft. la rr co > cj i<! 2 S3 ^ E -o OJ CJ -*3 ti 3 CQ D oo-^ E a ? E o OJ co .ti 3 J| DO I ° O • 2 2 8 w — C S « 2 c.2 O CU > C4-. >> o § -5 cu ed CL 3 cT cu cd .a"s 2 a 2 2 OJ « 2 '^1 3 a C E CJ 2 -a S BL. i ^ 6b o ii O w T3 fc 3 3 D. cj •-E >1 O OJ .3 CJ TJ co fc 5 ° 2 LU TJ O NO CN, CJ CO 3 r_r 6^ ON • , o •* 16" twQS Crt —* 3 ^jl cu CQ SO ^2 3 C o o LL. TJ O . ft. CQ ft. sic o > If O 3. •3 w H CQ ~< LU _ SSf co 13 2 u J= E 2 o a 2 K OH' « S3 CO 1 Z CQ CQ _ 4> cu .5 , o ,<2 .ti JS a a S2--° g _ a u « o s si ° & 5 = -5 g 8 w 1! £ » FT1 U 3 g " «*a te ,co c i& o •3 1 I I U UJ Ufa 2. <N u c 3 fa1 o o Is . I_fa O, cd O 00.2 c c 00 o •a o a i- O.Q a x c o. cu .= 2 a. o 00 4-. U 3 §§ C T3 C - O <L> o a 3 * c o fa . CU a O S3 3 2 fc- QJ tS u tr > S3 u u 1 co "O cu o Ci fa 2 o 2 S3 tu S3 CZ) o JS 3 11 BC UJ o. 3 j_r cu i-l " s ^ Js fa CQ Di t i" • CQ fa fa CQ — v c o _ "o CO CO JD K -» •° J-8 el " £ [i. n Hi •a oo i 2 " \£ !2 2 " " e3 73 Xl '1' to ._ i .1 1- CO s -t. "a, E °q cu < tN Q Co < tN S S3 I S3 8 3 s §• 3 * c 8"5 I f- O ll eo cd iS 3 Cui CO fa & U 4> 3.1 a Ts J= o a oo co a a vi s ae co > Q .S s •* CU tN Q CJ fa 13 CO co •r. * -o 1-B6 -2 j-a c3 •£ QJ eo 3 « l| a pi a cb o IS |! -J -2 E S O > j 2 •a 00 cu cu " ° c g co 3 2 > Q_ QJ 1 3 a _2 S3 | a. E ! tl> 1—' 2 •§ x Si » E > o. a o I s 5 S oo c ca .S 2 tt S3 .I ° QJ l-c to u 1 a & o k u 1'S "3 : Q £ x u.S .2? _ to ca g o a to E 2 3 S3 3 E " if Q 3 c -2 cu a 'ct — CO fa' u 3 r~ t/5 -S fl 52 w < CO Z Q 3 co •a' J 3 I J.00 M 111 CJ CO i g 3 5 a. ^ 3" « .2 : uE 5 c O. ' ri co 3 S3 cu 5 CO T3 i CU CU i -o _ ; 'C « I 5 .2 cu Xi o Q -o .S •a o-a 2 . ti — co 111 g 00.? . cu *^ r*> 2 ^ 5 S 3 u cu a o fa cu '» P 3 £ cu E o .ti o fa -o u « fN W =« o oo -53 o -2 ° co O co , Ifa CU 8 • - - ™ tfl co *o " — •= CU fU * !fl > H H 3 o -5 •s g & 3 1 1 'S "Si £ JS o 3 8 -•S 2 1 co o. I r2 3 .! o "to i S3 'ob ! Sfa O co o ed *0 QJ O > I3 O ^ a. S3 E H .2 ° co O C — <L> •_• u a CU on ob S3 g a ca o. T3 T3 & cu CO CU g S-ll a g CO *3 cu ra 00 j-00 t> CU S3J « CO • S .2 CU co t: E <L> <U D-fa I a •o • • is a a, E 3 3 ° s S c c 3 cu >> E JS 3 o. cu ca o •2 ^ >p cu -S ^ o co OO C OO fac^ 1-1 he's E S ~ c SPli 1 11 o C "O i3 co 2 '5b u ?P 2 •§ -s <a 1 S3 co u > 3 C | S a E S-42 | I S3 8 S « g 2 -o I > « 3 UH S to OH w Hi I i S" o e s u I ^ s s • •g §1 E o 3 O C. CJ CJ O.T3 ii CN co VO 3 CO CJ ^ o a life m ti o P on cj •S E o £ '•* •a i* "Sb 8 3 co .2 C oo oa 3. o 3 -S •o <•= O t3 0 « "K .S3 1 s t- co" >> CU •2 ^ co -3 3 -O 2 " O ,p a " ob .t. 3 5 -E I a 3 .2 o- 00 •o o S3 o e> ca CO CO 3 3 U < <g £ c/3 rn llo ^ I OJ * I I  <» CQ 3 1 1 *g 5° cj •£ * a ! u a •a -5 2x> •E ^ S 3 3 a O O. D. c E.-S E a o oo s ca 3 J= u. cj ca 3 3 E "2 5 .2 1 S>3 3 . — ej j2 J= sa 73 ca 63 r= a 3 -§ * g -o o |3 § § g.3 •- * "i a v « C 3 3 o -o C < § j= >8 CJ O t/3 <N CD "3 -4-» o C3 £ 4i o 15- . -=i -j-J ^ C C W Si 5 -TJ —H —C •g .2 Si CX gl ed cd < 52 O Ofi ^ 1> 4) (U C <D O O O w pq o u 2 J u w o o PL, M tq o 2 r-APPENDIX B BITEMARK CASES IN THE U.S. APPELLATE COURT APPENDIX B This appendix contains details of 103 U.S. Court cases that have involved bitemark evidence. The cases are contained within a table (Table B3). Also provided is a key to the abbreviations used in the Table and a full legal reference list for the cases [Lref]. Two additional tables (Table Bl and B2) are provided below and present a State by State summary of the cases and an index to the trends of bitemark admissibility. This appendix acts as a companion to the legal review that can be found in Section 2 of Chapter 1. Cases described in that section can be found in more detail here. TABLE Bl. State by State summary of indexed cases STATE CASES STATE CASES Alabama 43, 53 Nevada 12, 20, 25 Arizona 9, 66, 92 New Jersey 103 Arkansas 76, 82, 86 New York 17, 29, 30, 67, 90 California 3, 7, 8, 49, 89 North Carolina 18, 19, 37, 42 Connecticut 33,40 Ohio 27,81,83 Florida 31,32, 50,58, 60 Oklahoma 23,41,73,74 Georgia 36, 62 Oregon 5,71,77 Illinois 2, 4, 24,35,45,51,87, 75,80, 84, 94, 95,101 Pennsylvania 26, 70 Indiana 6,48 Rhode Island 34 Kansas 13,59 South Carolina 11 Louisiana 28,55 Tennessee 88 Massachuse tts 46 Texas 1,56, 65,68, 69, 72, 96 Michigan 64 Vermont 10 Military Cases 22, 63 Virginia 39, 57 Minnesota 87 Washington 47, 52, 91, 99, Mississippi 79, 85, 93, 97, 98, 100, 102 West Virginia 61 Missouri 14,15,16,21,38 Wisconsin 44 176 TABLE B2. Common objections to bitemark evidence admission from 103 U.S. Appeals. ISSUE OF ADMISSIBILITY ILLUSTRATIVE CASES FROM TABLE B3 Bitemark evidence not sufficiently reliable or established. Abuse of Court discretion in admitting testimony and evidence 4, 5, 6, 8, 10, 11, 14, 15, 17, 18, 23, 25, 26, 29, 31, 32, 40, 44, 45, 46, 50, 53, 57, 58, 61, 64, 67, 68, 70, 75, 80, 82, 82, 87, 88, 95,98 Arguments regarding the uniqueness of the human dentition 3,9, 15,20 Constitutional argument (5th Amendment). Improper seizure of exemplars 1,4, 17,27,48,51,56, 60, 94, 98 Photographs of bitemark evidence inflammatory 7, 10, 18,33,52, 86, 88 Inaccuracy of techniques. Errors in bitemark protocol 2, 9, 13, 19, 80, 97 Use of previous bitemarks or evidence of previous biting behaviour 22,30, 54, 86, 88, 89, 103 Defence requesting prosecution's testimony or funds for own witness 28, 34,41,43,73,74, 85,92 Witness prejudiced or other witness related objections 32,33,34, 58,61,62,91,97, 102 177 z o w 58 to c« o o CO H co p < CO o CN TJ cj 'o S3 CL> cas one E fc _C3 ca < US yea TS TJ 2 *- cj •S .2 E 00 TJ fc 6 T3 5 CD CM CJ C/l = - - - co « fc •e £ "a TJ —• B •— co*u fe •3 ob g.2 "a g 2S.t a. Q - ^ B O L ,K fc 4J CJ w CJ ^ 00 cj TJ g fl 8 ? co TJ ii 5 c * 1 = .22 s- "2 5 si Ifl^ 8 2.2 — a w aj o 'fc & c H ob | fl Z. CJ cj ^> 111 O S ^ ed o e 3 u o fl CJ •a "8 al < -a 3 fl § £ 6 ffS S3 9 S '" •- " aa f . - Ta •a -CJ TJ , a. j> 9- E 00 TJ fc CJ o u tU 4=1 03 O a m PQ W H Sg3 t_ Cfl NJ a 8 •5 <u s-g; C/3 TJ S3 ca S bo CJ .3 O - M X c3 CJ CO CJ '3 E C > O 2 S o 'co .CJ CO e-fc 2 BUB o c o CJ .3 U C 2 '35 Q 2 3 St .2 c& CJ •8 is £j~G 00 3 a TJ S 8 ,S a J2 2 a S 3 ,2>TJ oo Eo TJ J= Ha «J 3 a c; fl 8 1 1 *§ tl S S3 irQ ^ 2 g « J fc TJ §'3 u cu Q fl . e 3 3 CJ 8.1 'B S3 A X .2 '3 TJ 8 TJ TJ .tt g cj fc a 8 O g g> . TJ o.^ "c3 ca e CJ CJ ea u -••2 «fi'S o CJ ^ « o ' S3 m -S 5 2 CJ 2 E S.S 2 S3 ° J= fc cj ! 2 CJ : S3 s "S.TJ 2 § !f fl 3 CJ E 2 U ^3 E ^ eo E •| eg ^ 3 2 8 2 E cu eu s fl T^^ §2 ~0 CU C VI u Sue cfi cu S ^ ^= o T3 U w •— cu S .2 e to 5 2 ! cj o •5 CJ to cri td ss P z U o ^ CA CA o a. a. S - 3 "3 > li. ai cu CL,' Q Eg B S CJ CJ cj K CJ CJ O 3 *J co .5 g ^ •- 2 CQ > x> S3 . S TJ _ CJ E E •a 3 CQ 2 H CQ < > J2 cj ca •S-fc. 2 io s g F i < gas Ct-( 2 a J O 2 O.C3 o OJ H STJ 4J TJ CJ 3 a w. 2 'c ii "j O Cw U O 3 °- CJ 2 P & s < = !2 o o fa — • Dd, a, a CJ ' " •a •« -a « e . 2 cu s C < c/) Q tt, t=L o cu . i cu I S3 8 ! ° 3 ill ^TJ — 5 CQ 00 u fl | o £ 0 W "O 1-8 ^ = ^ a a log 1 2 u Sue Of O tc*;2 = o £ IO "SB 3 u CTTJ CO A g •8_> § JB '> § ' a> cd w J3 fcj JJ I"' k, co M w — ca o M 2 fl c ._ g fl 3 CJ CJ 3 fl w JJ CJ IB > .2 <s ij 2 « ° 5 3 c/j "! co &I y •? o : «2 cj 3 . ^cj CJ TJ 1/3 fl -3 cj cj . JA co ' fl § i Ej I 111 CJ a. c .s tS .1 = §•! U TD ,1.8 a ill's 'I I-a . Z I S3 TJ CJ O C co co | fl TJ U. 3 § • O 8 1 TJ -9 3 3 .3 3 5i « eg ' P ^ 5. S3 CJ TJ * a « § a aE & ca co c a -3 fc "2 I: a B CJ C/0 JO Xi CJ IS v. o i S fl s j3 a lie 2 ? c fl fl fl <u 5 o 8 fl S fc e S-a A* 13 P c5fl~ TJ TJ 3 .2, a -S Ifl > 2 o j? 2 „2 ca fl .« 5 S .2 .2 „ . ™ K > TJ e Xi - o o s g a CJ fl _ CJ fl -a fl -a o."tsfl §-j=.s O C 4= * 8. ti -• fe. §8 12 15 •| g« s « 1 c B fc a 'XJ cj ca Q CJ H JJ CJ TJ jr [/i o 2 .2 •a.s cj 3 3- S <. CJ TJ 1 -S J Jssfo^ •a a — c o o fl g j E •° S TJ 2 « 2 *- « .E ° E OOfl fc SBb O > ca ca cj 'S fl . fl U TJ 2_ > '5b e @ S ISE O cj ca •a 3 CJ E u o 2 3 M '5 •§ "3 ^ cj CJ CJ S3 '^ ° 3 CQ s to C -° 2 3 3 8 S .2 •§ CJ 3 TJ fl -a oo >. TJ TJ g" 3 |-8I 1 .3 co 3 .3 « 3 3 « 0 •§ Tel 2 O c ^ • » -3 Tj 2. e x: .S CJ CO 111 c—' tn U CJ P S ca « 3 c -•>S.S cw X) CO ca O CJ *J fl afl 8 T? -S g & S O 8 3 -3 S O. > •» fl 8 <N « fl tt cj 3 s g " a cufl'bi.E;2 S3 s:S"s u S3 S CQ . g E 2 u "8 2 ta ° 3 e 3T?T2 c SH-oo.fc'E S « I a > B * « '5 j s -C g » .2 | 5 ^ s £ s 8 CQ .5 is O CJ fc 3 * fl .2 > S '55 i? 3 >• "J 5! o Q co u J 3 o. •§«§ E.2 TJ •5 CO m s CJ qj XI Oi Xj ^co ? ^io TJ E s "e SB s 8 2 .2 tS fl co >« fi Q ej 3 M_( r O « ti. fl cj fl 5 1 ^ 2 3 -° JZ cj gfl 8 g S3 •a CJ CJ lis X fl CQ TJ < tt cj . S2 § 2 111 •23 u = •= is g«§ i 60 fl ? fl ? 'ill's ASIBE S 2 >2 S3 •S 3 "j a -r fl ?S J3 > 85° CJ -S S3 CJ u 9 (*~ io 2 fl <S o ^ X t.- u *S ^ U 8 I &l§ J d, M |J ol d.' Q Q X CQ E Tl § xi CQ 2 '? fl CQ e > T3 u S E a > U fc co O -;j X CQ fl g 2 3 ™ CO 111.11 CQ O CJ 8 00 X oo O co 2 fl o E S, a 3 .a -fl Oi 9.5 .2 *S > < C Q B u co CJ O fl o "5.'2 g S g -3 „ O .S fl 3 § >. ^ CJ Xt M O CJ 'S /j cu sa 2 U &o Q ci TJ „ CJ t • .tt 3 3 O s s 1 &i«31 B E -2 S s fc .a S3 . o »• xj M 3 3 13 O P CJ 00 c5 3. • CJ D. |oci< •g S3 o d Jj I •! 00^ 55 H O O ti zlcS W C 1 ca S cj 2 I „ J « « £ tL .2" o 2 g i& t! -3 2 'S „ 1 g 3 = :s O fl c _ o. *x ^ II all si hS?<u.QO! ON r--cj O to ca -s oo IJS s a |3 a. CJ O .3 'c? E 3 o 3 S3 M ' a B 8 8 o E O co «»"2 > 2 -ea s c Q eo eo CO O CL> CJ JD 3 So a a I "8 •a a I 8 • ZL B 3 73 O '5 cj 5 JS co 3 ° * J3 J= -3 CJ o « ^ 73 « •° 2 cj 'o 8 ti «-* o oo va S € 5 o i— £E' 2 3 CJ ed 5 E §3 1 J= JS S .<2 fa c co O E 3 1 E I i2 -3 co ' | ' .1 a •§> § co 3 CJ Si » 5 II J3 c= CJ o •a S 1 1 « B x co a eo 73 — cj S 3 '5b cd fa .2 u •a JS "= s-o E I" fa fa ° o sp u C CA C 2 ° O 3 'w — O OT S fa 3 C o. £ — p -a o c ' S 9 < , co e5 •o 3 < '5 pro a P E « <N 00 . -a 3 CJ 3 Sea g I'=lfa E CJ T3 •a E 2 S3 2 3 •M C CA , S3 -, CJ 1 CA a 3 & o CJ CJ 3 a 3 fa J2 ° -a c J2 3 S & CJ Q 2 CQ fa a .s S3 a to CJ g W 73 O CJ ^ .** fa a ^ a 3 >. S •2 S3 ti 3 flj « SB* a* §• - g a •a 3 1 • ca o S Mug ed CA X \3 fa co CO JJ g 2 * " 2 o. g a 3 o E Z -a CQ >. S3 JS * HI CO > — CJ E 2. "a? .a I •o -a d O 3 " l:^ -3 =a ~ CJ CJ 3 2 .3 3 •o S " 2 2> 3 a S-gl u ° u % S3 Cu I-si ^ cj CJ CQ co -3 •O 3 ss S o 15 2 1 S .E <5 3 S 3 -8 5 JS -3 g E o .E, I a a H a s a « 3 5 00 ; 2 .5 CO CJ 3 CJ CJ us 2 S-a -c -a B .a u S 'Sb-P Z<2 lis 8 5P cj ~ ~ c D. « '> ti 0- o. u 3 w CJ o»§S U 3 ca CQ S3 cd a C S <s § 11 8S, it S 3 .§•7 1 fa -o > JS JJ cj *S! 3 oo a a s 3 8-°? §• o 2 <5 E j= _ £ "S CO 73 fa CJ 4> 1 1 t o < cr-S a M o > 3 ! a a — -a 00 a -e •- >. S •§ i J 1 •2 a co is o S <aa.E3cg S3 > "2 « S fa CJ .1 SS -3 CO CJ CJ 7j -° ^ o s h CO 3 co 11 •o" £ it! co f- 3 fa cj E 2 « B W _ S S S3 3 5 JS fa '3 > <. O. O CO CJ CJ CQ £ o cd a> g ° P -o c <U cU CA •rt CJ CA 3 ^ s o Xj CJ 4-> C •" 3 „ I ^ co CS ^a-S.a* StS-Sfal S a^-gfa 1 53 fa, P 3 a CQ ".a o •a . a 2 ^ 3 CJ "3 3 CJ O a T3 3 2 7^ S •O -3 QJ Cd a i 3 u cyTJ a 5 * 3 S „ S c a, 2 a CQ CO -Z 5J CO S3 > 2 fa ,S .2 O fa Q.73 S CJ 2 CJ g CQ 3 I •3 2^ so > 1 CD Z. U S S3 ^ i cu o IJS-S n. u •S'a ss u g •a a a 2 « S > o ca CO 13 sau 3 fa -o U o CQ E a 8 3 o 2 o co o 3 tj _ CJ fa fa 'co CO 75 „ ^ a -B 2 •g * a •8 2 E _ S "g. O -i P . ~ CJ §1 S3 2 CQ u j> 3 Sa w '* a I •O 3 Sj 2 a .is 3 CJ 2 =a a j« a a-| ? 1 a ilil cj '? fa 5 '3 .3 JJ 3 B Q.3: B o 2 -I 9-B 3 | w ecS • - o CJ CH CJ -a 3 S 'Z JQ M fa S n.5 4J O o fa c a a cd Cu CJ E S3 & fa-3 d r-i cd C 1-E S .1 JJ a c -cu ^ fa « a ^ CA eu C ra ^ QJ CJ CA c .S g O CJ T3 •a -0 c 2 2 a a. CQ a fa H O j^" 00 u cj CJ eO cu CQ > CJ CM' fa fa fa oi Q fa ° ° g oa c a fa > "O "co 2 3 2 5 fafa^-S-2 g JS .S -b B 3 1 fi fa-SS33SFE8ifa c- k, - 3 £ I ° 1 11.2 8 8 I CQ o > B o J3 u o o sac 3 .S 2 2 .2 ^ XI O.J3 > CO B .3 " QJ 00 IJIal p V. 3 , tj JS — '• CQ js co P >• S tzi fa^<S X czi ™ O B o So 8'S . CJ CO k" ». 2 fa CO 5 3 « J o ^.D £ ^ c e- § = .2 cu oj a. ca CJ CO JS 2^3 > ° fa ,s s 1 .a o s Js cj CJ fa ^ e eo IS -S g-la on CQ oo < ti 2 S ti E b 3 C "T- O u fa (J fa < 2 S o t S o 2 P 3 u "7 Jo cd > ti -1 H Z cu o 2 S3 UJ CJ 3 C §CJ 1 3: a. 3 , .23 1 = •S.'S P JS 1—J •5 Z ^ a fa o 3 o CJ cj o T. Q a cj cs CO •> GO cj 3 i—i 3 CJ D, > i-3 CO js 13 CO i o 00 IIS I fj I 3-1 Z c 3 12 "G "3 CJ cd I I a P 5 ^ o 8 3 A | •3 * _5 2 3 CB tig £3 13 i & i e CJ •c o « E cd t> Ifl O •£ :| ;> 3 ™ s Er OJ CJ J o S 2 OD i— cj 2 fl fl 3 ,S fl . 3 <-i 3 fl e t 3 S u B e S J3 P- 3 3 X O. c CJ fl a "2 -2 cj cj to "B x 2 ° CO u i S 2 s S 2 CQ o - o. O-fl p 2 '3 o. * « 3 2 "2 5 > ' CJ cj * IS O CQ o CJ CO -2 O. TJ 1 s C cfl • <u fc 'E ts <= 3 3 cj O CJ _e co cj A -S3 fl.SS s CQ 3 O • CJ fl 'S : 2 = 3fl cj : c C4-, i <L> O I in , >-» t/> , V) <L> 2 & 3 3 •=> 3 CJ CJ fl fl c 2 o .5 *> ° « 3 J- a* «C >* 2 B cl 3.to 3 O « CO j-B ">S co fl -C b .3 =3 O S 2 .23 73 g „ t» c '6 o. S3 o. - £ ??e O 3 o cj c3 —i 3 O » ill U 8 £ O ^ iflfl 5=3-6 'Si 2 E 2 2 S CJ fl 3 Ifl' slf ca 73 2 to cj I S3 ob ^ -d '§.2 1 H " ca lie 3 CJ CO 2? Of to C fl Si O .2, is x ^* to fl fc to •- u 3 § »73 S 2 g « 1 ^ — T3 •a I "8 g OJ C CJ 3 .£ .JS 9 >• x?ig fc S < o o a o > •a 2 fl j i s .2 S .£P "H U £J U I S3? ca u 3 75 cj Sag u u u D fl is CQ ~ •« 3 CJ fc § S3 oo is CO u .^fl •— CO o ^- is 2 ^ w tj S™13.S 2Tj o ai o u is j .2, OO^XI O CJ JB Xj X1 3 £ cj o-o « _ s» jj °fl:S g'Sfeto.s - -llilllii-g IU £2 a> 8 •§ I gag-?'? *2 •a S3 £-2. S-8 § t: " 2 xi cj u g- co fl '5. 2-fl •£ S3- 1 ^ a> i| I XI XI j 1 a. a A l ^"S •-! 3 S > 11^ fl 3 CQ fl I tL> > <D fl | fl $J J ^ -°-I fl -S2 u U •o ail cj fc ca fc. •Sop1" 2 »° E TJ Ifl p S E o .2 S I §-.|fl ^3 ^ O 8.SS' 3 A E to CJ x N CJ O •fc cj -e 3 xj _a fl « «> fl •°x.i g.2 x; a • A a is 2 -o • to aj CO flj to Sf g P « -5 * .3 c O fl tt CHE S3 2 X S3 . ^ S3 §3 ifl ° S3 i* 3 .'-UV, TJ B o.5 op .3 o- S *a K ca CQ to M 5: Xj M S "a « fl « X CJ fl « 2 3 U vi O « ° 2 rj C3 3 " ,S -s o i> o S Q 8 X ifl s • fl ? •8 8 s ° E A x p 3 O. CJ o g fe-.tt oi 5 x: o *x o 2 « -S 3 CJ O. a Xj S3 j cfl 8.1 .1 «&i ^ 3 3 X CJ cjj -° X 10 2 2 o i A u fl S "S fj S3 CJ > .tt Xj ? X O Tj cj E S fl CJ CJ ^ — TJ >i CJ CO ?k 2 2 3 g cj cd 3 s |fl 8^ Cd "73 w <D tt a Ifl * CD cd £ Q o ^2 lieu §2 I J UH u. OH OH OH < tf Q 2 2 CQ E,2 fl o ^ Is _ S3 Z E <» .2 o cj +2 to tO *H oora 3 Uu ! S3 d : P E i 3 S fl ° *- .2 i S fn; •a oUffl'C o •S % 2 £t 8 -s 2 2 2 3 n 9-e § < w O St/i a> 3 a- o 2 o 3 a ~s 55 oi oo ^ s± c3 »: g. o 3 if "i J O SU CJ tt ~ ^5 u. -c . = ^ 2 8 •2 „ 2 b 1 oo o CJ S Q ci ? 5 •c CJ So & o "«fl • •—• CC ••— ^ ni « U v o C •c si ° .J 3 O So O ° ° s s S 8 ob ^•a |„ a >» SLs ^ is 2 SJ? I 155 Z3 < £ ci "1| i ° o .2 I"3 Al ? -O fc-CD U w O CAI (D 3 .23 3 fl E 22 s s| § « 8 g § a a A •* fl cj cj cj S 8 vi fl 3 '? •a u e £ 3 O 8 ** Vi "S> fl 'S fl .2 11 11 tu" cu M CL.fl H ^A Or w r. a._ b JJ 3 3 O XJ ¥11 73 i E -a '•3 o '5 ^ -o " CD G E 5 3 CQ |A O O u 2 x " A U 3 O CJ O Xj A g-1 a 73 C fl "" •o .23 2 •° I § S S3 - ° 3 sa « i C/J <U a o • > Vi 2. ^ 2 .tt1 If 2s 'u 5 ^ o O CM O C crt w Yi O -fl, 8 8 g fc 3 g > x „ a •2 u a •g t: ts a .23 S3 g> fc E xj S .3 U W r A g I o 3 3 2 'ca fj f o " .23 J 8-&A E o O cd —^ s CJ — xi A « 2 s 13 S 2 23 S *J" -fc cd O. Xj W w) 8* o •_' >> o 8 E . "£3 CD li 2 SE 1 ?, 8 S g -s £ 3 s S3 § •* -a cfl Vt • — u 3 CD « 5 « 3 u o ifl 2 3 O M u ^ 3 ^ £3 fc-.8.1 & 2 2 a CQ S ca * »al 2 .2, o 03 <D _-i 5 Al r» TO |Ac2 3 Si CQ -5 xf-b-S 73 g •;=; 1 -s g d 2 =|"2 3 S3 la : A O o > ,o 73 " cj § 3 cd fl fl OJ £3 fl 2 o I I 1 . O. t- tu p tu a h .£ 3 S CJ § 2 U CJ 3 S3 0 is E S Jj . E t3 A -2 S O ^ C 4> tU CD fl ill.-&w .S?"8 CD a g^ii 3^3 O m ^ 3 TO t/l ^» c s .2 2 H to TO « "^3 « 6 a _ "OS , 3-5 E 3 Tj .E g cd •= cj Vi C 2 "{= cd (— fc fc — E 3 e o CJ o .5 CJ -a u tc y CU C X CO Cd •— ",3 o ?„ 2 fl s5 s S 3 c -a - -3 I 2 v, 2 2 -o -81 «173 •a a J > S s a > 3 g»<j A o .a o c o j- c o 2 E 2 g 4? H a 5 CD xi tt XJ CQ W •S. § 8-S 1 -fc £? crt C O "O « *S 5 t5 °a C JM ^ A x> 73 fl S JJ 2 l.g-8 a -§ S 8 ^ S TO s«—' fc- -r* *J 4s s 2 -o 2 a o « " s g A 2- c a & 8 g A E 23 1.1 £ I § 8 g S32 g CQ f » •- fl ll I O-fl ,3 — 9-fl "S S •§"53 0 " _ 2 ca fl os * 5 O TO £3 1 CJ fl .2 2 - ca «j, feb&.E BC • fl-i = -2 fi 8 g. 73 •3 3 "JJ 2 'C .E A v ^ M ; E CQ fl fl 3 fl 'I 3 3 A fc 2 . • 11-3 u o fl Xj -3 TO tU c fl 5"° W JU E S .3 CJ tS H ' X) CJ fl CJ a tn 3 * fc ^ CJ fc 3 S ^-fl O cd tn " CJ M 0 CJ iJf »! ^A' o S ? •S CD TO TD fl I a 3 a 2 Xl _ JJ 2' ^ CQ • 2 u* S 3 -s ° a 0 X) tt fl §Sfl S3 S o • W 3 S tU ^ TO C T3 O "O « 2. S-c2'-§ S$ S fl fa o a xi s 0 S ofl I cn B o S 2 S3 m S S fl CJ o Z P CJ fc JU "S CU TO g "H. *o trt 5, — E 2-8 5§> ES 1^ § S3. cfl o 8 s 1 § cj DC 5 2 .23 ^§2xi g ooxj a -e i a S 11 8 •a s § Q s- g § I 2 O-OCL. 0 •£ x S3 - 3 y , 2 0 E • CJ •3 "J •3 M .a «s.s a tu x> CQ >• x> S3fl a " a "8 > Xl ^ 2 0:5 SH - -CJ 3 o a < u ^ _•• o I S o ° fl o 5 fl g. cj cj fl 9" ^ I J* o 23 P „ tt 1 3 * -llf-ISSz'fcu ^ . o _ a 2 c o.i & go <u -^2 a cd 3 3" o 1 on CJ < vi Z ci f Q £ 2 0 A s ^, a o a* 0 2 a S 3 •a « 2 § Ta 1 = ^2 °§ p CQ o CJ td CM" 00 6.1 .2 3 CO £ a «2, ll =i I a _ p a •3 co 00 f| c S '5 .a — i 5.p u c cj , CJ 73 i J3 Ego So 3 CJ o Si 8 OT 3 U C 3 h w cc c3 D. TS 3 * S 3 § 3-3 OT CJ tt > g cj E _ x u x u k n is cj ° I * -I CU CD 3 E > W _ §3 3 CL. WI Ji: si ill CD O ~ 5 - « 3 x OT C =• E r9 5. CJ o CJ OT S E 3 "S §J cj s Sj .2 3 -5 • -o tz CJ CJ OU 9 •r" j_. ccj J= -° E . ]> T. '» CJ JO u = 2 00 a « s S'e :SE>> x cj o CJ [_ c CJ j. 1 i«i 'S B o E CM S 3 X O cj u a 3 Si > co 3 5 s ES-2 8 x | JC 1 13 H C3- § •SS g fffljj ° a > CJ 3 u g-a 2 111 CJ S3 t: 3 £ o a 8 |I c OT S u E •o S 3 cj « CJ fa I O.S2 ° I o eg c |.s| "83 1 wi -a 8 2 00 -CJ CJ fa cj O g JJ. CJ c5 g a CJ O 73 fa C -H a-s„ CJ fa 83 s g cj > w ^ cj X CQ o a o CJ fa . OT 73 CQ cj •a & .2 3 oo s o a o JJ X 4J 2 * O CD E x = E Q fa 3 .2 8 S3 OT CJ C OT CM - ~ •s « o « OQ fa •J* > » a :s -a s 0 .3 3 ra — b e 8 |3 | 1 ! E 3 E x 3 a s Z, 3 -a •S « E 2 P 3 3 nj §••5 1 x S3 2 P SO § a -S t 00 73 CJ i .E 73 fa a E "a u x ca CT'OO CJ >. O. 3 x 2 CJ 3 CJ •3 > to CJ X 2 U T3 OT CQ o fj .5 » S * i E 73 3 fa • 3 fa > 3 1 CO O CJ fa W ™ CJ 1853 .1 g| I Ills'3 I 8-11 51 S'E ll|Dlf=fa 3 s "3 E i ° 3 CJ 73 o " a 3 E a. 73 S 73 8 a cj cj 3JJJJ3 .S « E 2 S3 cu O-cTl >, 3 1 « a s CJ CJ X 3 .1 O co co co '3 CJ E CJ X C CJ CVX 6 2 1 &2 3 8 CQ 00 co b. SgcS . CJ C 73 g 8 -23 S3 s Sj cj 73 u fa p 3 73 Q I S > S fa 2 * I? OT w CJ ! J3 3 fa CO « 3 fa 3 3 ' 3 3 o OT p cj 3 T3 CO 3 >, CJ 3 co seas CJ So E -E E — 2 S 2 o 1^ ll S 3 CL. X ' ^ a „ . fl 8 JE E •r -rSfa cj J£ fl fl fl ° S 8 S 3 a 8 CJ S OT ~° S.E'SS 8J 3 73 O 4) 3 -° , 3 CJ O ; g fa c ' t- fl ; " » 3 -.2 | S ti—i „ T3 O : 5*3 <a I ! ^ S -o H 73 1^75 , a -pja * fa S • TJ O L 2 CJ 3 . a p. 2 •2 jSa .E o • i u a co eg o a s : 73 i 73 OcS S S.73 CO fa1 S n 3 3 sf'8.3 l^fa S allffi fa g 1 73 2 §-3 g .75 CJ CJ S X 73 fl C 00 S ll -O cd 0 o t— fa— 1 -9 u 2 fa x M fl •a S3 > s 00 c s a oofl 3 3 ? fl co ^ CJ *SJ • S •3 g 73 . •—' Wl .2 3 3 ffix '5b3 -g fa 8 3 •£ O co > CJ CJ 00 _ 3 co S cj S fl ° 13 3 E>. It life C fe ci, o 73 _2 • g s Wl > f-OT _g -s Ifl3 3 a a s .2 &fa 3 CO *0 CD « P O ^ C S3 E u 2 fa .3 co 3 o 3 u P .5 Q O 73 O. S 3 -s ^ C fa 0 3 3 81 ril ? o-'ob3 fa S-3 uT u -I 00 w Cci J co" 2 co cd CD « .ts o <e CQ '? 2 3 0 r-° u cj if a .ts 2 o 03 o "? o fl 3 73 g § co -a *J CJ CJ . ._ OT CO —- CO M. *J U « 0.3 ^ « w ci n 3 O — c .=r > in c OQ x s co fa a ffi 00 |2H P S 2 c II wi C 3 2 flj 00 i 1 2 w h is-a CJ S Q S 3 ° fl,10 2 -2 g 2 _ CO ^ i , P. 2 • - <N D Q CJ Jo 2 d, x « P E M-a 13 fl Jl u o - S Jri 2. ^ oo ci ro oo •a * 5 <u — fl 2 8 ca • •S u I :vfa JS < oh i fa : 5 -o 11 Q •£ 1 -a " 8s 2 2 JS1! sj - T= « = o a « S c U o ° 5 _ u fe H r3 -3 2 .5 j5r2 — -S « 3 fl .2 — P a ^ .S w « "8 .2? 8 E 73 S -a •-CJ 2 u « s " B.X * 3 B u o <" 7= -zri OT P 2 -° 73 fl> is p S o ^ = 3 a S S g ^E°3 it: eu • 3 '5- § | w w n u 2 •£ 3 o ' "S a""° 2 > tt 2 73 K 2 73 fe - * >> J 3 CJ X 73 o s 3 a SJ CO £ CJ •Sis 3 € 3 — 75 X o. o J* H c3 fl fl „ •-s « £ ° 73 i £ 3 Q. S fe E .MS O " 73 8'S.i (8 O r u U K ox:0 g „ o 'si g j -a 3 - op .S 73 CO E 73 U JS. 73 % 3 » i OT o> II S g 73 * 3 E g s3 8> O -±3 •O J3 S3 • — fl s 3 o 3 = •8 3 S u SO SSI •I 8-g -2c cd C o •S fl '5 SP -8 .2-'fe cd o O » OT 2 0.3 X — fl * "2 u - J= 73 S3 cd 2 fl J-E. « g c > cu <u 5 CQ o •M% £ 5 is 0 « a " 00 ^ E S fa A 73 2 3 fl JD o ca .E — S 1 i I 3 s a u o 2 3 CJ CO — U -O .5 S? »> 1 § 5 S Ji a* S Ill 's J= • .„ O <u -3 o 2 s " 2 cd OT ^5 •S fl J3 cd t|ll sgfl §: o a u T3 a » i .2 .2 — fe 2 > fl cd > t- i> fe-s 151 B. g I c • ct. 73 u a , •g OT ™ E fe-" cd fl £ o tt « o Si t .fe; 7. 4> OT fa I o < fl a o Cd ^3 S u ,S • 73 S i S 3 E p o a CJ 73 1' O n J3 I Ct I ii 2 cSfafa g| O fa 4> . — 5 001 O O 73 fa < S3 A * 8 _ 73 — C cu 73 1) cd 3 S o c 2 tt fa -E a P & «3 73 .OOfl. jg & > fi fe u <C < o a o > cd 5 s 8 m -S g « E S3 -S H S 2 5-2 8 <5 S3 S .E o O fl 3 CJ — C O O O 73 fa fa -g " C D 1) u O co cj 4> -r-i 'p £ XJ 1 g •3 co C 3 3.2 g 73 j3 8 A fl O fej ° 2 S3 a ja fl |o«| g > -c3 3- 3 c3 OT r7" E B 2 « -5. 1 ^ 1 Si J3 O 3 > 8 E 73 3 i ^> fa <3 M 3 « ,S2 Q 73 OT u 73 §|fa « E cu *c U c 8 a t: §• § as *- 3 W 3 • U 00 8 S Sd B fl 2 c fa fe 2 S 3 « S 2 fl C2^ J3 3 > fa 73 > CJ CJ 2 U CJ I "5 "2 73 I 73 O O ca cj fe * 73 3 Hfl S. 5 13 « .2 3 -a OT aj OT CH " -S 3 IS •s -il 1 111 g fe 2 73 — — < 2 3 s fa o CJ CJ fl -° S3 cn « O 1> o -a > 4_> CA Is e fe C 73 CJ .5 _e 73 « fa OSS 3 fl' S .53 J=> cj S* ea « o x: nx ™ 3 ^ cy«j 'ca •8 I fal.3 S3T| J= S X> fl 5 i -H E. S3 ° o b & J3 CJ 2 (j 3 3 2 c fl a — -a o »2 " S fe OT fej OT fl fa CJ 3 cj £ 73 fa 11 fl 3 2 a 3 •§ E OT j| •3 Q 3 .2 CL, fl > cj OT .11 O ea '•5 "o o u CO Q, 2 2 H JS •5 § ed cu *•§ M77 73 cu 1 S a 5 S 2 is ~ °tS!M ^ 8.S 2 3 85 CQ OT 2 u 73 ^ . . u h u 3 Scf^S S« I « • 3 <« * 3 I  -3 OT 3 «& 3 2-8 8 1^2 Q I 3 73 u c 3 fa £ ' 3 OT = 2-2 2 J= CJ cd OT fa < fa U fa D D fa -J 3 fa fa CM CM X a ^ CM > <u OT 2 3 £ S 3 S. g | feC£ f S CJ > j— I OQ o C .1 >, 2-S > jo o a u «J g 00 E 73 73 fa o 3 3 •5 3 ° .5 oa J, oo o fl CJ 7^ O 3 — > 3 CJ CJ H 3 73 fa hS * >- .a ,P o o g Hfa e 3 CU O , o cw r. « o ^ = .g ( 11^ 5 ed P.3 ' H 00 &H < K ! ^ cd I c3 CO <*H s > o 3 <L> v_. ll12 = < fa o cj O CJ E 3 CJ : O cd cd .OT 3 ed OQ fl Q go *fa cd ^ S 3 5.1 2 E cj CJ O. > , 3 3 E 8 3 vo 3 3 3 O. 3 cxi P Sj cj 3 cj 1 O d. CQ cyo ft. Ci 3 8 CJ Xl •2 o -•e tt -2 3 3 O § g J E P 6 £ 1 2 73 0 ^ ta «fi s 1" 2 & &.E »gN cfaC^-ct'ZfaCJfa. ^ OT cd 8 fa §32 3 00 3 3 3 O S3 2 S» " & 3 " 3 O T cj s c/3 . ! 00 •a 3 O Xj fc fc o o c & S -3 s > CQ s cj fl cd L .2 S «3 III <S is" a g u " td £P o a t SS o 5 u M fl fl xi <g -a fl -s £ o CJ . cj 3 BO-" < 2 i a fl •S ja= •3 o ?! 3 2 2-o 2 cd <u ^2 a CU . CJ Is-. 'I? li •= -5 5 2 a Js =n Xj fl fl wf 'io fl E -E x= >• 11 ° S 3 .2 D S. " o-xi _H X fl Cl, CJ fl CO o CJ 44 3 3 2 <2 '<•> 3 » E S-lfl cd fl cj 3 c 3 CO ?> "3 2 3 O. cj 5" °-' o O §5 tU VI SB S3 .£? -2 i g.* -a s •as e« . tu i> o *5 fl 3 £ * 1 a :s 3 2 > S a 2 " S3 2 3 - fl ? H fl •5 S3 3 Sis a . 3 •S.i 2 CS >• o sgc. g co tt CJ >> M £ X. -S » 13 * 2s >. a a A s 2 o B-2 i P E ' •SQtS. x; „ cj ? 3 S ; fl .a •*-, op : o .o — -'WO 3- <« 3 tt fl g E°2 CA Xi fl "• CJ CJ tt ° S 3 o CJ cj • jj tt u XI 3 I '5! •o fl *3 fl co fl g 3 S «2 § O B03 3 i .tt CJ • 3 JJ 3 % 5 5 Z, e 13 3 E ti 8 &Jj 3 fl g g. 3 CU cd ^ -fc fl ?-J ° 9- S3 8 s S 'S 8 CJ < -3 fl § fl u 8 CJ.I C 00 j| CT\ fl ti £• 3 CU o 8 .2 S- s 1-= a . 2 I IS; S > CJ cb —. vi ja c x: is x o is fl o. cj cd •* 2 3 -j ^ cj H cj « 3 ja S-i sf * O _g = 3 oa S3 . o CJ 2 >^ S xi U 3 *• r U OD „H C TD C T3 . .2 U (j tU — "O w 00 -s - tu cz. tu CJ n X) cd tu ™ .5 <u 3 E .2 3 '3 2fl-S § Si § a 3 S 3 I ,w fl > s % « » 5 QfJ o B.P S.2 fl 3 A 'E1^ fl .5 CJ 3 XJ cl XJ -S A « O 3 2 S a c s1 e-a : CJ wj .5 2 g -tt o r •» CQ P xi K cj xi fffl 3 fl CJ S £3 I CJ 3 .23 TO 5 tU TO _ ^ fl IsS g flllll 1 s ? •if o •a x> u a wi "S 2"5 s '5 8^ 2 2 2- cj « 3 o O * 55 8 S CJ co J< g fc 3 O O Q S O O 3 O. fl fl OJ 3 cj ,*i I * Xj 00 00 S ,a <u 3« *j cj cd CO ig go* •si °- S- 3 E CJ CJ cj « O CJ W o cd fl O fc . «J s2 ' lo <B 3 fl fl s a o 5 2 •o 3 2 1-^ 1 Q S ' M-- E 2 « ii C 8 1 8A S C " 2 =3 co fl 3 ed 3 2 • 2 S ! & S3 « <U Z3 A S=J s Ifl fl cd -3 2 fc -a " <a ™ S * 3 2 > xi 1 I o t3 • S3 -^ 3 S 3 >> cj oj cd XI fl X> CJ § 5 Q o 1* ^! 3 si if 00 c s I li * s •s -I o o 2 > tu tu •s s •S JU o E cd CL> J_? P "O TO , TO 3 2 E 3 fl CJ fl 3 3 cS S'l « 0 =3 w 0 CJ J2 I's S 1 3 <s CJ fc fl fl ed Pa JS u 3 xi S g ti « A 8 t; •^wo-saoo-Sfc^ cj fl CU CJ 3 cj •s2 8 CQ Sfl .2 1 o cu.E 3 ° CO 2 3 s 18»i •S fl 2 E u '5 o .5 3 & .2 £V c — 2 3 c2 fl Cu Ci < 7; S » U O Cu uu Cu 1—J C-i < cu cu .5 tfi* ^i 3 O. •5 3 5 Q Q Q Q ^ I cu' Q Q Q > Ja o fl S3 § Z E III OQ a •o a "A3 fcllcS .2 E > fl CQ o A xi C/3 cu 2 o w JU >H « u ed 1: -7 n H. cj ed fc cj cj ZZ^ZQ 1 " O o ! * ti Z I J 3 U. 1 O CJ co 5 E S o "3 Z 5 <>• ci ti JS •» e . u. &u 43 A A" U 0 Q ^ S "ifl-O CJ <fc A S 3 1—1 CQ CQ uu < 00 d in 00 fe e j3 <8 o 5 » 111-OT Cfe S 8 ° u o fl I*5, § £ 8 1 a'? -5 •5 « « •- fa JO fe O CJ 4) cj cj cj a £ £ £ tt fe 2 o co e o '§ " .2 « 5 fl e 2 c 3 E E o o £ « c OT ii? s j— S3 to e-* 8.3 " " a 1 O. o u u 6 2 e S 3 4> * 73 S3 s .a "8 JH I a rr. CO 73 * 25 » 3 o s u Si -O fa CJ u o 00 2 fl -.E-S B. •3 2 3 g -§ -J •fe <4H CJ « o § u 3 g J= op 8 *fa * E § fl 0 > OT * « 5 3 1 ts I« " ! 8. g 'i § f II3 1 3 .a I £ a 2 3 o « -2, > 2? c •it § s cj „ g "5 TJ g a S3 S Is 0 si s ^ » 2 fe 2-S S--S S2-2 I fe OT 2 e cj CJ -a S o SJ J3 ca E . 2 B o OT. 2 e 73 00 > .S tt 73 3 . c3 R 00 7, 1 cj u 01 73 • CJ JJ OT 3 fa -a =3 S I 1) i> O g- to -5 co v 73 VI U o S- E C O T3 O W CJ e £ OT fl 5 3 8 fa O fa a .E CJ « fe S3 fa I CJ — 3 OT 2 73 € § S CJ cj 73 T3 ~ •- o CO CJ . fe 3 5 CD co fe H it fa I s faf § 2 3 a K r. tt 00 Sj E cj cj — 5 2 a OT «J 73 ^3 5 fa fl 3 S '7! 5 •2SZ£ 7=2-8 g .2 c -5 cj J3 Sa CJ S — 3 ^3 .0 73 t3 CJ ^ 73 O O 3 " .. — "> 73 ^ CJ eo fe tt I •§ 6 .B .a ?5 3 fe CJ S u 2 o > tt CJ «J 8 "3 I S3 '00 ^ 2^-8 £ 0 a CJcaoflco H>3 3fa z£ a I I > o I i fe 8 fe *3 .fe o S 2 3 Q ^3 fe OT 0 3 00 fc ja u 3 g J3 « Si 5 fa fa O. OT fa ?T ^ — H 3 S3 o E CQ CJ OT 3 „ •C co 8 fa ili'iisru-s •i"B«S I fill g 2 » " fa f OT fe cj J3 4> cj fa cci H it (5 6 3 K it a , 5 2.2 E 2 O .3 O n hfi. CJ fe 3 E CO X) CO vi g B 3 co . fa cj 2 00 O.T3 fa ecTI 8 A •I 3 fe 00 CJ fe fl CJ 1 3 cj 8 3 3 fl fl fe OT e P 8 « X a 3 S3 CQ 73 O. O. 00 o s fl j 11 CO c 0 8 00 3 13 S E 3 fe.*1 fa 3 OT fa O 3 CJ 8 O OT .fe E OT JJ 3 3 S. fl s fl C CO CJ a 'S 7. 3 u »• -5 fe 2 3 §• S 00 u <8 r s 3 2 ^ cj 3 fa 2 3 CQ ft £ « 2 s 73 fa g 8 • "S-^l 31 | 3 a a •- 5 I S3 lilt 8 2 cj 3 S OT ^ cq g S Su 3 3 s-s 03 s a fa » . cj OT Q cj .>;'£. S cj 73 3g a 73E 3- JJ fe E fl 3 Id 3 E S CJ 3 2 ll 11' a.2 ' O (L> C/3 TJ .ts 3 fa C/3 3 3 g a a I 73 s * w fe 73 fl Sj a 3 _c fe •O 73 X S3" E O 3 3 2 a . 35 s g I" a g s .=? g^Sfa 3 « « S j° (1) CO 'S U. Q5 S3 <« _ 0 3 ffl 7^ ES 3 E 2 S o fa CO S3 J I OT „ >. fe fe cT .a a o 3 c I S tt 5 00 o - cj 3 CJ 73 fa «3 D 1-1 in 3 CM' CM' D 1— CO vi I53l| E 2 S 0 c CQ fa fe CQ JD E I OT fa a 0 a 8 fe 8 .2 3 § a 3 £ a .2 .S I, g 8 a 1 1 i o-1 I fl .fi 1 H 0.5x>3 CT.fe fa S OT S J3 E S3 — fa > CJ 2 E fa o fa a fe 2 .a e '5 .2 fa 73 O ? fe fe fa CJ si" •• a. o • 5 fc ^ ^ fa o ; » o fl fe. fl •(-• r—1 8 3 'C 3 <N (fe oa CL. u ci 5 S S S CJ 2 g <» ' i 0 i O fa ! u ci E «j OT O . S 73 5 fa ^ 5? < cj fa ci OO s I s -a 1.2 « 3 tt i>CQ TO -J-* „v A g| E J 3 IS J3 fl co tS « fc fc « >, .2 1 u •= o S s a -S °- 3 | 5 CQ fl X cd 8 OJ h c/) I j- TO <U u -5 '£ U ti Ifc u Is Si aHf.| _ •C OOfl fl cj o co a 11J on o. o. TO 3 S Is CQ 8 _ a cj u a cu fl A •i ° TO _ 5 00 a. 3 w on S 3 fc-S . o tt. a 2- 2 « "> 3 x £ . fog 2 a i •* 2 "3 • cd w O cj _F5 co fl 3 -J 3 >, O o S -a -s 3 .tt S cj tt x ° X « III X fc ^fc C ~ 3 O 3 XI X CJ fl X, -fc fc .23=3 J S£ 3 fc fl S ° s fl :§ 2 A 3 oofl xl .S .2, s CJ I • — 3 I E 2 2 3 fl 3 .2 o.2 •a -S -a A 5 2 fl ]Sx> co CQ "S tU "O « § cj JJ a x> t A •» CJ co CJ X CJ o fl co 3 fl 3 >> fc. "3 BJ |-i CO co u X •1-8 CJ S3 O TO a _ S & ffl fl J -S § flea 13 fl •§ co XJ ST fl .2 8 co 3 2 I ss a £ >i C on « 2 . co "g ~a fc a g 3 .2 2 e -ob-o 8| as CJ 5 5" CJ cd s ^ S3 a " a-8?.2gtt|t3 CD TD n-, y cu 2 .5 °--2 s eg 2 & a-s ed CJ s> •— 3 R JJ fl o ,2 JJ 2 go § (a 3 & flCJ_fc3>3fcCJ <flfliocJcoO> a 2 g •o S fl Ifl-S 2^ 3 SJ g'g < .2 3 CQ CJ o 8 fl fl S sl 5? cj fl _ fc o. JJ a 5 11 i 2 c8 " % <u " *i c CJ 5 * .§ I JJ a 3 a ,2 e 3 . CJ O X -5 fc 8 1-5 JJ .a c e go i 8 g Jj a > A* cd Q. ? cj cj JJ 3) Sjfl «> H |S x s -2 00 3 MjJs .3.2 3 fl 2 o tt !e I 2 ! x: H ^, a sl o tt S3 " 3 2 a.s X X o 1 ^ a 2 B <U U O C 3 -O > O o ra a -fc fl cj o. ; .2 fl IS " cj cj S>-° fc 1 3 >, CJ •8 RJS'S-S E Sl 11 3 § I « & 5 , a xi •S2 u o • CA •<-• Is! IP OH fl fl JJ I I* 1*1 a u 2 w. _ 2 = 2 CJ 2 * 13' « tn C T3 (U TO TO T3 . 2 2 ha 2 ill CQ «> •tt „ fl tt cd cj |flfl fl — x? 2 aj K « O ed tu 0- O. B 2 g 2 .W ^ a S ^ § c S3 82 3. Cfc JJ 3 <2 ops •2-l.fc g -S" % S3 E 1 E X fl u .2 fl ? 2 I fc "3 2 3 u 2 •a t2 2 •s fl ? a g -S tl S JJ I'S I 1 E| JJ § i fl CJ fl 3 <fc 3 2 ° <2 If -8 o o 2 <ZI O-fl co CJ , U '5 £ T3 XJ UH O CQ *S TO O fl cd fl tl CJ cj s ° <o 1 cj .2 *; A D a u. E i, u JJ fl fl 00 CU <U IN O v> fc, co 3 O ^ > fl cd cd cj EES JJ * -o g CJ S oj cd xj fl 1-11 I g 8.1 J co 2 x> T> «J •g 2 S •* -3 fl •s 2,1 CJ CJ fl 2 S fl a g 2«f 8^ o 2 g TO *- vi •O tu C "O •S.l 2<2 o 2 o u CQ o c fl a as . * -o 81 fl -g '5 g § 3 tt -S fl a 55 oi e to ojz oi Si J2 3 2 ai di O if di a Q CJ CJ a 3 M CO I fc fc — — 3 OJ CJ fl „, cdoflej£Jfc8BO S co3 c.2-3w3.E=3 £2ofl<-,fl3*'" CJ 2 « CQ o ' 1| 2 <S.2f 2 e a CQ o 'C 12 3 CJ * gs IS S« E CJ OJ CQ c/5 « Sj3 °'1! JJ 3 0>*fc 1 ~~ CM O CJ o ti r-g| " g 2 % J2 cj &1 X3 a dJ on o Z •a J 0 • 8 n 3 o c=Jv5 IS E 3 la it Z 3 p.- o cJ2 3 .2 co fcJ CJ .2 fc CO Q g a „ o 2 xl u 3)2 g . 00 cd cj 3 cj S • 3."tt 'fc J n O -1 fc a. S3 < CJ 3 xi o s 55 o o S •23 -S ^ and D B'S Ecj'S o 3 ° CJ ti 3 . 3 3 N O CJ « ° OUR 2 o fc ^t-fc 3 r9 _ oo > Vi CJ ci 00 State claimed that they were not responsible for the funding of expert witnesses - only the basic tools of defence. The court noted that the defendant was able to secure an expert witness despite the ruling. There was no indication that DeVore wasn't sufficiently qualified or that the defendant was disappointed by his testimony. Summarising that, had court funds been supplied, the outcome would not have been different. Court stated that the BM evidence was permitted to demonstrate the nature of the crime as well as to corroborate the defendant's confession. Interestingly, Johnson stated, in mitigation, that he had been abused by his mother, including being bitten repeatedly. The court stated that the discovery had been correct and the defendant was given opportunity to cross-examine and object to Souviron's testimony. The experts were not questioned about their expertise in this case, nor was the defendant's forensic dentist called to testify against their opinions. The reliability of the bitemark evidence "was sufficient to exclude to a moral certainty every reasonable hypothesis of innocence". Court overruled and affirmed. Court upheld theKD, affirmed the RA and asked for a new sentence hearing on the H. The appeal on the BM was not part of the reversal reasoning. Appeal was overruled on all assignments of error. Appeal court agreed with lower court that the dental evidence should be entered. Yes. Stated that Standridge was refused funds for his own expert. Yes. Defendant admitted to biting and claimed that the introduction of the BM evidence was prejudicial. Yes. Claimed that all of the FDs analysis had not been properly disclosed. Yes Appeal claimed that original court should not have admitted the dental evidence. Glass stated that the highly unusual BM was consistent with the defendant's teeth. Glass also concluded that the appellant had a serious gum infection and an "abundance of calculus" -some of which was found deep in the bite wound. Defence stated that inconsistencies existed between the defendant's teeth and the BM, although he couldn't exclude Stand. Irons stated that the marks on the breast of Donna Phillips were made by the defendant's teeth. Souviron concluded that there was a match between McCrory's teeth and the marks on the deceased. Johnson stated that the bites "had to have been made by teeth identical in all of these characteristics to those I examined on Stinson" Rawson described the evidence as "overwhelming" and concurred with Johnson. d Z d Z d Z d Z P: Glass, R D: DeVore, D P: Irons, F P: Souviron, R P: Johnson, LT P: Rawson R D: FD present but never testified BM on victim's right breast. BMs on left thigh and right breast. One BM over right deltoid muscle. BMs on the breast, abdomen and upper pubic region. Eight in total. X ui x oi 2 X X >n oo 00 SO OO so OO Bobby Joe Standridge v. State of Oklahoma. Court of Criminal Appeals of Oklahoma [L41] State of North Carolina v. Bobby Ray Johnson. Supreme Court of North Carolina. [L42] McCroryv. State. Court of Criminal Appeals of Alabama. [L43] State of Wisconsin, v. Robert Lee Stinson. Court of Appeals of Wisconsin. [L44] •S a 73 E. & 3 OT "i 3 S f- co 1-^ _ 3 3 »1 5 =3 18 13 -§ I lis •c "5 ^ u c u u u n •in fa OT X) CJ cj O 73 3 S Or Is* .-Sag. S58. .1 i .a o S .S3 cj 73 ca cj £ 3 tt c = S o 73 ° tt <a 3 2 - £ ^ Z 2 Oo S t> fe 83 .s * Ji s S o S3 .5 S -S 'S 5 — fe — O C on c C u 2P B 2 o OT o S 73 SI 18 u o I s CQ o. S 3 CO p U O CJ CJ 8T3 c "g cj *3 on 3 o. fe o. C3 OT H" K o 3 •§ C o u 5 &J2 * 8 o. a cd E •£ •a " a S-gfa 8.1 S a g §• g. S a •§ 5 55 S i? E o ct! OT 2 = 11 o > J3 « 21? S 13 s ts 5^ as'* 3 *3 _ fa .2 H 73 -a OT TJ _ 3 CJ D..3 -3 c .2 5 5b73 73 a s CJ .2 'cj1 cj i- o. . 1 s-i o a OT ;0 5 .a 5 -o 3 CJ I Si a g •§ a feS-S "fa § O CJ g o 1 JS S JS S cl s? § .2 E .fe S 3 a 3 o f| „ - • • „ - * '-5 S S I ^<_e>ccJcJcj303cj Da cjfaxis a 3 DOTS o 73 „ 00 O £ •S -s * 3 3 S ^ fe E .2 -3 s « a 2 fa * a,g a? « fa. OU ^11 s on cq 2 -s ca <u t? .s a « > CJ 73 OT cj 3 s n -J .s"<a E 2 §-§2 n V u *U !=fa fa.S -o a "3 2 3* fa 00 3 a -o .s f °^ if PH t/5 c/J *J «2| 1? •« 2- ^ E *CJ CJ OT fa 5 CQ fe CJ fa fa OT V, S »|a » fa 43 3 ,H 'OT .3 o S OT 3 OT 3 . 8.1-SI1 .a a s 3 fa §• •S2 3 ^§ fa E 5 S o °" CJ CJ -rj . u C J "° fa fe CJ *3 fe fa 5 s his | §2 So' o OT CJ fe w "OT .S 5 i Q S » ° 2 e 3 S..2 ™ a 5 O fe< cj ca a ^ If « to c 5? o £ o o« 5 " O 3 OT 8-fa oo S -a s , -S o -o 2 o X) CJ n fe XI CJ OT tt o OT CJ g 8. S3,-^ • ^ WOT OT XI ^ o S ca « CJ a 2 fa x: 3 • fa fa fa 3 & J t»3. .s _ I' rl' 3 fe; fe cj X> 3 XI Xi O O. 3 fe OT - o S ; -s ^ 1. s CJ S 73 00 £-73 „ «> B M fa I 8.| E o 3 S 3 a s a- B S 2 « 73 xi a § a xi fe 3 : 73 2 o OT -g •s M o. " .2 | feb £ B jy o 3 cj .— x; o 73 XI O.C/5 fe 73 CJ o § tj 3 S SOT cS -E -o CJ fe 3 •° s a CJ fa g o Ch K J fa w ^ cj" S" Q U*« 3 CJ Jt* *£ '> 2 a" £ 2 cj fa .fc; ca o. _i 2 O a, oo a: cC DDD 2^ I OT O E ° • .3 fe CJ .2 "Si § > 'fe XI ° 3 3 3 fa 3 S S3 00 \S OT 3 '3 CJ 11 £ I O g^21 2 CN fa X) C3 73 2 • "S 'S JS > ° a fa -i s i D Si 13| d.>=H OH U UH H on 13 ii s 2 111 E s la §• § a t U O &o •g|3 oo is * a jj s P I- B 2 OJ fc OJ 'ob feb o fl i's c cd cu an cu cd tu cu ll <U TD o § . - & ° > fl cd £ : 2 a 5 a •s CJ «5 CJ •o a S ob 1-8 X) E <U W jg ^ 1 11 8 « c C C "1 cu cu cu flfj B. fc 3 -1 CJ OB'S •S t5 s 6 ^ OJ Ui — fj *ltd a * a f% <=• = 2 S o M •2 g t 5. 2 ° o 2 CJ to 13 fc 2 .H 3 "> '5 3 co o 'a § tu T3 Ui — O. B .<u ! fl J3 2 2 2 1? .2 o. x g OJ X cj OJ 3 —1 2 1 > 2 OJ fl 3 2 S 2 oo •c •o C fc . OJ fc 2 -S c E—1 -o </-> -S " T3 III ^ S "H oj fc- C tt E S •a > cd co O j 3JJ S | ? h a 3 xi <J 5uS m S2 ^'S B •SIS •s •§ -O o tu xj 3 "O /-N o c a EMU. o to o *olj aj E 3 o u c ^ ** u u u •-a « sa a , CR a i •§ 2 3 .1 ,0J 2 a ° ° § I-s .§ cj 5 o fl tt 2 < W u-2 8 •5 2 S CQ u g 2 g o OJ > oj a us " OJ fc Is I 8 ' 00 • fl. 3 I. M 1) — o"a - XI a-i tu B cu +3 f! t/1 a « S 2 Hi O •3 i 3 «j c ; 2 I -2 I O O OJ tt-Sel .1 § ° fl » 5 ^2 fl SS £ g S- 5 o X! S 2 2 2 . 2 fc « 711 J OT « -2J 1 ^ 2 § S § 3 fc OJ OJ OJ o fl > xi ^ A 3 M C| 3 a 3 O t/1 vi 0> TJ g .1 xi 2 | -a j2 J E 6 S " P 'S« II O. 2 o a OJ — c 2 i- E 5 . c 1 _j — T3 £ ail's a — cu 5 cu B -o a JJ * s 1 Ifl CL co J_, T3 3 C © ed > XJ w C cj t3 cu wi X> a fl s w _ " 2 A 2 a 2 S- > o> o. S fl M CU 1 = 1. a o ^ 2 a § " 2 E 0 3 5 1a « s ai ° Q -S .2 53 I £j fl u ^ ci j>i "8 OJ oo S o Ss ?'S fl Q OJ fl 2 3 8 o 6> °" Ifl I 3 2 OJ .2 O 2 S S OJ OJ S OJ 3 M fl 5 i ° T=; 2 o S fc- _i A -a n « 3 o < u j? S a a « ,a g« §Hg ^ a <K 13 I & 89 < 1 ™ o tj S p 2 < 3 « S g.! |< J 8 o •§..! ! o XI 112 2 'in ° B a llfl 6 O OJ CQ £ s x> a fo 8 fl E 2 -a 3 h £l§fl S g § ° <u *5 o ° cu s " 'a 2 •o 2 - -•o ~y 2 3 OJ z 3 o IN I-r- u. y co <u "rt cd E 2 fl E xi 3 2 'a -3 5 S S . a o.60 E 2 to OJ - 3 * fcfl.a lip "> tt »« 3 •fc fl OJ I 2 * S fc* OJ o fc. on fl « o c 3 tt "O |5 o •9 B xi < C/5 I 2 s CQ IS OJ a o cu s -5 2 OJ fl .S o ma nop cess ave no CJ -3 A CCJ fc fl la fl '? COU He a 2 •S 2 'C 3 •o * se -0 a 3 OJ 2 2 (2 * -S 8 xi v ta 2 • g 2 P C fc * ? « ca o = - ^ d 3 2 " OJ S OJ CJ g o tS fl > U ? 3 fl co OD 2 oj • — O S ceT 3 cB co c s .2 o OJ o 0- CQ fl CJ fl o i CO fc qj I So 2 a x: 3 ! 3 S-g i OJ fl 3 , fc OJ o fl : fl J= <a o [ fc* <J to X) i -a x; E .1 ! o ? 2 t3 ; ESS > CH CH cu on g ^ M fc! CH' Q Cj" fc" fc | .2 E CH C/0 CH CH' d 5 a « o a fc* X> C3 2 fc 2 o »E2 s g I g CQ >^ > oo M 0J St? . 03 fc I i «J xi 2 a S ob^ 2 O 'C o > -* 3 3 S 00 E— o a CQ 'C < a u X v\ <! tt CU 3 Q 5 8 OJ 2 E tt 2 3 ° il 2 „ « x: ob 5 i~ H°.s g 3 55 CQ o ti cd" E tS 2 o S I Jo £ ^ cu tu 5 r—i £ U C 00 a to oo 2 ci _ & fl co .2 * 2 u a = .2 cj CQ CJ O cj i" ° < w «-c ij a J= o CH OJ M Jj tt 'fc • n. 2"8a gS S ! o 3 1 a 8 & :cj^-. 2 i3 ^ o-S JJ 3 Q e-= OJ jS S a cn H 55 cj d, o ON =3 — E S fa •o 2 3 3 cj o. ca cn tn 4= tt & 3 2 o 11 s-s.*.| "° 2_ H 2 ii -5 •fa iS S S 3 in « o o e3 JC J= —> Ed •3 a 73 XI s •a « -cj .E -3 > .E co u C >. CJ fa ca •« cj E CJ . •o > 3 * e * a a TS a a CJ § & E OH O CJ 73, ^ 5^ 3 43 3. o- ca o S o - fef a «3 s ! u to m 3 * fa 73 O CJ cn S E Is* o. o- « cj o . S •3^2 £ cn cj ca a h o w fe a * § 1-5 a ° £03 .5 § Z. "3 a i..-g T3 >> -43 u £ 73 73 sla 1* O fe 11 • . „ -rj -77; « -5 -a ed cd T3 5 1 a -SP E a 3 73 O > CJ cj CJ u 1.1 u c8 • 00 0 8 ' s a 3 3 73 ^ is'i a a w * S >> g 73 -a .2 •o 8 > > 3 e 2 1 -3 "3 3 3 > fe TS • 00 * fe CJ 3 CJ CJ g sS 73 <n '? 73 U o a* 2 3 CQ * 5 J — Cfe fe — O OOfei CJ a 3 .2 S co O & 3 •a 3 H • a*' 0 = 2 "i ci 3 fa xj .g" o g 3 U 3 CJ 5 73 5 ooll .3 -tt 111.2 C 0073 fe u 'fe CJ "a „ 3 CJ c3 "*3 3 2 ti 73 g a ' . P §a O 5 B .S is la "S| *a 2*8* •e fa 5 .52 § all 3 73 £ ^ , 73 , 73 Sci O -8 fe 8 i u-fa I H 4j S3 .2 1 E ca cj Ji" 1 1 E -a -3 73 CJ 2 S cj CQ cj fefagg_fauJ=j-33fa„, Ufa cj -55 .E l> 0.73 xj 3 o C> obj x-e C" s fe CJ CN eg CJ < o S 73 S- 3 73 3 O 73 a -= 1 '3 « 3 & u CJ 3 E S 73 -a g u 5 s E S 73 E 8 " 2-2 CQ 2 fa S £_ o ca x? >xl0 .s till a fo X? O 5 -1 2 3 2 & 00 3 C" a j2 .a s w 73 . pal S3 Si Q 3 cT CJ CQ fa c So13 a E S • fa a ta ^ ,fafi 73 " 2 o 3 y 5 U CJ 2 g a 73 73 8 S g a .a § 2 '8 >• m S 0.73 00 « 3 8 c •-" U U W] Xj 73 I fe CJ ? ca 43 wi T3£ g 1 a E 3 E ^ a .a & 2 n i> E c O to .8 CJ a CH fa 3 "S cj ca s> E£ CJ fa 73 CB 3 a 73 s 43 a ca x? fa -s cd 0, ? a Q S §•5 22 _ fe CJ fa cj 5 .a « „ 5> s a •e a * 8 S B •O CJ 73 8 O fe —' fe C/3 CJ CJ O £ jS CJ O fa S CJ fa '-5 fa ta o> > ail U fe o •§ c° 2 2 g fa x> ? fa 73 .M 0 a p s ca fa o o §•&! a 3- a | a-»s S a « 2 fa •= •3 E o > £ w u fe 5b cn a ° c » as fa 2 73 -5 o 73 S ; a •§ ! M 3 gI k ^ ll » CQ s * % t3 E 8 $ E fe 73 "E s •° fe _g 73 E A 2 5 cn •s ? a cj xi < 43 0 •s *5 cd w » § «--s ca 8 CA c 8 73 S 3 S fa O "3 fe CJ ca 8 cn K cn '3 « J>„ cci fe 73 00 a 0 > 8 fe co fa 8 8 fe CO cn O 3 S3 3 S 3 2 73 ui S w cn 2 c-2 2 E •a "3 3 -i c&'i 1 u <u 73 fa 1 B - a O cn a Sa-S || CQ > a o c9x S a CQ cj « o 73 a ^ C 2 H o xi o « cn III-?! g §;a! ^ l.fe = a I a g xi s; a •§ < 233 2 I o 5 l.fli.f 43 O S 43 -T s a ^ g; a g 8 p & < < ^ o d. a 8 43 u 2c-o tt fa 3 .2 •* ca il lis cu K o < ci o-fa ja 2 Cu a ,9 ts .9 . fa ecf O 3 ecf tt .2 § a .a O 3 a Q o 3 fa fa -3 3 I cj H « a •E S S o CJ fe 0 aa -. fefa h»Q<3u. •a ^ -g.fe C7 ££2 E t— a u u o 2 3 *S — .2 & tt S ^ 2hU<ti a ao _ .2 2 CJ 2 If 'ill IICJ % I c 5 CU jVJ ra * fl 3 _ trt u O u 2 - • 2 "2 " •§ >> CO I -2 CU oj ^ = tt cj 3 fl O -c <_> S p "tu cu fe. S 2 B 2 2 2 lea g I 2 o CJ 3 > " O u «J U 3 3 H co fc fl O Z 2 11 e3 o V c u I'll ° 3 Q .a § fc •5 2 S3 _ L/, j_ 2 2 i CO o to X! u tt I§ « 8 .2 CJ s l« 2 * 1 s| A S 2 co CQ c 3 ^ O XJ O cj •a -5 ° 0 8 S ~ ed •g S3 1 8 CO u fc fl CJ .is 3 CJ CJ 60 J3 S * g a 2 2 -S 2 fj 3 3 OJ fc fc o •= 3 2 w S .2 -g <— V ed .2 *ej © "™ 2 3 .2 o O- CO fc cj 3 o X; CJ * 3 & ° 3 60 —• CJ 2 ij "S oo cj c: 3 -3 CJ tj 60 fc s •C 2 3 O 3 S 3 o a co CJ co 1 1 O.E tt fc 2 o J3 U fl ^ fl fc o S .2 S fl £ fc 3 fc u ° g 5 2 S fc fl -a S|'35 J, § if a 2 A a 2 S c l"Sfc cj fl O CJ fl 3 S/c2 S". o _ B £ -e „<o .s M I^UflSlS fl c ill Q IV) U '-3 cu cu 2 g •8-2 CJ fc Q. CQ 12 C u S i A i ^ 2 cu u •° g o 3 2 a 'g . 2 •= 2 fl •5 fl .2? 3 2 •r- e C r> Q S fl O fl g ' s a a •3 3 fl > fl CJ S 2 2 CQ fs a a Q • - o A - .a -tr •8 ^ E 2 >>-3 co a x> CJ fc _ fc fl -- — — cj CJ P 3 5 O CJ 60 o 3 •> 2 S ° g s o 2 CJ o fc B 5 x: CJ Q..3 ? fl a A o 83 .2 8 a || § 1 2 2 e E 2 £ 1 8 2* _ •a J. rS ' XJ *3 u "2 •3 o CJ CJ fc o. 2 tS 4j 3 OJ Q CQ S § g CJ CJ ' XI fl -3 =-f3 > 2 3 S 2 5 s£ 8 a „ _ Q fl io xi x: o-fl cj JJ o-Ms a XI g • .2" 8 g B 8 a a fj <2 u o xi g a H CJ _e CJ Cfc o cj 2 fl fl CO O CJ 60 A "8 * cj fl 2 fl fl fc co co u fl _ S Efj-S a c -.2 2 Xj u 60^ CO 111 11 til 13 •c 2 fc » «J CQ S -3 -3 B §3-cj fl •a § g s •S.B fc52 ^ S fc cj CJ fl S fl u 2 2 fc _ x> co gO " O fc fc 3 >,ja CJ — fl UX c io O0JJ 3 CJ O « S3 a 2 2 « •§ 2 o 3 111 2 g1 3 S oo fl Q > <u III IP I M - CQ fcJ 00 ^4 Cu Q 2- o co co .2 3 2 l?fl CJ S3 fl 9 ell S E CQ ? "2 u £ eo u" 1 g-B a .E -e I- - a 2 2 a 2 _ S co fl CJ 'CQ § 2 fc fc _ ffi oo x: fc 3 2 2 8 3 £ u ^ g cd = 2 |_5? ^ oo oo uu S3 fc M 3 i _CJ - .2 „ ^ E S3^ 2 0\ » 9 = J 2 00 fc] OO M Cl OJ OJ 3 Q o s fc CQ fl O 3 'fc tt Q o 3 pj o cj — OJ g CO 2 2 •g oo g. S § > oo u. ci CN ts ^ _ s.= a "So S^l _ " c -? S gfa 3 fa fa tt xi _ X 33 ca O c E fa 3 -2 8 2 § . •alls 8. tt a t 11 > 1 " < o cd o w Q JS O TD cd 5 Sfa 3 'g 73 CJ 3 CJ & CJ fa g xi S S Q fa •3 fa ° u ~ cj a. u ^ ^ 7j e s * _g .s o I  DQ is QJ JJ 3 E 3 tt o «J •o CQ .2 ,. j "3 3 Q § 8 fe 73 2 § 8 _ E £ "a -S tt sis fe ° 73 DQ o o u c •e c fa I S eg 3 E O. S O 3 c/3 ? 73 x: IIQ || 6-i CQ O 73 < ill CJ I >> 3 fa S M , g 2 3 o * <« _*> fa . fa 3 o tV} -H i—i oiHUfe CJ £3 IS 3 3 tt ,_ s IcS 1 § Ii tt 3 c3 ci cj S cj rS 5 O. E E M Ufa S 8*9 S3 5 cj c ii? 2 tt fe CJ cj fe CJ fe § 3 cj | o o b CJ fa fa *fa cj fa *o cd _o 73 E 73 3 a fa cn .Xj Q 3 fa O S._ s> CQ > CJ a 5 = s ^ .2 B |.& CJ .3 fe o.'a u cn 3 cn - O CJ cj X) cn CO S fa 73 fa 3 s o fa o".£ ._ 3 2 o ~ & « 55 fa s 2 S 2 S 8-g _ O. 3 CQ tt fa "a *•» _: *o » JS ja 8 1 S .2 3 £ 73 |«5 c fe2 life's » D &0 CJ </3 CJ oi izi C3" ^ cj IU' O 73 .52 fe cj —. 73 E 3 • a s cd 3 .3 <u 2S 3 •— cj tj > CJ fe O. £ S3 o 3 a ca « OJ 2 CJ •a z 00 ? If* . S * 3 73 « u fa a .2 b2S Q fa CQ E a 73 , •o u o 2 CO .p 1 a»J5| | a § | fa cj e cj 73 <2 3 S fa « > 5 73 73 Q ca Tj? x: cj CJ CJ • 73 fa a S3 •s a • fe o ? fa 2 11 •8 8^ _ o K " S w 3 fa CJ CJ O C — CJ ? (/I E 73 fe CJ 73 fa 2 3 T3 tyj o> cd .11 * 3 > 3 fa O cn 'S Q c3 3 V ^ WH iii to "S o 3 3 1 * _ <ts S > a S on o o g 2 tt °o a e 3 •£ |3| fa 2 ^ fa > TS Ti fa c S o O co ca cn _> .SPS -a „ 8"§-S IS fa § Q ^d O £f fe 73 3 a ? x. O o *n -o E -f 2 g tt S -0 3 o O cj S CJ "o 73 3 00 > .S 5 I-CJ -a '5 P S E >> s l8« cj ca fe cj O. > U 73 S- 3 fa •? U 3 CJ > cj i s 11 fa 73 CJ S3 73 CJ . 73 [2 fj fa U fa 2 cj — •a a 3 -s ^ SS s s It 3 ^ CD TI51 'fe ca « 2 E 3 fa 2 e s « sj a -fe1 •S jl & a "fi i 2 fa § fag/".-:?, cjg-88 a f fa o ||| 9 J-i 73 cj o Q (ii cu EC Q -, •if2*! « 55 "g tu Mi 2 H o g -a s Vi >'OhPa Os - O to B § = ss -° 1J3 2 -° > 8.S to •s ig ca I'T ass cfl o o 2 & rj cu > i s * fc >v Xl 2 g o •S3 u o e e .2 CJ CJ _ a cc tfl tu — S2 tt o a a. O CO O 2 CU - to g "t{ O 3 & >> ° s O 3 <L> £ o ca o ca ca fc B J= o ° 2 u ' <= c 3.-° ' 8 2 82 &-s a g-.§ ° .g-s •o 2 S- s £ _ ti •§ CJ O CJ CJ 3-.JJ. 3-.Si x Sj1 x 5? CJ fc CJ 3 •a & a 3 co -3 ca * cj CJ CJ fc E fl <" a° » H. 2 a & > a a 3 8 -f 2 2 > 3 CJ o 11 Xl o. a 8 a H ca S 'ill CO c S -d § u o - u & £ eg o. a a-gjf 8 g a a ca fl i.s 11 3 is a .j fl o E u 2 S S S 2 TI fc fc tt 2 tt 3 fl cj 5 g- S x> Jj o xj g cj § cj •- Xi C r \ CJ 3 > 3. CJ CJ ca 3 fl 5 3 a. 2 fl 5 -c3 ST OJ tU x: cfl *3 cfl tU - 2 cu "9 e 5 S r-& Il-g2 fc § <i 3 S oT'j J>2 S 8-o _ S3 _ s a O o CJ O = 2 2 2 JJ 2 = a S 3 2 M 2 tt fl B..E E a — P. s a CJ fl .S o to fl o.fl ° a u "8 3 "H -° - fc -* > § P-Q S3 a fl ^ OJ > cj fl -3 > fc ca fl JJ -CJ x> y E 2 £ M !> 2 fl 3 CJ >» cj i OJ •3 2 S c "8 3 3 O S fc I fl .£ & & « 1 3 S 8 ca .S fl fc > G 2 CJ a 3 8 3 3 E CJ tt 3 B fl 3 _j •2 I g a to ,t_ CL> , ~ fl.s1 3 £ ~ ^ cu is co 2 • I 2 8 .2 | C CQ cu o £ rj2 1- cfl r» i3 2 •= 3. - E tu X 1 ~ CD I6 o .5 cfl ^ CU 2 S ^ i _ 2 S a "Hi g a 1. cu cu to -tj o 5 flfl CJ ^J a 2 s fc 3 JJ r s cu cu •3 a "3 2 J: 2 « o 5 5 2 'fc tt ^ ca 8 2 i- 8 to 3 Xj ^ 2 <2 fl a _JJ u 2 CJ M fl S i) 2 ii P 2 8 E a £ fj fc t-< (U •a fl JJ g 1$ - -ll I §1 2 III 8 CU 3 o tt fl fl . CJ fc 2 XJ tu CQ 2 § _ 2 co X> — cj si Q § So fc c CJ « 0 1 al H fl 3" CJ to . fc|fl3fl'fc .a -a a - - § O JJ O fl CJ fl CJ . fc _2 -a 10 -3 32 _.-tsle" JJ J2 ™ o ° .2 -tt fc SJ 2 _ W ^ " 158 •S l.l •a _ CJ x> - D. cj o B a i JliillJaii ^ CJ _ II 3 _ CJ fc fc S fc'Q O C 2 o cu /-\ .O -r- t" ^!"E 111 'S a H 3 eg > 2 25 CQ u c3 O x> X on fl § -^2g«fcfc a 3 u g ||« 5 _ CJ x S g o S >* -a s 22cjO^ h M O « 2 fc 8 S3 2 s g °.fcH > on « cj fc ca „_ " Is 2 oo S 2 a .SP Jfl « O 3 cj -CJ fc E u - cj - 5 fl B.fl eoft Q »: CJ < ti fl •-OJ fc sis fl CQ _ o S3 I CJ t_) g cj 2 S3 cj^1 5 tt a. x v£ 3 « g fj a -a >,fl '? c ^ u cj o cj oS* -*-» iy) (U — £ CU 8 t fj > xl Cj CJ fl fc CO G fl fj 22s cu IS _J "S III i U '5! >> 2 * >. a, 2 S SH 8 . s 11P ? ra a CJ a I * I fj 2 _ — W vi '5b2 'C JJ -G u H fl |2 _o tu ttfl o fj „ .: - " •_; s _ a * u o 2 2 2 fl e o 0222. £2*2 Cj U<M O CJ fl O fl VI fj s 1 cj tt E 2 1-g -•g > -o cd cu cu t= 2 " c fl 3 u 2 2 3 ^ g O Mi1 % o % 3 s -2 w 1>2 g g°tt 2 A o .E _o CU u «Sfcll_'!f > u fl 2 " QJ fc O C cd c - fc o cj ~ " fcH QS-I a 1 ^ O •B * "O w 1 B « cd ^ « g •II 3^ tU T3 to — P 3 _< 2 u I sl ed to C o"< • 'g « JJ fl fl xi cd X) u " M-fl C .2 I}! - Sn 1 !i » E 21s u CQ cd •i c S1 S •= fj •S-S S3 S S3 >. p JJ 5 tt « 2 I 8 •& x>s JJ a 2 » 3 i) S3 2 3 o 3 fl fl ^ fj S 2 _ •g 3 A .2 x u re ii 3 CJ fc O.JJ rrvi cd X) cn CJ i£ fc-< 5 a 2 CO M o 2 g £ fl 2 3™ cj fc _ >• 2 e ^ 2 ?PJS a lis1 5 2-fc. CO s < 2 o fct li >• 1 ed cd cu 11 ' o E E o _ CJ _ > (Z> fci > _• O >> CO fc CO 1 1 C fc S fc —, 1 o _ : cj _ s ° CJ o JS .2 g * _ C cd « a |S ••2 S S Q CO CQ C u (fc 5 o. 2 A — c Cd 3 CJ ,0 O.— < CJ 3 2 a -8 ^ I 0 fl £ v, -g s S S 2 -a > '•§ - £ e .1 s g I § S Ifc > C cu 1 fc E 5 o 2 cd u fl CJ a -•a •§ 1 e „ -a d> ? = 2 Cj CJ o .tt CQ X) •5 fl cb CO CJ >> C XI 0 2 ^ cj CJ — S2 x: fc b 2 _ 8 .» I S •3 1 8 S 2 -3 « CU O O Ti -5 T3 2 1 is s •5 <§ .2 2 u CJ CJ DH S S3 O tU •3 -5 ~ fc O G S «-a -Sfc1 § tu "O cd _. •32*1 2 >fc ^ 3 o fl o CQ _3 fl OB 2 CJ fc xi Q c S3 a i * E e 2 =i o. « O-S. a! Q S o fl » ? e Si fc o o. n o fl . 0 u a g r, --t» o_ fc o ci -"2 2 CQ fc CJ 8 cj x s cj fl fl O. .tt g ni i •1 g fe- g .0 u E a s ts fc fj fl u •g ifl ci_ '> jo1 CQ JJ o u o e a 5 cj al 1 a a CJ |2 on xi .2* ® 2 .2 •= » cj a H •2 «» «§ifc 0 -2 fjl « fc p; a .a j S.Q --^ .t= fc Uu d « a 1 g g g CJ H S , CQ u 2 2 fc c E H cj u fj JJ •o fl 5 .2 fc fc V cd CJ fc OJ) co fl •fc cd co fc > 1 3 n -= I u o cu o o cog CU T3 ° "Out! X3 U cj fc S-2 O ° CJ o 0 2 'E cd CO fc g* -cj'i & T3 C cu o • CJ CJ S 5 fl fl cj CJ C - ,CJ > X & .SP c to JJ fl tt S = cu 2 m a X c CQ cj CJ 2 2 CQ > 2 tt cs -fc 5 CJ M CJ 2-£-x: cd < cd P5 -o O c O Er.tt J E I O O •— >> -—, .fffouop cS^ o ooH in 0*\ 3 73 III .N i si if § 73 cj • eg TS W CJ 3-° "2 loSS "•si : a I 3 00 • — ej ma H • S « 1 CLD £ i s £ ; a a s > S u e , -a a Us'? g * _ 73 cn fe *j it 1 -i oc,S a 73 s u —, H s « 2 S 3 > -c u » o c 2 o '3 fS 73 E C CJ 73 'Z £ E <3 . o <u c 'tt 3 .2 § 73 t3 _ CJ • — > _ 3 8 g 3 O. O 3 Q O- fe cj o o « a s fe-J * 3 §1 1 $ o 2 73 '> l._ o a CL CJ ,0 CJ fa 73 3 « ° .3 •g .2 fe -3 3 * E 55 .3, 8 ^ E fa 2 00 73 5 3 ? fa 6 &feH fe 77 .-S CJ .2 73 I a! cj <u O CJ = X3 IIU X S § P - "5 cci -c b i? 3° l^jj list 8.1 E u g cn * u CJ 73 fa O. cj 73 2 "3 !° U 73 fe 73 O 3 1= g CJ tP .1 a % §•5 0.. 73 CJ ft; CJ J- O •C 0 0» a K fa 3 -E tS ji 3 3 2 " ^ "2 tt " CJ OJ a u « o c — -— 3. a 3 .a .syp "a a 3 T3 > CJ 2 C3- fe cj g cl fe m a s a S3 .3 .ts £ O O ° 3 §f e fe-g D c3 -73 1 -K § 3 3 2 o o > fa .5 0 £ " U fa cj -fa 3 I g 3 -I 5 .2 2 -E 3 > s a I fe-gi|-i>£ SiE gf C/5 fa CJ S fa 73 fe fe o u cj •a * |a cd ^ cj at« s • 73 u o a b o o -3 CO CJ <u E s3 OB 3 CO ed « o .Jj 1 t) cd JZ ^ ej S Mi a u. s.ta a Oi) CJ ° fa • i .2 73 x J) r- . O fa I s 11 -73 CQ E E • fa .2 S 73 , tS E CO 73 fe •a 75 ™ 73 2 S3 E a fa cj 'fe fe <U qj 3 S fa:?-« 5 .•3 CJ s a g O cd o o :•§ aa £ ja 1! "ifa a §iS P "C 73 ? CJ fe •S 2 « 0 "3 3 cj .E g £ <A w *S •s I ^ •SI a. ca . . cj fa 73 CJ fa 2 3 5 " O £ 73 >• ° S CO 73 fe CJ > 32|^J.-ra 3 L « c S CJ * fe- fe 111. fe „ Q Q 2 X) > CJ 73 O I u g ^ cu oa K & 73 3 S 73 -5 cn „. 3 JJ CQ 0 73 co 1 ° a ||a g • 1 s l-Il g73 ^C *> ed § « <u S ? 0 g CQ cu c8 fcj cd i: 5 A ° fa . fe Q 3 a > a S z fa p feT 3 ss a oi ci CM' Q «! 12 1 ^ 1  fa -3 fa1 -a -a -° || CO > xi tS tS cn <J . fa ^ cn ^ b 00 g 75 -3 CQ s a § ji fj is CQ u a cn JJ E * .2 "S) O o lead g s a-s fe CJ CJ CJ L- fe • — CQ a. XJ > < a ° fa « t: 0 00 s H ^ e • Q.fei 2 c - a .f!l| -I'll |fa-E 3 _ x *= a . (- „ a is .2 slip S = o" S • ^ u o q « a *^ 3 ^ u o a S 3 1/3 i .2 > CJ .52 cn • >• 'iia. x a ja < o < o d. 3 ?- o _ fa 00 0 8 o 2 t?±t « O 5 « a fa 0 £ u 0 fa ^ fe* cn o C »i "3 „ 0 » B »" fe X) o 3-r^ CO £_ J < d. E '5. 0* o Q.; •5 a O w O _ cn tt So1 2 cj d OS 8 £ i-a 8 -a 3 i « J, 73 73 • « CJ I ?! .!2 Hi -i || CQ o. a Hi- is <& g ~ 2 s ^ gj g t> Z> <u J3 « S > <L> > t— (JJ '€ g g S o C W to S.afe § o-x; 73 o 2 ° £ £ * 73 -S3 48 l-S 1 •- % 8 = S"2 2.s a s J SEE CL, 1> g73 fe 2 u fe 73 C O m oo '>,•% 2 c > — o cj cj fe > a •a E •§ c „ 2 °»fe ^S3 f S3 fa 3 ,o •2 87373 £ wi 4> fe°sS s. '€ 5 g> ~ I 2 43 o OQ 2 -£Ji ° 1 C 73 VI .t: 3 2 73 S fe .2 X) 2fe •2 & u •a w S2 <*-< 3 o fl j= 73 E CJ CJ CJ 2 B T3 1.1 5 — -y C TO UL O 2 73 ^ - ° " ^ 5 •§ |? O wi _ 2 C ^ H*f i'l r > x; uj u D. > ^ w L. o 5 i <u fe^.H > 3 CJ s fe "co -S tS ^ -c a. o cix 75 73 2 fe Cop 00 ^ G O > To c3 -s -s • 111 .5 00 «J a J 73 2 » 73 S 73 3 - 3 •-Ufa 3" i -a § U S 03 .2 CQ 0. So Oco-St 18 c .11 fa 2 2 o 45 a 73 E .2 § tt § fa o- 3 Jl < s 8 «j a "s HT3 is ca ^ cj tt cj •—: o 73 7. o co a 3 -3 .„ 73 ^ " 13 .73 2 H S ^ IS o co fa fa £ o 73 W cj •K g w s 3 73 gl-s 2 I u 73 «5 3 CD5 .Ti co e c CA « a 2 fe CO fe fa fe 8 fa 2^ fe fe 73 JS VI > O fa fa fe >.4J 3 3 77 3 73 CO > 42 CQ ffi-a s •»'fi .2 .2 a 73 00 0 •s o 8 I g <- IE 33 S 73 U fa ifa fa S <4—1 u o T3 O 2f .1^ 5> o x> cj 5 SJ > 73 H C> fa •3 == OJ CO tj 2 fa 73 '? CQ O _ X) c £ w ea 0 w 8 0,73 S.i -s - §l.s a 0..2 s - g-.a'x. fe 00 cj O fa fa 1 XI c fe fe 73 JS CO 'S CJ fa ii 8 E >. s 2 73 t— S3 1 E S3 « 11 a c2 u •S -S 73 00 < 1 a 3 73 • 18 I a E si. a 1 P7= g I .2 E isZ S, fa C/3 . H • • • • cu Cu J Q Q S E §fa ^ 2 I S CQ ? .2,73 a fa .2 x o S 2 3 •§ .S CJ m fa CJ c— CJ fafa < g fa fa tt a" S 2 3 Q .2 ° fec3c-£fe-E 0 fe : « — 73 CJ .fe .c Tifa £2^0 |o gfe!.2-g ^ 2 § 2 S o o h « H < a cfl tt. r--•3 £ -° tt. " x: 5 "S a a o u *<2fl E o u u 'a a _i > u JJ "S CJ CJ to 3 M a a .2 e e-.i £ co S '> § 2 S3 fl O £ •IS « . c-4> (j (fc o A •->2 " c i > s a > u 2 ^ is a-5 i CQ 2 c " O 3 3 fc a 13 K 'i S -g -5 x> CJ fl fl O •S - tsi •- o U -3 S tt" *S t/i 5 <u «s J, a.& S S * 2 5« fl tt o ° y a 3 cn C O fl It 3* CU "O t s a ^ * g SI a § 2 S 2 § 82 <8 u cu > T3 - a £"° "g; xj § tB 1 ° JJ 2 fl ^ CJ ' = 1 a CJ 3 fc 2 a ° JJ to g • xi 2 A • « c <2 ifl 5 duo S X> 3 .2 -a 60 o JJ 2 S " ._• "• g DO CJ O S3 -5 « •* 2 .a o JJ •a 2 3 3 JJ o O A --<fc O g CJ fl > 3 o S? g in S O CH tt cn *o JJ O 3 fl CJ 2 Cfc !| '§ 81 & op c3 CD CJ fl 2 P. a 2 oo a •a S eal w •med i inal ence ained App affir orig sent rem; ttfl 5 §2 fl to fl <L> tt fl H 2 u tag*. 3 cd 3. g o .22 — —-a CJ 3. fl at E u — 3 CJ P < 8 g 2 a •3 fl . 8 ci, it: o = 2 , -S ^ g M *g ^ a "ET-lb 8 S | I '§ I S-3 o 2 g 3 8 CJ fl to OOfl al i> H a * § « « w co -5 TO T3 ll il 1 s s cj <G •o - I fc o 8 6 &• >, c ° ffS 00 o ' g .2 J .1 g" u S X « » 2 a fc O X tt . oo.S 2 -•s S Jj to 1 - E a g 2 : I a "8 f S 2 E 2 a | fc cj •- fl a = fc 3 fl O cj cj 3 2 CO fc 2 "3 E CJ > (S CJ fl J fl - a x: 2 <3 = o fj E £ fl cd cj _ S fc S * 2's B ca a „ Ills bite tu 2 u •a 2 CJ J= a o 2 i fied Xl E 3 CO cu 2 CU CO cu 2 3 O V in i § § 2 fl § I Cfc T3 E CJ « E fl fc-cd O Cu | -O 5 _ I a Elfc cj C cu 2 2 it ilka s a 2 "S fc 3 o 2 2 § o o 2. 2 3 cj « E" E » a » o -3 -i I i 2 fl ^•8 =d 2 § tt A a . TO —-l CQ 3 tt S 2 5 so £3 a — ^ to LLJ _ 5 a co cj ed - -ti tu fc B 1 cd 3 o 2 E u _ gfll M S 5 c >,* Be" JJ CO CO U 3 -g 1 ^ ii ra ro u -EES g 13 2 o 3 2 fc 3 tt O .tt CT" I 5*1 §S 8 t a ca to « " 1 •» I I "s S -a 2 2 CU CU cf 2 '1 •S -r 3 co -x .23 S 2 S 8 c . o 1)3 CO ( 3 £3. S fl. fl o o fl fc » » o E 2 e S g - cj CJ .2 CQ a. >. > 1 <K fl E S _ E -c CQ o ' 2 i-g fc u ffl x c S -o ° 2 1/1 u „- »J u fc 2fci=2 flc3s2 1|2 2 2 fc! 2 g ? ^•-•ttoacnxiaco^oo < BC O 1 < Vi < < vi O .2 H "S |X! fl fc psi fc- -fc o 0 8 CJ 2 u S^a h Q w O ci . x: g o .2 _ O 2 tt 5 tt tt 3 O a o 2 o fc< 1 2 ^u 0 a 11 8 •? " E I 2-ShhO< CU s 'a tu o. •si?2.! 2 fi 1 0 o.'« m 53 S o o < _ d, 3.5 2 o CM 73 -si — _ S S3 Sfc 55 o ci CU Ct_, Cu J£ «5 o -2-^fc fcJ u co o gli 1^ .S tu CU to wj ._i x= "S fj u * 8 a^fl ^ 1 5 & « Ci CJ to > O 00 ON g a c 8 S .2 ill"*, il J.I is ill 5 S3 53 _o si >*73 •9 3 -g' O fe fe" 2 fc 73 3 fe <J O -o CJ cd c2 o -O fc 11 fx; o « <fe .-s tfe 2 o 3 3 XI • 3 .3 cn .fe J2 §> tt O CJ cj XJ CJ 3 |- 5 a Q < 2 IS 8 til 5 S i &*l • 4> wi e c •O cn CJ cn 73 « CJ > fe £ CJ tt °- 2 3 O cn 3 s" ob >* fe — cj cf O o .= la .2 -o •3 3 3 3 •s 'CcS O fe 3 O. O a fe J£ ™ 'cj cd * fa _ 00 ' c o 111 fe .2 . Cfe —• 0 73 CJ 2 1 e o §3 •a 3 1 X ' fe cn . • fe O O _. r. — -3 3 60 S -3 o 5? « S | s f6 73 T—, wi •SB 5 73 tt " 3 fe O CJ u s XJ 3 >> O O > 73 CJ I'll CU O 4> , a 00 S fe cj •5 73 _ S 3 •2 cj g CJ cj ° 2 3 _ 2 E ° fc ° g cj „ -a 00 Cfe •- cd .3 — cj ^ 3 ^ £ eg „ g cj o .a X7 3 O ' 00 f 3 3 ^ O 3—2 73 3= 3 fe" 2 Tn a -« f £& _ Si.s £ .2 ra Wl O _ Jg Cu^ IcS fe E 3 cj cn fe ~ 00 cj CJ S g cj r3 a cP •5'Si = C8 O 3 fa -™ o CJ o. . £; o CJ CJ 73 - fa O O fe fe? -3 fa O 8 CJ fa g. fe a -a 73 fd CJ 73 fe 2 &>fa 2 c _ O 73 .fe O fe fa SCJOSEE"-O _fe XI O 73 CJ cd 73 Q. CJ cd 73 £ 75 -fe fa O XJ •s s = a tt Q o ra Sll„-3 = S-_2 > C « 73 O g S 73 5 2 g SS S 73 cj S 2 fa = •S fe -a » _ 1 -i E 73 I* cj fa fa i2 fa CJ cn CJ CJ is u *: ao c — ^ _=, cu c c u ra &H B 2l|| § g v- ?• ^3 ^ X> T3 (J crt g S « 00 e "TJ CJ D- CJ .3 < 73 cd 73 fa •i7 e tt E •= 3,5 no ° fc 'fe O cd o CJ fe fa 73 "5 O cn t-r B E td s | s ^ g S ra ^2 wi , p wi I 4J • fe TJ o u Q o XJ CJ cj E 2 >. cS H a .2 CJ a _ 5.2 S3 8-8 » '•H -9 2 > 3 Q CQ cd £ 73 fa « S- 2 3 f-B J a - 2 E fj g Bfi u_ » c2 S * 3 3 £ I 2 fa " D 73 O In XI • t= 3 M •3 cn O 3 .2 CJ X) fe £»»» o 3 -E 3 ° % c&.2 fe U tn JJ < fa 2 S. 2 CQ a S3 -I CJ S «J .2 ifal -7 > O <. cj cn 73 a! a 3 Cfe a ^ > 2 fa CO 8 D.fa pfa s ca 73 Q — u 6 s Ci 5 ai ed Q _ O ra Z E ! u ra i :f _s • fa CQ • ! 1 2 » , -5! « 2 t 73 3 cn S 2 3 : cj s; 3 2 72 73 2 a j2 cj-m 1 G c" wi too 2 fa1'*-& 0 u O. fa a o 73 « s _ § t= 2 & a,g:§ 3 ~ -g B. •* wi u tu Ul H O 73 CQ E fe TO — _ 2 •8 oS _ -0 O 73 ~ CJ C U •E fa •§ ra <s 5 S •2 _ CJ CJ cn -3 cj '5*73 >>c2 W5 XJ CJ » s ~ ~ Sfa Ifa fe 2 |H o a _ •S^SB 3 cn fj a 8-2 S-E>«<2 OJ ra •S Wl " s cd S fa 2 73 O CJ 3 I 2 ra u 73 S III o X •3 > « g _."S Sl73 tS tt 73 £ 3 S E •§ E , I 2. % 2 IICJ 3 g a! X Q M fc o 2 vi 2 al — cn CJ ^1 fa 2 > S O _ 7. -s 2 2 . tt fe 3 C e cn O S3 H S p-o O-o 0 2 * -K I'll: 1 °" 2 o CJ O. fe • fe fe fe ffl o o 2 S3 CQ & CJ Ox! CJ CJ 73 £ 3 O 73 O > 2 Ii ^ 4 s » •- j; •i. _. _ | 2 X > .5 < CJ X I 0 1 i _ • < 1 Vi >. Wl - § 3 M _ -5 73 73 la 0 •5 2 3 cj E S fe cd _ C/3 Vi _ 2 s. e 2 o 5 8 ° a & o g g fe 3 fe a|| U CJ S fe. OQ o S S o S3 CJ tt 00 cd 00 .,- fa fa E <-> u — H a u u fa fe» fe cn fe J tt . 73 fe- 3 cn ^ CB O u ?-|CJ § 2 c2 I ™ 13 1-1 § Z > vi D ci EaS, S _-X! „ c a. 5 o _ ca fi la ° 2. a u -o ^ cj CJ - -s •ft* CO 2 to I'S o S .S2 gl cd «i C5 cfl *J £5 fl * .2 A I - S o fl a. cj c fl 2fi « .5 cj fc fl Cj_, fc £ .1 _ c 2 o x: ^, op cj '5 -5 S CJ -g c fl >• g CJ +3 CJ fc fl CJ fl X) u« w CJ) CJ flcg f- x> w fl SB ~ §• CJ oo 11 co C <U - — -a K cd n> is s If ^ 2 fc ° XI CJ c » x; u |fl 00 > o fc t2 fl 'fc CJ cj -r; hs *• M SJ 8 S I .a 8 _'-\5 O. <L) uj s« oa .s 2 "a ~ to II .2 o fl c I §« S - c — cj £ fl CJ « .> cd -o §* cu = CJ .2 i"f g 2 § tt S ° 2 a « 3 — cj o a. c CJ fc cj H CJ CJ - g* 3 a — cj .2"3 H •S •SI ~ o o 2 £ 2 .2 O cu cj .5 2 -O .3 CJ fl 3 CJ O cr g o 2 _._ "13 fl B JJ o itt 12 ;s ill If I - _ D S a c •§ fl a a s 3> S A I cf I ii o a § B) I j- § t3 a .op a u 3 c fc * S3 S 2 s tt xi S § tt g| Cd TD CJ cu a, cu S s •fi 2 i o is £ 2 "> >>j= "S * "c XI CJ a 5 f 1 O O >> u E oi fc o ttt o -2 o <fc 9 CJ CJ o > fl o > JJ CJ S 2 S fl CJ E fisiP1 fl CJ c '3 2 CU ' cj fl O. " 2 Ofl § 5 > 2 oJ 5. £ .2 ^ CJ <, X) cn G. CJ fc XI 2 2 fl CJ CJ CJ u. c trt CJ cd fl 73 'r _ n.fc cj < aa 5 t\ •— fl cj 2 2 m § E *~ o cj > CJ 8 "A g-S o o cS tti « o xi 73 is CQ e cd fl ts 2 „ •8* 3 fl fc 00 CJ O S •- 2 c 2 " * s? § CJ CU TD os> 2 3 'to f -S X> fl -w ^Sfc s «1 .S .s » s d ofl 2 82 2 •? s S cj cd 2 s S 3 u | 8 A JJ 2 • fl fc CJ X) — S 8 x: cd .g c2 fl a x> c3 Q tu o- a. o . ,« ° • \s-gs ll-as." a s lfc fc I I tt CJ fc % fc I* CJ C/l fl fc fc u I  £ xi 2 , c fl 2 fl 1/1 fl a -s ^ a E g 2 u CJ R K fl 3 .2 12 a J " > 1 ||J5 a 'is? 9-S 1 g -C -O .£ 2 B «x fc CQ o « 5 x _ " tt « •tr o cd cu ? c B B tu a •= g •21.2^ 3 S > f o fc CJ fl cj cn _e cj « tfl *3 2 S 8 c S Ch to O cj fc £ <" c CQ ? CQ _=i tu o c tw tu Q CQ to j— tu CU "±3 —• a tt - I "s fl .S2 e a "i h tt c 3 o S to cj cd S 2 JJ 53 8 2 > — fl .2 »a » > c fl SP u C ifl ifl o u <2 & S t» fl u a s " 2 Q to S 73 fl O S e g_ cnfl ^ a ^ H O. 73 Q 1^ D CL o OS r 73 fc XI 0 w* a to fc sr 11 I (_ _ O 1 fc' Q I to CJ fl n-5-2 != 2 » 5 JZ .3 a . ^ 73 ci CQ fl x: o p S 2 a fl _«-° CQ 5 c _ 12 JIS*, fl.agc'flgccu fflu8ooSo8x " - CO as 1 g-£fc ° < o o CJ CJ & ->fc. SJ tt >^ 3) QT JS 2 o o cd °5 H Oi CJ O tl 111 <s |cJcj<fc g_7 75 ^ SS a -J ci m > > < > O g J I tt CJ o ~ 2 Z 2 o _; CJ a CJ .2-oo «! c» < ci _ Q ° 8 .a 2 _J I 5 2 d o o CN CJ-3 •a •s _ _ ill? 111 2 S « c CJ E iff | E 2 fc > •3 2 CJ sj z % • S3 >-S •3 i ( fl S3 ti « - g E H M XI «3 _ CJ > <J fl 2 •-,•§ -£8 _ s I C U oj _ U 8 fl .2? - t= » > C fc CJ cc O « O > !4 5 « A J§ ° fl _ s _, s g -1 e .1 -2 _> o o. in C eu O > <r s Iff ca > J= id to ob tu £ S-w cd CQ U5E S B-B • 00 I fl fc, E » Xt fl SAI 2 52 & I - & fl 5 5 "5 ii I 8 2 g octa £ O C c£? CJ ca -s a S a cu O CU CO = o E _ a 8 a oa - - 3 . a J 3" Sill ill jy o _ c " S3 &• * S - a ^ E Inste; vas itic tim's . ;> tl -S .2 fl > u cd cj fc* / orthc Dn the •s ^ / orthc Dn the x> ^2 made 1 he test caused bracke teeth. r. "S tU u fc> a O M T3 fc" o s 13 A S u i-; x= IB S •a • C CU cu 2 3 fl o cj "S X 8 CU Jj^ _ 3 O i^.2 2 .2 o. S3 cj fl fl CJ C 3 O § •O cu *0 cu cu cd Q 3 % 8 co O 2 | Id! 2 0 a 8 1- ° •o ,5 T3 J= — -g o lis '3 id — P. i cd i o <•> co s _ cu ?^ ^ CL> ll "3 - E 2 'is i » fc 2 I* I I If, S «2it. •= a fl fc fc re o £ 2 <-> o .2 'S 3 O u CJ CJ fl CJ 2 7, (O ^a. C ~ ed • ^ « 2 2-1 tu cu -<-> cu cd cu to -° j=" iPll •a - > - >. 2 " fl -5,TJ *J _ s if_ 8 tS XJ cn O .ti £ » ti 1)5 S « S 2 2 a S. S ^ . OX) CU « CJ) c ' ^J fc 2 * 2 x .S 'N S "--! 1 .2 c S "2 u i S 2 <S " a I 8 _ «-a c .2 "til " u u w cu 3 2 S3 a u •i 2 2 2 A cj cd fl x: x ^ u c _ "2^2 ° 3 2 c ,8 gs | fc CJ B|.! - e § 8 g S Q cn 2 S d. 2 t * •o C cn I £2. .2 E 2 cd aj fl cj o. ^ • xj S cj >^ 00 f 8 _ * 2 fl eg CJ CJ 3 . j= 00 O fl C CJ S fc ' 2cS |_ _. 2 _?<g fl =5 i? " 2- x O CJ _ D CO I 10 fc CK 3 S U CJ 2 2 n fc 2 s c <u cx £ _= I CQ O T3 3 cC <yi • cc S3 SS fc- a o g A 2 cn fl t3 5 2 2 o oa 3 9 2, o. ? fl « BOO •= 00? JJ «j jo fl g Sfc lb 'o S 3 o » e fc 5 .2 C ed O 2o 1 g-2 . cj fl ! 0. ,— O .fc ,> Q .2 - 1 S3 2 cn fl 3 I 2 § 0 " 2 3-1= 8 3 u s | o H u cj S fc. Q — 3 C 2 cn fl g c« £ 13 'o S fc. 2 fc» 2".2 cj « H ?: < fl co cn C I O cn 3 2 ^flM JJ 3 o J « _ 1 o u 3 _ § C5 (3 0 fj 5 „ fc o \o •1-1 O The only evidence in this case to link the defendant to the victim was the BMs. The prosecution had no semen, blood or other trace evidence. The court stated that they found little consensus on the number of points required for a "positive match". The testimony is "certainly open to defence council to attack the qualifications of the expert, the methods and data used to compare....the factual and logical base of the expert's opinions" The defendant should be open to challenge the reliability of the field". Majority also write "numerous scholarly authorities have criticised the reliability of this method of identifying a suspect". The BM evidence was crucial in case. The prosecution contested that the individual who bit the child was likely to have committed the other acts of physical abuse. Court accepted the BM evidence fully. BM evidence was not contended. The BM evidence was not disputed and the defence did not present an alternate expert witness. Court reversed the original verdict and death penalty. A new trial was advised. The appeal was based on issues other than the BM testimony which was accepted. Court overruled all the defendants arguments. Appeal based on other grounds. Appeal was based, mainly, on the ineffectivenes s of trial counsel. The Appeal court affirmed the original verdict. Yes. Appeal that the DA stated that BM evidence was as certain and reliable as fingerprint evidence. Stated that West's testimony was constitutionally unreliable. Stated that the dental evidence was the product of an illegal arrest and seizure of his dentures. Yes. Claimed that the BM evidence was being entered as proof of previous acts and therefore not admissible. No. No. West stated that he had been able to match casts (and a partial denture) from the defendant to BMs on the victim's body. Described as a "positive match". P FD examined casts from Keiser, the mother, the babysitter and the grandmother. Stated that Keiser was the most likely biter. D FD questioned the conclusions and accuracy of the testimony. FD stated that the BM were caused by the defendant. Kenney stated that one of the bite marks was consistent with the defendant's teeth. No. No. No. P: West, M P: UK D: UK P: West, M P: Kenney, J BMs on victim's right breast, between neck& shoulder, back of the right arm. BMon back of child BMs on the decedents shoulder, wrist and forearm. BMs on hand and arm. X el CA X _ o oo ON oo OS Eddie Lee Howard, JR v. State of Mississippi. Supreme Court of Mississippi. [L98] State of Washington v. Scott Kiser. Court of Appeals of Washington, Division One. [L99] Leonard James Walters v. State of Mississippi. Supreme Court of Mississippi. fLlOOl The People of The State of Illinois v. Steven Steward. Appellate Court of Illinois, First District, Second Division. [LlOl] CN O CN c3 o ^ C P CO _* S —. * E -S H § i-. ^5 to 0° 2-fj <s fj 5 v> S fe tE »2 5 — e o lu s-s-s i — s 2 |s -a T3 <+-<_£><-> c ° 3 » § a-SS fj' fj o jlfj O fe 5 u •a >• 3 » S " fj ffi •g 'O T3 ca £ • I 3 a 1 •S C IS § " o fill i fj o -5 S3 •O k- ^ O w -n i >T> S I fJ -a 3 S « H o o —'5 uss5 o "€ VI O CJ 3 fe 8 1 s « i § §) o fe ° 3 fe „ O 3 -O — > C fa fa cj o. o 3 .5 3 -cj fj o "a M & _e 3- cj fj il 3 a. 3 „• a fj •s -a _ « fe fj 8 -2 § S „ oS88g'§aE 3 |ll •o S ca a o B 8" — 3 5"! —1 tt 3 13 '3 <-> v. I 8 * i >» R o > m v, — a 3 a„- CJ TS c2 >>2 ^ -S « Q o. S •> o fj I § 11 .3 cj fj S fj S •If I c2 " co fj -5 !>_ i fj fe CJ CJ fe " cn •a v. ! E i o ! -M L 'r ^ -3 c* IIS 5b ca CO '5 fj s oa fi e CJ c? 3 JJ vi 3 2 2 E E oa fj .2 a fJ E •8 § 3 CJ ra oj CJ J> cj cj aa fi si* ^ 3 fe 1> € u -s *S a fj s -"8 § ^ 3 1 fj Ji •o « 2 g <s VI CJ (j 3 2 — 2 -s « •— VI > > o « 3 CXJS 111!. ra \s § I E ts a -5 o > >^ CO X5 C •ti « o u to O " •§ & J ---fell um 5 s cu O 15 §^ 2. 3 ov 5 2 .2 — -5 o >• < §• 8 fa m % Ilea o -a's 3 c" O _, JS r/S •— fe CN S vi 30 3 CJ Vi 3 E 0 * CQ ,2 3 3 fe «J CJ vi C u 2 • — CJ fell a 3 5 -2 8 T5 > 3 u u «j . Cc, fe •S 3 ° — -S .a CJ fe J= fe r° ,_ 1 fc O 2 D-fe? ? _-3 _ 3 a CJ| cd & 3 ™ o .2 §.E a ill •al I _ E ||| „, > c u > o 0.0 o X =3 « oj ra to 3 I -o-B I CJ R CJ 73 ^ - 2 srs 2 ~ a 1 ~ - fj n ;* .2 ^ 3»fe ii <i£ s C -2 . cj u co E § g S VI 2 VI fe _-Si___SfJc5fJ_ Xi I a I If. j_, u 2 oa « fj fi •o ~ 2 u S W to O Ii U 3 J O ^fe fe S 8 S *- rti o •a.fe VI CJ 0 3 2 P -o • - • 3 a fj o E o o 8 2 jj • S."go « -s " 5 "3 -1 u J 2- u I I 3 * CJ flj £ fi 2 I •§ CQ fi S •S"S_ 0 "O -3 3 _ E fe *0 "vi o fi 1 CJ s 'fe .S CJ 3 fe fe cj s" a & fa o CJ XJ 3 oa § _ _ a - < an &r> s fa 3 2 fe o 2 CJ 3 fe vi i'ti > !> O O O« IZ c fe 8 2 tyj > cz> Z < Ci o KEY YR Year of opinion publication CT Crime type B Burglary AR Aggravated Rape FD Forensic Dentist ATR Attempted Rape BM(s) Bitemark(s) AN Arson P Prosecution / State expert RO Robbery D Defence expert SA Sexual Assault SB Sexual Battery AS Aggravated Assault H Murder A Assault AG Aggravated Burglary CA Child Abuse AT Aggravated Battery KD Kidnapping UK Unknown AK Aggravated Kidnapping AH Attempted Murder MD Medical Doctor AB Armed Burglary RA Rape LEGAL REFERENCES The references are presented in a format that can be directly accessed by the LEXSEE™ feature of LEXIS™. Information on obtaining full transcripts for these cases can be found at www.lexis.com. [Ll] Doyle v. State, 159 Texas, C.R.310, 263 S.W.2d 779 (Jan. 20, 1954) [L2] People v. Johnson, 8 111. App.3d 457, 289 N.E.2d 772 (Nov. 16, 1972) [L3] People v. Marx, 54 Cal. App.3d 100, 126 Cal. Rptr. 350 (Dec. 29, 1975) [L4] People v. Milone, 43 111. App.3d 385, 356 N.E.2d 531 (Apr.7 1976) [L5] State v. Routh, 30 Or. App.3d 901, 568 P.2d 704 (Sep. 12, 1977) 204 [L6] Niehaus v. State, 265 Ind. 655, 359 N.E.2d 513 (Jan 25, 1977) [L7] People v. Watson, 75 Cal. App.3d 384, 142 Cal. Rptr. 134 (Nov. 28, 1977) [L8] People v. Slone, 76 Cal. App.3d 611, 143 Cal. Rptr 61 (Jan. 6 1978) [L9] State v. Garrison, 120 Ariz. 255, 585 P.2d 563 (Sept. 20 1978) [L10] State v. Howe, 136 Vt. 53, 386 A.2d 1125 (Mar 15, 1978) [Ll 1] State v. Jones, 273 S.C. 723, 259 S.E. 2d 120 (Oct. 11 1979) [L12] Deutscher v. State, 95 Nov. 669, 601 P.2d 407 (Oct. 18 1979) [L13] State v. Peoples, 227 Kan. 127, 60S P.2d 135 (Jan. 19, 1980) [L14] State v. Kleypas, 602 S.W.2d 863 (Mo. App.) (July. 10, 1980) [L15] State v. Sager, 600 S.W.2d 541 (Mo. App.) (May 5, 1980) [L16] People v. Geer, 624, S.W.2d 143; (Mo. App.) (Sep. 22, 1981) [L17] People v. Middleton, 54 N.Y.2d 42, 429 N.E.2d 100 (Oct. 27, 1981) [L18] State v. Temple, 302 NCI., 273 S.E.2d 273 (Jan. 6, 1981) [L19] State v. Green, 305 N.C. 463, 290 S.E.2d 625 (May 4, 1982) [L20] Bludsworth v. State, 98 Nev. 289, 646 P.2d 558 (June 18, 1982) [L21] State v. Turner, 633 S.W.2d 421 (Mo. App) (Mar. 2 1982) [L22] United States v. Martin, 13 M.J. 66 (CMA 1982) (Apr. 19, 1982) [L23] Kennedy v. State, 640 P.2d 971 (Oklahoma) (Feb. 3, 1982) [L24] People v. Queen, 108 111. App.3d 1088, 440 N.E.2d 126 (July 13, 1982) [L25] Aguilar v. State, 98 Nev. 18, 639 P.2d 533 (Jan. 28, 1982) [L26] Commonwealth v. Graves, 310 Pa. Super 184; 456 A.2d 561 (Feb. 4, 1983) [L27] State v. Sapsford, 22 Ohio App.3d 1 (Nov. 9, 1983) [L28] State v. Stokes, 433 So.2d, 29 (La. 1983) (May 23,1983) [L29] People v. Bethune, 484 N.Y.S. 2d 577. 105 A.D.2d 262 (Dec. 31, 1984) [L30] People v. Smith, 63 N.Y.2d 41, 468 N.E.2d 879 (July 2,1984) [L31] Bradford v. State, 460 So.2d 926 (Fla. App. 2d Dist. 1984) (Nov. 30, 1984) [L32] Bundy v. State, 455 So.2d 330 (Florida Sup. Ct.) (June 21, 1984) [L33] State v. Asherman, 193 Conn. 695, 478 A.2d 227 (July 17, 1984) [L34] State v. Adams, A.2d 218 (R.I. 1981) (Aug. 21, 1984) [L35] People v. Williams, 128 111. App.3d 384, 470 N.E.2d 1140 (Oct. 22, 1984) [L36] Smith v. State, 253 Ga. 536, 322 S.E.2d 492 (Nov. 16,1984) 205 [L37] State v. Carter, 74 N.C.App 437, 328 S.E.2d 607 (May 7, 1985) [L38] State v. Dickson, 691 S.W.2d 334 (Mo. App. 1985) (April 2,1985) [L39] Tuggle v. Commonwealth, 230 Va. 99, 334 S.E.2d 838 (Sept. 6, 1985) [L40] State v. Ortiz, 198 Conn. 220, 502 A.2d 40Q (Dec. 31, 1985) [L41] Standridge v. State, 701 P.2d 761 (Okl. Cr. 1985) (June 6. 1985) [L42] State v. Johnson, 317 N.C. 343, 346 S.E.2d 596 (Aug. 12, 1986) [L43] McCrory v. State, 505 So.2d 1272 (Ala. Cr. App) (Dec. 9, 1986) [L44] State v. Stinson, 134 Wis. 2d 224, 397 N.W.2d 136 (Oct. 28, 1986) [L45] People v. Pante, 147 111. App.3d 1039, 498 N.E.2d 889 (Oct. 3. 1986) [L46] Commonwealth v. Cifizzari, 397 Mass. 560, 492 N.E.2d 357 (May 14, 1986) [L47] State v. Bingham, 105 Wash. 2d 820, 719 P.2d 109 (Wash. 1986) (May 15, 1986) [L48] Wade v. State, 490 N.E.2d 1097 (Ind. 1986) (April 3, 1986) [L49] People v. Walkey, 177 Cal. App. 3d 268, 223 Cal. Rptr. 132 (Cal. App., 4th Dist) (Jan. 1986) [L50] Jackson v. State, 511 So.2d 1047 (Fla. App.) (Aug. 7, 1987) [L51] People v. Dace, 153 111. App. 3d 891, 506 N.E.2d 280 (Apr. 3 1987) [L52] State v. Kendrick, Al Wash. App. 620, 736 P.2d 1079 (May 11, 1987) [L53] Handley v. State, 515 So.2d 121, Court of Criminal Appeal of Alabama (June 30, 1987) [L54] People v. Wachal, 156 111. App. 3d 331, 509 N.E.2d 648 (May 29,1987) [L55] State v. Vital, 505 So.2d 1006 (La. App.) (Apr. 9,1987) [L56] Marquez v. State, 725 S.W.2d 217 (Tex. Cr. App.) (Jan. 14, 1987) [L57] Harwardv. Commonwealth, 5 Va. App. 468, 364 S.E.2d 511 (Jan. 19,1988) [L58] Mitchell v. State, 527, So.2d 179 (Fla. Sup. Ct) (May 19, 1988) [L59] State v. Pierce, Slip opinion not designated for publication, Supreme Court of Kansas [L60] DuBoise v. State, 520 So.2d 260 (Fla. Sup. Ct) (Feb 4, 1988) [L61] State v. Armstrong, 369 S.E.2d 870 (W.Va.) [L62] Bromley v. State, 380 S.E.2d 694 (Ga. 1989) (June 30, 1989) [L63] United States v. Sergeant Rickey J. Covington. ACM 27337 [L64] People v. Marsh, 441 N.W.2d 33 (Mich. App. 1989) (May 15,1989) [L65] Chaney v. State, 775 S.W.2d 722 (Texas App. Dallas) (July 5, 1989) 206 [L66] State v. Richards, 166 Ariz. 576, 804 P.2d 109 (Aug. 7, 1990) [L67] People v. Calabro, 555 N.Y.S2d 321, 161 A.D.2d 375 (May 15, 1990) [L68] Spence v. State, 795 S.W.2d 743 (Tex. Crim. App.) (June 13, 1990) [L69] Litaker v. State, 784 S.W.2d 739 (Tex. App.) (Feb. 21, 1990) [L70] Commonwealth v. Henry, 524 Pa, 135, 569 A.2d9 929 (Feb. 8, 1990) [L71] State v. Moen, 110 Ore. App. 372; 822 P.2d 762 (Dec. 18, 1991) [L72] Salazar v. State, Slip opinion (Tex. App. Houston) (Jan, 16, 1991) [L73] Wilhoit v. State, 809 P.2d 1322 (Ct. of Crim. App. Of Okla) (April 16,1991) [L74] Washington v. State, 863 P.2d 673; 1992 Okla. Crim. App. [L75] State v. Burgos, 53 111. 2d 218; 606 N.E.2d 1201 (Nov 19, 1992) [L76] Harris v. State, Slip opinion (Arkansas App.) (Nov 18, 1992) [L77] State v. Lyons,124 Ore. App. 598; 863 P.2d 1303; 1993 Ore. App. (Nov. 30,1992) [L78] Davis v. State, 611 So. 2d 906 (Miss. Sup. Ct.) (Dec. 17, 1992) [L79] Williams v. State, 829 S.W.2d 216, (Tex. Crim. App. En Blanc) (April 15, 1992) [L80] People v. Holmes. 234 111. App. 3d 931, 601 N.E. 2d 985 (Sept. 8, 1992) [L81] State v. Hill, 64 Ohio St. 3d 313 (Dec, 11, 1992) [L82] Verdict v. State, 315 Ark. 436, 868 S.W.2d 443 (Dec. 20, 1992) [L83] State v. Welburn, Slip opinion (Ohio App.) (Nov. 17, 1993) [L84] People v. Gallo, 260 111. App. 3d 1032, 632 N.E.2d 99 (Mar. 18,1994) [L85] Harrison v. State, 635 So.2d 894 (Miss. Sup. Ct.) (Apr. 14, 1994) [L86] Kinney v. State, 315 Ark.481, 868 S.W.2d 463 (Jan. 10,1994) [L87] State v. Hodgson, 512 N.W.2d 95 (Minn. Sup. Ct.) (Feb. 11, 1994) [L88] State v. Cazes, 875 S. W.2d253;Tenn (Feb. 14,1994) [L89] State v. Noguera, 512 U.S. 1253; 114 S.Ct. 2780; (Feb. 20, 1994) [L90] People v. Brown, 162 Misc. 2d 555, 618 N.Y.S.2d 188 (N.Y. Co. Ct.) (Oct. 6,1994) [L91] State v. Warness, 11 Wash. App. 636, 893 P.2d 665 (May 1, 1995) [L92] State v. Krone, 182 Ariz. 319, 897 P.2d 621 (Ariz. Sup. Ct) (June 22, 1995) [L93] Naples v. State, 666 So. 2d 763 (1995) [L94] State v. Payne, 282 111. App. 3d 307; 667 N.E.2d 643; 199 111. App. [L95] State v. Shaw, 278 111. App. 3d 939; 664 N.E.2d 97; 199 111. App. [L96] Rios v. State, Unpublished opinion. 04-96-00375-CR 207 [L97] Banks v. State, 725 So. 2d 711; (1997) [L98] Howard v. State, 697 So. 2d 415; (1997) [L99] State v. Kiser, 87 Wash. App. 126; 940 P.2d 308; (1997) [L100] Walters v. State, 720 So.2d 856; (1998) [LlOl] State v. Steward, 179 ///. 2d611; 705 N.E.2d447; (1998) [L102] Brewer v. State, 725 So.2d 106; (19978) [L103] State v. Fortin, 318 N.J. Super. 577; 724 A.2d 818; (1999) APPENDIX C CASE MATERIALS 209 APPENDIX C This appendix contains an example of a bitemark overlay and a life-sized bitemark photograph used in this study. Chapter 3 describes the production of these materials. 210 Spare sheet for examination of overlay. II£ 213 APPENDIX D DATA 214 SECTION 1 STUDY ONE DATA 215 EXAMINER 1 - ABFOSTD1 CASE NUMBER FIRST EXAMINATION SECOND EXAMINATION SUSPECT A SUSPECT B SUSPECT A SUSPECT B 1 Positive Excluded Positive Excluded 2 Excluded Positive Excluded Positive 3 Excluded Positive Positive Excluded 4 Positive Positive Excluded Positive 5 Excluded Positive Positive Excluded 6 Excluded Positive Positive Excluded 7 Positive Excluded Positive Excluded 8 Excluded Positive Excluded Positive 9 Positive Excluded Positive Excluded 10 Excluded Positive Excluded Positive Contingency Tables Forced Decision Model - First Examination Forced Decision Model - Second Examination Gold Standard - Known Positive Gold Standard - Known Positive Pos. Excluded Total Pos. Excluded Total Overlay Positive 7 4 11 Overlay Positive 7 3 10 Result Excluded 1 8 9 Result Excluded 1 9 10 Total 8 12 20 Total 8 12 20 Statistical Analysis of First Examination Data Sensitivity % Specificity % PPV % NPV % PREV % Accuracy % TD FPR % FNR % PTP+ % PTP-% 87.5 66.6 63.6 88.9 40 75 20 33.3 12.5 63.4 11.1 PA% PA+% PA-% Kappa SE 75 74 76 0.51 0.214 Statistical Analysis of Second Examination Data Sensitivity % Specificity % PPV % NPV % PREV % Accuracy % TD FPR % FNR % PTP+ % PTP-% 87.5 75.0 70 90 40 80 20 25.0 12.5 70 10 PA% PA+% PA-% Kappa SE 80 78 82 0.60 0.219 Comparison Between Examination 1 and 2 Exam 1 Kappa Results From Intra-Examiner Data Exam 2 Pos. Excluded Total Kappa SE P PA% Positive 7 3 10 0.30 0.222 0.089 65 Excluded 4 6 10 Total 11 9 20 216 EXAMINER 2 - ABFOSTD1 CASE NUMBER FIRST EXAMINATION SECOND EXAMINATION SUSPECT A SUSPECT B SUSPECT A SUSPECT B 1 • Positive Excluded Positive Excluded 2 Excluded Positive Excluded Positive 3 Positive Excluded Excluded Positive 4 Excluded Excluded Excluded Excluded 5 Excluded Positive Excluded Excluded 6 Positive Excluded Positive Excluded 7 Positive Excluded Positive Excluded 8 Excluded Positive Positive Excluded 9 Positive Excluded Positive Excluded 10 Excluded Positive Excluded Excluded Contingency Tables Forced Decision Model - First Examination Forced Decision Model - Second Examination Gold Standard - Known Positive Gold Standard - Known Positive Pos. Excluded Total Pos. Excluded Total Overlay Positive 8 1 9 Overlay Positive 5 2 7 Result Excluded 0 11 11 Result Excluded 3 10 13 Total 8 12 20 Total 8 12 20 Statistical Analysis of First Examination Data Sensitivity % Specificity % PPV % NPV % PREV % Accuracy % TD FPR % FNR % PTP+ % PTP-% 100 91.7 88.9 100 40 95 20 8.3 0 55.9 0 PA% PA+% PA-% Kappa SE 95 94 96 0.90 0.222 Statistical Analysis of Second Examination Data Sensitivity % Specificity % PPV % NPV % PREV % Accuracy % TD FPR % FNR % PTP+ % PTP-% 62.5 83.3 71.4 76.9 40 75 20 16.7 37.5 71.4 23.1 PA% PA+% PA-% Kappa SE 75 67 80 0.47 0.222 Comparison Between Examination 1 and 2 Exam 1 Kappa Results From Intra-Examiner Data Pos. Excluded Total Kappa SE P PA% Exam 2 Positive 5 2 7 0.38 0.219 0.041 70 Excluded 4 9 13 Total 9 11 20 217 EXAMINER 3 - ABFOSTD1 CASE NUMBER FIRST EXAMINATION SECOND EXAMINATION SUSPECT A SUSPECT B SUSPECT A SUSPECT B 1 Positive Excluded Positive Excluded 2 Excluded Positive Excluded Positive 3 Excluded Positive Excluded Positive 4 Excluded Excluded Excluded Excluded 5 Excluded Positive Excluded Positive 6 Positive Excluded Positive Excluded 7 Positive Excluded Positive Excluded 8 Excluded Positive Excluded Positive 9 Positive Excluded Positive Excluded 10 Excluded Positive Excluded Positive Contingency Tables Forced Decision Model - First Examination Forced Decision Model - Second Examination Gold Standard - Known Positive Gold Standard - Known Positive Pos. Excluded Total Pos. Excluded Total Overlay Positive 8 1 9 Overlay Positive 8 1 9 Result Excluded 0 11 11 Result Excluded 0 11 11 Total 8 12 20 Total 8 12 20 Statistical Analysis of First Examination Data Sensitivity % Specificity % PPV % NPV % PREV % Accuracy % TD FPR % FNR % PTP+ % PTP-% 100 91.7 88.9 100 40 95 20 8.3 0 88.9 0 PA% PA+% PA-% Kappa SE 95 94 96 0.90 0.222 Statistical Analysis of Second Examination Data Sensitivity % Specificity % PPV % NPV % PREV % Accuracy % I'D FPR % FNR % PTP+ % PTP-% 100 91.7 88.9 100 40 95 20 8.3 0 88.9 0 PA% PA+% PA-% Kappa SE 95 94 96 0.90 0.222 Comparison Between Examination 1 and 2 Exam 1 Kappa Results From Intra-Examiner Data Pos. Excluded Total Kappa SE P PA% Exam 2 Positive 9 0 9 1.00 0.224 0.000 100 Excluded 0 11 11 Total 9 11 20 218 EXAMINER 4 - ABFOSTD1 CASE NUMBER FIRST EXAMINATION SECOND EXAMINATION SUSPECT A SUSPECT B SUSPECT A SUSPECT B 1 Positive Excluded Positive Excluded 2 Excluded Positive Excluded Positive 3 Excluded Excluded Excluded Excluded 4 Excluded Excluded Excluded Excluded 5 Excluded Excluded Excluded Excluded 6 Positive Excluded Positive Excluded 7 Excluded Excluded Excluded Excluded 8 Excluded Excluded Excluded Excluded 9 Excluded Excluded Excluded Excluded 10 Excluded Excluded Excluded Excluded Contingency Tables Forced Decision Model - First Examination Forced Decision Model - Second Examination Gold Standard - Known Positive Gold Standard - Known Positive Pos. Excluded Total Pos. Excluded Total Overlay Positive 3 0 3 Overlay Positive 3 0 3 Result Excluded 5 12 14 Result Excluded 5 12 14 Total 8 12 20 Total 8 12 20 Statistical Analysis of First Examination Data Sensitivity % Specificity % PPV % NPV % PREV % Accuracy % TD FPR % FNR % PTP+ % PTP-% 37.5 100 100 70.6 40 75 20 0 62.5 100 29.4 PA% PA+% PA-% Kappa SE 75 55 83 0.42 0.182 Statistical Analysis of Second Examination Data Sensitivity % Specificity % PPV % NPV % PREV % Accuracy % I'D FPR % FNR % PTP+ % PTP-% 37.5 100 100 70.6 40 75 20 0 62.5 100 29.4 PA% PA+% PA-% Kappa SE 75 55 83 0.42 0.182 Comparison Between Examination 1 and 2 Exam 1 Kappa Results From Intra-Examiner Data Pos. Excluded Total Kappa SE P PA% Exam 2 Positive 3 0 3 1.00 0.224 0.000 100 Excluded 0 17 17 Total 3 17 20 219 EXAMINER 5 - ABFOSTD1 CASE NUMBER FIRST EXAMINATION SECOND EXAMINATION SUSPECT A SUSPECT B SUSPECT A SUSPECT B 1 Positive Excluded Positive Excluded 2 Excluded Positive Excluded Positive 3 Excluded Excluded Excluded Excluded 4 Excluded Excluded Excluded Excluded 5 Excluded Excluded Excluded Excluded 6 Positive Excluded Positive Excluded 7 Positive Excluded Positive Excluded 8 Excluded Positive Positive Excluded 9 Excluded Positive Positive Excluded 10 Excluded Excluded Excluded Excluded Contingency Tables Forced Decision Model - First Examination Forced Decision Model - Second Examination Gold Standard - Known Positive Gold Standard - Known Positive Pos. Excluded Total • Pos. Excluded Total Overlay Positive 5 1 6 Overlay Positive 5 1 6 Result Excluded 2 12 14 Result Excluded 3 11 14 Total 7 13 20 Total 8 12 20 Statistical Analysis of First Examination Data Sensitivity % Specificity % PPV % NPV % PREV % Accuracy % TD FPR % FNR % PTP+ % PTP-% 71.4 92.3 86.1 82.9 40 85 20 7.7 28.6 86.9 17.1 PA% PA+% PA-% Kappa SE 85 77 89 0.66 0.222 Statistical Analysis of Second Examination Data Sensitivity % Specificity % PPV % NPV % PREV % Accuracy % TD FPR % FNR % PTP+ % PTP-% 62.5 91.7 83.3 78.6 40 80 20 8.33 37.5 83.3 21.4 PA% PA+% PA-% Kappa SE 85 77 89 0.66 0.222 Comparison Between Examination 1 and 2 Exam 1 Kappa Results From Intra-Examiner Data Pos. Excluded Total Kappa SE P PA% Exam 2 Positive 4 2 6 0.52 0.224 0.010 80 Excluded 2 12 14 Total 6 14 20 220 EXAMINER 6 - ABFOSTD1 CASE NUMBER FIRST EXAMINATION SECOND EXAMINATION SUSPECT A SUSPECT B SUSPECT A SUSPECT B 1 Positive Excluded Positive Excluded 2 Excluded Positive Excluded Positive 3 Excluded Excluded Excluded Positive 4 Excluded Excluded Excluded Excluded 5 Excluded Excluded Excluded Excluded 6 Positive Excluded Positive Excluded 7 Positive Excluded Positive Excluded 8 Excluded Positive Excluded Positive 9 Excluded Excluded Excluded Excluded 10 Excluded Excluded Excluded Excluded Contingency Tables Forced Decision Model - First Examination Forced Decision Model - Second Examination Gold Standard - Known Positive Gold Standard - Known Positive Pos. Excluded Total Pos. Excluded Total Overlay Positive 5 0 5 Overlay Positive 5 1 6 Result Excluded 3 12 15 Result Excluded 3 11 14 Total 8 12 20 Total 8 12 20 Statistical Analysis of First Examination Data Sensitivity % Specificity % PPV % NPV % PREV % Accuracy % TD FPR % FNR % PTP+ % PTP-% 62.5 100 100 80 40 85 20 0 37.5 100 20 PA% PA+% PA-% Kappa SE 85 77 89 0.67 0.211 Statistical Analysis of Second Examination Data Sensitivity % Specificity % PPV % NPV % PREV % Accuracy % TD FPR % FNR % PTP+ % PTP-% 62.5 91.7 83.3 78.6 40 80 20 8.33 37.5 83.3 21.4 PA% PA+% PA-% Kappa SE 85 77 89 0.66 0.222 Comparison Between Examination 1 and 2 Exam 1 Kappa Results From Intra-Examiner Data Pos. Excluded Total Kappa SE P PA% Exam 2 Positive 5 1 6 0.88 0.222 0.000 95 Excluded 0 14 14 Total 5 15 20 221 EXAMINER 7 - ABFOSTD1 CASE NUMBER FIRST EXAMINATION SECOND EXAMINATION SUSPECT A SUSPECT B SUSPECT A SUSPECT B 1 Positive Excluded Positive Excluded 2 Excluded Positive Excluded Positive 3 Positive Excluded Positive Excluded 4 Excluded Excluded Excluded Excluded 5 Excluded Positive Excluded Positive 6 Positive Excluded Positive Excluded 7 Positive Excluded Positive Excluded S Excluded Positive Excluded Positive 9 Positive Excluded Positive Excluded 10 Excluded Positive Excluded Positive Contingency Tables Forced Decision Model - First Examination Forced Decision Model - Second Examination Gold Standard - Known Positive Gold Standard - Known Positive Pos. Excluded Total Pos. Excluded Total Overlay Positive 8 1 9 Overlay Positive 8 1 9 Result Excluded 0 11 11 Result Excluded 0 11 11 Total 8 12 20 Total 8 12 20 Statistical Analysis of First Examination Data Sensitivity Specificity PPV NPV PREV Accuracy TD FPR FNR PTP+ PTP-% % % % % % % % % % 100 91.7 88.9 100 40 95 20 8.3 0 88.9 0 PA% PA+% PA-% Kappa SE 95 94 96 0.90 0.222 Statistical Analysis of Second Examination Data Sensitivity % Specificity % PPV % NPV % PREV % Accuracy % TD FPR % FNR % PTP+ % PTP-% 100 91.7 88.9 100 40 95 20 8.3 0 88.9 0 PA% PA+% PA-% Kappa SE 95 94 96 0.90 0.222 Comparison Between Examination 1 and 2 Exam 1 Kappa Results From Intra-Examiner Data Pos. Excluded Total Kappa SE P PA% Exam 2 Positive 8 0 8 1.00 0 0.000 100 Excluded 0 12 12 Total 8 12 20 222 SECTION 2 STUDY TWO DATA 223 SECTION 2 - PART 1 DATA FROM ABFO DIPLOMATES 224 EXAMINER 1 - ABFO CASE NUMBER SUSPECT A SUSPECT B FDM LEVEL ROC FDM LEVEL ROC 1 Positive 1 Excluded 1 2 Excluded 5 Positive 1 3 Don't Know 4 Excluded 3 4 Excluded 5 Positive 1 5 Positive 3 Positive 3 6 Positive 1 Excluded 5 7 Positive 3 Positive 3 8 Excluded 3 Positive 1 9 Positive 1 Positive 2 10 Positive 2 Positive 2 Forced Decision Model ROC Analysis Gold Standard - Known Positive Level Sensitivity % Specificity % Youden's Index Overlay Result Pos. Excluded Total 5 100 0 0.00 Positive 8 5 13 4 100 25.0 0.25 Excluded 0 6 6 3 100 33.3 0.33 Total 8 11 19 2 75 66.7 0.42 1 62.5 83.3 0.46 Sensitivity % Specificity % PPV % NPV % PREV % Accuracy % TD FPR % FNR % PTP+ % PTP-% 100 54.5 61.4 100 42 74 19 45.5 0 61.4 0 AUC% SIG (50 %) PA% PA+% PA-% Kappa SE BCO 79.2 0.002 74 76 71 0.50 0.199 1 225 EXAMINER 2 - ABFO CASE NUMBER SUSPECT A SUSPECT B FDM LEVEL ROC FDM LEVEL ROC 1 Positive 2 Don't Know 3 2 Don't Know 3 Positive 2 3 Don't Know 3 Don't Know 3 4 Don't Know 3 Don't Know 3 5 Don't Know 4 Don't Know 4 6 Don't Know 3 Excluded 5 7 Don't Know 3 Excluded 5 8 Don't Know 3 Don't Know 3 9 Don't Know 3 Don't Know 3 10 Don't Know 3 Don't Know 3 Forced Decision Model ROC Analysis Gold Standard - Known Positive Level Sensitivity % Specificity % Youden's Index Pos. Excluded Total 5 100 0 0.00 Overlay Positive 2 0 2 4 100 16.7 0.17 Result Excluded 0 2 2 3 87.5 25.0 0.13 Total 2 2 4 2 25.0 100 0.25 1 0 100.0 0 Sensitivity Specificity PPV NPV PREV Accuracy TD FPR FNR PTP+ PTP-% % % % % % % % % % 100 100 100 100 50 100 4 0 0 100 0 AUC% SIG (50 %) PA% PA+% PA-% Kappa SE BCO 66.7 0.095 100 100 100 1.00 0.50 2 226 EXAMINER 3 - ABFO CASE NUMBER SUSPECT A SUSPECT B FDM LEVEL ROC FDM LEVEL ROC 1 Positive 1 Excluded 5 2 Positive 5 Positive 2 3 Excluded 5 Positive 3 4 Positive 3 Excluded 3 5 Positive 2 Excluded 3 6 Positive 1 Excluded 5 7 Excluded 4 Excluded 2 8 Excluded 5 Positive 2 9 Excluded 3 Excluded 3 10 Excluded 5 Positive 1 Forced Decision Model ROC Analysis Gold Standard - Known Positive Level Sensitivity % Specificity % Youden's Index Pos. Excluded Total 5 100 0 0.00 Overlay Positive 5 4 9 4 100 45.5 0.45 Result Excluded 3 8 11 3 87.5 45.5 0.33 Total 8 12 20 2 62.5 81.8 0.44 1 37.5 100.0 0.38 Sensitivity % Specificity % PPV % NPV % PREV % Accuracy % TD FPR % FNR % PTP+ % PTP-% 62.5 66.6 55.5 72.7 40 65 20 33.3 37.5 55.6 27.3 AUC% SIG (50 %) PA% PA+% PA-% Kappa SE BCO 81.8 0.001 65 59 70 0.29 0.222 4 227 EXAMINER 4 - ABFO CASE NUMBER SUSPECT A SUSPECT B FDM LEVEL ROC FDM LEVEL ROC 1 Positive 2 Excluded 5 2 Excluded 3 Positive 2 3 Excluded 5 Positive 3 4 Excluded 5 Excluded 5 5 Positive 5 Excluded 5 6 Positive 2 Excluded 5 7 Positive 1 Excluded 4 8 Excluded 3 Positive 2 9 Positive 2 Excluded 3 10 Positive 3 Excluded 3 Forced Decision Model ROC Analysis Gold Standard - Known Positive Level Sensitivity % Specificity % Youden's Index Pos. Excluded Total 5 100 0 0.00 Overlay Positive 6 3 9 4 87.5 50.0 0.38 Result Excluded 2 9 11 3 87.5 58.3 0.46 Total 8 12 20 2 75.0 100 0.75 1 12.5 100 0.13 Sensitivity % Specificity % PPV % NPV % PREV % Accuracy % TD FPR. % FNR % PTP+ % PTP-% 75 75 66.7 81.8 40 75 20 25 25 66.7 18.2 AUC% SIG (50 %) PA% PA+% PA-% Kappa SE BCO 88.0 0.000 75 71 78 0.49 0.222 2 228 EXAMINER 5 - ABFO CASE NUMBER SUSPECT A i SUSPECT B FDM LEVEL ROC FDM LEVEL ROC 1 Positive 2 Excluded 5 2 Excluded 3 Positive 3 3 Excluded 3 Excluded 5 4 Excluded 3 Excluded 5 5 Excluded 5 Excluded 3 6 Positive 2 Excluded 5 7 Positive 1 Excluded 4 8 Excluded 5 Positive 4 9 Excluded 4 Excluded 5 10 Excluded 5 Excluded 5 Forced Decision Model ROC Analysis Gold Standard - Known Positive Level Sensitivity % Specificity % Youden's Index Pos. Excluded Total 5 100 0 0.00 Overlay Positive 5 0 5 4 87.5 66.7 0.54 Result Excluded 3 12 15 3 62.5 75.0 0.38 Total 8 12 20 2 37.5 100 0.38 1 12.5 100.0 0.13 Sensitivity Specificity PPV NPV PREV Accuracy TD FPR FNR PTP+ PTP-% % % % % % % % % % 62.5 100 100 80 40 85 20 0 37.5 100 20 AUC% SIG (50 %) PA% PA+% PA-% Kappa SE BCO 81.3 0.001 85 77 89 0.67 0.211 4 229 EXAMINER 6 - ABFO CASE NUMBER SUSPECT A SUSPECT B FDM LEVEL ROC FDM LEVEL ROC 1 Positive 2 Excluded 5 2 Excluded 5 Positive 1 3 Excluded 5 Positive 4 4 Excluded 4 Excluded 4 5 Excluded 3 Positive 2 6 Positive 1 Excluded 5 7 Positive 1 Excluded 4 8 Excluded 4 Excluded 5 9 Positive 2 • Excluded 3 10 Excluded 4 Positive 1 Forced Decision Model ROC Analysis Gold Standard - Known Positive Level Sensitivity % Specificity '% Youden's Index Pos. Excluded Total 5 100 0 0.00 Overlay Positive 7 1 8 4 87.5 33.3 0.21 Result Excluded 1 11 12 3 87.5 83.3 0.71 Total 8 12 20 2 87.5 100 0.88 1 50 100 0.50 Sensitivity % Specificity % PPV % NPV % PREV % Accuracy % TD FPR % FNR % PTP+ % PTP-% 87.5 91.7 87.5 91.6 40 90 20 8.3 12.5 87.5 8.3 AUC% SIG (50 %) PA% PA+% PA-% Kappa SE BCO 89.6 0.000 90 88 92 0.79 0.224 2 230 EXAMINER 7 - ABFO CASE NUMBER SUSPECT A SUSPECT B FDM LEVEL ROC FDM LEVEL ROC 1 Positive 1 Excluded 5 2 Excluded 5 Positive 2 3 Excluded 5 Excluded 3 4 Excluded 5 Excluded 5 5 Excluded 5 Positive 2 6 Positive 2 Excluded 5 7 Positive 2 Excluded 4 8 Excluded 5 Excluded 5 9 Positive 3 Excluded 3 10 Excluded 5 Positive 1 Forced Decision Model ROC Analysis Gold Standard - Known Positive Level Sensitivity % Specificity % Youden's Index Pos. Excluded Total 5 100 0 0.00 Overlay Positive 6 1 7 4 87.5 75.0 0.63 Result Excluded 2 11 13 3 87.5 83.3 0.71 Total 8 12 20 2 75 100 0.75 1 25 100 0.25 Sensitivity % Specificity % PPV % NPV % PREV % Accuracy % TD FPR % FNR % PTP+ % PTP-% 75 91.7 85.7 84.6 40 85 20 8.3 25 87.5 8.3 AUC% SIG (50 %) PA% PA+% PA-% Kappa SE BCO 91.1 0.000 85 80 88 0.68 0.222 2 231 EXAMINER 8 - ABFO CASE NUMBER SUSPECT A SUSPECT B FDM LEVEL ROC FDM LEVEL ROC 1 Positive 1 Excluded 5 2 Excluded 5 Positive 1 3 Positive 1 Excluded 5 4 Excluded 3 Positive 3 5 Excluded 3 Positive 2 6 Positive 2 Excluded 5 7 Positive 1 Excluded 5 8 Excluded 5 Positive 3 9 Positive 1 Excluded 3 10 Excluded 5 Positive 1 Forced Decision Model ROC Analysis Gold Standard - Known Positive Level Sensitivity % Specificity % Youden's Index Pos. Excluded Total 5 100 0 0.00 Overlay Positive 8 1 9 4 100 63.6 0.64 Result Excluded 1 10 11 3 100 63.6 0.64 Total 9 11 20 2 87.5 100 0.88 1 62.5 100 0.63 Sensitivity % Specificity % PPV % NPV % PREV % Accuracy % TD FPR % FNR % PTP+ % PTP-% 88.9 90.9 86.7 92.5 40 90 20 9.1 11.1 86.7 7.5 AUC% SIG (50 %) PA% PA+% PA-% Kappa SE BCO 97.7 0.000 90 89 91 0.80 0.224 2 232 EXAMINER 9 - ABFO CASE NUMBER SUSPECT A SUSPECT B FDM LEVEL ROC FDM LEVEL ROC 1 Positive 2 Excluded 5 2 Excluded 5 Positive 1 3 Excluded 5 Excluded 5 4 Excluded 5 Excluded 5 5 Excluded 5 Excluded 5 6 Don't Know 4 Don't Know 4 7 Excluded 5 Excluded 5 8 Positive 3 Excluded 5 9 Excluded 5 Excluded 5 10 Excluded 5 Excluded 5 Forced Decision Model ROC Analysis Gold Standard - Known Positive Level Sensitivity % Specificity % Youden's Index Pos. Excluded Total 5 100 0 0.00 Overlay Positive 2 1 3 4 37.5 83.3 0.21 Result Excluded 5 10 15 3 25.0 91.7 0.17 Total 7 11 18 2 25.0 100 0.25 1 12.5 100 0.13 Sensitivity % Specificity % PPV % NPV % PREV % Accuracy % TD FPR % FNR % PTP+ % PTP-% 28.6 90.1 66.7 66.6 39 67 18 9.1 71.4 66.7 33.4 AUC% SIG (50 %) PA% PA+% PA-% Kappa SE BCO 62.0 0.188 67 40 77 0.22 0.201 2 233 EXAMINER 10 - ABFO CASE NUMBER SUSPECT A SUSPECT B FDM LEVEL ROC FDM LEVEL ROC 1 Positive 2 Don't Know 3 2 Excluded 3 Positive 2 3 Positive 3 Don't Know 3 4 Positive 3 Don't Know 3 5 Don't Know 4 Don't Know 4 6 Positive 3 Excluded 5 7 Positive 3 Excluded 5 8 Excluded 3 Don't Know 3 9 Excluded 3 Don't Know 3 10 Positive 3 Don't Know 3 Forced Decision Model ROC Analysis Gold Standard - Known Positive Level Sensitivity % Specificity % Youden's Index Pos. Excluded Total 5 100 0 0.00 Overlay Positive 4 1 5 4 100 18.2 0.18 Result Excluded 3 4 7 3 87.5 27.3 0.15 Total 7 5 12 2 25.0 100 0.25 1 0 100 0 Sensitivity % Specificity % PPV % NPV % PREV % Accuracy % TD FPR % FNR % PTP+ % PTP-% 57.1 80 66.5 72.9 41 100 12 20 42 66.5 27.1 AUC% SIG (50 %) PA% PA+% PA-% Kappa SE BCO 67.6 0.083 67 67 67 0.35 0.273 2 234 SECTION 2 - PART 2 DATA FROM ASFO MEMBERS 235 EXAMINER 1 - ASFO CASE NUMBER SUSPECT A SUSPECT B FDM LEVEL ROC FDM LEVEL ROC 1 Positive 2 Excluded 5 2 Excluded 5 Excluded 5 3 Excluded 5 Excluded 5 4 Excluded 5 Excluded 5 5 Excluded 5 Excluded 5 6 Positive 1 Excluded 5 7 Excluded 5 Positive 2 8 Excluded 3 Positive 2 9 Excluded 3 Excluded 5 10 Excluded 5 Excluded 5 Forced Decision Model ROC Analysis Gold Standard - Known Positive Level Sensitivity % Specificity % Youden's Index Pos. Excluded Total 5 100 0 0.00 Overlay Positive 3 1 4 4 50.0 83.3 0.33 Result Excluded 5 11 16 3 50.0 83.3 0.33 Total 8 12 20 2 37.5 91.7 0.29 1 12.5 100.0 0.13 Sensitivity % Specificity % PPV % NPV % PREV % Accuracy % TD FPR % FNR % PTP+ % PTP-% 37.5 91.7 75 68.7 40 55 20 8.3 62.5 75 31.3 AUC% SIG (50 %) PA% PA+% PA-% Kappa SE BCO 68.2 0.080 70 50 79 0.32 0.199 4 236 EXAMINER 2 - ASFO CASE SUSPECT A SUSPECT B NUMBER FDM LEVEL ROC FDM LEVEL ROC 1 Positive 2 Excluded 5 2 Excluded 5 Positive 1 3 Excluded 5 Excluded 5 4 Positive 3 Positive 3 5 Don't Know 4 Don't Know 4 6 Positive 1 Excluded 5 7 Positive 3 Excluded 5 8 Excluded 5 Positive 2 9 Positive 3 Positive 3 10 Excluded 5 Excluded 5 Forced Decision Model ROC Analysis Gold Standard - Known Positive Level Sensitivity % Specificity % Youden's Index Pos. Excluded Total 5 100 0 0.00 Overlay Positive 6 3 9 4 87.5 66.7 0.54 Result Excluded 1 8 9 3 75.0 75.0 0.50 Total 7 11 18 2 50.0 100 0.50 1 25.0 100.0 0.25 Sensitivity % Specificity % PPV % NPV % PREV % Accuracy % TD FPR % FNR % PTP+ % PTP-% 85.7 72.7 64.9 89.7 37 77.8 18 27.3 14.3 64.9 10.3 AUC% SIG (50 %) PA% PA+% PA-% Kappa SE BCO 84.9 0.000 78 75 80 0.56 0.230 4 237 EXAMINER 3 - ASFO CASE NUMBER SUSPECT A SUSPECT B FDM LEVEL ROC FDM LEVEL ROC 1 Positive 1 Excluded 3 2 Excluded 5 Positive 2 3 Excluded 5 Don't Know 3 4 Excluded 5 Excluded 5 5 Excluded 3 Don't Know 3 6 Positive 1 Excluded 5 7 Positive 2 Excluded 5 8 Don't Know 3 Positive 2 9 Positive 2 Excluded 5 10 Excluded 5 Excluded 5 Forced Decision Model ROC Analysis Gold Standard - Known Positive Level Sensitivity % Specificity % Youden's Index Pos. Excluded Total 5 100 0 0.00 Overlay Positive 6 0 6 4 87.5 66.7 0.54 Result Excluded 1 10 11 3 87.5 66.7 0.54 Total 7 10 17 2 75.0 100 0.75 1 25.0 100 0.25 Sensitivity Specificity PPV NPV PREV Accuracy TD FPR FNR PTP+ PTP-% % % % % % % . % % % 85.7 100 100 91.0 41 94.1 17 0 14.3 100 9.0 AUC% SIG (50 %) PA% PA+% PA-% Kappa SE BCO 89.6 0.000 94 92 95 0.88 0.241 2 238 EXAMINER 4 - ASFO CASE NUMBER SUSPECT A SUSPECT B FDM LEVEL ROC FDM LEVEL ROC 1 Positive 2 Excluded 5 2 Excluded 5 Positive 2 3 Excluded 5 Excluded 5 4 Excluded 5 Excluded 5 5 Excluded 5 Positive 2 6 Positive 1 Excluded 3 7 Don't Know- 4 Excluded 5 8 Excluded 3 Positive 1 9 Excluded 5 Excluded 5 10 Excluded 5 Excluded 5 Forced Decision Model ROC Analysis Gold Standard - Known Positive Level Sensitivity % Specificity % Youden's Index Pos. Excluded Total 5 100 0 0.00 Overlay Positive 5 0 5 4 75.0 83.3 0.58 Result Excluded 2 12 14 3 62.5 83.3 0.46 Total 7 12 19 2 62.5 100 0.63 1 25.0 100 0.25 Sensitivity % Specificity % PPV % NPV % PREV % Accuracy % TD FPR % FNR % PTP+ % PTP-% 71.4 100 100 86.2 36 89 19 0 28.6 100 13.9 AUC% SIG (50 %) PA% PA+% PA-% Kappa SE BCO 83.3 0.001 89 83 92 0.76 0.223 2 239 EXAMINER 5 - ASFO CASE NUMBER SUSPECT A SUSPECT B FDM LEVEL ROC FDM LEVEL ROC 1 Positive 1 Excluded 3 2 Don't Know 3 Positive 1 3 Excluded 5 Excluded 5 4 Excluded 5 Don't Know 2 5 Excluded 5 Excluded 3 6 Positive 1 Excluded 5 7 Positive 1 Excluded 5 8 Don't Know 3 Positive 1 9 Excluded 2 Positive 2 10 Excluded 5 Excluded 5 Forced Decision Model ROC Analysis Gold Standard - Known Positive Level Sensitivity % Specificity % Youden's Index Pos. Excluded Total 5 100 0 0.00 Overlay Positive 5 1 6 4 87.5 58.3 0.46 Result Excluded 3 8 11 3 87.5 58.3 0.46 Total 8 9 17 2 75.0 83.3 0.58 1 62.5 100 0.63 Sensitivity % Specificity % PPV % NPV % PREV % Accuracy % TD FPR % FNR % PTP+ % PTP-% 62.5 88.9 83.3 72.7 47 76 17 11.1 37.5 83.3 27.2 AUC% SIG (50 %) PA% PA+% PA-% Kappa SE BCO 86.5 0.000 76 71 80 0.52 0.235 1 240 EXAMINER 6 - ASFO CASE NUMBER SUSPECT A SUSPECT B FDM LEVEL ROC FDM LEVEL ROC 1 Positive 1 Excluded 5 2 Positive 1 Excluded 5 3 Don't Know 4 Don't Know 4 4 Excluded 5 Excluded 5 5 Excluded 3 Excluded 2 6 Positive 1 Excluded 5 7 Excluded 5 Positive 1 8 Don't Know 2 Excluded 3 9 Positive 1 Excluded 5 10 Excluded 5 Excluded 5 Forced Decision Model ROC Analysis Gold Standard - Known Positive Level Sensitivity % Specificity % Youden's Index Pos. Excluded Total 5 100 0 0.00 Overlay Result Positive 3 2 5 4 62.5 46.2 0.09 Excluded 5 7 12 3 62.5 69.2 0.32 Total 8 9 17 2 50.0 76.9 0.27 1 37.5 84.6 0.22 Sensitivity % Specificity % PPV % NPV % PREV % Accuracy % TD FPR % FNR % PTP+ % PTP-% 37.5 77.8 59.9 58.4 47 59 17 22.2 62.5 59.9 41.6 AUC% SIG (50 %) PA% PA+% PA-% Kappa SE BCO 62.5 0.176 59 46 67 0.16 0.245 3 241 EXAMINER 7 - ASFO CASE NUMBER SUSPECT A SUSPECT B FDM LEVEL ROC FDM LEVEL ROC 1 Excluded 3 Excluded 5 2 Excluded 5 Positive 2 3 Excluded 5 Excluded 5 4 Excluded 5 Excluded 5 5 Excluded 5 Excluded 5 6 Positive 1 Excluded 5 7 Excluded 3 Excluded 5 8 Don't Know 3 Don't Know 3 9 Excluded 5 Excluded 5 10 Excluded 5 Excluded 5 Forced Decision Model ROC Analysis Gold Standard - Known Positive Level Sensitivity % Specificity % Youden's Index Pos. Excluded Total 5 100 0 0.00 Overlay Positive 2 0 2 4 62.5 91.7 0.54 Result Excluded 5 11 16 3 62.5 91.7 0.54 Total 7 11 18 2 25.0 100.0 0.25 1 12.5 100.0 0.13 Sensitivity Specificity PPV NPV PREV Accuracy TD FPR FNR PTP+ PTP-% % % % % % % % % % 28.6 100 100 68.6 39 72 18 0 74.4 100 31.4 AUC% SIG (50 %) PA% PA+% PA-% Kappa SE BCO 78.1 0.008 72 44 81 0.33 0.175 4 242 EXAMINER 8 - ASFO CASE NUMBER SUSPECT A SUSPECT B FDM LEVEL ROC FDM LEVEL ROC 1 Don't Know 2 Excluded 5 2 Excluded 5 Positive 1 3 Excluded 5 Don't Know 5 4 Excluded 3 Excluded 3 5 Excluded 4 Excluded 4 6 Positive 1 Excluded 5 7 Excluded 3 Excluded 5 8 Excluded 5 Excluded 2 9 Don't Know 3 Positive 3 10 Excluded 5 Excluded 5 Forced Decision Model ROC Analysis Gold Standard - Known Positive Level Sensitivity % Specificity % Youden's Index Overlay Result Pos. Excluded Total 5 100 0.0 0.00 Positive 2 1 3 4 87.5 66.7 0.54 Excluded 3 11 14 3 75.0 75.0 0.50 Total 5 12 17 2 50.0 100 0.50 1 25.0 100 0.25 Sensitivity % Specificity % PPV % NPV % PREV % Accuracy % TD FPR % FNR % PTP+ % PTP-% 40 34.6 72.1 73.9 35 76 17 8.3 60 72.1 26.0 AUC% SIG (50 %) PA% PA+% PA-% Kappa SE BCO 84.9 0.000 76 50 85 0.36 0.230 4 243 EXAMINER 9 - ASFO CASE NUMBER SUSPECT A SUSPECT B FDM LEVEL ROC FDM LEVEL ROC 1 Positive 3 Excluded 4 2 Excluded 5 Positive 2 3 Excluded 5 Excluded 5 4 Excluded 5 Excluded 5 5 Positive 2 Don't Know 4 6 Positive 2 Excluded 5 7 Positive 2 Excluded 5 8 Excluded 5 Positive 3 9 Positive 3 Excluded 5 10 Positive 3 Excluded 5 Forced Decision Model ROC Analysis Gold Standard - Known Positive Level Sensitivity % Specificity % Youden's Index Pos. Excluded Total 5 100 0 0.00 Overlay Positive 6 2 8 4 87.5 75.0 0.63 Result Excluded 1 10 11 3 75.0 83.3 0.58 Total 7 12 19 2 37.5 91.7 0.29 1 0 100.0 0 Sensitivity % Specificity % PPV % NPV % PREV % Accuracy % I'D FPR % FNR % PTP+ % PTP-% 85.7 83.3 75.1 90.9 37 84.2 19 16.7 14.3 75.1 9.15 AUC% SIG (50 %) PA% PA+% PA-% Kappa SE BCO 83.3 0.000 84 80 87 0.67 0.228 4 244 EXAMINER 10 - ASFO CASE NUMBER SUSPECT A SUSPECT B FDM LEVEL ROC FDM LEVEL ROC 1 Positive 2 Excluded 5 2 Excluded 3 Positive 2 3 Excluded 5 Positive 3 4 Excluded 5 Excluded 5 5 Positive 5 Excluded 5 6 Positive 2 Excluded 5 7 Positive 1 Excluded 4 8 Excluded 3 Positive 2 9 Positive 2 Excluded 3 10 Positive 3 Excluded 3 Forced Decision Model ROC Analysis Gold Standard - Known Positive Level Sensitivity % Specificity % Youden's Index Pos. Excluded Total 5 100 0 0.00 Overlay Result Positive 6 3 9 4 87.5 50.0 0.38 Excluded 2 9 11 3 87.5 58.3 0.46 Total 8 12 20 2 75.0 100 0.75 1 12.5 100 0.13 Sensitivity % Specificity % PPV % NPV % PREV % Accuracy % TD FPR % FNR % PTP+ % PTP-% 75 75 66.7 81.8 40 75 20 25 25 66.7 18.2 AUC% SIG (50 %) PA% PA+% PA-% Kappa SE BCO 88.0 0.000 75 71 78 0.49 0.222 2 245 SECTION 2 - PART 3 DATA FROM GENERAL DENTAL PRACTITIONERS 246 EXAMINER 1 - GDP CASE NUMBER SUSPECT A SUSPECT B FDM LEVEL ROC FDM LEVEL ROC 1 Positive 2 Positive 3 2 Positive 2 Positive 2 3 Positive 3 Excluded 5 4 Excluded 5 Excluded 5 5 Positive 3 Positive 3 6 Positive 1 Excluded 5 7 Excluded 5 Excluded 5 8 Positive 3 Positive 2 9 Excluded 3 Excluded 5 10 Don't Know 4 Don't Know 4 Forced Decision Model ROC Analysis Gold Standard - Known Positive Level Sensitivity % Specificity % Youden's Index Pos. Excluded Total 5 100 0 0.00 Overlay Positive 4 6 10 4 87.5 50 0.38 Result Excluded 2 6 8 3 75.0 58.3 0.33 Total 6 12 18 2 50 91.7 0.42 1 12.5 100.0 0.13 Sensitivity % Specificity % PPV % NPV % PREV % Accuracy % TD FPR % FNR % PTP+ % PTP-% 66.7 50 46.0 70.1 39 55.6 18 50 33.3 46.0 29.9 AUC% SIG (50 %) PA% PA+% PA-% Kappa SE BCO 77.1 0.008 56 50 60 0.14 0.213 2 247 EXAMINER 2 - GDP CASE NUMBER SUSPECT A SUSPECT B FDM LEVEL ROC FDM LEVEL ROC 1 Positive 3 Excluded 4 2 Excluded 5 Positive 2 3 Excluded 5 Excluded 5 4 Excluded 5 Excluded 5 5 Positive 2 Don't Know- 4 6 Positive 2 Excluded 5 7 Positive 2 Excluded 5 8 Excluded 5 Positive 3 9 Positive 3 Excluded 5 10 Positive 3 Excluded 5 Forced Decision Model ROC Analysis Gold Standard - Known Positive Level Sensitivity % Specificity % Youden's Index Pos. Excluded Total 5 100 0 0.00 Overlay Result Positive 6 2 8 4 87.5 75.0 0.63 Excluded 1 10 11 3 75.0 83.3 0.58 Total 7 12 19 2 37.5 91.7 0.29 1 0 100.0 0 Sensitivity % Specificity % PPV % NPV % PREV % Accuracy % TD FPR % FNR % PTP+ % PTP-% 85.7 83.3 75.1 90.9 37 84.2 19 16.7 14.3 75.1 9.15 AUC% SIG (50 %) PA% PA+% PA-% Kappa SE BCO 83.3 0.000 84 80 87 0.67 0.228 4 248 EXAMINER 3 - GDP CASE NUMBER SUSPECT A SUSPECT B FDM LEVEL ROC FDM LEVEL ROC 1 Excluded 5 Positive 2 2 Excluded 5 Positive 1 3 Positive 3 Positive 3 4 Excluded 5 Excluded 3 5 Excluded 5 Positive 2 6 Positive 1 Excluded 5 7 Excluded 3 Excluded 5 8 Excluded 5 Positive 2 9 Positive 2 Excluded 5 10 Positive 3 Excluded 5 Forced Decision Model ROC Analysis Gold Standard - Known Positive Level Sensitivity % Specificity % Youden's Index Pos. Excluded Total 5 100 0 0.00 Overlay Positive 5 4 9 4 75.0 58.3 0.33 Result Excluded 3 8 11 3 75.0 58.3 0.33 Total 8 12 20 2 62.5 91.7 0.54 1 25.0 100.0 0.25 Sensitivity % Specificity % PPV % NPV % PREV % Accuracy % TD FPR % FNR % PTP+ % PTP-% 62.5 66.6 55.5 72.7 40 65 20 33.3 37.5 55.6 27.3 AUC% SIG (50 %) PA% PA+% PA-% Kappa SE BCO 77.6 0.010 65 59 70 0.29 0.222 2 249 EXAMINER 4 - GDP CASE NUMBER SUSPECT A SUSPECT B FDM LEVEL ROC FDM LEVEL ROC 1 Positive 1 Excluded 5 2 Excluded 5 Positive 2 3 Positive 3 Excluded 3 4 Excluded 3 Excluded 5 5 Don't Know 4 Excluded 5 6 Positive 1 Excluded 5 7 Excluded 5 Excluded 5 8 Excluded 5 Positive 1 9 Don't Know 4 Don't Know 4 10 Positive 1 Excluded 5 Forced Decision Model ROC Analysis Gold Standard - Known Positive Level Sensitivity % Specificity % Youden's Index Pos. Excluded Total 5 100 0 0.00 Overlay Result Positive 4 3 7 4 62.5 50 0.13 Excluded 2 8 10 3 50.0 66.7 0.17 Total 6 11 17 2 50.0 91.7 0.42 1 37.5 91.7 0.29 Sensitivity % Specificity % PPV % NPV % PREV % Accuracy % I'D FPR % FNR % PTP+ % PTP-% 66.7 72.3 62.9 75.8 41 70 17 27.3 33.3 62.9 24.2 AUC% SIG (50 %) PA% PA+% PA-% Kappa SE BCO 64.1 0.152 71 62 76 0.38 0.214 2 250 EXAMINER 5 - GDP CASE SUSPECT A SUSPECT B NUMBER FDM LEVEL ROC FDM LEVEL ROC 1 Positive 2 Excluded 5 2 Excluded 3 Positive 2 3 Don't Know 4 Don't Know 4 4 Excluded 5 Excluded 5 5 Excluded 5 Positive 3 6 Positive 2 Excluded 4 7 Positive 3 Positive 3 8 Positive 3 Excluded 5 9 Excluded 5 Excluded 5 10 | Excluded 3 Positive 3 Forced Decision Model ROC Analysis Gold Standard - Known Positive Level Sensitivity % Specificity % Youden's Index Pos. Excluded Total 5 100 0 0.00 Overlay Result Positive 6 2 8 4 75.0 41.7 0.17 Excluded 2 8 10 3 75.0 66.7 0.42 Total 8 10 18 2 37.5 100.0 0.38 1 0.0 100.0 0.00 Sensitivity % Specificity % PPV % NPV % PREV % Accuracy % TD FPR % FNR % PTP+ % PTP-% 75 0 80.0 74.7 80.3 44 77 18 20.0 25.0 74.6 19.7 AUC% SIG (50 %) PA% PA+% PA-% Kappa SE BCO 74.0 0.029 78 75 80 0.55 0.236 3 251 EXAMINER 6 - GDP CASE NUMBER SUSPECT A SUSPECT B FDM LEVEL ROC FDM LEVEL ROC 1 Positive 2 Excluded 5 2 Excluded 5 Positive 2 3 Excluded 5 Excluded 5 4 Excluded 5 Excluded 5 5 Excluded 5 Excluded 3 6 Positive 1 Excluded 4 7 Positive 3 Excluded 5 8 Positive 3 Excluded 5 9 Positive 2 Excluded 5 10 Positive 2 Excluded 3 Forced Decision Model ROC Analysis Gold Standard - Known Positive Level Sensitivity % Specificity % Youden's Index Pos. Excluded Total 5 100 0 0.00 Overlay Result Positive 6 2 8 1 4 87.5 75.5 0.63 Excluded 2 10 12 3 87.5 83.3 0.71 Total 8 12 20 2 50.0 91.7 0.42 1 12.5 100 0.13 Sensitivity % Specificity % PPV % NPV % PREV % Accuracy % TD FPR % FNR % PTP+ % PTP-% 75.0 83.3 75.0 83.3 40 80 20 16.7 25.0 75.0 16.7 AUC% SIG (50 %) PA% PA+% PA-% Kappa SE BCO 85.9 0.000 80 75 83 0.58 0.224 •3 252 EXAMINER 7- GDP CASE NUMBER SUSPECT A SUSPECT B FDM LEVEL ROC FDM LEVEL ROC 1 Positive 2 Excluded 3 2 Don't Know 3 Positive 1 3 Excluded 3 Excluded 5 4 Excluded 5 Excluded 5 5 Excluded 5 Positive 2 6 Positive 2 Excluded 5 7 Positive 2 Excluded 5 8 Excluded 3 Positive 2 9 Positive 2 Excluded 3 10 Excluded 3 Excluded 5 Forced Decision Model ROC Analysis Gold Standard - Known Positive Level Sensitivity % Specificity % Youden's Index Pos. Excluded Total 5 100 0 0.00 Overlay Result Positive 7 0 7 4 87.5 50 0.38 Excluded 1 11 12 3 87.5 50 0.38 Total 8 11 19 2 87.5 100 0.88 1 12.5 100 0.13 Sensitivity % Specificity % PPV % NPV % PREV % Accuracy % I'D FPR % FNR % PTP+ % PTP-% 87.5 100 100 91.7 42 77 19 0 12.5 100 8.3 AUC% SIG (50 %) PA% PA+% PA-% Kappa SE BCO 90.6 0.000 95 93 93 0.89 0.228 2 253 EXAMINER 8 - GDP CASE NUMBER SUSPECT A SUSPECT B FDM LEVEL ROC FDM LEVEL ROC 1 Positive 2 Excluded 5 2 Positive 3 Positive 2 3 Excluded 5 Positive 2 4 Excluded 5 Excluded 5 5 Excluded 3 Positive 3 6 Don't Know 4 Excluded 5 7 Positive 2 Excluded 5 8 Positive 2 Positive 2 9 Positive 2 Excluded 5 10 Don't Know 4 Don't Know 4 Forced Decision Model ROC Analysis Gold Standard - Known Positive Level Sensitivity % Specificity % Youden's Index Pos. Excluded Total 5 100 0 0.00 Overlay Result Positive 6 3 9 4 100 58.3 0.58 Excluded 0 8 8 3 75.0 66.7 0.42 Total 6 11 17 2 62.5 83.3 0.46 1 0.0 100.0 0.00 Sensitivity % Specificity % PPV % NPV % PREV % Accuracy % TD FPR % FNR % PTP+ % PTP-% 100 72.7 66.4 100 35 82 17 27.3 0 66.4 0 AUC% SIG (50 %) PA% PA+% PA-% Kappa SE BCO 82.3 0.000 82 80 84 0.65 0.227 4 254 EXAMINER 9 - GDP CASE NUMBER SUSPECT A SUSPECT B FDM LEVEL ROC FDM LEVEL ROC 1 Positive 2 Excluded 3 2 Positive 3 Positive 1 3 Excluded 5 Excluded 5 4 Excluded 5 Excluded 5 5 Excluded 3 Positive 2 6 Don't Know 4 Excluded 5 7 Positive 2 Excluded 5 8 Positive 2 Excluded 2 9 Positive 2 Excluded 3 10 Don't Know 4 Excluded 5 Forced Decision Model ROC Analysis Gold Standard - Known Positive Level Sensitivity % Specificity % Youden's Index Pos. Excluded Total 5 100 0 0.00 Overlay Positive 5 3 8 4 87.5 50 0.38 Result Excluded 0 10 10 3 75.0 58.3 0.33 Total 5 13 18 2 75.0 91.7 0.67 1 12.5 100.0 0.13 Sensitivity Specificity PPV NPV PREV Accuracy TD FPR FNR PTP+ PTP-% % % % % % % % % % 100 71.4 66.3 100 36 79 18 28.6 0 66.3 0 AUC% SIG (50 %) PA% PA+% PA-% Kappa SE BCO 82.3 0.001 79 71 83 0.57 0.207 2 255 EXAMINER 10 - GDP CASE NUMBER SUSPECT A SUSPECT B FDM LEVEL ROC FDM LEVEL ROC 1 Positive 2 Excluded 3 2 Don't Know 3 Positive 1 3 Excluded 3 Excluded 5 4 Excluded 5 Excluded 5 5 Excluded 5 Positive 2 6 Positive 2 Excluded 5 7 Positive 2 Excluded 5 8 Excluded 3 Positive 2 9 Positive 2 Excluded 3 10 Excluded 3 Excluded 5 Forced Decision Model ROC Analysis Gold Standard - Known Positive Level Sensitivity % Specificity % Youden's Index Pos. Excluded Total 5 100 0 0.00 Overlay Positive 7 0 7 4 87.5 50 0.38 Result Excluded 1 11 12 3 87.5 50 0.38 Total 8 11 19 2 87.5 100 0.88 1 12.5 100 0.13 Sensitivity % Specificity % PPV % NPV % PREV % Accuracy % TD FPR % FNR % PTP+ % PTP-% 87.5 100 100 91.7 42 77 19 0 12.5 100 8.3 AUC% SIG (50 %) PA% PA+% PA-% Kappa SE BCO 90.6 0.000 95 93 93 0.89 0.228 2 256 

Cite

Citation Scheme:

        

Citations by CSL (citeproc-js)

Usage Statistics

Country Views Downloads
United States 43 5
India 34 16
Germany 13 52
South Africa 12 0
Canada 12 0
United Kingdom 11 2
China 8 6
Indonesia 6 0
Malaysia 5 4
France 4 0
Romania 2 0
New Zealand 2 0
Pakistan 1 1
City Views Downloads
Unknown 44 64
Ashburn 16 0
Hyderabad 12 0
Pretoria 12 0
Montreal 6 0
Beijing 5 1
London 5 0
Marietta 4 0
Seattle 4 0
Delhi 3 1
Washington 3 0
Greater Sudbury 3 0
Sunnyvale 3 0

{[{ mDataHeader[type] }]} {[{ month[type] }]} {[{ tData[type] }]}
Download Stats

Share

Embed

Customize your widget with the following options, then copy and paste the code below into the HTML of your page to embed this item in your website.
                        
                            <div id="ubcOpenCollectionsWidgetDisplay">
                            <script id="ubcOpenCollectionsWidget"
                            src="{[{embed.src}]}"
                            data-item="{[{embed.item}]}"
                            data-collection="{[{embed.collection}]}"
                            data-metadata="{[{embed.showMetadata}]}"
                            data-width="{[{embed.width}]}"
                            async >
                            </script>
                            </div>
                        
                    
IIIF logo Our image viewer uses the IIIF 2.0 standard. To load this item in other compatible viewers, use this url:
http://iiif.library.ubc.ca/presentation/dsp.831.1-0089349/manifest

Comment

Related Items