"Medicine, Faculty of"@en . "DSpace"@en . "UBCV"@en . "Junaid, Kathryn Anne"@en . "2009-05-25T20:22:49Z"@en . "1998"@en . "Master of Science - MSc"@en . "University of British Columbia"@en . "The purpose of this study was to determine whether teachers could use the\r\nMovement Assessment Battery for Children Checklist (MABC Checklist) to identify\r\nschool age children with motor coordination difficulties. The use of this checklist would\r\nenable teachers to select the most appropriate children for referral to occupational and\r\nphysiotherapy services.\r\nIn this study teachers completed the MABC Checklist on one hundred and three\r\nrandomly selected school-aged children whose parents consented to their child's\r\nparticipation. A physiotherapist then administered the Movement Assessment Battery for\r\nChildren Test (MABC Test) to the same children.\r\nThe relationship between the teachers' scores on the MABC Checklist and the\r\nMABC Test was examined using the Pearson product-moment correlation (one-tailed\r\ntest)(r = 0.51). The degree of concordance in decision-making between teachers and the\r\nphysiotherapist identifying the same children with and without motor difficulties was\r\ndetermined by calculating the following characteristics of the checklist scores: sensitivity,\r\nspecificity, positive predictive value and negative predictive value.\r\nThe sensitivity at both the fifteenth and fifth cut-off points was low so that many\r\nchildren who had motor problems and those \"at risk\" for motor problems based on the\r\nMABC Test, were not detected using the MABC Checklist. Because of the low\r\nsensitivity of the MABC Checklist, it would not be of great value as a screening tool.\r\nFurther revisions of the MABC Checklist as well as more detailed training in its use\r\nwould be needed before it could be considered as a screening tool to enable teachers to\r\nidentify children with motor difficulties."@en . "https://circle.library.ubc.ca/rest/handle/2429/8163?expand=metadata"@en . "3023098 bytes"@en . "application/pdf"@en . "TEACHERS' USE OF THE MABC CHECKLIST TO IDENTIFY CHILDREN WITH MOTOR DIFFICULTIES by Kathryn Anne Junaid BSR.PT., University of British Columbia, 1982 A THESIS SUBMITTED IN PARTIAL FULFILMENT OF THE REQUIREMENTS FOR THE DEGREE OF MASTER OF SCIENCE in THE FACULTY OF GRADUATE STUDIES (School of Rehabilitation Sciences) We accept this thesis as conforming to the required standard THE UNIVERSITY OF BRITISH COLUMBIA September, 1998 \u00C2\u00A9 Kathryn Anne Junaid, 1998 In p resent ing this thesis in partial fu l f i lment of t h e requ i rements f o r an advanced degree at the Univers i ty of Brit ish C o l u m b i a , I agree that t h e Library shall make it f reely available f o r reference and study. I fu r ther agree that permiss ion f o r extensive c o p y i n g o f this thesis f o r scholar ly pu rposes may b e g ran ted by the head o f m y d e p a r t m e n t o r by his o r her representat ives. It is u n d e r s t o o d that c o p y i n g o r pub l i ca t i on of this thesis fo r f inancial gain shall n o t be a l l o w e d w i t h o u t my w r i t t e n permiss ion . D e p a r t m e n t of S^Wa.W vVi\- ^evayvce-g The Univers i ty o f Brit ish C o l u m b i a Vancouver , Canada Date \u00E2\u0080\u0094 v ^ O c j f o - y^K- \C\CV& , DE-6 (2/88) ABSTRACT The purpose of this study was to determine whether teachers could use the Movement Assessment Battery for Children Checklist (MABC Checklist) to identify school age children with motor coordination difficulties. The use of this checklist would enable teachers to select the most appropriate children for referral to occupational and physiotherapy services. In this study teachers completed the MABC Checklist on one hundred and three randomly selected school-aged children whose parents consented to their child's participation. A physiotherapist then administered the Movement Assessment Battery for Children Test (MABC Test) to the same children. The relationship between the teachers' scores on the MABC Checklist and the MABC Test was examined using the Pearson product-moment correlation (one-tailed test)(r = 0.51). The degree of concordance in decision-making between teachers and the physiotherapist identifying the same children with and without motor difficulties was determined by calculating the following characteristics of the checklist scores: sensitivity, specificity, positive predictive value and negative predictive value. The sensitivity at both the fifteenth and fifth cut-off points was low so that many children who had motor problems and those \"at risk\" for motor problems based on the MABC Test, were not detected using the MABC Checklist. Because of the low sensitivity of the MABC Checklist, it would not be of great value as a screening tool. Further revisions of the MABC Checklist as well as more detailed training in its use would be needed before it could be considered as a screening tool to enable teachers to identify children with motor difficulties. T A B L E OF CONTENT Abstract ii Table of Content iii List of Tables iv List of figures v Acknowledgment vi Dedication vii Chapter 1 Introduction 1 Statement of the Problem 1 Justification of the Study 3 Purpose of the Study 4 Definition of Terms 5 Chapter 2 Literature Review 7 The Development of Motor Skills 7 Persistent Motor Delays & Secondary Concerns 9 Screening Tests 12 Teachers' Application of Screening Tests for Motor Problems 13 The Movement Assessment Battery for Children (MABC) 14 The Research Questions 21 Chapter 3 Methods 22 Participants 22 Procedure 25 Measurements 27 Examiners 29 Data Analysis 29 Chapter 4 Results 32 Chapter 5 Discussion 41 Limitations of the Study 46 Summary and recommendations 48 References 50 Appendix 56 iii LTST OF TABLES TABLE 1 TABLE 2 TABLE 3 TABLE 4 TABLE 5 TABLE 6 Age and Gender of Children in the Sample 24 Munro's Descriptive Terms for the Strength of a Correlation Coefficient 30 Results using the \"At Risk\" or Fifteenth Percentile Cut-off Point 37 Results using the \"Motor Impairment\" or Fifth Percentile Cut-off Point 38 Results using the \"At Risk\" or Fifteenth Percentile Cut-off Point (Outliers Removed) 39 Results using the \"Motor Impairment\" or Fifth Percentile Cut-off Point (Outliers Removed) 40 iv LTST OF FIGURES FIGURE 1 Age and Gender of Children in the Sample. FIGURE 2 Scatter Plot FIGURE 3 FIGURE 4 FIGURE 5 FIGURE 6 Sensitivity and Specificity for the \"At Risk\" (15th percentile) Cut-off Point Sensitivity and Specificity for the \"Motor Impairment\"(5th percentile) Cut-off Point Sensitivity and Specificity for the \"At Risk\" (15th percentile) Cut-off Point (Outliers Removed) 23 33 37 38 39 Sensitivity and Specificity for the \"Motor Impairment\"^411 percentile) Cut-off Point (Outliers Removed) 40 v ACKNOWLEDGMENT I would like to thank the following people for their help through the process of this thesis: Dr. Susan Harris whose dedication to physiotherapy and ability to teach the stones got me into the master's program and whose constant guidance, encouragement and enthusiasm kept me working till the end. I am fortunate to have found such an amazing individual for a mentor. My thesis committee members, Dr. Anne Carswell and Dr. Alison Fulmer, for their insight and input in the initial development of the study and for their constant advice throughout. Dr. Jonathan Berkowitz whose ability to keep things simple helped me tremendously with the statistical analysis. Coquitlam School District, Dr. David Oborne and Pat Elliot, for allowing me to use Coquitlam Schools and for their assistance in organizing the data collection. The children and their teachers for their time and effort in participating in the study. Sunny Hill Health Centre for Children and my colleagues for their understanding and endless support. My husband, Sard, for his tremendous support, tolerance and endurance over the four years it took me to complete this thesis. My daughters, Sara and Esme for their countless reminders to \"get a grip\" and \"get a life\". DEDICATION I dedicate this thesis in memory of my parents, Mary and Keith Jordan who taught me so many things including the value of education. vii CHAPTER I INTRODUCTION For the past decade, there has been increasing concern over children whose motor performance in school interferes with their ability to succeed in the classroom (Lam & Henderson, 1987). It has been estimated that approximately 8% to 15% of children in elementary schools have some form of motor coordination problem (Cratty, 1986). The Ministry of Education in British Columbia endorses a policy of the inclusion of all children, regardless of ability or disability, into the general classroom. (British Columbia Ministry of Education, 1995). This diversity of children's skills in the classroom has made it exceedingly challenging for teachers to estimate the severity of children's problems and this includes the severity of motor difficulties. Statement of the Problem Teachers are often the first to notice that a child has difficulty with fine motor or gross motor tasks but, at present, there is no screening test available for teachers to identify which children would benefit from further assessment. Although there are many screening tools used by teachers to assist them in identifying children at risk for academic or learning difficulties, there are few instruments that adequately screen for motor impairment (Compton, 1984). Consultation with physiotherapists working in the school system in British Columbia revealed that no screening tool is used by teachers to identify the degree of a child's motor difficulties before a referral is made to physiotherapy or occupational therapy. Although it has been shown that teachers can make accurate and reliable judgements regarding children's skills without the use of a standardized screening test to support their suspicions (Henderson & Hall, 1982), all children who are suspected of having motor difficulties in B.C. currently are referred to physiotherapy or occupational therapy. This method of referral taxes the limited physiotherapy and occupational therapy resources and results in long wait-lists. Until recently, no valid and reliable screening tool for children with motor difficulties was available. Tests that were available were comprehensive diagnostic tests usually administered by physiotherapists and occupational therapists. In British Columbia, there are two commonly administered tests of motor abilities that examine both fine motor and gross motor abilities: the Peabody Developmental Motor Scales or PDMS (Folio & Fewell, 1982) and the Braininks-Oseretsky Test of Motor Proficiency or BOTMP (Bruininks, 1978). Widely used throughout British Columbia, the PDMS and the BOTMP have been criticized by clinical physiotherapists and occupational therapists for their lengthy administration time, their lack of ability to identify a child's specific area of motor impairment and their limited usefulness in developing specific intervention programs. These tests have also been criticized for their lack of psychometric robustness (Hinderer, Richardson & Atwater, 1989; Verderber & Payne, 1987; Wilson, Polatajko, Kaplan & Faris, 1994). 2 Recently a third test has been developed: The Movement Assessment Battery for Children (MABC) (Henderson & Sugden, 1992). The MABC is comprised of an observational checklist known as the MABC Checklist and a performance test labelled the MABC Test; the latter is a standardized, norm-referenced test of motor performance. The MABC Checklist is purported to be a quick and easy-to-use screening tool that documents the observations and concerns of the child's classroom teacher. It was designed to reflect activities in which children participate routinely within the school environment and skills that are necessary for academic achievement. It was therefore thought worthwhile to examine the usefulness of this checklist as a screening tool for use by teachers in selecting appropriate children for further testing. To examine the accuracy of the MABC Checklist as scored by classroom teachers, these scores were compared to those obtained on the MABC Test as administered by a physiotherapist for the same group of seven and eight year old children. Justification of the Study In young children, motor difficulties can be associated with immaturity as well as with chronic disabilities, such as Developmental Coordination Disorder (DCD). Although there is no known cure for persistent motor problems such as DCD, early identification of young children with motor difficulties may assist in the prevention of secondary problems, such as difficult behavior and poor self-esteem (Polatajko, Macnab, Anstett, Malloy-Miller, Murphy & Noh, 1995; Schoemaker, Hijlkema & Kalverboer, 1994). Providing children with compensatory strategies for school work and encouraging participation in 3 individualized or non-competitive sports or leisure activities may prevent such secondary problems. If teachers using the MABC Checklist can reliably identify children with motor difficulties, then the MABC Checklist would be a useful screening tool to identify children with motor problems who would benefit from further assessment. The consistent use of this checklist by teachers could provide a quick method of passing on useful information to therapists regarding the child's ability to perform motor tasks that are routinely required in the school environment. These classroom contextual tasks are not examined by the administration of the MABC Test or by the administration of either the PDMS or the BOTMP. Combining the results of the MABC Checklist and MABC Test would assist teachers and therapists to work collaboratively to design Individualized Education Plans (IEP) or treatment programs for children with motor problems. Purpose of the Study The Movement Assessment Battery for Children (MABC) is comprised of the MABC Checklist and a modified version of the Test of Motor Impairment- Henderson (TOMI-H)(Stott, Moyes, & Henderson, 1984; Sugden, 1985). The checklist of the MABC battery purports to be a screening tool that allows teachers to identify children at risk for motor difficulties. The purposes of the study were: 1) to examine the relationship between teachers' scores on the MABC Checklist and the physiotherapist's scores on the MABC Motor Performance Test; and 4 2) to examine the degree of concordance in decision-making by detennining if the teachers and physiotherapist, using these two measures, identify the same children with motor difficulties and those without motor difficulties. Definitions of Terms The terms used in this proposal were defined as follows: Norm-referenced test: A test that judges individual performance in relation to group norms (Domholdt, 1993). Criterion-referenced test: A test in which scores are interpreted on the basis of absolute criteria (e.g., the number of items answered correctly) rather than on relative criteria, such as how the rest of the normal group performed (Derstine, 1994). Correlation coefficient: A correlation coefficient is a statistical summary of the degree and direction of the relationship or association between two variables (Glass & Hopkins, 1984). Sensitivity: Sensitivity is the proportion of children who truly have a motor coordination problem in the screened population who are identified by the screening test. Sensitivity is a measure of the probability of correctly diagnosing a child, or the probability that any given child will be identified by the screening test (Last, 1983). Specificity: Specificity is the proportion of children who truly do not have a motor coordination problem who are so identified by the screening test. It is a measure of the probability of correctly identifying a child without a motor problem with a screening test (Last, 1983). 5 Positive predictive value: Positive predictive value is the portion of children correctly identified as positive by the checklist compared to all the children who tested positive (Carran & Scott, 1992; Domholdt, 1993). Negative predictive value: Negative predictive value is the proportion of the children correctly identified as negative by the checklist compared to all the children who tested negative (Carran & Scott, 1992; Domholdt, 1993). 6 CHAPTER II LITERATURE REVIEW This chapter examines recent literature on the identification of children with motor difficulties. This includes issues related to the development of motor skills and the secondary problems that arise from poorly developed motor skills. Issues related to screening children with motor difficulties, and teachers' ability to screen for those difficulties are also reviewed. Finally the MABC was examined with the purpose of establishing whether the MABC Checklist could serve as a suitable motor screening test for classroom teachers in British Columbia. The Development of Motor Skills The motor performance that is required for full participation in school programs becomes more demanding with age. This expectation is supported by studies that have shown a positive relationship between motor skills and age in school-age children (Butterfield & Loovis, 1993; Strohmeyer, Williams & Schaub-George, 1991; Thomas, 1980). When competence in complex motor tasks is expected in the school setting around the age of seven or eight, motor problems that interfere with academic and athletic performance, such as poor handwriting and difficulties with sports, become apparent (Laszlo & Sainbury, 1993). It is at this time that teachers notice those children with motor difficulties and refer them to physiotherapy or occupational therapy for assessment of these more complex motor skills. Much research has examined the relationship between age and the development of motor skills (Fischman, Moore and Steele,1992; Gabbard & Hart,1993; Hay, Fleury, Bard 7 & Teasdale, 1994; Magalhaes, Koomar & Cermak, 1988; Parker, Larkin & Ackland, 1993; Speigel, Steffens, Rynders & Braininks, 1990; Sugden & Sugden,1991). The development of motor skills and functional movement starts early in life. Two studies by von Hofsten (1980, 1983) documented the ability of four month old infants to anticipate the location and reach successfully for a moving object. Cratty (1986) described distinct age-related stages in the development of two-handed catching. In a study on the motor skills of 6 to 12 years olds by Hughes and Riley (1981), a task requiring children to perform jumping jacks was omitted as it was too difficult for six year olds to perform, again supporting the perception that the ability to perform complex motor skills increases with age. School therapists have also noticed that teachers refer many children aged seven or eight for assessment of their abilities to perform more difficult manual dexterity tasks in the classroom and more coordinated gross motor tasks in sports activities. This study will therefore examine the motor skills of children aged seven or eight years because it is at this age that children are expected to perform more complex tasks. In the investigation of age-related motor skills, researchers have found it important to specify the conditions under which the task is being performed. Although models of motor control and motor learning are still evolving, the work of Adams (1971), Gentile (1972), Higgens (1991), Saltzman & Kelso (1987), Schmidt (1982) and others have brought together knowledge from the fields of psychology and kinesiology to further the understanding of motor skill acquisition. Traditionally physiotherapists have analyzed motor skills in terms of properties such as time, force, direction, and sequencing. We now know from the work of psychologists and kinesiologists that the context in which the movement occurs, such as environment, motivation and attitude should also be taken into 8 account. Equally important are the cognitive aspects of motor learning such as attention, retention and recall (Bandura, 1977). Persistent Motor Delays and Secondary Concerns One reason for identifying motor difficulties in school-age children is that there is evidence to suggest that these problems not only persist but also lead to secondary complications. Losse and colleagues found that children who had motor difficulties when first tested, still had below average motor skills ten years later (Losse, Henderson, Elliman, Hall, Knight & Longmans, 1991). These children also had a higher incidence of academic failure, more mental illness and a higher rate of poor self-esteem. Blondis, Snow, Roizen, Opacich, and Accardo (1993) examined five year old children with motor-delay in a three year follow-up study. This study showed significant motor gains by the children with motor-delay at the end of the first year, but this group remained significantly delayed compared to children in the control group when tested after three years. Cantell, Smyth and Ahonen (1994) reported on a ten year follow-up study of Finnish children who were diagnosed as \"clumsy\" at the age of five. Almost half of the children identified with motor problems at five years continued to have significant difficulty with motor skills. The adolescents with persistent motor difficulties had fewer social hobbies and lower academic achievement. However, although it was thought that they were less physically and socially competent than their peers, the adolescents did not have poor 9 opinions regarding their social acceptance or self-worth. The authors suggested that this may reflect a difference in cultural values, Le. that outstanding performance in sports is not highly regarded among Finnish people. An eight year follow-up study by Knuckey and Gubbay (1983) reported that children with mild and moderate motor difficulties, ages ranging from six to 12 at the commencement of the study, were found to catch up to their peers. However, no report was given on whether or not interventions were received during the eight year period. Children with severe motor coordination difficulties had less favorable outcomes. The study also reported that while the children with mild and moderate impairments improved their motor skills, they had less skilled jobs than the control group and their academic achievement was lower, even when there was no difference in intelligence. The most commonly reported social problem related to motor difficulties is one of poor self-esteem (Willoughby, King & Polatajko, 1995). Reviewing the current literature it would appear that the relationship between motor skills and social development depends not only on the specific aspect of social development that is studied but how that aspect is defined. The following studies have examined specific aspects of social development and their relationship to motor skills. Willoughby, Polatajko and Wilson (1995) used Harter's model of the development of self-esteem to distinguish feelings regarding academic and scholastic competence from feelings associated with acceptance by peers and adults (Harter, 1986). In this study, they found that children with learning disabilities (LD) and developmental coordination disorder (DCD) did have significantly lower levels of self reported physical and cognitive competence than the control group. However, a 10 more favorable result was that children with LD and DCD did not report significantly lower levels of maternal or peer acceptance. Research from Schoemaker, Hijlkema, and Kalverboer (1994) supported other studies that found children with motor difficulties to be more introverted, more anxious and to judge themselves less physically and academically competent. When they compared the children with severe motor difficulties to children with social and emotional problems, there was less socially negative behavior among children with severe motor problems. In an earlier study by Zimmer, mixing socially was found to be a problem in preschool children with motor delays (Zimmer 1981, as cited in van Rossum, van der Born & Vermeer, 1992). Contrary to those findings, however, when van Rossum, van der Born, and Vermeer (1992) examined social inhibition, defined as a term to indicate the phenomenon of social neglect or withdrawal from the peer group, they found no relationship between motor skills and social inhibition in first, third and fifth grade students. However, they did find that students who had received intervention for their motor delays scored lower on the motor performance test and scored higher on the social inhibition scale than a control group of matched classmates without motor delays. Thus, social inhibition may be an important outcome to assess when evaluating the efficacy of intervention programs for children with motor difficulties. The evidence provided in these studies indicated that issues relating to age, social development and environment should be taken into account when selecting a screening test for identifying children with motor difficulties. 11 Screening Tests. According to Meisels (1988), screening tests have been used as a means to identify children with developmental delays for the last thirty years. The need for teachers to have a reliable test for screening motor difficulties has also been recognized for many years by professionals in the fields of physical education and educational psychology (Henderson, 1987; Johnston, Crawford, Short, Smyth & Moller, 1987; Meisels, 1989; Stott, Moyes, & Henderson, 1984). Many screening tests have been developed to identify infants and preschool children with developmental delays, but these tests were not designed to identify motor problems in school-aged children. Most school screening tests focus on the identification of educational problems such as poor reading abilities and poor comprehension (Compton, 1984). A few screening tests have been developed to examine motor difficulties in school-age children, such as Gubbay's test (Gubbay, 1975), the South Australia Motor Coordination Screening Test (SAM Test; Johnston et aL, 1987) and more recently, the MABC Checklist (Henderson & Sugden, 1992). Both Gubbay's test and the SAM Test require teachers to administer standardized fine and gross motor items described in each test. It was thought that teachers would be reluctant to perform specific tests but would be more likely to complete a checklist from classroom observations. No screening test is presently used by teachers to identify children with motor difficulties in schools in British Columbia. The present referral system requires a teacher to refer all children suspected of having motor problems to an occupational therapist or a physiotherapist for assessment. This could result in the administration of a diagnostic test of motor proficiency which is time consuming and expensive. It may also result in many children with minor motor 12 concerns, who might have been eliminated from the wait-list by a screening test, undergoing lengthy diagnostic tests and delaying service to children with more significant problems. Screening tests are designed to identify children who are at risk for developmental delays in a specific area and who may need further assessment to establish a diagnosis (Meisels, 1988). In order to be useful, a screening test should be brief and easy to use, as well as reliable and valid (Meisels & Shonkoff, 1989). Screening tests are often the first phase in a series of assessments needed by a child. It is therefore important that the results of the screening test will direct the appropriate children to further assessment. Screening tests may be administered to apparently asymptomatic populations to determine those at risk, but in the case of school- aged children with motor difficulties, their classroom teacher may already suspect that a child has a problem. This screening test would be used to assess the severity of the problem to determine if further assessment is warranted. Teachers' Application of Screening Tests for Motor Problems. The most valuable place to perform a screening test to identify school-aged children with motor problems would be in the school environment. Teachers in elementary school often use screening tools to perform preliminary assessments of academic skills such as reading, spelling and mathematics. It would therefore seem appropriate that teachers could administer preliminary screening for motor difficulties as welL There is evidence in the literature to support the idea that teachers make accurate and reliable judgements regarding children's motor skills. Henderson and Hall (1982) examined the abilities of 13 weU-informed teachers to recognize motor impairment in their students and correlated their observations with those of a pediatrician and a psychologist using more structured methods. The results showed teachers to be very accurate (agreement 98%). This finding has been further substantiated by Lam and Henderson (1987) and Sugden and Sugden (1991). No valid and reliable screening tool for motor impairment was available for teachers until recently. Now the Movement Assessment Battery for Children (MABC) has been developed and is purported to include a screening tool for teachers to use to identify children with motor difficulties. Therefore the usefulness of this test was examined with the purpose of ascertaining whether the checklist would be an appropriate screening test for teachers' use to identify children with motor problems in British Columbian schools. The Movement Assessment battery for Children (MABC ) The Movement Assessment Battery for Children, developed by Henderson and Sugden (1992), is comprised of a performance test (the MABC Test) and an observational checklist (the MABC Checklist) as well as guidelines for intervention. The MABC Test The MABC Test is a normative-referenced, standardized test of motor performance designed to identify children with motor impairment (Henderson & Sugden, 1992). The edition of this motor performance test used in the MABC is an updated version of the Test 14 of Motor Impairment- Henderson revision 1984 (TOMI-H) (Stott, Moyes & Henderson, 1984). Studies establishing the validity of the TOMI-H (Stott et aL, 1984) were referenced in discussing the validity of the MABC Test in the MABC manual A study by Henderson and Hall (1982) demonstrated that there was a high degree of concordance among teachers using classroom observation, pediatricians using a neurodevelopmental assessment and scores on the TOMI in identifying children with motor coordination difficulties (agreement = 89%). Another study by Lam and Henderson (1987) also supported the validity of the MABC Test. Because children with mental handicaps also have difficulties performing age-appropriate motor skills, the first part of this study compared the motor skills of a group of boys with known mental handicap and a group of the same-aged boys using the TOMI-H. There was a significant difference between the scores of each group on the TOMI-H. This study also examined the relationship between the boys with mental handicaps' scores on the TOMI-H with their scores on an earlier version of the MABC Checklist, then called the Teacher Checklist (Keogh, Sugden, Reynard & Calkin, 1979). A strong positive relationship was found between the scores on the TOMI-H and scores on the Teacher Checklist (r = 0.88, p <0.001). Riggen, Ulrich and Ozman (1990) also found the TOMI-H to be a reliable and valid test of motor impairment. In their study, they examined reliability of the TOMI-H as a tool measuring motor impairment by calculating the standard error of the measure (SEM) which they found to be 0.86. Calculation of the concurrent validity, using the Bruininks-Oseretsky Test of Motor Proficiency-Short Form as the standard, resulted in an 88% agreement between the two tests. However, some criticisms of the TOMI-H (Stott, 15 Moyes & Henderson, 1984) relating to gender bias were found in a study by Causgrove, Dunn & Watkinson (1996). Significantly more boys than girls aged 9 to 12 were found to be able to pass the catching and throwing tasks. Eight-five percent of the boys passed the catching item compared to only 49% of the girls and 50% of the boys passed the throwing item while only 22% of the girls were successful This difference will have to be given consideration when analyzing the data in studies using the MABC Test with children in this age range. The fact that the MABC Test and its predecessor, the TOMI-H, have frequently been used to identify children with motor impairment in other research studies supports the use of the MABC Test to measure motor impairment in the proposed study (Cermak, Ward, & Ward, 1986; Henderson, Rose & Henderson, 1992; O'Brien, Cermak & Murray, 1988; Piek & Coleman-Carman, 1995; Sims, Henderson, Hulme & Morton, 1996). The positive relationship between the MABC Test's scores and children's motor performance found in the described studies adds further support of the use of the MABC Test as a measure of motor impairment in this study. The MABC Checklist The checklist is a criterion-referenced test that can be filled out by the classroom teacher over a period of one to two weeks to allow time for careful observation of the child's movements in classroom and playground settings. This checklist was originally developed by Sugden (Sugden, 1972 as cited by Henderson & Sugden, 1992) to enable teachers to assess motor difficulties in kindergarten children. It was revised in 1984 and then modified once again before becoming part of the MABC package (Henderson & 16 Sugden,1992). It is primarily designed for elementary school teachers to enable them to envisage the child's motor problems in relation to the demands of the school environment (Wright & Sugden, 1996). The tasks required to be performed on the MABC Checklist are analyzed by examining both the child and the environment. The checklist looks into situations where the child is stationary or moving and the environment is either fixed or changing. It is therefore possible to distinguish between occasions when a child is in control of his/her actions and occasions when a child must modify his/her movements in response to the demands of the changing environment (Wright, Sugden, Ng, and Tan, 1994). It was reported in the manual that teachers were able to complete the checklist accurately and found no difficulty with the instructions given in the manual or with the scoring. The usefulness of the MABC Checklist for teachers was also supported in the study by Wright et aL (1994). In a pilot study to this study, teachers confirmed that the checklist was a user-friendly tool that they could complete without difficulty. Meisels (1988) noted that a good screening tool must be both reliable and valid as well as quick and easy to adrninister. The reliability of a test \"concerns a test's repeatability, that is, how often a test yields identical results on repeated tests.\" (Meisels, 1988, p. 530). The test-retest reliability of the MABC Checklist was examined and is reported in the manual (Henderson & Sugden, 1992). Fifty teachers were asked to complete a checklist on a child. Of these 41 checklists were returned. A second checklist was then scored and the degree of agreement between the two sets of scores was explored. Correlation coefficients were reported for the total scores on the four motor sections as well as the four motor section sub-scores. These data demonstrated a high 17 correlation r= 0.89 (p< 0.001) for the total scores. The correlation coefficients for each of the four motor sections were also statistically significant (Section 1, r = 0.88; Section 2, r = 0.84; Section 3, r = 0.77; and Section 4, r = 0.76). Evidence to support the reliability of the checklist was also found in the study by Wright and colleagues (1994) in which the results demonstrated a high level of agreement between test and retest. Reliability data were reported on 120 children's checklist scores. The correlation coefficients for all sections were statistically significant, except for those 8 year old girls on Section 2 (r >0.33 to r < 0.86; p< 0.05). Validity concerns \"the crucial relationship between the test and some external criterion against which the accuracy of the test is being measured.\" (Meisels, 1988, p. 530). Using a valid test is important because all screening tests result in some degree of false positive and false negative errors. Research on the construct validity of a test helps determine the degree and type of errors with which the tester should be aware when using the test. As reported in the manual, the validity of the MABC Checklist was explored in two different studies. In one study, the relationship between the teachers' judgements of the children's motor skills and the children's scores on the MABC Checklist was examined. The 47 children identified by their teachers as having motor difficulties were then compared with other children randomly selected from the same classrooms using the MABC Checklist. A one-way analysis of variance showed the difference between the two groups to be highly significant (F = 85.2, d.f. = 1348; p < 0.0001). These results support the view that the MABC Checklist is valid test of motor impairment. Construct validity was also examined by detennining the relationship between the MABC Checklist and the TOMI-H. (Henderson & Sugden, 1992). Agreement between 18 the teachers' identification of children whose abilities were typical and identification of children within normal limits on the TOMI-H was given as high although specificity was not reported. Only two of the 32 children were not identified by the checklist who were found to have motor problems when tested on the TOMI-H. Teachers using the MABC Checklist identified more children as having motor problems than using the TOMI-H. Seven out of 16 children identified on the MABC Checklist scored below the 15th percentile on the TOMI-H. As described in the manual, there could be several reasons for this, Le. teachers were overzealous in their scoring of motor problems, or that the TOMI-H misses children about whom the teacher is concerned. The authors stated that a more likely explanation was that the screening test did not differentiate the cause of the deficit to be a motor problem and therefore included children who had attention deficit, hyperactivity and other problems that may have interfered with their ability to perform motor skills (Henderson & Sugden, 1992). Studies by Wright et aL (1994) and Wright and Sugden (1996) also found the MABC Checklist to be a suitable screening tool for children in Singapore. These two studies formed part of a larger project that investigated the appropriateness of using the MABC Checklist and MABC Test to identify children with motor difficulties in Singapore. In the study by Wright et aL (1994), the MABC Checklist correctly identified children who did not have a motor problem but there were concerns with the suggested cut-off points for identifying children with motor impairment. Although not fully explained in the article (Wright et a! 1994), these difficulties were related to specific items in the checklist on which teachers in Singapore had difficulty commenting, such as the child's ability to ride a 19 bicycle or to use blocks and beads. As a result, it was suggested that some adaptations to the checklist be made for its use in Singapore. In the study by Wright and Sugden (1996), 64 children, between the ages of six and nine, were selected by the MABC Checklist as having motor problems (n = 427). When these children were tested on the MABC Test, only 17 were confirmed as having motor problems although approximately half had borderline scores. This study goes on to suggest that the cut-off points of 5% and 15% for the UK data may need to be adjusted for Singaporean children. Wright and Sugden (1996) also discussed the possibility that teachers in Singapore may be more accepting of younger children's motor difficulties than they are of those of older children as teachers tended to identify more children as having problems as they got older: 1% of 6- and 7-year olds were identified whereas 14% of 9 year olds were identified. As the normative data for the MABC Test were originally collected on English and Canadian children, it is less likely that these cultural differences will occur in the proposed study. A recent study ( Piek & Edwards, 1997) examined the identification of children with developmental coordination disorder by class and physical education teachers. Using the MABC Checklist as a screening tool, they found that classroom teachers only identified 25% of the children with a motor coordination problem while physical education teachers identified 49 % of the children with motor problems. Neither this study nor the study by Wright and Sugden (1996) examined the individual classification decisions, ie. sensitivity of the checklist, so the true number of children not identified or the false negative decisions using the checklist as a screening tool can not be determined. 20 The Research Questions. The MABC Checklist was proposed for use as a screening test for teachers to identify children with motor problems in schools in British Columbia. The MABC Checklist has been shown to be valid and reliable and to be quick and easy to administer. Teachers have shown themselves to be able to make reliable judgements regarding children's motor skills. The importance of early identification of motor problems in school-aged children was supported by evidence that these motor skill problems persist over the lifespan and may add secondary problems such as social, academic and economic difficulties, over time. The following research questions were therefore proposed. Research Question I What is the relationship between teachers' scores on the MABC Checklist and a physiotherapist's scores on the MABC Motor Performance Test in identifying school-age children with motor difficulties? Research Question II Do teachers and physiotherapists using these two different measures identify the same children as having motor difficulties and the same children as not having motor difficulties? 21 CHAPTER m METHODOLOGY This study design was a cross-sectional study using a stratified random sample with replacement of 103 children aged seven and eight. It examined the association between teachers' scores on the MABC Checklist and children's scores on the MABC Test as administered and scored by a physiotherapist. This chapter describes the methodology used in the study. Subjects, data collection, and data analyses are also described. Participants A stratified random sample of 164 children was selected from ten elementary schools in School District 43 (Coquitlam) in British Columbia. Originally it was decided to select 12 children from each school, using a table of random numbers, hoping that 10 signed consent forms would be returned by the parents . However, some school personnel suggested that, due to the high ratio of parents whose first language was not English in their schools, a sample of 20 randomly selected children would be more prudent. In schools where there was a high percentage of Chmese-speaking children, a notice in Cantonese was included in the pack sent home suggesting that it may be useful to ask an English speaking friend to assist the parent to fill out the consent form. In the four schools where larger samples were requested, the return rate for the consent forms ranged from 35% to 55%. This confirmed the teachers' concerns that in schools with a high enrolment of students from new immigrant families where English was a second language, consent would be low. In the other six schools, the return rate ranged from 75% to 100%. The random sample from the Coquitlam School District was thought to be representative of 22 other school districts in the Lower Mainland in British Columbia and contained a cross-section of children from different socio-economic backgrounds as well as from different cultural backgrounds. The teachers received a total of 110 consents from parents who agreed to allow their children to participate in the study. Because three children had passed their ninth birthday by the time of the test they were excluded from the study. Also four checldists were not returned. All the children in the study were aged seven or eight at the time of testing and were attending a Grade 2, Grade 3 or Grade 2/3 split classroom. The sample consisted of 22 girls and 33 boys aged seven and 21 girls and 27 boys aged eight. (Fig.l; table 1). FIGURE 1 Age and gender of children in the sample H female Bmale age 7 age 8 23 TABLE 1 AGE AND GENDER OF CHILDREN IN THE SAMPLE GENDER female male Row total 7 years 22 33 55 53.4% 8 years 21 27 48 46.6% Column total 43 60 103 41.7% 58.3% 100% Because it is the policy of the Ministry of Education in British Columbia to place all children, including those with severe physical and mental handicaps, into integrated classrooms, children who have been identified by the Coquitlam School District of Coquitlam, B.C. as having other disabilities were excluded from participation. This excluded from the study children with a medical diagnosis such as cerebral palsy or spina bifida and children with severe mental handicaps, severe behavior problems or severe language disorders. These children were already identified within the school system and it would serve no purpose to subject them to further motor screening. The incidence of motor coordination difficulties in young children has been reported to be 8% to 15 % (Cratty, 1986), therefore a sample size of 100 children was chosen to increase the likelihood of having some children with motor coordination difficulties present within the sample. 24 Procedure Prior to the commencement of the study, approval was received from the University of British Columbia (UBC) Ethics Review, Behavioural Sciences Screening Committee. Permission for the students and teachers to participate in the study was obtained from the Coquitlam School District Superintendent, the school principals of the selected schools and the parents of the selected children. Permission from individual school teachers was not sought as it is the prerogative of the school principal to accept or refuse participation in research projects. The schools were selected by choosing every third elementary school in the school district starting at a randomly selected point in the list of schools. A letter explaining the purpose of the study and the procedure involved was sent to each of these individuals along with the appropriate consent form. Following receipt of the signed consents, the classroom teachers of the selected children were provided with information regarding the study and written instructions on how to complete the MABC Checklist. Because this screening test was not designed to require any specific training, no instructional course was given. This ensured that the use of the checklist in this study closely resembled how it would be used in the community if adopted as a screening tool to be used in the classroom. Teachers were given an opportunity to seek clarification of any items on the checklist prior to the commencement of the study. The study was commenced in the spring in order to give the classroom teachers sufficient time to know their students. They were asked to complete the MABC Checklist 25 on the selected children in their class over a period of two weeks. The completed checklists were collected, in most cases prior to the administration of the MABC Test. The physiotherapist administered the MABC Test individually to the selected children in each classroom. In order to prevent examiner expectation bias, the physiotherapist had no prior knowledge of how the children had been scored on the checklist. The MABC Checklists and MABC Test score sheets were held at Sunny Hill Health Centre for Children until all the children had been tested. After the circulation and collection of the initial checklists, seven out of ten randomly selected teachers who had been requested to fill out a follow-up checklist received a second checklist. The purpose of this second checklist was to assess test-retest reliability of the MABC Checklist when used as a screening test by classroom teachers. The second checklist was completed within two weeks of the teacher completing the first checklist. The results showed that for the seven teachers (20% of the teachers involved in the study) who completed the second checklist there was 97.3% agreement between their first and second scores. Inter-rater reliability demonstrates the consistency among different examiners' ratings on the administration of the same test. Although only one examiner was used in this study, the investigator was asked to establish this examiner's reliability administering the MABC Test. Prior to the study, inter-rater reliability of the examiner was established by a second physiotherapist, experienced in the administration of standardized tests, simultaneously scoring the MABC Test on a group of typical seven and eight year old students (n=4). Agreement between the examiner and the second physiotherapist was found to be 96.5%. Inter-rater reliability of the physiotherapists scores on the MABC 26 Test was assessed by the same method at one school during the study. Seven children were scored by both physiotherapists at this school in accordance with the number of parental consents received. Agreement between the examiner and the second physiotherapist was found to be 92.7%. Measurements The MABC is comprised of the MABC Test and the MABC Checklist, which were both used in this study. The M A B C Test The MABC Test is a norm-referenced test of motor impairment. It is divided into four age-bands, each of which contain eight motor items which are placed in three categories: manual dexterity, ball skills, and balance. The scores on an individual item range from 0 to 5 and are ordinal High scores indicate a greater degree of difficulty with the item. The sum of the individual scores on the eight motor items provides a total impairment score. This test may be administered by a professional from either a rehabilitation therapy or educational background. The MABC Test was standardized simultaneously in the UK and Canada with approximately 600 children in each country. Because there was no significant difference between the two groups of children, the test items were thought to be appropriate for children in British Columbia. The MABC Test uses two cut-off points to identify a child with motor problems. Total impairment scores, at or below the 5 t h percentile, are indicative of a definite motor impairment, while total impairment scores between the 6 t h and 15th percentiles are 27 classified as borderline or \"at risk\". The rationale for these cut-off points is that not all children with motor difficulties have motor impairments. Children in the motor impairment range have more severe problems and may benefit from adapted physical education programs or classroom strategies such as a reduced written output program. Children in the \"at risk\" category may have motor difficulties due to immaturity, attention deficit or hyperactivity problems and may benefit from increased exposure to physical activities or further medical assessment. The MABC Checklist The MABC Checklist is a criterion-referenced screening tooL In the checklist, there are five sections. The first four relate to movement skills. Section 1 assesses the stationary child in a stable environment and includes items such as threading beads, writing and cutting. In Section 2, the child is moving in a stable environment such as running to kick a stationary ball and walking around the classroom avoiding collision with stationary objects. Section 3 involves the stationary child in a moving environment such as catching a ball or hitting a moving ball with a bat. Section 4 deals with the moving child in a moving environment, such as running to catch an approaching ball and moving around the classroom without colliding with other moving students. The final section is made up of items that relate to the child's behavior while participating in motor activities. Section 5 is scored and analyzed separately from the rest of the checklist. Each item on the first four sections of the checklist is measured on a four-point ordinal scale, Very well = 0, Just OK = 1, Almost = 2, and Not Close = 3. There are twelve items in each of four motor 28 sections on the checklist so that the total scores on the checklist range from 0 to 144. The summation of the scores on each section provided the interval data required of the data analysis. Examiners The classroom teachers filling out the MABC Checklists were qualified elementary school teachers with knowledge of the development of motor skills acquired by children in the primary grades. The physiotherapist who administered the MABC Test had twenty-two years' experience in paediatric physiotherapy as well as fifteen years' experience in administering standardized tests of motor performance. The physiotherapist was enrolled in the M.Sc. program in the School of Rehabilitation Sciences at the University of British Columbia. Data Analysis The following hypotheses were derived from the research questions stated in Chapter II. A description of the data analyses relating to these hypotheses is then given. Null Hypothesis I There is no relationship between the teachers' scores on the MABC Checklist and the therapist's scores on the MABC Test in identifying children with motor difficulties. Null Hypothesis II There is no agreement between classroom teachers and the physiotherapist in categorizing children with and without motor difficulties. 29 The first hypothesis was tested by calculating the correlation coefficient between the teachers' scores on the MABC Checklist and the physiotherapist's scores on the MABC Test. The data were examined to see if the assumptions of correlation coefficients were met. Given that the relationship between the scores on the checklist and the scores on the MABC Test were expected to be linear and that the data were continuous, a Pearson product moment correlation coefficient was used to calculate the relationship. A scatterplot of the data was examined to verify that the relationship formed a straight line. Strength of the coefficient was interpreted using Munro's descriptive terms for the strength of correlation coefficients (Munro, Visintainer, & Page, 1986). As both the MABC Checklist and the MABC Test measure motor skills, a moderate to high correlation was expected. TABLE 2 Munro's Descriptive Terms for the Strength of a CorrelationCoefficient .00- .25 little, if any correlation .26- .49 low correlation .50- .69 moderate correlation .70- .89 high correlation .90- 1.00 very high correlation Null hypothesis n was analyzed using 2x2 contingency tables. The sensitivity and specificity of the MABC Checklist were described by examining the proportion of children, identified by the checklist to have motor difficulties, who were found also to have problems when tested with the MABC Test (Sensitivity = a /a+b xlOO; specificity = b /b+d x 100). The positive and negative predictive values were also calculated (Positive 30 predictive value = a/a+b xlOO; negative predictive value = c/c+d xlOO). A cut-off point at the 15th percentile was used to identify children whose motor difficulties placed them in an \"at risk\" category. The percentage agreement between the MABC Checklist and MABC Test results was calculated. A cut-off point at the 5 t h percentile was used to identify children with a definite motor problem on both the checklist and performance test. The percentage agreement of this cut-off point was also calculated. These cut-off points were established when the MABC Test was originally developed using a sample of British children and a sample of Canadian children. It was anticipated that these cut-off points would be appropriate for the sample of children in this study. 31 CHAPTER IV RESULTS The data analysis for each research question described in Chapter JJI is reported in this chapter. The first purpose of the study was to examine the relationship between the teachers' scores on the MABC checklist and the physiotherapist's scores on the MABC Test. This was examined using a Pearson product-moment correlation coefficient, (one-tailed test). A one-tailed test was used because the hypothesis was stated in the null form and the measurement tools only identify children with motor difficulties. The correlation coefficient provides the examiner with information regarding the strength of the relationship between the two different measurement tools. The correlation between the scores on the MABC Checklist and the MABC Test for the total sample (103 seven and eight year old boys and girls) was r = 0.51 (p = 0.000). The scatter plot illustrates this relationship (Figure 2). 32 FIGURE 2 SCATTER PLOT of M A B C Test Scores and M A B C Checklist Scores 40-I 30- o 20- \u00E2\u0080\u00A2 \u00C2\u00B0 \u00E2\u0080\u00A2 10-0-o o B n \u00C2\u00B0 oop o \u00C2\u00BB \u00E2\u0080\u009E \" \u00C2\u00B0 8 \u00C2\u00B0 * m t f \u00C2\u00B0 _o ^fi, & \" \u00C2\u00B0 D \u00C2\u00B0 n fc V . / -a o -10 -20 0 20 40 60 80 100 CHECK The scatter plot revealed two outliers. Points on the scatter plot are considered to be outliers when one variable or a combination of the variables has an extreme value that unduly influences the statistics (Tabachnick & Fidell, 1989). When these two outliers were removed from the sample the correlation coefficient increased (r = 0.60, p = 0.000) but remained in the moderate range according to Munro's descriptive terms for the 33 a correlation coefficient. The stability of the correlation was further examined by calculating the correlation coefficients independently for the group of seven year olds and the group of eight year olds and independently for boys and girls with and without the outliers. These results are presented in Table 3. TABLE 3 Stability of the Correlation Coefficient TOTAL SAMPLE Pearson r p-value A G E 7 YEARS (n=55) r = 0.63 p = 0.003 8 YEARS (n=48) r = 0.29 p = 0.023 NO OUTLIERS 7 YEARS (n=54) r = 0.69 p = 0.000 8 YEARS (n=47) r = 0.42 p = 0.002 TOTAL SAMPLE GENDER MALE (n=60) r = 0.61 p = 0.000 FEMALE (n=43) r = 0.29 p = 0.029 NO OUTLIERS MALE (n=59) r = 0.69 p = 0.000 FEMALE (n=42) r = 0.41 p = 0.004 34 The correlation coefficients remain in the moderate range for seven year olds and males but decrease to the low range for both eight years olds and females. The second purpose of the study was to examine the degree of concordance in decision-making by the teachers and physiotherapist using the two different tools. This was examined by calculating the sensitivity, specificity, positive predictive value and the negative predictive value at both the \"at risk\" and \"motor impairment\" cut-off points of both tools.( Tables 4, 5, 6, and 7; Figures 3, 4, 5, and 6). The sensitivity of the MABC Checklist refers to the percentage of the children correctly identified by the Checklist as having a motor problem. The sensitivity allows the examiner to assess the reliability of the result on the MABC Checklist. High sensitivity indicates that the MABC Checklist does reliably identify children with motor problems. Low sensitivity indicates that the MABC checklist misses children with motor problems. In this study the sensitivity for the \"at risk\" (15th percentile) cut-off point was 14.3% and for the \"motor impairment\" cut-off point (5th percentile), the sensitivity was 11.1% indicating that the MABC Checklist has low sensitivity. With the two outliers removed the sensitivity at both cut-off points remained the same. Specificity of the MABC Checklist refers to the percentage of children correctly identified as not having a motor problem. Low specificity indicates to the examiner that the checklist incorrectly identifies some children who do not have a problem as having a problem. The specificity at the \"at risk\" (15th percentile) cut-off point was 97. 8% and at the motor impairment (5 th percentile) cut-off point was 98.9% indicating that the MABC has very high specificity. With the two outliers removed the specificity at both cut-off points increased to 100%. 3 5 The positive predictive value of the MABC Checklist refers to the portion of children with motor difficulties correctly identified by the Checklist compared to all the children identified by the MABC Checklist as having motor difficulties. The positive predictive value tells the examiner how likely a positive result on the MABC Checklist is a true result. The positive predictive value in this study was 50 % at both the \"at risk\" and \"motor impairment\" cut-off points indicating that only half of the children identified by the checklist truly had a motor problem. With the removal of the two outliers the positive predictive value increased to 100% for both cut-off points indicating that all the children identified by the checklist truly had a motor problem. This also indicates that the both the outliers were false positives at the \"at risk\" cut-off point and that one of the outliers at the \"motor impairment\" cut-off point was a false positive. The negative predictive value of the MABC Checklist refers to the portion of children without motor difficulties correctly identified by the MABC Checklist compared to all the children identified by the MABC Test as not having motor difficulties. The negative predictive value tells the examiner how likely a negative result on the MABC Checklist is a true result. The negative predictive value at the \" at risk\"(15th percentile) cut-off point was 88% and at the \"motor impairment\"(5th percentile) cut-off point was 92%. This indicates that the MABC Checklist and the MABC Test had very good agreement when classifying the children who did not have motor problems. With the two outliers removed, the negative predictive values remained the almost same. 36 r 1 > dd O O B ffl n 00 H Sensitivity and Specificity for the \"At Risk\"(15th percentile) Cut-off Point M A B C TEST at risk\" p o i\u00E2\u0080\u0094t 14 not \"at r isk\" 89 FIGURE 3 TABLE 4 RESULTS USING THE \"AT RISK \" OR 15th PERCENTILE CUT-OFF POINT: Sensitivity = 2/14 = 14.3% Specificity = 87/89 = 97.8% Positive predictive value = 2/4 = 50.0% Negative predictive value = 89/99 = 87.9% Sensitivity and Specificity for the Motor Impairment(5th percentile) Cut-off Point MABC TEST r 1 > n n w o GO H impairment no impairment impairment i no impairment 8 93 9 94 101 F I G U R E 4 T A B L E 5 R E S U L T S U S I N G T H E \" M O T O R I M P A I R M E N T \" O R 5th P E R C E N T I L E C U T - O F F P O I N T : Sensitivity = 1/9 = 11.1% Specificity = 93/94 = 98.9% Positive predictive value =1/2 = 50.0% Negative predictive value = 93/101 = 92.1% 38 Sensitivity and Specificity for the \"At Risk' or 15th Percentile Cut-off Point (Outliers Removed) 5 O n w o r oo H M A B C TEST 'at risk\" not \"at risk\" 87 99 14 87 RESULTS WITH OUTLIERS REMOVED FIGURE 5 TABLE 6 RESULTS USING THE \"AT RISK\"OR 15th PERCENTILE CUT-OFF POINT (OUTLIERS REMOVED) Sensitivity = 2/14 = 14.3% Specificity = 87/87 = 100% Positive predictive value = 2/2= 100% Negative predictive value = 87/ 99 = 87.9% 39 Sensitivity and Specificity for the \"Motor Impairment\" or 5th Percentile Cutt-off Point (Outliers Removed) fe o o w o p I\u00E2\u0080\u0094H oo H MABC TEST impairment no impairment 92 100 92 FIGURE 6 TABLE 7 RESULTS USING T H E \"MOTOR IMPAIRMENT\" OR 5 t h PERCENTILE CUT-OFF POINT (OUTLIERS REMOVED) Sensitivity = 1/9 = 11.1% Specificity = 92/92 = 100% Positive predictive value = 1/1 = 100% Negative predictive value = 92/100 = 92% 40 CHAPTER V DISCUSSION The purpose of this study was to determine whether teachers using the MABC Checklist as a screening tool could identify children with motor difficulties. In this chapter the results are examined in more detail Some comparisons between this study and other studies using the MABC Checklist are made. The limitations of the study and the relevance of the results to teachers and therapists are discussed. Finally, possible future directions in the identification of children with motor difficulties are explored. Using Munro's descriptive terms (1986), the correlation coefficient between the teachers' scores on the MABC Checklist and the physiotherapist's scores on the MABC Test (r = 0.51, p = 0.000, and with the outliers removed, r = 0.60, p = 0.000) is considered to be in the moderate range. Both the MABC Checklist and MABC Test are purported to test motor skills and they are included in the same battery of tests because they examine motor skills from a different perspective. The MABC Test examines a few very specific activities that differentiate a child's motor skills from the norm, while the MABC Checklist uses observations of a large number of activities of different difficulty and complexity in different environments. The results indicate that only one quarter to one-third of the variance between the two sets of scores can be explained by motor difficulties. When the stability of the correlation coefficient was examined further it was found to be stable in the moderate range for males and seven years olds. However the correlation coefficient dropped to the low range for females and eight years olds (with and 41 without the outliers). This low correlation coefficient found with females and eight years olds suggests that the MABC Checklist is not an effective screening tool for these two groups. This finding regarding females and eight year olds was also reported in a study by Wright and Sugden (1996). They found a lower correlation coefficient for eight years old girls in a study of seven and eight year old children (n = 103) detennining the test-retest reliability of the MABC Checklist. In general this study's results demonstrate that although the overall correlation coefficient is in the moderate range it is not stable and that the correlation may not be sufficiently strong to consider the use of the MABC Checklist as a screening tooL The correlation coefficient illustrated by the scatter plot identified two outliers. Outliers are values that do not fall within the same range as most of the other values in the distribution (Portney & Watkins,1993). These two outliers have extreme values on one variable and therefore have a greater influence on the best fit line than the other points within the cluster (Tabachnick & FidelL 1989). Because these two outliers are of clinical interest, the data have been examined with and without them. The checklist scores for the two outliers were completed by the same teacher. When she was asked what her concerns were regarding these two children, she reported that she did not think they had motor coordination problems. She thought that they were noticeably immature in all areas of development compared to the other children in her class. Another extreme point was noted in the upper right-hand corner of the scatter plot. This case was also examined as a possible outlier. Children with severe mental or physical handicaps and children with severe language or behavior disorders were excluded from the sample. Children with learning disabilities were included in the sample because 42 of the high rate of co-morbidity between learning disabilities and motor coordination difficulties. This point on the scatter plot represented a child with severe learning disabilities who was considered by the school he attended to be within the same population as the rest of sample. Although he had extreme scores on both measures, these scores did not unduly influence the correlation coefficient and therefore this point was not removed. A correlation coefficient in the moderate range is not sufficient to determine the usefulness of the MABC Checklist as a screening tool The correlation does not determine if the Checklist and Test are identifying the same children with and without motor difficulties. Other studies have not examined this problem. In one study by Wright et aL, (1994), the sample of children with motor difficulties was determined by administering the MABC Checklist. They compared this group with an age and gender-matched group of children without motor difficulties using the MABC Test. This method of selection does not account for children with motor difficulties who were missed by the MABC Checklist. Information regarding the sensitivity, specificity, positive and negative predictive values of the checklist must also be considered when deciding the usefulness of a screening tool. In the present study, the sensitivity or the ability of the teachers using the Checklist to correctly identify children with motor problems was found to be poor. At the 15th percentile, the checklist failed to identify 12 out of the 14 children that the MABC Test identified as being \"at risk\" for motor problems. At the 5 t h percentile, the sensitivity of the teachers using the MABC Checklist was also poor. Removing the outliers did not change this result. The MABC Checklist missed eight out of nine children identified by the MABC Test as having a motor impairment. This result was very different from those 43 reported in the manual by Henderson and Sudgen (1992) where teachers using the MABC Checklist over-identified children as having motor problems. Only seven of the 16 children identified by the MABC Checklist were confirmed by the MABC Test. The low sensitivity found in this study means that teachers using the MABC Checklist would miss children who need help rather than over identify them. Therefore the MABC Checklist would not be of great value as a screening tool in the school system. Results of the present study show that the specificity, or the ability of the teachers using the Checklist to correctly identify children who do not have motor problems, was very accurate. At the 15th percentile, the checklist identified all but two of the same children without motor problems as the MABC Test. At the 5 t h percentile, there was only one child that the MABC Checklist identified as having motor impairment for whom the MABC Test did not corroborate. This result improved to 100% agreement between the MABC Checklist and MABC Test when the outliers were removed. This is a highly meaningful characteristic of a screening test in that if this screening tool identifies a child as not having a problem then the teacher can be almost certain that the MABC Test score would agree with this result. Using the positive predictive value, the results of the study first indicated that only 50% of the children identified by the MABC Checklist as having a motor concern, at either the 5th or 15th percentile cut-off points, truly had a motor concern. When the two outliers were removed from the sample this result improved to 100% . This indicates that although the MABC Checklist did not identify all the children with motor difficulties it was accurate regarding those it did identify. Using the negative predictive value, the results showed that the MABC Checklist will correctly identify a high proportion of 44 children without motor concerns compared to the total number of children the MABC Checklist identifies. Probably the most important characteristic of a screening tool is its sensitivity. If the sensitivity is low, many children who require services will not be identified. All the individual classification decisions were calculated both with and without the two outliers because outliers can dramatically change the number of false positive and false negative results in a sample (Tabachnick & Fidell, 1989). In this study, they effected the specificity and positive predictive values and, to a lesser extent, the negative predictive values. The sensitivity was found to be low with and without the outliers in the sample. The low sensitivity result indicates that the MABC Checklist would not be a reliable screening tool for identifying motor coordination difficulties. When considering this result it should be noted that the MABC Checklist was not originally designed as screening test, although its authors have used it as such (Wright et aL, 1994, Wright & Sudgen, 1996). It was intended to be part of a battery of tests and it does contribute information regarding a child's ability to function in different environments that is not examined by the MABC Test. Another important issue when considering the appropriateness of a screening tool is how well the sample used in the study represents the population at large. As noted earlier, the incidence of motor coordination problems in school-aged children had been report to be as high as 15% and as low as 8% (Cratty, 1986). Using the MABC Test as the standard in this study, 14 out of 103 children were identified as being \"at risk\" or 13.6% . Nine out of 103 children were identified as having a motor impairment or 8.7%. These results are very close to those reported in other prevalence studies. 45 Using the MABC Checklist, four out of 103 children(3.88%) were identified as being \"at risk\" and two out of 103 children(1.9%) were identified as having a motor impairment. These results are somewhat different from those reported by Wright et al. (1994) and Piek and Edwards (1997). In Wright et al's study (1994), the MABC Checklist identified 15 % of children (64 out of 427) as having motor difficulties or being \"at risk\", while in Piek and Edwards' study the MABC Checklist identified 18.7% of the children as having motor problems. Possible reasons that teachers did not identify the expected number of children with motor problems in the present study are discussed under limitations of the study. Limitations of the Study The 32 teachers who participated in this study were not specifically taught how to complete the checklist. They were simply given the instructions that were included with the test kit. The teachers provided some feedback regarding their ability to complete the checklist. They stated that they did not know if a child could or could not perform some of the items regarding activities of daily living such as the child's ability to dress. They also felt that they could not answer some of the other questions simply from observation and needed to test the child in order to give correct information. The intra-rater reliability of a sample of teachers in this study demonstrated that in spite of their concerns, teachers were consistent in their scoring of the MABC Checklist. In some schools, the classroom teachers did not teach the children physical education, therefore, they were not confident 46 to report the child's gross motor skills with complete accuracy. The other commonly mentioned criticisms from the teachers were that the checklist was too long and time-consuming to complete. These difficulties with the MABC Checklist are similar to those noted by teachers in Singapore who were involved in a study by Wright et aL(1994). This feedback suggests that teachers might have filled out the checklist differently if they had received more specific instruction. It may be worthwhile to replicate this study with the inclusion of teacher education in the administration of the MABC Checklist to verify if this makes a difference to the results. Also the large number of teachers (n=32) involved in this study may have effected the reliability of the MABC Checklist scores. However because inter-rater reliability between teachers was not assessed this could not be confirmed. The sample of children who participated in this study came from one school district in the Lower Mainland of British Columbia. Although Coquitlam School District is the second largest school district in BC, its ethnic diversity may be different from northern and rural areas of BC. The results of this study, therefore, may not be generalized to school districts outside the Lower Mainland. Staff from some of the schools in the study reported that more than 50% of the children in their school were from non-English speaking homes and a few of the children in the study had lived in Canada less than one year. These cultural and language differences may have affected the children's exposure to some of the tasks on the checklist. Also this may have influenced the teachers to be more lenient in their scoring of the MABC Checklist. 47 Summary and recommendations The results of this study did not support the independent use of the MABC Checklist by teachers to identify children with motor coordination problems. The results indicated that many children with motor problems would be missed if teachers were to use the MABC Checklist as a screening tool Knowing that the problems regarding the number of referrals to physiotherapy and occupational therapy services are unlikely to be resolved by increased government funding, future investigators must examine other avenues to resolve the difficult issues around service provision to children with motor coordination difficulties. This may be achieved by modifying the MABC Checklist as suggested by the teachers in this study and by providing the teachers with specific training in the use of the MABC Checklist. Further analysis of the MABC Checklist scores by section may reveal that certain sections of the Checklist would provide a more sensitive screening tool than the use of the whole Checklist. Future solutions may involve the development of a test for preschool children that would differentiate children with immature motor development from children with motor coordination difficulties prior to entering the school system. In keeping with the Ministry of Health's commitment to early intervention programs, these programs have considerably more occupational therapy and physiotherapy time available to assess children with motor difficulties. Earlier identification of motor coordination difficulties before the development of complex motor tasks requires more research, but if this could be done, it would reduce the need for a screening tool for school-aged children and allow school resources to be 48 directed to the development of interventions for motor coordination difficulties and to the prevention of the secondary problems that may develop. 49 REFERENCES Adams, J. A. (1971). A closed-loop theory of motor learning. Journal of Motor Behavior. 3(2). 111-149. Bandura, A. (1977). Social learning theory. Englewood Cliffs, NJ: Prentice HalL Blondis, T. A., Snow, J. H., Roizen, N . J., Opacich, K. J., & Accardo, P. J. (1983). Early maturation of motor-delayed children at school age. Journal of Child Neurology. 8. 323-330. British Columbia Ministry of Education. (1995). Special Education Services: A manual of policies, procedures and guidelines. Bmininks, R. H. (1978). Bruininks-Oseretsky Test of Motor Proficiency. Manual. Toronto, ON: American Guidance Service Inc. Butterfield, S. A., & Loovis, E. M . (1993). Influence of age, sex, balance, and sport participation on development of throwing by children in Grade K-8. Perceptual and Motor Skills. 76, 459-464. CantelL M . H. M. , Smyth, M . M. , & Ahonen, T. P. (1994). Clumsiness in adolescence: Education, motor, and social outcomes of motor delay detected at 5 years. Adapted Physical Activity Quarterly. 11. 115-129. Carran, D. T., & Scott, K.G. (1992). Risk assessment in preschool children: Research implications for the early detection of educational handicaps. Topics in Early Childhood Special Education. 12(2). 196-211. Causgrove Dunn, J., & Watkinson, E. J. (1996). Problems identifying children who are physically awkward using the TOMI. Adapted Physical Activity Quarterly. 13. 347-356. Cermak, S. A., Ward, E. A., & Ward, L.W. (1986). The relationship between articulation disorders and motor coordination in children. American Journal of Occupational Therapy. 40(8), 546-550. Compton, C. (1984). A guide to 75 tests for special education. Belmont, CA: David S. Lake Publishers. Cratty, B. J. (1986). Perceptual and motor development in infants and children. Englewood Cliffs, NJ: Prentice HalL 50 Derstine, S. (1994). Tests of infants and child development. In J. S. Tecklin (Ed.), Pediatric physical therapy (pp. 16-39). Philadelphia. PA: J. B. Lippincott. Domholdt, E. (1993). Physical therapy research: Principles and application. Philadelphia: W.B. Saunders, Co. Fischman, M . G., Moore, J. B., & Steele, K. H. (1992). Children's one-hand catching as a function of age. Research Quarterly for Exercise and Sport. 63, 349-355. Folio, M . R., & Fewell, R. R. (1982). Peabodv Developmental Motor Scales and Activity Cards. Hingham, MA: D L M Teaching Resources. Gabbard, C , & Hart, S. (1993). Foot-tapping speed in children ages 4 to 6 years of age. Perceptual and Motor Skills. 77. 91-94. Gentile, A. M . (1972). A working model of skill acquisition with application to teaching. Quest. 17. 3-23. Glass, G., & Hopkins, K. (1984). Statistical methods in education and psychology. Needham Heights, MA: Allyn and Bacon. Gubbay, S. S. (1975). The clumsy child. Philadelphia: W.B. Saunders. Harter, S. (1986). Processes underlying the construction, maintenance, and enhancement of the self-concept in children. In J. Suls & A. G. Grenwald (Eds.), Psychological perspectives on the self. Hillsdale, NY: Erlbaum. Hay, L., Fleury, M. , Bard, C , & Teasdale, N. (1994). Resolving power of the perceptual and sensorimotor systems in 6- to 10-year old children. Journal of Motor Behavior. 26, 36-42. Henderson, L., Rose, P., & Henderson, S. (1992). Reaction time and movement time in children with a developmental coordination disorder. Journal of Child Psychology and Psychiatry. 33(5), 895-905. Henderson, S. E. (1987). The assessment of \"clumsy\" children: Old and new approaches. Journal of Child Psychology and Psychiatry. 28(4), 511-527. Henderson, S. E., & HalL D. (1982). Concomitants of clumsiness in young children. Developmental Medicine and Child Neurology. 24,448-460. Henderson, S. E., & Sugden, D. A. (1992). Movement Assessment Battery for Children. Manual Sidcup, Kent: The Psychological Corporation Ltd. Higgens, S. (1991). Motor skill acquisition. Physical Therapy. Tl, 123-139. 51 Hinderer, K. A., Richardson, P.K., & Atwater, S.W. (1989). Clinical implications of the Peabody Developmental Motor Scales: A constructive review. Physical & Occupational Therapy in Pediatrics. 9(2), 81-105. Hughes, J. E., & Riley, A. (1981). Basic gross motor assessment. Physical Therapy. 61.503-511. Johnston, O., Crawford, J., Short, H., Smyth, R., & Moller, J. (1987). Poor coordination in 5 year olds: A screening test for use in schools. Australian Paediatric Journal. 23, 157-161. Keogh, J., Sudgen, D. A., Reynard, C , & Calkin, J. (1979). Identification of clumsy children: Comparisons and comments. Journal of Human Movement Studies. 5, 32-41. Knuckey, N . W., & Gubbay, S. S. (1983). Clumsy children: A prognostic study. Australian Paediatric Journal 19. 9-13. Lam, Y.Y. , & Henderson, S. (1987). Some applications of the Henderson revision of the test of motor impairment. British Journal of Educational Psychology. 57. 389-400. Last, J. M . (Ed). (1983). A dictionary of epidemiology. Toronto: Oxford University Press. Laszlo, J. I., & Sainsbury, K. M . (1993). Perceptual-motor development and prevention of clumsiness. Psychological Research. 55,167-174. Losse, A., Henderson, S. E., Elliman, D., Hall D., Knight, E., & Jongmans, M . (1991). Clumsiness in children - Do they grow out of it? Developmental Medicine and Child Neurology. 33, 55-68. Magalhaes, S. C , Koomar, J. A., & Cermak, S.A. (1988). Bilateral motor coordination in 5-to 9-year old children: A pilot study. American Journal of Occupational Therapy. 43, 437-443. Meisels, S. J. (1988). Developmental screening in early childhood: The interaction of research and social policy. Annual Review Public Health, 9, 527-50. Meisels, S. J. (1989). Can developmental screening tests identify children who are developmentally at risk? Pediatrics. 83,578-585. Meisels, S. J., & Shonkoff, J. P. (1989). Handbook of early childhood intervention. New York: Cambridge University Press. 52 Munro, B. H., Visintainer, M . A., & Page, E. B. (1986). Statistical methods for health care research. Philadelphia: J. B. Lippincott Co. O'Brien,V., Cermak, S. A., & Murray, E. (1988). The relationship between visual-perceptual motor abilities and clumsiness in children with and without learning disabilities. American Journal of Occupational Therapy. 42(6), 359-364. Parker, H. E., Larkin, D., & Ackland, T. R. (1993). Stability and change in children's skill Psychology Research. 55, 182-189. Piek, J. P., & Coleman-Carman, R. (1995). Kinaesthetic sensitivity and motor performance of children with developmental coordination disorder. Developmental Medicine and Child Neurology. 37, 976-984. Piek, J. P., & Edwards, K. (1997). The identification of children with developmental coordination disorder by class and physical education teachers. British Journal of Educational Psychology. 67. 55-67. Polatajko, H., Macnab, J., Anstett, B., Malloy-Miller, T., Murphy, K., & Noh, S. A. (1995). A clinical trial of the process-orientated treatment approach for children with developmental co-ordination disorder. Developmental Medicine and Child Neurology. 37, 310-319. Portney, L.G., & Watkins, M P.(1993). Foundations of clinical research. Englewood Cliffs, NJ: Prentice HalL Riggen, K J . V Ulrich, D.A., & Ozman, J.C. (1990). Reliability and concurrent validity of the test of motor impairment - Henderson revision. Adapted Physical Activity Quarterly. 2, 249-258. Saltzman, E., & Keslo, J. A. S. (1987). Skilled actions: A task-dynamic approach. Psychological Review. 94(1), 84-106. Schmidt, R. A. (1982). Motor control and learning. Champaign, IL: Human Kinetics Publishers. Schoemaker, M . M. , Hijlkema, M . G., & Kalverboer, A. F. (1994). Physiotherapy for clumsy children: An evaluation study. Developmental Medicine and Child Neurology. 36, 143-155. Sims, K , Henderson, S. E., Hulme, C , & Morton, J. (1996). The remediation of clumsiness. 1: An evaluation of Laszlo's kinaesthetic approach. Developmental Medicine and Child Neurology. 38, 976-987. 53 SpeigeL A. N. , Steffens, K. M. , Rynders, J. E., & Bruininks, R. H. (1990). The early motor profile: Correlation with the Bniininks-Oseretsky Test of Motor Proficiency. Perceptual and Motor Skills. 71. 645-646. Stott, D. H., Moyes, F. A., & Henderson, S. E., (1984). The Henderson Revision of the Test of Motor Impairment: A comprehensive approach to assessment. Adapted Physical Activity Quarterly. 3. 204-216. Strohmeyer, H. S., Williams, K., & Schaub-George, D. (1991). Developmental sequences for catching a small ball: A prelongitudinal screening. Research Quarterly for Exercise and Sport. 62. 257-266. Sugden, D.A. (1985). Test of Motor Impairment - Henderson Revision. By D.H. Stott, F. A. Moyes, S. E. Henderson. Published 1984 by Brook Educational Publ. Ltd., Guelph, Ontario. Adapted Physical Activity Quarterly. 2, 167-169. Sugden, D., & Sugden, L. (1991). The assessment of movement skill problems in seven and nine year old children. British Journal of Educational Psychology. 61. 329-345. Tabachnick, B. G., & FideU, L. S. (1989). Using multivariate statistics. New York, N.Y: Harper Collins Publishers, Inc. Thomas, J. R. (1980). Acquisition of motor skills: Information processing between children and adults. Research Quarterly for Exercise and Sports. 51.158-173. van Rossum, J. H. A., van den Born, S., & Vermeer, A. (1992). Social inhibition and motor skill performance in first, third and fifth grade children. Acta Paedopsychiatrica. 55.107-113. Verderber, J., & Payne, V. (1987). A comparison of the long and short forms of the Bruininks-Oseretsky Test of Motor Proficiency. Adapted Physical Activity Quarterly. 4. 51-59. Von Hofsten, C. (1980). Predictive reaching for moving objects by human infants. Journal of Experimental Child Psychology. 30 . 369-382. Von Hofsten, C. (1983). Catching skill in infancy. Journal of Experimental Psychology: Human Perception and Performance. 9, 75-85. Willoughby, C , King, G., & Polatajko, H. (1995). A therapist's guide to children's self-esteem. American Journal of Occupational Therapy. 50, 124-132. 54 Willoughby, C , Polatajko, H., & Wilson, B. (1995). The self-esteem and motor performance of young learning disabled children. Physical & Occupational Therapy in Pediatrics. 14,1-29. Wilson, B. N. , Polatajko, H. J., Kaplan, B.J., & Fans, P. (1994). Use of the Braininks-Oseretsky Test of Motor Proficiency. American Journal of Occupational Therapy. 49(1). 8-17. Wright, H. C , & Sugden, D. A. (1996). A two step procedure for the identification of children with developmental co-ordination disorder in Singapore. Developmental Medicine and Child Neurology. 38. 1099-1105. Wright, H. C , Sugden, D. A., Ng, R., & Tan, J. (1994). Identification of children with movement problems in Singapore: Usefulness of the Movement ABC Checklist. Adapted Physical Activity Quarterly. 11, 150-157. 55 APPENDIX PARTICIPANTS TEST SCORES > CO c CD SI CO co cn m cn CD c CD O 0 0 0 0 CO 0 0 CD C D CO CO CO CM CM > CO SL CD JO CO CM CM CN > CO CO CO O XI 03 E CM CD tn CO CO o cn o 00 CO 00 CM CM tri in co\" m m CD c CD ra CS c CD .\u00C2\u00A3> CM o <0 in oo co o \u00C2\u00AB in > a in co \u00E2\u0080\u00A2a u CD E CO CO CO o cu co co o oi oo o cri io o iri CD TJ C CD TO 00 00 00 CD TO CO IO o> o CO co m o> co co co co > CO sz CD J3 CM CO T5 CD CO CO CO CO > ID tn CO \"0 O n co E id CM o CD 10 co CO CO o CM CO CO CD u CO CM CM CO CO CM CM CO m co o \u00E2\u0080\u00A2h m co o CM CO JC CD CD in co TJ CD in > CD OT 3 CO E CD CO O CD co o CD CD c CD CD O co oo co CM CM co co o CM O CM O oo O) oo CM 00 cn cn oo o CO 00 "@en . "Thesis/Dissertation"@en . "1998-11"@en . "10.14288/1.0088596"@en . "eng"@en . "Rehabilitation Sciences"@en . "Vancouver : University of British Columbia Library"@en . "University of British Columbia"@en . "For non-commercial purposes only, such as research, private study and education. Additional conditions apply, see Terms of Use https://open.library.ubc.ca/terms_of_use."@en . "Graduate"@en . "Teachers\u00E2\u0080\u0099 use of the MABC checklist to identify children with motor difficulties"@en . "Text"@en . "http://hdl.handle.net/2429/8163"@en .