Open Collections

UBC Theses and Dissertations

UBC Theses Logo

UBC Theses and Dissertations

A study of the variables associated with the acceptance and rejection of A.E.R.C. abstracts Pipke, Ingrid 1981

Your browser doesn't seem to have a PDF viewer, please download the PDF to view this item.

Item Metadata

Download

Media
831-UBC_1981_A8 P56.pdf [ 11.77MB ]
Metadata
JSON: 831-1.0055836.json
JSON-LD: 831-1.0055836-ld.json
RDF/XML (Pretty): 831-1.0055836-rdf.xml
RDF/JSON: 831-1.0055836-rdf.json
Turtle: 831-1.0055836-turtle.txt
N-Triples: 831-1.0055836-rdf-ntriples.txt
Original Record: 831-1.0055836-source.json
Full Text
831-1.0055836-fulltext.txt
Citation
831-1.0055836.ris

Full Text

A STUDY OF THE VARIABLES ASSOCIATED WITH THE ACCEPTANCE AND REJECTION OF A.E.R.C. ABSTRACTS by INGRID PIPKE B.A., University of V i c t o r i a , 1975 A THESIS SUBMITTED IN PARTIAL FULFILLMENT OF THE REQUIREMENTS FOR THE DEGREE OF MASTER OF ARTS i n THE FACULTY OF GRADUATE STUDIES (Department of Administrative, Adult and Higher Education) We accept t h i s thesis as conforming to the required standard THE UNIVERSITY OF BRITISH COLUMBIA .October 1981 (T) Ingrid Pipke, 1981 I n p r e s e n t i n g t h i s t h e s i s i n p a r t i a l f u l f i l m e n t o f t h e r e q u i r e m e n t s f o r an advanced degree a t t h e U n i v e r s i t y o f B r i t i s h C o l u m b i a , I a g r e e t h a t t h e L i b r a r y s h a l l make i t f r e e l y a v a i l a b l e f o r r e f e r e n c e and s t u d y . I f u r t h e r agree t h a t p e r m i s s i o n f o r e x t e n s i v e c o p y i n g o f t h i s t h e s i s f o r s c h o l a r l y p u r p o s e s may be g r a n t e d by t h e head o f my department o r by h i s o r h e r r e p r e s e n t a t i v e s . I t i s u n d e r s t o o d t h a t c o p y i n g o r p u b l i c a t i o n o f t h i s t h e s i s f o r f i n a n c i a l g a i n s h a l l n o t be a l l o w e d w i t h o u t my w r i t t e n p e r m i s s i o n . Department o f Administrative, Adult and Higher Education The U n i v e r s i t y o f B r i t i s h C o l u m b i a 2075 Wesbrook P l a c e V ancouver, Canada V6T 1W5 Date 19th October 1981 DE-6 (2/79) i ABSTRACT Adult education i s a f i e l d of p r a c t i c e which has given r i s e to an emerging d i s c i p l i n e concerned with the creation of i t s own body of know-ledge. The f i e l d and the d i s c i p l i n e e x i s t i n a r e c i p r o c a l r e l a t i o n s h i p where information i s d i f f u s e d both ways. One method for disseminating information i s the Adult Education Research Conference (A.E.R.C.) which promotes research i n the d i s c i p l i n e and encourages professional c o l l a -boration among adult educators. Information dissemination processes are v i t a l to the d i s c i p l i n e and f i e l d , and are studied through meta-research. In the present study, ab-s t r a c t s submitted to Steering Committees for the Adult Education Research Conference i n 1978, 1979, and 1980 were examined to c l a r i f y variables associated with acceptance or r e j e c t i o n . The study was grounded i n s o c i a l science l i t e r a t u r e focusing on v a r i a b l e s associated with the acceptance or r e j e c t i o n of manuscripts submitted for p u b l i c a t i o n . A 41-item instrument was developed to assess the c h a r a c t e r i s t i c s of A.E.R.C. abstracts. As A.E.R.C. abstracts are judged " b l i n d " ( i . e . , authors are unknown to judges), the study examined " i n t e r n a l " abstract v a r i a b l e s . These concerned the content (adult education focus and methodological o r i e n t a t i o n ) , the research processes employed, and the composition of the abstract. Procedures aimed at measuring the r e l i a b i l i t y and v a l i d i t y of the instrument were executed. Expert judges (the 1981 A.E.R.C. Steering Committee) attested to the content v a l i d i t y of the instrument. For t e s t -r e t e s t purposes, 97 abstracts were coded twice and 20 were coded three times to y i e l d a mean item s t a b i l i t y - a c r o s s - t i m e c o e f f i c i e n t of r=.68. Inter-judge r e l i a b i l i t y was established by having f i v e judges code nine randomly selected abstracts. A repeated measures analysis of variance showed that the f i v e judges made consistent decisions concerning 37 of the 39 v a r i a b l e s . During a second procedure, the coding decisions of the researcher were compared with those of the judges. "Researcher-judges" data were subject to analysis of variance which revealed acceptable l e v e l s of agreement on 37 v a r i a b l e s ; the two " u n r e l i a b l e " r e s u l t s stemmed from the non-conforming decisions of a judge, not the researcher. During p i l o t procedures, scales and coding c r i t e r i a were systematically r e f i n e d . It was concluded that the f i n a l form of the instrument was content v a l i d and r e l i a b l e . Using t h i s instrument, 329 accepted and rejected A.E.R.C. abstracts were coded on 39 v a r i a b l e s . Item means of abstracts accepted and rejected i n 1978, 1979, and 1980 d i f f e r e d s i g n i f i c a n t l y on nine, s i x , and nine v a r i -ables r e s p e c t i v e l y . Variables d i f f e r e n t i a t i n g between accepted and re-jected abstracts were entered into discriminant function equations f o r 1978, 1979, and 1980. P r o f i l e s f o r accepted abstracts d i f f e r e d by year. In 1978, accepted abstracts were pr i m a r i l y written i n an a c t i v e voice, had a clear and l o g i c a l argument, were oriented towards use of a p a r t i c u l a r research methodology, had " c l e a r l y i d e n t i f i e d " instrumentation and implications for the f i e l d , and did not focus on agency sponsorship of adult education pro-grammes. In 1979, accepted abstracts were methodologically oriented, focused on programme planning issues but not agencies, had a c l e a r l y defined inductive t h e o r e t i c a l development, and were not w e l l anchored i n the i i i l i t e r a t u r e . The 1980 " p r o f i l e " showed that accepted abstracts focused on foundations of adult education or c h a r a c t e r i s t i c s of adults and l e a r n -ing, had " c l e a r l y i d e n t i f i e d " data c o l l e c t i o n procedures, used higher-order (e.g., multivariate) data analysis, and only moderate amounts of dysfunctional jargon. Separate discriminant function equations for each year s u c c e s s f u l l y c l a s s i f i e d 81 percent of abstracts i n 1978, 71 percent i n 1979, and 78 percent i n 1980. It was s i g n i f i c a n t that, i n general, variables associated with acceptance did not have the same, or even a sim-i l a r , e f f e c t i n each of the years studied. Judges appeared to weight variables d i f f e r e n t l y by year. This r a i s e s questions concerning the ab-s t r a c t s e l e c t i o n process and the e l e c t i o n of Steering Committee members. i v TABLE OF CONTENTS Page ABSTRACT i LIST OF TABLES i x LIST OF FIGURES x i ACKNOWLEDGEMENTS x i i Chapter 1 BACKGROUND TO THE PROBLEM 1 INTRODUCTION 2 Brief History 2 The F i e l d 3 The D i s c i p l i n e 5 Information Dissemination 7 META-RESEARCH 10 THE PROBLEM 14 2 LITERATURE REVIEW 16 External Variables 17 Internal Variables 18 Summary 22 Implications for the Present Study 23 3 ADULT EDUCATION RESEARCH CONFERENCE 27 H i s t o r i c a l Overview and Issues 27 The A.E.R.C. Process 32 V Chapter Page 4 INSTRUMENT DEVELOPMENT 35 INTRODUCTION 35 General Considerations 35 VARIABLE SELECTION 36 DEPENDENT VARIABLE 38 INDEPENDENT VARIABLES 39 Content Variables 39 Adult Education Focus 39 Coding 41 Methodological Orientations 41 Coding 42 Process Variables 43 Step One: Theoretical Development 45 * Coding . 45 Step Two: L i t e r a t u r e Review 46 Step Three: Research Design 48 Step Four: Methodology 49 Step Five: Results . 54 Step Six: Conclusions arid Discussion 55 State of the Research 58 Compositional Variables 59 Coding 59 PILOT TESTING 62 Interjudge R e l i a b i l i t y 62 V a l i d i t y , 66 v i Chapter Page 5 PROCEDURES 68 Preliminary Considerations 68 Coding 69 S t a b i l i t y Over Time 70 Data Preparation and Analysis 72 Discriminant function analysis 72 6 RESULTS 74 COMPARISON OF ACCEPTED AND REJECTED ABSTRACTS: BIVARIATE ANALYSIS 74 1978 74 Abstract C h a r a c t e r i s t i c s 77 Content v a r i a b l e s 77 Process v a r i a b l e s 78 Compositional variables 79 1979 79 Abstract C h a r a c t e r i s t i c s . 81 Content variables 81 Process v a r i a b l e s 82 Compositional v a r i a b l e s 83 1980 83 Abstract C h a r a c t e r i s t i c s . 85 Content v a r i a b l e s 85 Process v a r i a b l e s 86 Compositional variables 86 Consistency Over Three Years 87 Ch a r a c t e r i s t i c s of a l l Abstracts by Year 89 v i i Chapter Page COMPARISON OF ACCEPTED AND REJECTED ABSTRACTS: DISCRIMINANT FUNCTION ANALYSIS . . . ' 89 1978 . 91 Corr e l a t i o n matrix 91 Discriminant function analysis 92 1979 . . . ' 96 Corr e l a t i o n Matrix 96 Discriminant function analysis 97 1980 99 Cor r e l a t i o n matrix 99 Discriminant.function analysis 101 Summary 103 7 SUMMARY, CONCLUSIONS, IMPLICATIONS AND LIMITATIONS . . 106 SUMMARY 107 CONCLUSIONS I l l Acceptance of Abstracts I l l Judges "Weighting" Abstracts 112 Ch a r a c t e r i s t i c s of Abstracts 114 Content of abstracts 115 Research processes 117 Composition of abstracts 118 DISCUSSION 120 Rela t i v e Impact 120 POSSIBLE IMPLICATIONS FOR A.E.R.C 122 v i i i Chapter Page C a l l for Papers , 122 Length of Abstracts 122 Abstract Selection Process 123 Abstract Guidelines 124 E l e c t i o n of Judges 124 LIMITATIONS OF THE STUDY 125 REFERENCE NOTES 127 REFERENCES 128 APPENDICES 133 Appendix 1: A.E.R.C. Coding Schedule 133 Appendix 2: A.E.R.C. Abstract Form 139 Appendix 3: C o r r e l a t i o n Matrix of Variables Associated with A.E.R.C. Abstracts 1978 140 Appendix 4: Co r r e l a t i o n Matrix of Variables Associated with A.E.R.C. Abstracts 1979 145 Appendix 5: Co r r e l a t i o n Matrix of Variables Associated with A.E.R.C. Abstracts 1980 150 X X LIST OF TABLES Table Page 1 C h a r a c t e r i s t i c s of L i t e r a t u r e Relevant to A.E.R.C. Abstract Acceptance 23 2 Process Variables (and Their Scaling) Employed i n a Study of A.E.R.C. Abstract Acceptance 44 3 Interjudge R e l i a b i l i t y Indices for 39 Variables Related to Acceptance of A.E.R.C. Abstracts 64 4 S t a b i l i t y Across Time Indices f o r 39 Variables Related to Acceptance of A.E.R.C. Abstracts 71 5 Means and S.D.'s of Variables Associated with Accepted and Rejected A.E.R.C. Abstracts f o r 1978 . . 75 6 Means and S.D.'s.of Variables Associated with Accepted and Rejected A.E.R.C. Abstracts f o r 1979 . . 80 7 Means and S.D.'s of Variables Associated with Accepted and Rejected A.E.R.C. Abstracts f o r 1980 . . 84 8 Extent to Which the Mean Process, Content, and Compositional Scores of Accepted and Rejected A.E.R.C. Abstracts were S i g n i f i c a n t l y D i f f e r e n t i n 1978, 1979, and 1980 88 9 Variable Means for a l l Abstracts Submitted to A.E.R.C. i n 1978, 1979 and 1980 90 10 Interactive E f f e c t s of Variables Associated with Acceptance of A.E.R.C. Abstracts f o r 1978 94 11 Percentage of 1978 A.E.R.C. Abstracts Correctly Assigned to Accept and Reject Groups by D i s -criminant Function Analysis 95 12 Inte r a c t i v e E f f e c t s of Variables Associated with Acceptance of A.E.R.C. Abstracts f o r 1979 97 13 Percentage of 1979 A.E.R.C. Abstracts Correctly Assigned to Accept and Rej ect Groups by D i s -criminant Function Analysis 99 14 Interactive E f f e c t s of Variables Associated with Acceptance of A.E.R.C. Abstracts f o r 1980 101 X Table Page 15 Percentage of 1980 A.E.R.C. Abstracts Correctly Assigned to Accept and Reject Groups by Dis-criminant Function Analysis 103 16 Process, Content and Compositional Variables Which Entered Discriminant Function Equations i n 1978, 1979, and 1980 104 x i LIST OF FIGURES Figure Page 1 Hypothesized E f f e c t s of Variables Influencing Selection of Manuscripts Where the Author i s I d e n t i f i e d 25 2 Hypothesized E f f e c t s of Variables Influencing Selection of A.E.R.C. Abstracts where Judges are " B l i n d " 26 3 A.E.R.C. Steering Committee Members from 1978 -1981 33 x i i ACKNOWLEDGEMENTS A completed thesis i s the goal of any graduate student. Hopefully, the process i s enjoyable and stimulating. However, i t can be f r u s t r a t i n g and arduous. This thesis has been a l l of those, and now, f i n a l l y , i t i s fi n i s h e d . Throughout the year, many in d i v i d u a l s offered help and assistance. Family and friends provided encouragement. Fellow students empathized. Thanks go to Keith Foster, Byron Burnham, and Dan Pratt (my course advisor). My committee deserves s p e c i a l thanks. Dale Rusnell and Todd Rogers devoted time to reading and c r i t i q u i n g the various dra f t s of t h i s t h e s i s . Thanks also to Todd for steering me clear of p o t e n t i a l problems that could have developed following Dale Rusnell's decision to leave the u n i v e r s i t y . A f t e r Dale Rusnell departed, i t was discovered that u n i v e r s i t y requirements created a need to secure a new reader from the Adult Education D i v i s i o n . W.S. G r i f f i t h o ffered to read the thesis at t h i s l a t e stage and did so the day before i t was typed. Thus, he should not be held responsible for decisions concerning the design, analysis or other a t t r i b u t e s of the study. Alan Knox, Harland Copeland, Bob Bruce, and Wayne Schroeder gave information concerning the A.E.R.C. The four judges and the 1980-81 A.E.R.C. Steering Committee gave time and energy during the instrument development process. John C o l l i n s took time out from so-called retirement to explain d i f f i c u l t a n a l y s i s . Roger, my " s i g n i f i c a n t other", had to remain at arms length from this work but i n close p h y s i c a l and psychological proximity to the w r i t e r . His caring encouragement provided a foundation for completion. 1 CHAPTER 1 BACKGROUND TO THE PROBLEM This i s a study of variables associated with the acceptance of papers for " p u b l i c a t i o n " at the Adult Education Research Conference (A.E.R.C). It f a l l s w ithin the developing t r a d i t i o n of meta-research i n adult educa-t i o n . The need for and timeliness of the study stems from the s t a t e of adult education research, demands for the development of a " d i s c i p l i n e " , 1 and the maturing of the A.E.R.C. as an instrument for disseminating adult education research findings. Part of the need for the present study a r i s e s from the fact adult education i s both a s o c i a l science d i s c i p l i n e and f i e l d of p r a c t i c e which exist i n a . r e c i p r o c a l r e l a t i o n s h i p where information i s d i f f u s e d both ways. Issues a r i s i n g from the nature of the " d i s c i p l i n e " , the f i e l d , and more general processes associated with p u b l i c a t i o n of s c i e n t i f i c findings, pro-vide a preface to the present i n v e s t i g a t i o n . Most s o c i a l science d i s c i -p l ines study t h e i r research and dissemination processes. Moreover, much s o c i a l science meta-research straddles d i s c i p l i n a r y boundaries. 1 " D i s c i p l i n e " i s used here i n the lay sense of the term. For present purposes, a d i s c i p l i n e i s deemed to r e f e r to the presence of an organized body of knowledge with more-or-less agreed upon boundaries, domains of inquiry, and basic concepts. The question of what c o n s t i -tutes a d i s c i p l i n e i n the s c h o l a r l y sense i s contentious. Kliebard (1965) suggests that i t consists of "organized i n t e l l e c t u a l resources" and " c e r t a i n a t t r i b u t e s which uniquely q u a l i f y them f o r teaching and learning". Some argue that education i s not a d i s c i p l i n e ; thus how could adult education be a d i s c i p l i n e ? For the purposes of t h i s study, i t i s not necessary to become embroiled i n t h i s controversy. The term as used below, merely distinguishes between the creation of knowledge (about adult education) and the f i e l d of p r a c t i c e . 2 INTRODUCTION For adult educators, the need to study research and dissemination processes p a r t l y stems from the nature of adult education which has a b r i e f h i s t o r y . B r i e f History During the nineteenth century there was l i t t l e organized education for adults. The f i r s t schools for adults, described by Pole (1816), were la r g e l y under the auspices of the church; adult education was viewed as the means by which adults could acquire l i t e r a c y s k i l l s to b u i l d s e l f -worth and to increase r e l i g i o s i t y . The purpose of adult education has t y p i c a l l y changed to r e f l e c t the s o c i o - c u l t u r a l context i n which i t occurs. After World War I adult edu-cation was an instrument for s o c i a l reconstruction (Ministry of Re-construction, 1919) exemplified i n i n s t i t u t i o n a l forms by the English Workers' Educational Association and the Canadian Antigonish Movement (which lead to the development of c r e d i t unions and co-operatives i n the Maritimes). Adult education was also used to "Americanize" immigrants moving to the United States during the early 1900's and, more recently, to provide remedial education for adults with l i t t l e schooling, to t r a i n labour through vocational and t e c h n i c a l education, to supply opportunities to adults for s e l f - f ul'f illment, and to increase i n d i v i d u a l effectiveness i n s o c i a l and family r e l a t i o n s h i p s , and suchlike. Examining the "modern era", Cotton (1968) i d e n t i f i e d three epochs i n adult education. The f i r s t period (1919-1929), the age of "idealism", saw adult education as an instrument for s o c i a l reform and reconstruction. 3 During the l a t t e r part of the decade, the American Adult Education Associa-t i o n was established (1926). This event p u b l i c l y defined adult education as an independent f i e l d of p r a c t i c e i n the U.S.A. During the age of "realism" (1930-1946) i d e a l s were adjusted to economic and s o c i a l r e a l i t i e s . L i t e r a t u r e by p r o f e s s i onal adult educators demanded more s p e c i f i c d e f i n i t i o n s and descriptions of the f i e l d . The f i r s t graduate programme of adult educa-t i o n was established at Columbia University i n 1930. The t h i r d period (1947-1964), " p r o f e s s i o n a l i z a t i o n " , was characterized by the increased pro-f e s s i o n a l i z a t i o n and i n s t i t u t i o n a l i z a t i o n of adult education. The rapid growth of the d i s c i p l i n e during these years was evidenced by an expansion of graduate programmes, the establishment of the Commission of Professors of Adult Education, the organization of a National Seminar for Adult Educa-t i o n Research, and an increasing body of l i t e r a t u r e concerning adult educa-t i o n as a f i e l d of p r a c t i c e and professional study. The F i e l d Adult education occurs i n a diverse array of formal and informal sett i n g s . The most conspicuous part of the f i e l d i s the formally-organized i n s t i t u t i o n s c l a s s i f i e d by Schroeder (1970). Type I agencies have adult education as t h e i r primary or c e n t r a l function (e.g., proprietary schools); Type II agencies serve the educational needs of youth but serve adults as a secondary function (e.g., community co l l e g e s ) ; Type III agencies view adult education as a function a l l i e d to some non-educational community need (e.g., l i b r a r i e s , museums); Type IV agencies regard adult education as a subordinate function employed to further a s p e c i a l i n t e r e s t (e.g., labour unions, business and industry, churches). Of these types, those which 4 regard adult education as t h e i r primary function form the smallest group. The other three recognize t h e i r involvement (though sometimes minimal) i n adult education a c t i v i t i e s , but most of t h e i r funding, resources, and energies are devoted towards development of primary functions. Thus, adult education was once described as marginal (Clark, 1958) because i t occupied a peripheral p o s i t i o n i n many i n s t i t u t i o n s . Numerous i n d i v i d u a l s plan programmes and design i n s t r u c t i o n for adult c l i e n t e l e s . Houle (1970) c l a s s i f i e d leadership i n the f i e l d as a pyramid consisting of three l e v e l s . At the base, the largest group consists of volunteers and lay leaders from community organizations. The second l e v e l includes i n d i v i d u a l s involved as part-time providers of adult education services either as part of regular, or supplemental employment (e.g., night school i n s t r u c t o r s , l i b r a r i a n s ) . At the apex i s the smallest group composed of i n d i v i d u a l s who s p e c i a l i z e i n and consider adult education to be t h e i r primary p r o f e s s i o n a l concern. This group includes, amongst others, d i r e c t o r s of adult education i n various organizations (e.g., u n i v e r s i t i e s , colleges, museums), di r e c t o r s of t r a i n i n g i n business and industry, educational spe-c i a l i s t s for voluntary organizations, and professors of adult education. Adult educators are dispersed through a bewildering v a r i e t y of settin g s . Almost any adult i n s t i t u t i o n , organization, or s o c i a l instrument has some educational function or element. It i s impossible to count and d i f f i c u l t to catalogue the numbers and types of adult educators i n a modern society l i k e Canada. But casual and systematic observation suggests that i t i s d i f f i c u l t to disseminate information to these people, some of whom f a i l to recognize they are part of the f i e l d . 5 Individuals i n the f i e l d organize and administer programmes or i n s t r u c t adult learners. Most adult educators are programme planners or i n s t r u c t o r s . In contrast, persons i d e n t i f i e d with the " d i s c i p l i n e " systematically study t h e o r e t i c a l problems associated with adult education or, as i s often the case, p r a c t i c a l s i t u a t i o n s experienced by p r a c t i t i o n e r s . Adult education scholars are l a r g e l y involved with research a c t i v i t i e s i n i n t e r n a t i o n a l organizations and u n i v e r s i t i e s . Persons involved with the d i s c i p l i n e are part of the small group at the apex of Houle's pyramid for whom the study of adult education has become a primary professional concern. The D i s c i p l i n e Adult education i s referred to as an emerging d i s c i p l i n e . In i t s e f f o r t s to b u i l d a unique body of knowledge, i t has u t i l i z e d two procedures fo r acquiring knowledge (Jensen, 1964). The f i r s t involves scrutiny of experiences gained from the f i e l d of p r a c t i c e . By studying the f i e l d , the adult educator formulates p r i n c i p l e s or generalizations to explain pheno-mena and thereby provides guidelines and controls for p r a c t i t i o n e r s . In the second procedure, the adult educator "borrows and reformulates" know-ledge from other d i s c i p l i n e s . This involves the screening of theory and research from other d i s c i p l i n e s and adapting i t to adult education. Know-ledge from d i s c i p l i n e s such as psychology, sociology, h i s t o r y , and admin-i s t r a t i o n has been reformulated for adult education purposes. At some point however, an emerging d i s c i p l i n e uses l e s s "borrowed" knowledge and more of i t s "own". Moreover, i t generates new knowledge based on i t s own previous research. For t h i s process to occur, a d i s c i p l i n e must have an established research base. Pr i o r to about 1955, adult educators 6 did not conduct much research (see UNESCO, 1972). The f i r s t major review of adult education was published as a t o p i c a l issue i n the Review of  Educational Research (American Educational Research Association, 1950). In subsequent years, the Adult Education Association of the U.S.A. published inventories or l i s t i n g s of adult education research. In 1959, the p u b l i c a -t i o n of An Overview of Adult Education Research (Brunner et a l . , 1959) marked a milestone for adult education research. Its purpose was to iden-t i f y "any generalizations on which p o l i c y could be based, and which could be offered for the guidance of those preparing to be professional adult education workers on either a f u l l or part-time b a s i s " (p. i v ) . Recently, a c i t a t i o n study showed that adult education researchers now c i t e more of t h e i r "own" research than that of other d i s c i p l i n e s (Boshier & Pickard, 1979). It i s e s s e n t i a l that adult education develop a unique body of know-ledge, both as a s o c i a l science d i s c i p l i n e and a f i e l d of p r a c t i c e . Its importance however i s d i r e c t l y r e l a t e d to the extent to which knowledge i s disseminated throughout the d i s c i p l i n e and f i e l d . Knowledge gained from d i s c i p l i n a r y research can be v i t a l to the f i e l d i n several ways. For example, research on teaching techniques can be applied by the p r a c t i t i o n e r . Research on adult motivation to p a r t i c i p a t e i n educational a c t i v i t i e s can aid the programme planner reach p o t e n t i a l p a r t i c i p a n t s . Reciprocally, problems from the f i e l d can act as stimulants for d i s c i p l i n a r y research. For example, the problem of "drop-outs" has been the subject of adult edu-cation research; s p e c i a l c h a r a c t e r i s t i c s of the older adult have stimulated 7 research designed to improve "teaching" directed at t h i s audience. The f i e l d and the d i s c i p l i n e exist i n a r e c i p r o c a l r e l a t i o n s h i p ; each depends upon and reinforces the other. Information Dissemination The t i e that binds the d i s c i p l i n e and f i e l d of p r a c t i c e i s the communi-cation network which e x i s t s between them. The extent to which information and knowledge i s r e c i p r o c a l l y communicated between d i s c i p l i n e and f i e l d determines the growth and development of each. There are a v a r i e t y of methods by which knowledge i s disseminated. Communications systems between scholar and p r a c t i t i o n e r are both informal and formal. The former i s con-ducted primarily through interpersonal channels; the l a t t e r uses more formal channels such as journals, publications, and conferences. For current pur-poses, dissemination systems include printed matter, conference attendance, and information r e t r i e v a l systems, thus f a l l i n g i n the formal category. The printed form remains the most important method f o r dispersing information. Adult education l i t e r a t u r e most commonly appears i n books, pamphlets, p e r i o d i c a l s , and unpublished materials. Verner (1960) i n a re-view of adult education l i t e r a t u r e , discussed h i s t o r i c a l , survey, research, and general writings. Some works endure over time, for example, Bryson's Adult Education (1936), Lindeman's The Meaning of Adult Education (1926), the Report of the B r i t i s h M i n i s t r y of Reconstruction (1919), Brunner et a l . ' s (1959) An Overview of Adult Education Research, arid Adult E d u c a t i o n —  Outlines of an Emerging F i e l d of U n i v e r s i t y Study (Jensen et a l . , 1964). Recent work such as the Faure report Learning to Be (1972) also shows signs of having a l a s t i n g impact on the f i e l d . But much l i t e r a t u r e i s of a l i m i t e d value to either the d i s c i p l i n e or the f i e l d (Verner, 1960). 8 Along with books, journals and p e r i o d i c a l s d i f f u s e l i t e r a t u r e through-out adult education. Adult Education (USA) focuses on philosophy, theory, and research as does i t s B r i t i s h counterpart Studies i n Adult Education. Some journals are concerned with comparative education as, for example, Convergence; others r e l a t e to s p e c i f i c i n t e r e s t s i n the f i e l d , for example, the Canadian Journal of University Continuing Education, Adult L i t e r a c y , and Educational Gerontology; while others are devoted s p e c i f i c a l l y to the general p r a c t i t i o n e r , as, for example, L i f e l o n g Learning — the Adult Years and Adult  Education (N.I.A.E.). Conferences and conventions are the second method of disseminating know-ledge. Their popularity i s evidenced by the l a v i s h numbers of p a r t i c i p a n t s and money spent on these events. The popularity of "conferencing" among p r a c t i t i o n e r s and scholars i s p a r t i a l l y due to the i n s t i t u t i o n a l l y dispersed nature of adult education. These events provide the opportunity for i n s t i -t u t i o n a l l y and geographically dispersed adult educators to exchange ideas, news, information and find i n g s . Although time i s spent attfedi^fg p'apersT or symposia sessions, much knowledge i s gained through informal or s o c i a l a c t i v i t i e s . Major conferences which a t t r a c t scholars and p r a c t i t i o n e r s i n -clude the Adult Education Association Conference (USA) (held i n St. Louis, 1980, Anaheim, 1981) and the Adult Education Research Conference (held i n Vancouver, 1980; De Kalb, 1981; Lincoln, 1982). Another method f o r knowledge dissemination i s the information r e t r i e v a l system which has benefitted from advancing computer technologies. The Edu-c a t i o n a l Resources Information Center (E.R.I.C.), for example, provides easy access to a wide range of s i g n i f i c a n t educational ( p a r t i c u l a r l y 9 unpublished) documents. Other information services include the National Educational Associations' Adult Education Clearinghouse (N.A.E.C.) and the School Research Information System (S.R.I.S.). As i n other f i e l d s and d i s c i p l i n e s , adult education i s experiencing an "information explosion". P r a c t i t i o n e r s , scholars and students are inundated with books, journals, conference publications, and ephemera. But, does quantity ensure quality? Are there any controls which d i s t i n g u i s h the meritorious from the mediocre? For the d i s c i p l i n e , are there screens to ensure that s c i e n t i f i c a l l y sound research i s disseminated while non-sense i s inhibited? These questions have led researchers i n various d i s c i p l i n e s to study information dissemination networks and q u a l i t y control systems. Thus, the "gatekeeping" function of editors and the manuscript referee system have evoked sc h o l a r l y i n t e r e s t . In a study of evaluation patterns i n the natural sciences and humanities, Zuckerman and Merton (1971) maintained that referees are "an example of status judges who are charged with evaluating the q u a l i t y of role-performance i n a s o c i a l system" (p.66). Examples of status-judges include teachers who assess the q u a l i t y of student work, art c r i t i c s , supervisors i n industry, and journal and "conference" ed i t o r s . Zuckerman and Merton studied r e j e c t i o n rates for " s c i e n t i f i c " and "human-i s t i c " journals, status differences of scholars submitting manuscripts, questions concerning anonymity (of manuscript authors) i n the judging pro-cess, the duration of the referee process, and the influence of referees, editors, and authors on the review process. They concluded that despite 10 i t s imperfections, the system of monitoring s c i e n t i f i c work before i t enters into the archives of science means that much of the time s c i e n t i s t s can b u i l d upon the work of others with a degree of warranted confidence. It i s i n t h i s sense that the structure of authority i n science, i n which the referee system occupies a cen t r a l place, provides an i n s t i t u t i o n a l basis for the comparative r e l i a b i l i t y and cumulation of knowledge (1971, p. 99). Other studies, described below, have examined variables which deter-mine acceptance or r e j e c t i o n of manuscripts (e.g., Chase, 1970; Gottfredson, 1978), the review process (e.g., Rodman & Mancini, 1977), and the degree of interjudge agreement on manuscript acceptance (e.g., Scott, 1974). Studies l i k e these are becoming more prominent i n s o c i a l science l i t e r a t u r e . Apparently, many authors believe that i t i s important to moni-tor research processes. Thus, meta-research, the systematic study of research, i s a type of monitoring employed by d i s c i p l i n e s as they mature. At f i r s t , d i s c i p l i n e s appear to struggle to create boundaries and basic concepts. Secondly, they study r e l a t i o n s h i p s and processes within bounda-r i e s . Eventually, there i s a body of knowledge capable of analysis by meta-researchers. Meta-research has undoubtedly been boosted by developments i n information processing and systems theory but s t i l l appears to r e f l e c t the maturing of a d i s c i p l i n e . META-RESEARCH Through meta-research, adult educators can c r i t i c a l l y examine research a c t i v i t i e s and outcomes to f a c i l i t a t e the understanding, p r e d i c t i o n and control of s c i e n t i f i c a c t i v i t i e s that b u i l d theory, and, i n the long term, influence the f i e l d of p r a c t i c e (Kerlinger, 1977). As an emerging d i s c i -p l i n e , adult education has given r i s e to several types of meta-research. 11 Sork (1980), reviewed more than 100 examples of meta-research i n adult edu-cation and i d e n t i f i e d s i x types: Type I — Inventories of Research; Type II — General Reviews of Research: Type III — C r i t i c a l Reviews of S p e c i f i c Topics; Type IV — Research Agendas or Taxonomies of Needed Research; Type V — Focused Cr i t i q u e s of Research Methodology; and Type VI — Frameworks or Paradigms for Understanding and Improving Research. Type I — Inventories of Research contain r e g i s t e r s of research whose primary purpose i s to make known who i s doing what work. There are several d i f f e r e n t forms of research inventories. These include annual "Research Re-views" sponsored by the A.E.A. and published i n Adult Education (e.g., Kap-lan, 1955-1959). From 1967-73, the E.R.I.C. Clearinghouse on Adult Educa-t i o n compiled and published annual research r e g i s t e r s . Currently, the E.R.I.C. Clearinghouse on Adult, Career and Vocational Education publishes periodic inventories. Other organizations and i n s t i t u t i o n s have developed s i m i l a r studies including inventories of thesis and d i s s e r t a t i o n research, non-degree research i n Canada, and studies completed at s p e c i f i c u n i v e r s i t i e s . Type II — General Reviews of Research summarize complete work, form generalizations, and judge whether or not progress has been made. Brunner's (1959) Overview of Adult Education Research remains as the most outstanding example. Although subsequent reviews for assessing the general d i r e c t i o n of research have been published, none have the comprehensiveness of Brunner's work. Type I I I — C r i t i c a l Reviews on S p e c i f i c Topics involve state-of-the-art reviews which focus on s p e c i f i c topics and emphasize findings and t h e i r generalizations. Sork i d e n t i f i e s t h i s as a neglected form of meta-research 12 i n adult education. The topic of adult development and learning has received the greatest i n t e r e s t . Recent reviews concern needs (Monette, 1977) and adult learning projects (Tough, 1978). Type IV — Research Agendas or Taxonomies of Needed Research are developed s p e c i f i c a l l y to stimulate research a c t i v i t i e s i n the f i e l d . Brunner (1960) published an early l i s t of research needs followed by Houle (1962), Kreitlow (1968, 1975), Knox (1977), and others. The e f f e c t of such taxonomies on adult education research i s questionable. However, these agendas do i d e n t i f y research questions which adult educators consider to be within t h e i r domain of study. Type V — Focused Critiques of Research Methodology examine methodolo-gies used i n adult education research. This type of meta-research received impetus i n the l a s t decade as adult educators became more aware of standard s o c i a l science research methods. Thus, there have been reviews concerning the advantages and disadvantages of p a r t i c i p a t o r y research methodologies ( H a l l , 1975; Lindsey, 1976), the use of methodologies associated with grounded theory (Mezirow, 1971), and the use of factor analysis i n adult motivation studies (Boshier, 1976). Type VI — - Frameworks or Paradigms for Understanding and Improving  Research consider a t t r i b u t e s of the research process and the publishing behaviour of researchers. This area of meta-research increased i n the 1970's with studies focusing on content analysis (Dickinson & Rusnell, 1971), c i t a t i o n patterns (Boshier & Pickard, 1979), and p u b l i c a t i o n a c t i -v i t i e s of professors (Long, 1977). Other studies centre on fundamental assumptions of, or suggest improvements f o r , research i n adult education. 13 Sork concluded that meta-research i n adult education "has made an important contribution to the process of separating knowledge from ideas" (p. 24) and i s an e s s e n t i a l element i n i t s development as a d i s c i p l i n e . Most of Sork's references were to Type I and II research. Although Type VI research has increased i n the past decade, the number of reported studies remains small. Only f i v e studies concerned p u b l i c a t i o n and dissemination processes. Most concern Adult Education, the American jo u r n a l of research and theory. Dickinson and Rusnell (1971) conducted a content analysis; Long and Agyekum (1974) focused on the kind, content, and authorship of published a r t i c l e s ; Boshier and Pickard (1979) analyzed c i t a t i o n patterns of published a r t i c l e s ; and Lee (1979) studied the contribution of graduate students to the jou r n a l . A further study looked at the p u b l i c a t i o n a c t i v i t y ( i n a number of journals) of selected members of the Commission of Professors (Long, 1977). Research re l a t e d to the acceptance/rejection of manuscripts for these journals i s yet to be conducted. Furthermore, there i s no or l i t t l e reference to studies concerning the nature of research "published" i n major conferences or forums. This meta-research o r i e n t a t i o n i s a f l e d g l i n g area of study i n adult education. There have been a few content analyses of journals, only one known c i t a t i o n study and a small amount of research on dissemination systems. There does not appear to have been any study of va r i a b l e s that influence the acceptance/rejection of s c i e n t i f i c work submitted for "pu b l i c a t i o n " . In adult education, t h i s probably stems from i t s newness as a f i e l d of study, the existence of few "major" journals, and the ephemeral and dispersed nature of the l i t e r a t u r e (Verner, 1960). 14 There has also been a complete absence of work concerning the Adult Education Research Conference, a v e h i c l e for the dissemination of research findings. Moreover, no writer i n adult education has studied variables associated with the acceptance or r e j e c t i o n of manuscripts submitted for p u b l i c a t i o n at conferences (or i n j o u r n a l s ) . The purpose of the present project was to study variables associated with the acceptance or r e j e c t i o n of papers submitted for " p u b l i c a t i o n " at the Adult Education Research Con-ference. The present study thus f a l l s within Type VI of Sork's meta-research typology. THE PROBLEM The A.E.R.C. i s an instrument for the dissemination of knowledge to North American and "f o r e i g n " adult educators. Despite t h i s primary r e s -p o n s i b i l i t y , no guidelines have been set to influence the nature of the knowledge disseminated. As w e l l , no studies have evaluated the processes or outcomes of t h i s conference. The purpose of the present thesis was directed toward p a r t i a l l y remedying t h i s d e f i c i t . The s p e c i f i c aims of the study were: (i) to examine the c h a r a c t e r i s t i c s of abstracts submitted for A.E.R.C's held i n 1978, 1979, and 1980; and ( i i ) to i d e n t i f y v a r i a b l e s associated with acceptance/re-j e c t i o n of abstracts submitted to the A.E.R.C. i n 1978, 1979, and 1980. 15 Before describing the nature of the A.E.R.C, l i t e r a t u r e concerning the dissemination of s c i e n t i f i c information i n rel a t e d s o c i a l science d i s -c i p l i n e s i s b r i e f l y presented. This review was conducted with the expecta-t i o n that v a r i a b l e s which predict acceptance/rejection of submitted manu-s c r i p t s might be p r e d i c t i v e of acceptance/rejection of abstracts submitted to A.E.R.C. 16 CHAPTER 2 LITERATURE REVIEW The paucity of l i t e r a t u r e i n adult education concerning information dissemination and va r i a b l e s associated with the acceptance/rejection of manuscripts led to the examination of studies i n rela t e d s o c i a l science d i s c i p l i n e s . Researchers i n psychology, sociology, economics, p o l i t i c a l science, and education have systematically studied information dissemina-t i o n processes. As w e l l , the growing number of new journals and the i n -creasing volume of manuscripts submitted for p u b l i c a t i o n have led research-ers and editors to examine e d i t o r i a l p o l i c i e s and review processes. Re-searchers are asking questions about e d i t o r i a l gatekeeping, p u b l i c a t i o n lagtime, interjudge agreement, and c r i t e r i a f o r pub l i c a t i o n . Of p a r t i c u l a r i n t e r e s t to the present i n v e s t i g a t i o n were those studies which focused on variables associated with the acceptance or r e j e c t i o n of manuscripts. In view of the d i v e r s i t y of the dissemination l i t e r a t u r e , the review was organized i n terms of the following three questions: 1. What i s the primary concern of the a r t i c l e (e.g., c r i t e r i a f o r manuscript acceptance, interjudge agreement)? 2. In determining acceptance or r e j e c t i o n of manuscripts, what type of v a r i a b l e s does the study employ? 3. What type of analysis (e.g., mu l t i v a r i a t e , b i v a r i a t e , univariate) i s used i n t h i s study? 17 After reviewing a modest body of l i t e r a t u r e i t was apparent that variables influencing the acceptance or r e j e c t i o n of manuscripts could be c l a s s i f i e d into two major types: those "external" and those " i n t e r n a l " to the manuscript. "External" variables are those embodied i n the judges, contributors, or judging process; " i n t e r n a l " v a r i a b l e s are those i n c o r -porated i n the manuscript. External Variables Several studies examined the influence of external variables on the l i k e l i h o o d of p u b l i c a t i o n . T y p i c a l studies include one by Crane (1967) who attempted to assess the influence of the editor's "awareness of s c i e n t i s t s locations i n the academic s t r a t i f i c a t i o n system" (p. 195). Variables studied included: anonymity vs. non-anonymity ( i . e . , the contributor i s known/unknown to the reviewers) and academic c h a r a c t e r i s t i c s of editor and contributor ( i . e . , academic and i n s t i t u t i o n a l a f f i l i a t i o n , p r o f e s s i o n al age, doctoral o r i g i n ) . Crane concluded that academic c h a r a c t e r i s t i c s of editors ; and contributors a f f e c t the evaluation of s c i e n t i f i c a r t i c l e s ; anonymity does not change t h i s r e s u l t . Abramowitz et a l . (1975) focused on the reviewer's p o l i t i c a l o r i e n t a -t i o n as an independent v a r i a b l e . They hypothesized that manuscript referees would "bias t h e i r inferences about the q u a l i t y of a p o l i t i c a l l y relevant empirical paper i n a d i r e c t i o n congruent with t h e i r own p o l i t i c a l convic-t i o n s " (p. 189). The authors concluded that referee judgements were v a l i d when concerned with s p e c i f i c aspects of the manuscript's q u a l i t y ( i . e . , w r i t i n g , methodology). However, reviewer decisions to recommend the manu-s c r i p t ' s p u b l i c a t i o n are subject to p o l i t i c a l bias. 18 Rather than looking at editor/contributor c h a r a c t e r i s t i c s , Rodman and Mancini (1977) examined e d i t o r i a l procedures and, i n p a r t i c u l a r , "equity i n the process and procedures that lead to an e d i t o r i a l d e c i s i o n " (p. 369). The authors studied "sponsored submission" ( i . e . , someone of sp e c i a l status submits the manuscript f o r the author); " i n s i d e track sub-mission" ( i . e . , contributor's who have a s p e c i a l r e l a t i o n s h i p to the e d i t o r ) ; and "back region communication" ( i . e . , communication hidden from contribu-tors but b u i l t into the e d i t o r i a l procedure). Rodman and Mancini used des-c r i p t i v e s t a t i s t i c s and concluded that these "neglected areas" r a i s e ques-tions about pr o f e s s i o n a l c o n f l i c t of i n t e r e s t and e d i t o r i a l o b j e c t i v i t y . Internal Variables A second group of studies focused on " i n t e r n a l " manuscript v a r i a b l e s . Frantz (1968) asked e d i t o r i a l board members of educational psychology, per-sonnel, and counselling journals to rank c r i t e r i a f o r manuscript evaluation. C r i t e r i a included: contribution to knowledge, design of the study, o b j e c t i -v i t y i n reporting r e s u l t s , topic s e l e c t i o n , writing s t y l e and r e a d a b i l i t y , and other i n t e r n a l v a r i a b l e s . Two others that he considered, reputation of the author and i n s t i t u t i o n a l a f f i l i a t i o n , were external v a r i a b l e s . Hartung and Latta (1969) asked journal editors to consider three questions raised by prospective authors. The t h i r d concerned factors i n f l u e n c i n g an editor's decision to accept manuscripts. The fa c t o r s , i n order of frequency, were: qu a l i t y of the w r i t i n g , topic and content, appeal to readers, timeliness, research vs. opinion, author a member of the association publishing the journ a l , i l l u s t r a t i o n s , and length. With the exception of an author's mem-bership i n the association, a l l these v a r i a b l e s were i n t e r n a l to the manu-s c r i p t . 19 Another study concerned with c r i t e r i a for manuscript acceptance involved a survey of editor's opinions of requirements for p u b l i c a t i o n i n psychology journals. Wolff (1970) l i s t e d f i f t e e n v a r i a b l e s including: contribution to knowledge, research design, o b j e c t i v i t y i n reporting r e s u l t s , s t a t i s t i c a l analyses, w r i t i n g s t y l e and r e a d a b i l i t y , and t h e o r e t i c a l model. Most of the v a r i a b l e s i d e n t i f i e d were i n t e r n a l to the a r t i c l e ; external v a r i a b l e s were author's status and reputation and i n s t i t u t i o n a l a f f i l i a t i o n . Chase (1970), i n a study of "the operations of the evaluation and r e -cognition systems of science" (p. 262), l i s t e d ten c r i t e r i a f o r s c i e n t i f i c p u b l i c a t i o n . Selected professors from the natural and s o c i a l sciences, were asked to judge the " e s s e n t i a l i t y " of the c r i t e r i a . In order of "importance", v a r i a b l e s were ranked as follows: l o g i c a l r i g o r , r e p l i c a b i l i t y of research techniques, c l a r i t y and conciseness of w r i t i n g s t y l e , o r i g i n a l i t y , mathemati-c a l p r e c i s i o n , coverage of s i g n i f i c a n t e x i s t i n g l i t e r a t u r e , compatibility with generally accepted d i s c i p l i n a r y e t h i c s , t h e o r e t i c a l s i g n i f i c a n c e , per-tinence to current d i s c i p l i n a r y research, and a p p l i c a b i l i t y to p r a c t i c a l / applied problems i n the f i e l d . Chase concluded that mathematical and tech-n i c a l c r i t e r i a were stressed i n the natural sciences while l o g i c o - t h e o r e t i c a l standards were emphasized i n the s o c i a l sciences. Similar v a r i a b l e s were i d e n t i f i e d i n two studies concerning p u b l i c a t i o n i n economics journals (Coe & Weinstock, 1967; Weber, 1972). Variables i d e n t i f i e d as the chief impediments to p u b l i c a t i o n were: no s i g n i f i c a n t addition to the current body of knowledge, s u p e r f i c i a l i t y , inadequate r e -search, and poorly written. McCartney (1973), i n an e d i t o r i a l concerning the review process i n sociology, i d e n t i f i e d patterns within reviewer's comments. 20 Reviewer's c r i t i c i s m s and comments focused on several major problem areas: conceptual and t h e o r e t i c a l (e.g., c l a r i t y and p r e c i s i o n of concepts); methodological and design (e.g., problems i n construction and use of measures and s c a l e s ) ; a n a l y t i c and i n t e r p r e t i v e (e.g., incomplete a n a l y s i s ) ; and writing (e.g., lack of attention to s t y l e ) . Focusing on interjudge r e l i a b i l i t y , Scott (1974) attached an appraisal sheet to each manuscript submitted for review to the Journal of Personality  and S o c i a l Psychology. Manuscripts were sent to two reviewers. Of the seven a t t r i b u t e s l i s t e d on the appraisal sheet, i n t e r r e f e r e e agreement ( i n t r a c l a s s c o r r e l a t i o n c o e f f i c i e n t ) was s i g n i f i c a n t for s i x : importance of the present contribution, attention to relevant l i t e r a t u r e , design and a n a l y s i s , s t y l e and organization, succinctness, and recommendation (accept/ r e j e c t ) . Silverman and C o l l i n s (1975) examined publishing r e l a t i o n s h i p s i n higher education. Their study focused on authors' ra t i o n a l e s for p u b l i c a -t i o n , c r i t e r i a used i n s e l e c t i n g journals for proposed submission, desired standards of author and editors i n the review process, and s p e c i f i c c r i t e r i a for manuscript s e l e c t i o n . P a r t i c i p a n t s i n the sample rated the c r i t e r i a "which should i d e a l l y characterize the s t y l i s t i c and compositional elements of manuscripts" (p. 375). The analysis suggested that process and content variables are c r i t i c a l to review decisions. Process included organizational v a r i a b l e s (e.g., c l a r i t y and conciseness of w r i t i n g , appropriate use of s t a t i s t i c s , v a l i d i t y of l o g i c used, s p i r i t e d s t y l e ) , and norms of scholar-ship (e.g., t h e o r e t i c a l grounding, review of l i t e r a t u r e , r e p l i c a b i l i t y ) . Content included v a r i a b l e s such as i n t e r e s t to readers, timeliness of topic, a p p l i c a b i l i t y to p r a c t i c e and applied problems i n the f i e l d , and contribu-t i o n to basic knowledge. 21 The studies reviewed thus f a r were l a r g e l y concerned with c r i t e r i a for p u b l i c a t i o n . A l l employed simple u n i - or b i v a r i a t e s t a t i s t i c s and, as far as can be established, did not use any p a r t i c u l a r methodological or t h e o r e t i c a l stance to guide d a t a - c o l l e c t i o n and a n a l y s i s . A recent study, more sophisticated than those reviewed above, was conducted by Gottfredson (1978). Gottfredson advanced t h i s type of information d i s -semination l i t e r a t u r e i n a series of studies designed to investigate three major aspects of the peer-evaluation system i n psychology: "the r e l i -a b i l i t y of peer judgements of a r t i c l e q u a l i t y " ; "the c r i t e r i a upon which assessments of a r t i c l e q u a l i t y are l i k e l y to be made"; and " r e l a t i o n s h i p s between peer judgements of a r t i c l e q u a l i t y and the number of c i t a t i o n s made to a r t i c l e s following p u b l i c a t i o n " (p. 920). A p r i n c i p a l components analysis yielded nine i n t e r p r e t a b l e groups of a t t r i b u t e s of journal a r t i -c l e s . The f i r s t consisted of practices to avoid. "The problem has not been considered c a r e f u l l y enough", "the experiment conducted does not address the stated question", or "the author uses l o f t y s c i e n t i f i c jargon when p l a i n English w i l l do" loaded on t h i s component. Components II and III addressed "do's": r e l a t i n g to s c i e n t i f i c or substantive matters (e.g., " i t attempts to unify the f i e l d " , " i t deals with an important t o p i c " ) , and r e l a t i n g to s t y l e and composition (e.g., " i t i s well written", " i t avoids u n r e a l i s t i c speculation"). Other components focused on the importance of o r i g i n a l i t y and heurism; t r i v i a l i t y ; s c i e n t i f i c advancement; "data-grinders" (e.g., " i t contains more data but no new i n s i g h t s " ) ; "ho-hum" research (e.g., the author uses p r e c i s e l y the same procedures as everyone e l s e ) ; and narrowness of research concerns. The nine components, which accounted for 49.6 percent of the variance, led Gottfredson to conclude "... that 22 p r e s c r i p t i v e norms for s c i e n t i f i c evaluation exist and transcend subdis-c i p l i n a r y bounds" (p. 924). In the second study of the s e r i e s , the author used these c r i t e r i a "to achieve r e l i a b i l i t y of peer evaluations of psycho-l o g i c a l work" (p. 924). Judges were asked to evaluate selected a r t i c l e s using an evaluation scale based on the c r i t e r i o n emerging from the f i r s t study. The findings showed only moderate agreement across the judges. Summary The nature of the l i t e r a t u r e reviewed i s summarized i n Table 1 i n terms of study focus, type of a n a l y s i s , and d i s c i p l i n e . A l l but one of the studies shown i n Table 1 focused on manuscripts submitted to journals. Only one study (McReynolds, 1971) concerned s e l e c t i o n of papers for a con-ference; i t focused on the interjudge r e l i a b i l i t y of committee members i n -volved i n s e l e c t i n g papers for a meeting of the American Psychological Association. Variables i d e n t i f i e d with s e l e c t i o n processes ( i . e . , acceptance or r e j e c t i o n ) are external or i n t e r n a l to the manuscript. External variables are those inherent i n the editor, reviewer, and contributor (e.g., academic and i n s t i t u t i o n a l a f f i l i a t i o n , membership i n professional organizations) or the reviewing process (e.g., anonymity vs. non-anonymity, sponsored sub-missions). Internal variables are those embodied i n the manuscript, usually related to w r i t i n g and compositional processes and content c r i t e r i a concerned with, for example, theoretical/conceptual basis, methodological and empirical basis, and o r i g i n a l i t y and a p p l i c a b i l i t y to the f i e l d . These variables have been i d e n t i f i e d i n several studies from s o c i a l science d i s c i p l i n e s , either as a d i r e c t (e.g., manuscript c r i t e r i a ) or related (e.g., interjudge r e l i a b i l i t y ) focus of the study. 23 .Table 1 Characteristics of Literature Relevant to A.E.R.C. Ab stract Acceptance Variable External/ Internal Internal Author Focus/Foci of the Study Type of Analysis Crane (1979) Abramowitz, et a l (1975) Rodman &, Mancini (1977) Frantz (1968) Hartung & Latta (1969) Wolff (1970) Coe & Weinstock (1967) Chase (1970) McReynolds (1971) Weber (1972) McCartney (1973) Scott (1974) Silverman & C o l l i n s (1975) Gottfredson (1978) Acceptance/Rejection of Manuscripts Acceptance/RejectIon of Manuscripts E d i t o r i a l Procedures C r i t e r i a for Manuscript Selection C r i t e r i a for Manuscript Selection C r i t e r i a for Manuscript Selection Manuscript Review Process C r i t e r i a for Manuscript Selection Interjudge R e l i a b i l i t y Manuscript Review Process Reviewer Comments and Criticisms Interjudge R e l i a b i l i t y Rationales for Publication C r i t e r i a for Selecting Journals Standards i n Review Process C r i t e r i a for Manuscript Selection C r i t e r i a for Manuscript Selection R e l i a b i l i t y for Peer Judgments A r t i c l e Quality and Citations Univariate Bivariate Univariate Univariate Univariate Univariate/ Bivariate Univariate Univariate Bivariate Univariate Bivariate Bivariate Multivariate Discipline Sociology & Economics Psychology Higher Education Education Psych-ology & Counselling Education Psychology Economics Natural & Social Sciences Psychology Economics Sociology Psychology Higher Education Psychology Another conclusion a r i s i n g from t h i s review concerns analytic st r a t e -gies employed to predict manuscript acceptance. In many of the studies, judges ranked variables according to their importance or influence i n 24 the judging process. In others, c o r r e l a t i o n a l techniques were used with acceptance treated as dependent, and i n t e r n a l or external v a r i a b l e s as i n -dependent. The main focus was on the extent to which judges made consistent judgements concerning the worth of manuscripts. Few authors took t h e i r work to a l o g i c a l conclusion of pr e d i c t i n g acceptance. Moreover, those that came close to t h i s goal used simple b i v a r i a t e s t a t i s t i c s that f a i l e d to portray v a r i a b l e i n t e r a c t i o n s which undoubtedly influence acceptance. This apparent defect probably arose because few authors (other than journal editors) had access to large pools of rejected manuscripts necessary for such analyses. There i s also a notable absence of theory i n t h i s l i t e r a t u r e . G a r f i e l d (1979) presumed he had a t h e o r e t i c a l basis for his c i t a t i o n studies; informa-t i o n processors also employ "theory". But i n the l i t e r a t u r e surveyed for th i s study, there was a notable absence of theory; i f anything, researchers merely revealed c o r r e l a t i o n s . The data speak for themselves; at t h i s early stage, researchers are more i n c l i n e d to proceed i n d u c t i v e l y than deductively. Implications f o r the Present Study Despite the absence of theory i n t h i s area, many of the authors c i t e d above distinguished between i n t e r n a l and external v a r i a b l e s that influenced acceptance. The l i t e r a t u r e c i t e d appears to be t y p i c a l of situa t i o n s where both i n t e r n a l and external v a r i a b l e s i n t e r a c t to determine acceptance (Fig. 1). 25 ACCEPTANCE/REJECTION Figure 1 Hypothesized E f f e c t s of Variables Influencing Selection of Manuscripts Where the Author i s I d e n t i f i e d The d i s t i n c t i o n between i n t e r n a l and external variables that influence acceptance i s useful for the present study. The extent to which external v a r i a b l e s operate depends upon whether the adjudication process i s b l i n d (e.g., author's names known or unknown). For present purposes, the s i t u -a t i o n diagrammatically presented i n Figure 1 i s inappropriate because A.E.R.C. abstracts are reviewed b l i n d . Thus, the s i t u a t i o n pertaining to the s e l e c t i o n of A.E.R.C. abstracts can be better portrayed as follows (Fig. 2). 26 Internal Variables H ACCEPTANCE/REJECTION Figure 2 Hypothesized E f f e c t s of Variables Influencing Selection of A.E.R.C. Abstracts where Judges are " B l i n d " Although external variables possibly have an e f f e c t on A.E.R.C. abstract acceptance, they cannot be studied for two reasons; judges have no d i r e c t knowledge of authors and, more pragmatically, "external" informa-t i o n i s not c o l l e c t e d p r i o r to, during, or a f t e r the A.E.R.C. Authors merely submit an abstract accompanied by a facing page showing t h e i r name and address. Facing pages are removed before abstracts are sent to judges. 27 CHAPTER 3 ADULT EDUCATION RESEARCH CONFERENCE The Adult Education Research Conference i s an important v e h i c l e for knowledge dissemination within the d i s c i p l i n e and f i e l d of adult education. The h i s t o r y of the conference and the process for submission, s e l e c t i o n , and presentation of conference papers are described i n what follows. H i s t o r i c a l Overview and Issues The Adult Education Research Conference celebrated i t s 21st birthday i n 1980 and has become the largest annual meeting of adult education r e -searchers anywhere i n the world. The notion of gathering adult education researchers together was o r i g i n a l l y conceived i n the l a t e 1950's by mem-bers of the Commission of Professors (associated with the Adult Education Association of the U.S.A.) who perceived the need for a stronger research o r i e n t a t i o n within the f i e l d . Early meetings, known as the National Seminar on Adult Education Research, consisted l a r g e l y of students at the University of Chicago and s t a f f members from the now defunct Center for the Study of L i b e r a l Education for Adults. The National Seminar had no c o n s t i t u t i o n , by-laws, or dues, and was run on the good w i l l of interested members and i n s t i t u t i o n s who supplied postage, stationery, and s e c r e t a r i a l services. Over the years, t h i s organization evolved into the A.E.R.C. which functions as informally as i t s predecessor. Recently, however, because of a need to obtain taxation and other advantages, the A.E.R.C. assumed the 28 accoutrements of a formal a s s o c i a t i o n . During the annual business meeting of the 1976 A.E.R.C. i n Toronto, the group decided to incorporate. The membership adopted a formal c o n s t i t u t i o n and by-laws which o u t l i n e the purposes, membership, dues, d i r e c t o r s and t h e i r duties, meetings, e l e c t i o n s , and other concerns of the organization. The purposes of the A.E.R.C, as defined i n A r t i c l e II of the c o n s t i t u -t i o n , are: (1) To promote the improvement of research and evaluation i n adult education; (2) to fos t e r p r o f e s s i o n al c o l l a b o r a t i o n among persons who promote research, conduct research or u t i l i z e research findings i n the f i e l d of adult education. To t h i s end, A.E.R.C sponsors an annual conference which provides a forum f o r f l e d g l i n g researchers, recent graduates, and veterans of adult educa-t i o n . In recent years, the organization has become s u f f i c i e n t l y important for researchers to t r a v e l to " f a r - o f f " places l i k e San Antonio and Van-couver. The A.E.R.C. conference s i t e , as i n most national organizations, i s chosen with the aim of accommodating the needs of researchers dispersed from coast to coast. A l i s t i n g of c i t i e s that have recently hosted A.E.R.C i l l u s t r a t e s t h i s point. In e a r l i e r years there was a tendency to hold the A.E.R.C. i n the American mid-West; i n recent years i t has moved across the continent: (1982 - Lincoln, Nebraska) 1981 - DeKalb, I l l i n o i s 1980 - Vancouver, B r i t i s h Columbia 1979 - Ann Arbor, Michigan 1978 - San Antonio, Texas 1977 - Minneapolis, Minnesota 1976 - Toronto, Ontario 1975 - St. Louis, Missouri 1974 - Chicago, I l l i n o i s 1973 - Montreal, Quebec 1972 - Chicago, I l l i n o i s 1971 - New York, New York 1970 - Minneapolis, Minnesota. Recently, an increasing number of foreign researchers have presented papers at A.E.R.C. For example, the 1980 Vancouver conference hosted r e -searchers from Sweden, Great B r i t a i n , Nigeria, the United States, and Canada. For North American and, increasingly, f o r foreign adult educators, the Adult Education Research Conference i s an instrument for the dissemination of knowledge. The opportunity t h i s event provides for dialogue between adult educa-tors and researchers i s of c e n t r a l concern to the membership. Adult educa-tors have r e i t e r a t e d t h i s concern i n debates re l a t e d to a possible amal-gamation or a f f i l i a t i o n of A.E.R.C. with other organizations. One p o s s i -b i l i t y would be a closer r e l a t i o n s h i p with the Adult Education Association (U.S.A.) and the sharing of conference s i t e s and dates. Although loosely a f f i l i a t e d with the A.E.A., the A.E.R.C. maintains i t s own conference and or i e n t a t i o n . Other questions concern amalgamation with the American Edu-c a t i o n a l Research Association. In 1968 A l l e n Tough asked A.E.R.C. p a r t i -cipants f o r t h e i r opinions on t h i s issue. Respondents indicated t h e i r 30 preference f o r "the autonomy, the smallness and the cohesiveness that was possible by r e t a i n i n g A.E.R.C." (Copeland & Long, 1973). A second survey by the 1972-73 A.E.R.C. Steering Committee showed s i m i l a r preferences. A hi g h l i g h t i n the debate followed at the 1973 Montreal A.E.R.C. At the request of the Steering Committee, Copeland and Long asked: "What pro-f e s s i o n a l r e l a t i o n s h i p ( s ) should adult education researchers/evaluators seek to e s t a b l i s h with researchers and evaluators i n other f i e l d s of edu-cation?" Their statement suggested nine advantages and disadvantages of a close r e l a t i o n s h i p and included a l t e r n a t i v e "A.E.R.C. - A.E.R.A. r e l a -tionship patterns". Whether or not the membership adopted a p a r t i c u l a r " r e l a t i o n s h i p pattern" i s unclear, although Plan H, "A.E.R.C. should become an incorporated organization of professional adult education r e -searchers/evaluators. Informal or f o r a l [ s i c ] r e l a t i o n s h i p s could be established with A.E.R.A. as desired by both A.E.R.C. and A.E.R.A." (p. 7), r e f l e c t s current A.E.R.C. status. Nevertheless, the amalgamation issue l i n g e r s as r i s i n g conference costs and reduced expense funds are causing members to once again voice t h i s concern ( R o c k h i l l , 1978; Copeland, 1980). In 1981 A.E.R.C. remains as an autonomous organization — an i n d i c a t i o n of the importance which the membership a t t r i b u t e s to i t s own research confer-ence. During i t s 22 years, the s i z e and stature of the A.E.R.C. have ex-panded i n North America and abroad. As noted, there i s an increasing i n -volvement by foreign researchers who bring perspectives from d i f f e r e n t c u l t u r a l contexts. These exchanges have, u n t i l recently, occurred only on an informal, i n d i v i d u a l basis with l i t t l e formal exchange between adult education organizations. But i n 1978 an o f f i c i a l representative from the 31 United Kingdom Standing Committee on Uni v e r s i t y Training and Research i n Adult Education (SCUTREA) and the Porec Conference (organized by the Androgogical Center i n Zagreb) attended the San Antonio conference and thus began a process which "would r e s u l t i n much useful face-to-face i n t e r -a c t i o n with prominent researchers i n adult education ... (and the) ... more rapid d i f f u s i o n of abstracts and s i g n i f i c a n t research findings":'(Kidd, 1977, p. 1). Subsequently, European organizations have i n v i t e d A.E.R.C. to send representatives to meetings such as the International Seminar on Adult Education Research i n Sweden (1979) and the SCUTREA conference (Manchester, 1979). At times, A.E.R.C. pa r t i c i p a n t s have been asked to support o f f i c i a l representation at these conferences (Ann Arbor, 1979). Though some of the membership agreed to l i m i t e d support, a substantial number were opposed to expending A.E.R.C. monies f or i n t e r n a t i o n a l t r a v e l (Fellenz, 1979). As the A.E.R.C. "comes of age" (theme of the 1980 conference), i t has experienced growing "pains". But, from i t s inauspicious beginning as the National Seminar on Adult Education Research, the A.E.R.C. has expanded into an organization of i n t e r n a t i o n a l dimensions. It has grown from a small group of adult educators i n Chicago to a membership dispersed from coast to coast. I t has matured from an informally organized body of re-searchers to an organization structured by a c o n s t i t u t i o n and by-laws. The membership has faced issues concerning the a b i l i t y of A.E.R.C. to es t a b l i s h and r e t a i n i t s own i d e n t i t y or to amalgamate with larger organi-zations. Moreover, i n recent years, i t s stature has increased as foreign researchers and educators, i n d i v i d u a l l y and as organizations, have p a r t i -cipated i n the A.E.R.C. Despite coming of age, A.E.R.C. has not assumed the r i g i d i t y of adulthood; i t s conference r e f l e c t s the inf o r m a l i t y and f a m i l i a r i t y of the adult educators and researchers who gather to.discuss and report t h e i r research i n t e r e s t s and concerns. The A.E.R.C. Process The v e h i c l e by which the A.E.R.C. f u l f i l l s i t s purposes i s i t s annual conference. This event consists of paper and symposia presentations, .poster-sessions and research exchanges, the annual business meeting, a graduate student award, and a v a r i e t y of e x t r a - c u r r i c u l a r events. Conference attend-ance i s the only formal membership requirement for the A.E.R.C. The A.E.R.C. i s arranged by a four-person executive committee of which two new members are elected annually to insure continuity. Nominations are received p r i o r to, and during the short business meeting associated with the A.E.R.C. In recent years there has been some unhappiness associated with the e l e c t i o n of judges. 2 Figure 3 shows the names of judges associated with the A.E.R.C. i n recent years. It i s t h i s body's r e s p o n s i b i l i t y to organize the conference and publish i t s proceedings. Each year the A.E.R.C. ex-ecutive issues a " C a l l f o r Papers" mailed to a l l former attendees (on a mailing l i s t of approximately 1,500 names) and advertised i n various adult education journals. Abstracts of papers offered for possible presentation are received and evaluated by the four executive members (judges) who meet at the Annual Conference of the A.E.A. (U.S.A.) which i s usually held i n October or November. These abstracts are judged b l i n d , that i s , the judges are not t o l d who the authors are. There i s no handbook to guide the judging process. The overlapping judges pass on t h e i r understanding of the judging procedures, thereby ensuring some continuity from one year to the next. 2 I n view of the fa c t A.E.R.C. moves from c i t y to c i t y and anyone attending the business meeting i s deemed to be a member, i t i s possible for " l o c a l s " to elect " t h e i r own" people to the Steering Committee. At recent annual meetings of the A.E.R.C. various prominent p a r t i c i p a n t s have exhorted voters to c r i t i c a l l y appraise a l l candidates. 33 1978 Kreltlow Davie 1979 Cunningham Cunningham Pennington Pennington 1980 Spikes Spikes Fellenz Fellenz 1981 Boshier Boshier Simpson Simpson Merriam Compton Figure 3 A.E.R.C. Steering Committee Members from 1978 - 1981 The 1980 and 1981 A.E.R.C. Steering Committees used the following pro-cedure to select abstracts for the conference. A l l abstracts were read before the meeting with committee members making t h e i r i n d i v i d u a l choices. At t h e i r meeting, the members made a " f i r s t cut" whereby abstracts were un-animously chosen as accepted. Because an i n s u f f i c i e n t number (necessary f o r the A.E.R.C. programme) of abstracts was selected, the committee then made a second and t h i r d cut u n t i l the required number was obtained. Thus, i f on the f i r s t round the committee unanimously agreed on only 25 acceptable ab-s t r a c t s , then second and t h i r d selections were made u n t i l the necessary 40-45 abstracts were chosen. In addition, the committee chose "alternate" ab-s t r a c t s i n case some of the accepted papers could not be presented at the conference. 34 Successful authors were n o t i f i e d that t h e i r abstract had been accepted; they then had to provide a copy of the e n t i r e paper for i n c l u s i o n i n the conference proceedings. Currently, about 150 abstracts enter the judging process; about 45 are accepted. (There are also some symposia but they f a l l outside the scope of t h i s study.) U n t i l 1981, the executive s p e c i f i e d that abstracts must not exceed 250 words; no other requirements were promul-gated . Much s o c i a l science research involves development of r e l i a b l e and v a l i d instruments. Usually, i t i s the behaviour of human beings or animals that i s measured by s o c i a l s c i e n t i s t s . The present study involved measure-ment of A.E.R.C. abstracts, i n p a r t i c u l a r , v a r i a b l e s l i k e l y to be associated with t h e i r acceptance or r e j e c t i o n . Because the problem required the i d e n t i -f i c a t i o n and q u a n t i f i c a t i o n of "acceptance" variables and much of the coding work was done by one researcher, i t was e s s e n t i a l that a r e l i a b l e and v a l i d measuring instrument be developed. The following chapter describes con-si d e r a t i o n s pertaining to the instrument, i t s development, and variables selected for study. 35 CHAPTER 4 INSTRUMENT DEVELOPMENT INTRODUCTION The f i r s t purpose of t h i s chapter i s to describe general considera-tions pertaining to the development of the instrument constructed to accomplish the goals of t h i s study. The second purpose i s to discuss issues r e l a t i n g to the development and use of the instrument, focusing on va r i a b l e s , t h e i r o p e r a t i o n a l i z a t i o n , r e l i a b i l i t y , and v a l i d i t y . General Considerations As noted i n Chapter i , the purpose of th i s study was to understand and predict acceptance of A.E.R.C. abstracts. P r i o r to con-s t r u c t i n g a draft instrument i t was apparent that the study would be en-hanced i f : i . v a r i a b l e s were cast on equal-interval scales that would render data s u i t a b l e for parametric analysis; i i . scale-points were properly "anchored" (by providing clear l a b e l s or examples for each scale-point); i i i . where possible, v a r i a b l e s were operationalized i n accord with extant s o c i a l science and adult educa-t i o n theory; i v . p i l o t t e s ting was employed to ensure that the f i n a l instrument was r e l i a b l e and v a l i d ; and v. l i b e r a l c r i t e r i a were employed for coding abstracts (as suggested by a cursory examination of abstracts).. 36 VARIABLE SELECTION The d e s c r i p t i v e and l a r g e l y a t h e o r e t i c a l nature of previous manu-s c r i p t s e l e c t i o n l i t e r a t u r e and the absence of studies concerning the s e l e c t i o n of adult education or other papers f o r conference presentation led to a decision to employ numerous, l a r g e l y i n d u c t i v e l y derived v a r i -ables. Any v a r i a b l e , no matter how inconsequential, that appeared to be associated with the acceptance of papers, was e l i g i b l e for i n c l u s i o n i n t h i s study. Variables were derived by:: i . examining relevant research concerning acceptance/rejection of journal a r t i c l e s ; (see Chapter 2) i i . conferring with A.E.R.C. o f f i c e r s ; i i i . attending the 1980 A.E.R.C; and i v . conferring with knowledgeable adult education researchers. Some var i a b l e s noted below may appear t r i t e . For example, could "the number of words i n the abstract t i t l e " be a predictor of acceptance? But, i n the absence of previous work upon which to base judgements concerning t r i t e n e s s or any other a t t r i b u t e , many p o t e n t i a l predictors were included. Many would possibly be eliminated i n the f i r s t steps of the planned d i s -criminant function a n a l y s i s . With regard to the foregoing, the following variables were considered: Dependent v a r i a b l e (y) — acceptance/rejection Independent va r i a b l e s (x) — Content Variables — Foundations — Characteristics/Adult Learning — Agency or I n s t i t u t i o n 37 — Programme Planning — Instruction/Techniques — Adult Education D i s c i p l i n e — Theoretical/Conceptual — A r c h i v a l / H i s t o r i c a l — Empirical/Hard Data — Admonitional/Prescriptive — Methodological Process Variables — N o . of 'Direct C i t a t i o n s — No. of Authors Cited — Deductive — Inductive — State of the Research — Cumulative L i t e r a t u r e — Novelty of Research — Research Design — Data C o l l e c t i o n — Instrumentation — Instrument R e l i a b i l i t y — Instrument V a l i d i t y — Sample or Population — Type of analysis — Results — Conclusions — Research Implications — Theoretical Implications — Implications for the F i e l d 38 Compositional Variables — No. of Words i n the T i t l e — No. of Words i n the Abstract — O r i g i n a l Form — Attachments — Presentation — Voice — Jargon — Funding Source — Flow of the Argument A complete d e s c r i p t i o n of these 39 variables i s presented i n Appendix 1. For some, the o p e r a t i o n a l i z a t i o n i s obvious: "no. of words i n the t i t l e " c l e a r l y r e f e r s to the number of words (including to, but, and other con-jun c t i o n s ) . Others were less c l e a r . These are described i n the discussion that follows. DEPENDENT VARIABLE As described i n the previous chapter, four A.E.R.C. judges selected abstracts i n a b l i n d review process. Accepted abstracts were subsequently published i n the conference programme. For the purposes of t h i s study, p u b l i c a t i o n i n the conference programme s i g n i f i e d acceptance; i f not l i s t e d , the abstract was c l a s s i f i e d as rejected. The dependent v a r i a b l e was thus dichotomous. Accepted abstracts were coded 2; rejected abstracts were coded 1. 39 INDEPENDENT VARIABLES In considering i n t e r n a l v a r i a b l e s that might predict acceptance, i t was apparent from a preliminary review of the abstracts to be coded that some va r i a b l e s were rel a t e d to content, others to the research process, and others to compositional aspects. Accordingly, the independent v a r i -ables were organized into three major classes: content, process, and compositional. Content Variables The f i r s t group of variables r e l a t e d to the abstract content, topi c , or area of study. What was the abstract about? What was i t s focus? What area of adult education did the abstract concern? As well as i d e n t i f y i n g the p a r t i c u l a r focus of each abstract, the o v e r a l l methodological o r i e n t a -t i o n taken i n the study was considered as a possible relevant v a r i a b l e . Thus, two v a r i a b l e s were involved TT— a d u l t education focus and methodologi-c a l o r i e n t a t i o n . Adult Education Focus Previous researchers have attempted to c l a s s i f y areas'of adult education (Lee, 1979; Long & Agyekum, 1974; Dickinson & Rusnell, 1971). Each of these studies involved the c l a s s i f i c a t i o n of content i n the j o u r n a l Adult Education. The process of c l a s s i f y i n g A.E.R.C. abstracts into primary areas of adult education was s i m i l a r . The model which underlay the c l a s s i f i c a t i o n employed herein was based on Verner's (1962) d i s t i n c -t i o n between method and techniques and the associated d i f f e r e n t i a t i o n of programme planning and i n s t r u c t i o n . This d i s t i n c t i o n was amplified i n Boshier's (1978) model which enabled Lee (1979) to c l a s s i f y Adult Education 40 a r t i c l e s by subject. This scheme i s an improvement on the c l a s s i f i c a t o r y system used by Dickinson and Rusnell (1971) for t h e i r content analysis of Adult Education. The v a r i a b l e s for c l a s s i f y i n g areas of adult education were as follows: Variable 1 — Foundations of Adult Education — Studies i n this cate-gory were concerned with the functions of adult education; i t s philosophy ( i . e . , r a t i o n a l e and p r i n c i p l e s ) ; i n t e r n a t i o n a l perspectives; l i f e l o n g education; public p o l i c y ; and basic concepts. Variable 2 — C h a r a c t e r i s t i c s of Adult Learners and Adult Learning — These studies focused on the adult l i f e - c y c l e ; p h ysiological/psychological determinants of behaviour; theories of learning; or differences between adults and c h i l d r e n that have implications for learning and motivation. Learning project studies i d e n t i f y i n g c h a r a c t e r i s t i c s of learners were i n -cluded i n t h i s category. Variable 3 — Agency or I n s t i t u t i o n a l Sponsors — These abstracts were concerned with sponsors of adult programmes such as u n i v e r s i t i e s , community groups, or government. The focus of the abstract was the sponsor; a unique way i n which an agency conducted a needs-assessment or programme evaluation was not included i n t h i s v a r i a b l e . Variable 4 — Programme Planning, P a r t i c i p a t i o n , Administration and  Methods — This v a r i a b l e referred to the i n i t i a t i o n and maintenance of edu-c a t i o n a l a c t i v i t i e s including p a r t i c i p a t i o n concerns, administrative concepts, and p r a c t i c e s . Studies included those concerning i n d i v i d u a l , group, and community methods; needs and needs analysis; programme goals; the budgeting and marketing of programmes; t h e i r evaluation; and_participationj drop-out and persistence. 41 Variable 5 — Design and Management of Instruction: Techniques and  D e v i c e s — This v a r i a b l e referred to materials, procedures, s t r a t e g i e s , and/or systems for e s t a b l i s h i n g a r e l a t i o n s h i p between learning tasks and learners. Studies coded "yes" on t h i s v a r i a b l e were concerned with objec-t i v e s e t t i n g , analysis into learning tasks and techniques; techniques and devices; evaluation of learning; and evaluation of i n s t r u c t i o n . V a r iable 6 -— Adult Education as a D i s c i p l i n e and F i e l d of Study — These studies were concerned with issues re l a t e d to the d i s c i p l i n e of adult education. Abstracts on topics such as meta-research, dissemination of knowledge about the d i s c i p l i n e , and the t r a i n i n g of adult educators were coded "yes" on t h i s v a r i a b l e . Coding Each v a r i a b l e was coded dichotomously (see Appendix 1). Most abstracts could be coded i n a s i n g l e category. However, a number of abstracts con-cerned two (and sometimes more) areas of adult education. For example, a study concerning the use of group discussion and a discussion groups was coded "yes" on Variable 4 (Programme Planning) and Variable 5 ( I n s t r u c t i o n ) . 3  Methodological Orientations S o c i a l science textbooks writers use various frameworks to c l a s s i f y "types" of research. Some d i s t i n g u i s h between ex post facto and experi-mental research (e.g., Campbell & Stanley, 1963); others organize chapters around "empirical", " h i s t o r i c a l " , and "methodological" f o c i (e.g., Kerlinger, 1973). The recent handbook of research i n adult education (Long & Hiemstra, 1980) has chapters on "survey research", "grounded theory", " h i s t o r i c a l The d i s t i n c t i o n between group discussion (a technique) and the discussion group (a method) a r i s e s from Verner's (1962) conceptual scheme. 42 research", and "experimental research". This mix of methodologies and t h e o r e t i c a l approaches w i l l r a i s e the i r e of some c r i t i c s . But i t does provide a minimal framework for c l a s s i f y i n g adult education research. Casual examination of A.E.R.C. abstracts suggested that they could be r e l i a b l y and v a l i d l y coded on f i v e dichotomous v a r i a b l e s as follows: Variable 7 — Theoretical/Conceptual — Theory was the primary focus of the study. Variable 8 — A r c h i v a l / H i s t o r i c a l — The abstract reported a study which investigated, recorded, analysed, and interpreted events of the past for the purpose of making generalizations about the past, present, and future. Variable 9 — Empirical/Hard Data — The primary focus of the study was the gathering and analysis of data. Variable 10 — Admonitional/Prescriptive — The abstract exhorted readers to adopt a p a r t i c u l a r stance (for example, i n favour of grounded theory or m u l t i v a r i a t e s t a t i s t i c s ) . The tone of the e n t i r e abstract, not j u s t the conclusions, had to be p r e s c r i p t i v e and hortatory. Variable 11 — Methodological — The intent of the research was c l e a r -l y to i n v e s t i g a t e a use of a p a r t i c u l a r methodology. Empirically-oriented abstracts focusing only on instrument development were also included i n t h i s category. The use of innovative methodology as an i n c i d e n t a l adjunct to a larger problem was not considered feoi.'be a-methodological focus. Coding. Each v a r i a b l e was coded dichotomously (see Appendix 1). P i l o t t e s t i n g procedures determined that most abstracts could be coded i n a s i n g l e cate-gory (e.g., an empirical study). Some were coded "yes" on two v a r i a b l e s . 43 For example, an abstract advocating the use of a p a r t i c u l a r q u a l i t a t i v e methodology was coded "yes" on both methodological and admonitional/pre-s c r i p t i v e . A small number could not be f i t t e d into any category (e.g., a d e s c r i p t i o n of a p a r t i c u l a r A.B.E. outreach programme). Process Variables The v a r i a b l e s included under process were those t y p i c a l l y associated with the canons of research. Six steps commonly employed when designing, completing, and reporting research were considered. Although each step may be employed to varying degrees by d i f f e r e n t researchers, a u t h o r i t i e s (e.g., Kerlinger, 1973) appear to support the notion that any research project w i l l involve: 1. The development of a theory to guide hypothesis formulation and data gathering a c t i v i t i e s (theoretical^dev.elopment); 2. The review of l i t e r a t u r e relevant to the problem ( l i t e r a t u r e review); 3. The design of a plan to gather data relevant to the problem and desired analysis (research design); 4. The implementation of procedures f o r data c o l l e c t i o n , i n s t r u -ment development, and sample s e l e c t i o n (methodology); 5. The generation of r e s u l t s relevant to the theory and problem ( r e s u l t s ) ; and 6. The c r e a t i o n of conclusions and discussion based on the pre-v i o u s l y generated r e s u l t s (conclusions and discussion). Table 2 contains b r i e f descriptions of each of the s i x steps l i s t e d above and shows the scale categories used when coding these v a r i a b l e s . A seventh v a r i -able, "State of the Research" (see Table 2) r e f e r s to the extent to which each abstract reported f i n i s h e d or " i n progress" research. M. Table 2 Process Variables (and their Scaling) Employed i n a Study of A.E.R.C. Abstract Acceptance Steps in the Research Process 1. Theoretical Development 2. Literature Review Variables Deductive Inductive Number of direct c i t a t i o n s Scale Categories Not deductive = 1 Possibly deductive = 2 Probably deductive = 3 Def i n i t e l y deductive = 4 Not inductive = 1 Possibly inductive = 2 Probably inductive = 3 Def i n i t e l y inductive = 4 Actual count Number of authors cited Actual count 3. Research Desig 4. Methodology 5. Results 6. Conclusions & Discussion 7. State of the Research Cumulative Literature Novelty of Research Research Design Data Collection Instrumentation Instrument R e l i a b i l i t y Instrument V a l i d i t y Sample or Population Analysis used Type of Analysis Results Conclusions Research Implications Theoretical Implications Implications for the Fi e l d State of the Research Not at a l l cumulative = 1 Sl i g h t l y cumulative = 2 Moderately cumulative = 3 Extremely cumulative = 4 Is an elaboration of old ideas = 1 Breaks new ground or presents new ideas Not i d e n t i f i e d = 1 Ex post facto (including h i s t o r i c a l ) - 2 Quasi-experimental = 3 Experimental c 4 Not i d e n t i f i e d - 1 Barely i d e n t i f i e d - 2 P a r t i a l l y i d e n t i f i e d = 3 E x p l i c i t l y i d e n t i f i e d = 4 No = 1 Yes = 2 Analyzed data but "type" unclear = 2 Univariate — frequencies only = 3 Bivariate — chi-square analysis, one-way ANOVA, t-test - 4 Multivariate — regression, factor analysis, discriminant function analysis AID 3 = 5 Not i d e n t i f i e d = 1 Barely i d e n t i f i e d = 2 P a r t i a l l y i d e n t i f i e d = 3 E x p l i c i t l y i d e n t i f i e d = 4 Not i d e n t i f i e d = 1 Barely i d e n t i f i e d - 2 P a r t i a l l y Identified = 3 E x p l i c i t l y identfied = 4 Conceptual phase = 1 Planning phase = 2 Operational phase « 3 Analytical phase = 4 Results and Conclusions phase = 5 Implications phase = 6 45 Step One: T h e o r e t i c a l Development Coding. As noted i n Table 2, two v a r i a b l e s were associated with t h i s step i n the research process. Marx (1963) described four types of "meta-theory", that i s , r e l a t i o n s h i p s between the conceptual (theory) and empiri-c a l (data) l e v e l s . C r i t i c a l to t h i s study was the d i s t i n c t i o n between de-ductive and inductive theory construction. Deductive theory describes a s i t u a t i o n where a formally organized theory guides research. Resulting data are used to modify and produce new and better theory. Inductive theory, which consists of summary statements of empirical r e l a t i o n s h i p s , describes an inverse s i t u a t i o n . Data are c o l l e c t e d , analyzed, and t h e o r e t i c a l s t a t e -ments made, a f t e r the data "have spoken". Numerical codes assigned to each category are shown below. Variable 12 — - Deductive Theory Not deductive — This code was used where the research was not deductively derived. For example, i f the research was d e f i n i t e l y i n -ductively derived, or i f no t h e o r e t i c a l base was apparent, the study was coded i n t h i s category. = 1 Possibly deductive — This code was used where the study possibly flowed from an e x i s t i n g theory. For example, an abstract with the statement "An exploratory attempt was made to i n v e s t i g a t e the e f f i c a c y of a t h e o r e t i c a l model which predicts p a r t i c i p a t i o n i n continuing p r o f e s s i o n a l education" would be coded i n t h i s category. = 2 Probably deductive — This code was used where the theo-r e t i c a l framework employed was most l i k e l y deductively derived. A study drawing t h e o r e t i c a l guidance from M i l l e r ' s force f i e l d analy-s i s of p a r t i c i p a t i o n i n adult education was an example i n t h i s cate-gory. = 3 D e f i n i t e l y deductive — This code was used where the study d e f i n i t e l y flowed from extant theory. For example, a study of teacher-learner interactions based on Rogerian congruence theory was coded " d e f i n i t e l y deductive". = 4 The second v a r i a b l e concerned the extent to which the study flowed from an i n d u c t i v e l y derived t h e o r e t i c a l stance. 46 Variable 13 — Inductive Theory Not inductive — This code was used where the research was not i n d u c t i v e l y derived. I f , for example, the study was d e f i n i t e l y deductively derived or, i f there was no evidence of inductive processes, the abstract was coded i n t h i s category. = 1 Possibly inductive — This code was used where the study showed some evidence of inductive t h e o r e t i c a l development. For example, an abstract s t a t i n g "research i d e n t i f i e d 23 s k i l l s asso-ciated with establishment of psychological climate and management of learning groups" or a paper focusing on learning projects data were coded as "possibly inductive". = 2 Probably inductive — This code was used where the r e -search showed strong evidence of inductive processes. For example, an abstract which stated "In an e f f o r t to construct empirically derived scenarios concerning the immediate future of adult educa-t i o n , a Delphi survey was conducted" was coded i n t h i s category = 3 D e f i n i t e l y inductive — This code was used where the research was d e f i n i t e l y based on inductive processes. Recent "grounded theory" studies of adult basic education organizations or works of f i c t i o n were examples from t h i s category. = 4 Step Two: L i t e r a t u r e Review A l i t e r a t u r e review serves to explain the t h e o r e t i c a l r a t i o n a l e for the problem and locate the study i n an e x i s t i n g body of research. Four va r i a b l e s were examined to assess the extent to which abstracts conformed to t h i s step i n the research process. These were: number of d i r e c t c i t a -t ations; number of authors c i t e d ; cumulativeness of the l i t e r a t u r e ; and novelty of the research. Variable 1 4 — N u m b e r of d i r e c t c i t a t i o n s C i t a t i o n s d i r e c t l y c i t e d i n the abstract (00 - 99) (enter the actual count) Refers to the number of d i r e c t references made to studies, a r t i c l e s , instruments (e.g., Boshier's E.P.S.). For example, "previous p a r t i c i p a -t i o n studies ..." i s not a d i r e c t c i t a t i o n . For a c i t a t i o n to be d i r e c t , someone must be named. If an author i s c i t e d more than once for d i f f e r e n t contributions (e.g., Tough, 1974; 1978) t h i s counts as two c i t a t i o n s . S e l f - c i t a t i o n s are also included. 47 Variable 15 — Number of authors c i t e d The number of d i f f e r e n t authors c i t e d (00 - 99) (enter the actual count) Refers to the i n d i v i d u a l authors c i t e d . If there are co-authors, each i s counted i n d i v i d u a l l y (e.g., Johnston & Rivera' counts as two authors). If an author i s c i t e d more than once (e.g., Verner, 1962; Verner & Booth, 1964) t h i s counts as only one c i t a t i o n for Verner. An i n s t i t u t i o n i s an author (e.g., UNESCO, 1972). Variable 16 — Cumulative L i t e r a t u r e — This v a r i a b l e referred to the cumulativeness of l i t e r a t u r e i n adult education and rela t e d d i s c i p l i n e s and focused on the content, not the methodology of the research. For example, a study of m i d - l i f e c r i s e s using a content analysis of novels by male authors stems from a substantial body of knowledge although the research approach was "new". This abstract was considered to stem from an "extremely cumulative" body of knowledge. L i b e r a l c r i t e r i a were used to code t h i s v a r i a b l e as follows: Not at a l l cumulative — This code was used when studies did not appear to stem from any "known" body of research or approach to the problem. Examples of abstracts coded i n t h i s category included a study of the war metaphor i n adult basic education or an analysis of e x i s t e n t i a l themes i n adult education. = 1 S l i g h t l y cumulative — This code was used when reference was made to not more than one "antecedent" piece of l i t e r a t u r e or recognizable idea/model/theoretical o r i e n t a t i o n i n adult education or another d i s c i p l i n e . Examples i n t h i s category included studies concerning the future of adult education. = 2 Moderately cumulative — This code was used when r e f e r -ence was made to at least two "antecedents" or a modest body of knowledge known to e x i s t i n adult education or elsewhere. Examples i n t h i s category included: meta-research l i t e r a t u r e (e.g., an h i s t o r i c a l analysis and taxonomy of meta-research i n adult education); l i t e r a t u r e on margin (e.g., a study t e s t i n g the theory of margin using a population of widows); comparative education (e.g., a study on the a s c r i p t i o n of needs i n adult education i n Alberta and Quebec); or learning projects (e.g., an abstract concerning the learning pro-j e c t s of low income, urban a d u l t s ) . = 3 48 Extremely cumulative — This code was used when reference was made to three or more "antecedents" or to a substantial body of knowledge. T y p i c a l of l i t e r a t u r e i n t h i s category were studies focusing on p a r t i c i p a t i o n , the adult l i f e - c y c l e , programme evalua-t i o n , motivational orie n t a t i o n s , group dynamics,or needs-assessment. = 4 Variable 17 — Novelty of Research for Adult Education — In order to develop a s o l i d body of knowledge, researchers are encouraged to r e p l i c a t e and b u i l d on previous research. However, novel approaches to research stimulate growth. Casual observation and consultation with present and former members of the A.E.R.C. steering committee suggested that some v a r i -ance i n acceptance was due to the unconventional, new, or "catchy" nature of a project. This dichotomous v a r i a b l e was scaled as follows: Is an elaboration of old ideas — This code was used when the research followed " t r a d i t i o n a l " methodologies and/or w e l l -established areas of study i n adult education. Examples i n t h i s category included abstracts describing a d e s c r i p t i v e study on l e a r n -ing projects; continuing professional education needs assessment; research on competencies of adult i n s t r u c t o r s ; or studies u t i l i z i n g the Education P a r t i c i p a t i o n Scale (Boshier, 1977). = 1 Breaks new ground or presents new ideas — This code was used when the study appeared to employ an innovative methodology or concern a "novel" problem or heretofore neglected area of research. Examples included a study which examined the "value of Jean-Paul Sartre's p h i l o s o p h i c a l play The F l i e s i n terms of i t s e x i s t e n t i a l e t h i c a l value for adult educators"; an abstract based on the use of "autobiographical material to investigate the l i f e perspectives of adults"; and a study on "the a r t i s t as educator" which u t i l i z e d interviews with a r t i s t s and "aesthetic theory with s p e c i a l r e f e r -ence to the communicative function of a r t " . = 2 Step Three: Research Design t A research design i s the plan, structure, and strategy of the study. Its basic purpose i s to provide answers to questions i n s c i e n t i f i c a l l y de-f e n s i b l e ways. The v a r i a b l e associated with t h i s step of the research pro-cess was "research design". For present purposes three design categories were considered: ex post facto, quasi-experimental, and experimental. 49 Variable 18 — Research Design Ex post facto — The researcher examined the e f f e c t s of a n a t u r a l l y occurring treatment a f t e r i t had occurred. This cate-gory was broadened to also include h i s t o r i c a l research and Campbell and Stanley's (1963) pre-experimental designs ( i . e . , one-shot case, study, one-group pretest-posttest design, s t a t i c group comparison). This category also included survey research, content analyses, and d e s c r i p t i v e or case studies. Most adult education research was coded i n this category. = 2 Quasi-experimental — The researcher manipulated a treatment and c o n t r o l l e d f or some, but not a l l sources of i n -t e r n a l v a l i d i t y . Included i n t h i s category were time-series, equi-valent time samples, equivalent materials samples, and non-equivalent c o n t r o l groups designs (Campbell & Stanley, 1963). If an abstract stated " F i f t y adults p a r t i c i p a t e d , the experimental groups attended a one day workshop; the co n t r o l group did not attend. Pre- and post-tests were administered", the abstract was coded quasi-experimental. There was no mention of random assign-ment, a c r i t i c a l element i n an experimental design. = 3 Experimental — The investigator manipulated at least one independent v a r i a b l e and c o n t r o l l e d for a l l sources of i n t e r n a l v a l i d i t y . These designs included: pretest-posttest c o n t r o l group, Solomon four-group, and posttest only co n t r o l group (Campbell & Stanley, 1963). For the purposes of t h i s study, f a c t o r i a l designs i n which randomization was used, were also included i n t h i s cate-gory. = 4 Coding. As shown i n Table 2, the three design categories were coded as 2, 3, and 4 r e s p e c t i v e l y . If the design was not apparent, or no design was c a l l e d f o r , the abstract was coded 1. Step Four: Methodology Methodology r e f e r s to formal procedures c a r r i e d out by the researcher. The researcher s e l e c t s a sample, c o l l e c t s data, tests hypotheses, and analyzes data. Methodology i s not l i m i t e d to empirical research; concep-t u a l / t h e o r e t i c a l or a r c h i v a l / h i s t o r i c a l studies are also based on sound research s t r a t e g i e s . For example, an h i s t o r i c a l l y focused study included information on primary and secondary data sources and subsequent analysis s t r a t e g i e s . 50 As noted i n Table 2, s i x variables were associated with t h i s step i n the research process: data c o l l e c t i o n , instrumentation, instrument r e l i a b i l i t y , instrument v a l i d i t y , sample or population, and type of analysis. Variable 19 — Data C o l l e c t i o n — This v a r i a b l e was scored as follows: Not i d e n t i f i e d — This code was used when an abstract gave no information concerning data c o l l e c t i o n , for example, "Data were gathered". = 1 Barely i d e n t i f i e d — This code was used when an abstract made only vague statements concerning data c o l l e c t i o n , for example, "100 interviews were conducted"; "questionnaires were mailed to state-wide college administrators"; "a grounded theory approach was used"; or "workshop p a r t i c i p a n t s completed an a t t i t u d e scale". = 2 P a r t i a l l y i d e n t i f i e d — This code was used when an abstract i d e n t i f i e d several procedures i n the data c o l l e c t i o n pro-cess, for example, "Trained interviewers conducted 20 minute t e l e -phone interviews" or "a grounded theory approach, u t i l i z i n g in-depth interviews with college administrators". = 3 E x p l i c i t l y i d e n t i f i e d — This code was used when the abstract c l e a r l y explained a l l data gathering procedures. For example, the following abstract was coded i n t h i s category: "data c o l l e c t i o n was of two types: a paper and p e n c i l instrument com-prised of semantic d i f f e r e n t i a l items ... and open ended questions to assess ... and video-taping of each session with s e n s i t i v e recording of verbal and non-verbal reactions to the ongoing ex-perience". = 4 Variable 20 — Instrumentation — This v a r i a b l e concerned the extent to which instruments used i n the data c o l l e c t i o n process were described. To q u a l i f y , the abstract had to i d e n t i f y and/or describe an instrument(s) used to c o l l e c t data. Frequently used instruments include personality t e s t s , a t t i t u d e scales, i n t e r e s t inventories, i n t e l l i g e n c e and aptitude t e s t s , and interview schedules. Other instruments include r a t i n g scales or recording devices used i n connection with, for example, a Bales or 51 Flanders Interaction analysis. As with some previous v a r i a b l e s , l i b e r a l c r i t e r i a were used for coding. However, instrument i d e n t i f i c a t i o n was not based on naming generic types ( i . e . , p ersonality t e s t ) ; the name or some content was necessary i n order to be coded. Thus, a statement such as "Subjects completed a personality inventory" was coded "not i d e n t i f i e d " . Coding. This v a r i a b l e was scaled as follows: Not i d e n t i f i e d — No mention of s p e c i f i c instrumentation. = 1 Barely i d e n t i f i e d — Only a "bare-bones" d e s c r i p t i o n i s given (e.g., a questionnaire concerning attitudes towards continu-ing professional education was administered). = 2 P a r t i a l l y i d e n t i f i e d — Some content of the instrument i s described (e.g., the f i r s t question asked respondents to rank order statements concerning the need for continuing professional education). = 3 E x p l i c i t l y i d e n t i f i e d — S p e c i f i c d e t a i l concerning at le a s t one instrument i s given. The name or author of an instrument (e.g., 16 P.F., Eysenck Personality Inventory) i s s u f f i c i e n t if. the instrument i s known and has an established reputation. Where the experimenter has used two or more instruments only one need be " e x p l i c i t l y " i d e n t i f i e d . = 4 Variable 21 — Instrument R e l i a b i l i t y — This referred to the s t a b i -l i t y and/or consistency of a measuring instrument. To be " p o s i t i v e l y " coded on t h i s v a r i a b l e , an abstract had to contain a d e s c r i p t i o n of the type of r e l i a b i l i t y — t e s t - r e t e s t , p a r a l l e l forms, or. i n t e r n a l consistency. Coding. The following four-point scale was used to score instrument r e l i a b i l i t y : Not i d e n t i f i e d — No information given. = 1 Barely i d e n t i f i e d — Reference to the f a c t instrument i s r e l i a b l e but no evidence of having tested i t s r e l i a b i l i t y i n the present study (e.g., ... an instrument with known r e l i a b i l i t i e s was employed . . . ) . =2 52 P a r t i a l l y i d e n t i f i e d — Oblique reference to the fa c t instrument r e l i a b i l i t y procedures were employed i n the present study, r e s u l t s are probably a v a i l a b l e but are not revealed i n the abstract (e.g., a s i x week test r e - t e s t r e l i a b i l i t y procedure was employed . ..) . =3 E x p l i c i t l y i d e n t i f i e d — Actual type of r e l i a b i l i t y tests and/or r e s u l t s are revealed (e.g., a s i x week test r e - t e s t procedure showed that the instrument was r e l i a b l e r = .67, p.< .05). = 4 Variable 22 — Instrument V a l i d i t y — This referred to the degree to which an instrument a c t u a l l y measured what i t was designed to measure. A "high" code on t h i s v a r i a b l e was obtained i f an abstract described the types of v a l i d i t y tested f o r . Types of v a l i d i t y include: content, c r i t e r i o n - r e l a t e d , and construct. Coding. The scale f or t h i s v a r i a b l e , based on a l i b e r a l coding c r i -t e r i o n was: Not i d e n t i f i e d — No information given at a l l Barely i d e n t i f i e d — Reference to the fa c t the instrument i s v a l i d but no evidence of having tested the v a l i d i t y i n the pre-sent study (e.g., an instrument with known v a l i d i t y was employed). = 2 P a r t i a l l y i d e n t i f i e d — Oblique reference to the fa c t instrument v a l i d i t y procedures were employed i n the present study, r e s u l t s are probably a v a i l a b l e but not revealed i n the abstract (e.g., instrument v a l i d i t y was determined). = 3 E x p l i c i t l y i d e n t i f i e d — Actual type of v a l i d i t y pro-cedures and/or r e s u l t s are revealed (e.g., content v a l i d i t y was determined by submitting the instrument to a panel of judges). = 4 Variable 23 — Sample or Population — This v a r i a b l e concerned the extent to which an abstract described either the study population or sample. It included the number and demographic c h a r a c t e r i s t i c s of subjects. The strategies used (e.g., s t r a t i f i e d random sample, table of random numbers) to draw the sample were also included i n t h i s v a r i a b l e . 53 Coding. The following scale was used on t h i s v a r i a b l e : Not i d e n t i f i e d — No de s c r i p t i o n i s given. = 1 Barely i d e n t i f i e d — Only a "bare-bones" d e s c r i p t i o n given (e.g., t o t a l s i z e only — 100 administrators). = 2 P a r t i a l l y i d e n t i f i e d — The t o t a l number plus two other pieces of information concerning the S's (e.g., 100 women, 1 8 - 3 5 years). = 3 E x p l i c i t l y i d e n t i f i e d — The number and three or more a d d i t i o n a l pieces of information concerning S's or selection/sampling procedures (e.g., 100 female Baptist high school teachers were randomly selected). = 4 Variable 24 — Type of Analysis — This v a r i a b l e concerned two ques-tion s : was some form of data analysis mentioned i n the study? If so, what was the "highest"type of analysis — unclear, univariate, b i v a r i a t e , or multivariate? The c r i t e r i a for coding t h i s v a r i a b l e were les s l i b e r a l than for other v a r i a b l e s . The abstract had to e x p l i c i t l y describe the type of analysis used. For example, the statement "appropriate m u l t i v a r i a t e s t a t i s -t i c s were used" was not considered m u l t i v a r i a t e analysis, rather i t was categorized as "type unclear". Coding. For the purposes and nature of the research reported, a c l a s s i f i c a t i o n scheme which would y i e l d meaningful data was as follows: Some form of data analysis was mentioned i n th i s study: No. = 1 Yes. = 2 "Highest" type of data a n a l y s i s : > Analyzed data but "type" unclear. = 2 Univariate — Frequencies only. = 3 B i v a r i a t e — Chi-square a n a l y s i s , c o r r e l a t i o n s , One-way ANOVA, t - t e s t . = 4 Mu l t i v a r i a t e — Regression, factor a n alysis, d i s -criminant function analysis, AID 3. = 5 54 Although the conceptual foundations of t h i s s c a l i n g might be questioned i t i s contended that t h i s item meets assumptions for o r d i n a l i t y . Where an abstract did not reveal or allude to any type of analysis i t was assigned the lowest value (one). Where " a n a l y s i s " was alluded to, but the "type" was unclear (e.g., "appropriate analyses were conducted") i t was coded two; un i v a r i a t e analyses were coded three; b i v a r i a t e analyses were coded four; m u l t i v a r i a t e analyses were coded f i v e . Step Five: Results The " r e s u l t s " represent the phase i n the research process where data are presented. As the f o c a l point of the research, the data are c r i t i c a l l y analyzed and reported i n t h i s phase. The researcher presents data relevant to the research hypothesis. The data are often presented i n tables or fig u r e s explained with a written commentary. Research r e s u l t s appear i n abstracts, although i n an abbreviated form. Abstracts contain " r e s u l t s " statements, although tables and figures are not presented. Variable 25 — R e s u l t s — This v a r i a b l e considered the extent to which the author reported outcomes a r i s i n g from data c o l l e c t i o n and an a l y s i s . A l -though l i b e r a l c r i t e r i a were used i n coding t h i s v a r i a b l e , the abstract had to contain some statement, whether very general or more s p e c i f i c , which re f e r r e d to the research r e s u l t s . Coding. This v a r i a b l e was scaled as follows: Not i d e n t i f i e d — No r e s u l t s were given = 1 Barely i d e n t i f i e d — If the word r e s u l t ( s ) appears, code "barely" because the researcher acknowledges t h i s element exists (e.g., r e s u l t s of the study w i l l be discussed). = 2 P a r t i a l l y i d e n t i f i e d — A general " r e s u l t " statement or only one r e s u l t appears (e.g., r e s u l t s indicated female teachers have a more negative a t t i t u d e towards continuing educa-t i o n than male teachers). = 3 E x p l i c i t l y i d e n t i f i e d — A d e f i n i t e statement of two or more r e s u l t s appears (e.g., r e s u l t s indicated a more negative a t t i t u d e towards formal continuing education by female teachers than male teachers. Female teachers, however, spend a greater amount of time on i n d i v i d u a l learning p r o j e c t s ) . = 4 As indicated, i f the statement "Results of the study w i l l be discussed" appeared, the abstract was coded "barely i d e n t i f i e d " . The author was given c r e d i t for acknowledging that r e s u l t s are i n t e g r a l to the study. For a "higher" code the abstract required one or two s p e c i f i c statements report-ing a ctual r e s u l t s . Step Six: Conclusions and Discussion This step, the f i n a l phase of the research process, contained several elements. The f i r s t i s the " f i n d i n g s " which are f a c t u a l statements based on data analyzed. A second i s a discussion of the l i m i t a t i o n s and weaknesses of the study; In the conclusions, the researcher explores questions raised i n the study or states whether the research hypothesis i s accepted or r e -jected. A f i n a l element consists of statements which suggest areas or problems for further i n v e s t i g a t i o n or which draw implications for research, theorizing, or the f i e l d . In an abstract a l l or some of the above-mentioned elements w i l l appear i n an abbreviated form. Coding. For t h i s study, four v ariables represented t h i s phase: con-clusions; implications for research; implications for theorizing; and implications for the f i e l d of practice.-56 Variable 26 — Conclusions — - This v a r i a b l e acknowledged the presence or absence of conclusions stated i n the abstract. It did not question the v a l i d i t y of the conclusions, only the extent to which the abstract reported them. These may have been broad or s p e c i f i c statements. The v a r i a b l e was scaled as follows: Not i d e n t i f i e d — No conclusions were given. = 1 Barely i d e n t i f i e d — If the word conclusion(s) appears, code "barely" because the researcher acknowledges t h i s element exi s t s (e.g., conclusions w i l l be discussed). = 2 P a r t i a l l y i d e n t i f i e d — A general statement of a con-c l u s i o n appears (e.g., i t can be concluded that a conference i s a successful means of disseminating information). = 3 E x p l i c i t l y i d e n t i f i e d — A d e f i n i t e statement of two or more conclusions appears (e.g., i t can be concluded that the Semantic D i f f e r e n t i a l i s both a r e l i a b l e and v a l i d measure of the a t t i t u d e change which conference p a r t i c i p a n t s underwent). = 4 As with preceding v a r i a b l e s , l i b e r a l c r i t e r i a were employed. For example, i f a sentence such as "Conclusions w i l l be discussed" appeared i n the ab-s t r a c t , the author was given c r e d i t for acknowledging t h e i r importance. Variable 27 — Implications for Research — Implications allowed the author to extrapolate from the present study to other areas such as further research. The abstract had to contain a general or s p e c i f i c statement con-cerning implications. The following scales were used for coding: Not i d e n t i f i e d — No implications were mentioned. = 1 Barely i d e n t i f i e d — The researcher "barely" acknow-ledges implications a r i s e from the study (e.g., implications for future research w i l l be considered) but does not state what they are. = 2 P a r t i a l l y i d e n t i f i e d — A general statement of i m p l i -cations f o r research appears (e.g., further studies must be con-ducted to determine the extent of i n d i v i d u a l learning p r o j e c t s ) . At l e a s t one actual i mplication i s noted. = 3 57 E x p l i c i t l y i d e n t i f i e d — A d e f i n i t e statement of at le a s t two implications for research appears (e.g., further studies, u t i l i z i n g more precise c r i t e r i a than those i n t h i s study, must be conducted to determine the extent of i n d i v i d u a l learning projects. As we l l , learners must . . . ) . = 4 Variable 28 — Implications for Theorizing — The same c r i t e r i a for coding applied to t h i s v a r i a b l e : Not i d e n t i f i e d — No implications were mentioned. = 1 Barely i d e n t i f i e d — The researcher "barely" ac-knowledges implications a r i s e from the study (e.g., implications for future theorizing w i l l be discussed) but does not discuss any. = 2 P a r t i a l l y i d e n t i f i e d — A general statement of at l e a s t one implic ation for theorizing appears (e.g., force f i e l d analysis w i l l be a valuable t o o l i n understanding adult p a r t i c i p a t i o n ) . = 3 E x p l i c i t l y i d e n t i f i e d — A d e f i n i t e statement of at lea s t two implications for theorizing appears (e.g., force f i e l d a nalysis applied to adult p a r t i c i p a t i o n suggests the need to re-evaluate this concept. Furthermore . . . ) . = 4 Variable 29 — Implications for the F i e l d of Practice — As i n the pre-vious two v a r i a b l e s , the following coding scales were employed: Not i d e n t i f i e d — No implications were mentioned. = 1 Barely i d e n t i f i e d — The researcher "barely" acknow-ledges implications a r i s e from the study (e.g., implications for the p r a c t i c e of adult education w i l l be discussed) but does not a c t u a l l y state any. = 2 P a r t i a l l y i d e n t i f i e d — A general statement of at least one implication for the f i e l d of p r a c t i c e appears (e.g., t h i s study indicates the need to develop a futures orientation:" i n adult education). = 3 E x p l i c i t l y i d e n t i f i e d — A d e f i n i t e statement of at lea s t two implications for the f i e l d of p r a c t i c e appears (e.g., t h i s study indicates the need f or programme planners to give greater consideration to macro-level (e.g., community, s o c i e t a l and global) needs data, and develop a . . . ) . =4 58 These v a r i a b l e s were coded l i b e r a l l y . If a vague sentence such as "Implications for the f i e l d of p r a c t i c e w i l l be discussed" was i n the abstract, the author was credited with acknowledging that implications arose, though none were a c t u a l l y stated. State of the Research The preceding nineteen variables described the research process. Individual or groups of variables focused on a p a r t i c u l a r phase of t h i s process. Coding procedures demanded that the researcher judge the extent to which each v a r i a b l e was present i n the abstract. An a d d i t i o n a l v a r i a b l e , "state of the research", concerned the "completeness" of the research or the most advanced; stage to which the research had progressed. Coding. Variable 30 — State of the Research — The following scale was used: Conceptual phase — Evidence that the nature of the problem and v a r i a b l e s have been conceptualized, but no evidence that data gathering operations have been performed. Planning phase — Plans for implementation of research procedures are revealed but no evidence of actual implementation i s presented. = 1 = 2 Operational phase — Researcher has implemented pro-cedures and gathered data. There i s no evidence of data analysis. = 3 A n a l y t i c a l phase — Data were gathered -and analyzed but no e x p l i c i t r e s u l t s were revealed. = 4 Results and Conclusions phase — E x p l i c i t r e s u l t s and conclusions are described. Mere i l l u s i o n to, or a ;statement saying that there are r e s u l t s and conclusions i s inadequate. Actual r e s u l t s and conclusions must be described. = 5 Implications phase — E x p l i c i t implications for theory, future research or p r a c t i c e are described. Mere i l l u s i o n to, or a statement saying that there are implications i s inadequate. Actual implications must be described. = 6 59 L i b e r a l coding c r i t e r i a were not applied to t h i s v a r i a b l e . For example, i f the abstract stated "Results and conclusions w i l l be discussed", i t was coded i n the " A n a l y t i c a l Phase"; no e x p l i c i t r e s u l t s and conclusions were a c t u a l l y presented. These research phases applied to empirical, h i s t o r i c a l , and conceptual studies. An abstract with a theoretical/conceptual focus was coded as research i n the "conceptual phase". For example, an abstract which concerned a theory of paradigm-transition learning was research i n the conceptual phase. The abstract advanced a theory, but i t was not tested. Well-designed a r c h i v a l / h i s t o r i c a l studies also followed phases or states of research. Compositional Variables The two groups of variables described thus far focused on the "content" ( i . e . , topic or areas of study of the abstract) and the "process" ( i . e . , r e l a t e d to the research process) of the research. The next group of v a r i -ables concerned the "composition" of an abstract. Composition ref e r s to s t y l e ( i . e . , the author's w r i t i n g s t y l e , grammar) and presentation ( i . e . , layout, neatness). The following variables r e l a t e d to composition: number of words i n the t i t l e , number of words i n the abstract, abstract presented (on the o r i g i n a l form), attachments (added to the a b s t r a c t ) , abstract pre-sentation, abstract "voice", jargon, funding source revealed, and c l a r i t y and l o g i c a l flow of the argument. Coding The coding for most of these variables was straightforward. Examples included Variables 31 and 32 — Number of words i n the t i t l e or abstract; Variable 33 — abstract presented on the o r i g i n a l form ( i . e . , the standard form issued i n the A.E.R.C. C a l l for Papers); Variable 34 — attachments 60 added to the abstract ( i . e . , were a d d i t i o n a l sheets stapled onto the form?); and Variable 35 — funding source ( i . e . , did the abstract mention the funding source of the research?). Others required some judgement on the part of the coder. Variable 36 — Abstract Presentation — This v a r i a b l e r e f e r r e d to the physical presentation of the abstract: Sloppy (e.g., gross typing errors; crossing out; bad j u s t i f i c a t i o n on typing). = 1 Not very neat (e.g., some but not gross errors i n typing and layout). = 2 Moderately neat (e.g., no typo's but spacing, etc., not p e r f e c t ) . = 3 Very neat (e.g., 100% error free; impeccable neatness and layout). = 4 Variable 37 — Abstract Voice -— The dominant "type" of verb used by the author was used to determine the "voice" of the abstract. In the coding process, abstracts written p r i m a r i l y i n an a c t i v e voice were distinguished from those written i n the passive voice. Passive voice — t r a n s i t i v e verbs a t t r i b u t e the verbal a c t i o n to the person or object (e.g., It i s contended that; Chi-square analysis was performed; A questionnaire was completed by p a r t i c i p a n t s ) . = 1 Active voice — the subject performs the action re-presented by the verb (e.g., The author contends that; The researcher performed a chi-square a n a l y s i s ; P a r t i c i p a n t s completed a questionnaire). = 2 Variable 38 Jargon — According to the Concise Oxford Dictionary (Sykes, 1976) jargon i s speech f a m i l i a r only to a group or profession. Adult education has some fu n c t i o n a l jargon stemming from d i f f e r e n t areas of study. Examples include: "motivational o r i e n t a t i o n s " ; "the adult's mar-gin"; "higher order needs"; " e x p e r i e n t i a l learning"; and "needs assessment 61 strategy". However, authors often "dress-up" t h e i r w r i t i n g by using un-necessary jargon. The following sentence from an abstract i l l u s t r a t e s t h i s point: "At the same time an important connection was seen i n T. Kuhn's thesis that the development of s c i e n t i f i c knowledge i s character-ized by phases of accumulation within the p r e v a i l i n g paradigm and revolu-tions i n which the whole basis of the paradigm i s challenged by an altered conception of the f i e l d " . In t h i s example, the jargon was dysfunctional; i t confused the meaning of the sentence. Functional jargon, on the other hand, served to c l a r i f y meaning. For example, the phrase "the adult's margin" i s unique to adult education. This phrase l a b e l s McClusky's (1963) concept and i s buttressed by empirical research. It i s a short way of l a b e l l i n g hypothesized r e l a t i o n s h i p s between an adult's "power" and "load"; i n b r i e f i t describes an aspect of the adult learner that many writers con-sider helps d i s t i n g u i s h adult from pre-adult education. Its use i s i i - . -f u n c t i o n a l . The following scale was used for t h i s v a r i a b l e : Extensive (e.g., extensive use of unnecessary jargon to "dress-up" the abstract — involves use of non-standard jargon). = 1 Moderate (e.g., two or three usages of unnecessary . overly pompous jargon). = 2 Rare (e.g., one or bare minimum use of unnecessary jargon). = 3 None (e.g., abstract i s cleanly written i n " p l a i n " language using only "standard" adult education or s o c i a l science jargon). = 4 Variable 39 — C l a r i t y and Logical Flow of the Argument — This v a r i -able r e f e r r e d to the author's a b i l i t y to c l e a r l y and l o g i c a l l y o u t l i n e the nature of the problem, methodology, r e s u l t s , and conclusions of the study. C l a r i t y of the argument or flow of the abstract was considered to be 62 independent of the substance or o v e r a l l content. For example, the abstract could have been c l e a r l y and l o g i c a l l y written yet devoid of substantive information, i . e . , c l e a r but naive. This v a r i a b l e was scaled as follows: A copy of the coding schedule used i s presented i n Appendix 1. The schedule consisted of the v a r i a b l e s , t h e i r c r i t e r i a , and a b r i e f descrip-t i o n of the "anchoring" points for each v a r i a b l e category or l e v e l . The extent to which the v a r i a b l e i d e n t i f i c a t i o n and coding system would r e s u l t i n an instrument s u f f i c i e n t l y r e l i a b l e and v a l i d to be used i n the substantive study was examined i n a series of p i l o t studies. The extent to which expert judges made consistent coding decisions and the degree to which the instrument was content v a l i d were examined. In addition to the p i l o t study, issues re l a t e d to s t a b i l i t y across time were also ex-amined i n the main study (see Chapter 5 "Procedures", p. 68). Interjudge R e l i a b i l i t y In t h i s study, interjudge r e l i a b i l i t y r e f e r r e d to the agreement among f i v e judges trained to use the coding schedule. The following steps out-l i n e the interjudge r e l i a b i l i t y procedures followed: 1. Two professors of adult education with extensive knowledge of research methodology c r i t i c i z e d the f i r s t draft of the coding schedule, paying p a r t i c u l a r attention to the instrument, the d e f i n i t i o n of v a r i a b l e s , coding categories, and the proposed format. Recommended changes were i n -corporated i n a second coding schedule. Moderately clear Extremely clear Not at a l l clear Only s l i g h t l y c l e a r = 1 = 2 = 3 = 4 PILOT TESTING 2. A panel of f i v e judges (the p r i n c i p a l i n vestigator, two pro-fessors, and two adult education doctoral students with knowledge of research methodologies) coded a sample of f i v e abstracts. The judges were trained to use the instrument. This involved an explanation of the schedule the coding of an abstract, followed by a comparison of responses and a discussion. After a d d i t i o n a l c l a r i f i c a t i o n of v a r i a b l e s , the f i v e judges separately coded the f i v e abstracts. Responses were compared and points of c l a r i f i c a t i o n discussed. 3. Changes recommended by the judges i n step two were .incorporated into the f i n a l coding schedule (see Appendix 1). Following an explanation of the changes, the judges independently coded f i v e new abstracts. Table 3 shows r e l i a b i l i t y indices on each v a r i a b l e for two aspects of interjudge r e l i a b i l i t y — " i n t e r j u d g e " which shows the extent to which judges agreed among themselves and "researcher-judges" which shows the ex-tent to which the researcher agreed with the other four judges. Although judges coded a t o t a l of eleven abstracts, r e l i a b i l i t y indices were based only on the l a s t nine to control for t r a i n i n g e f f e c t s . , Training occurred pr i m a r i l y during the coding of the f i r s t and second abstracts. Changes i n -corporated into the f i n a l coding schedule were based on judge's recommenda-tions. Therefore, judges required no a d d i t i o n a l t r a i n i n g before coding the second group of abstracts. Table 3 Inter-judge R e l i a b i l i t y Indices for 39 Variables Related to Acceptance of A.E.R.C. Abstracts 6h Variable Inter-judge Researcher-judges F-Ratio F-Prob. F-Ratio F-Prob. Foundations 1.35 1.86 Characteristics/Adult Learning .57 .33 Agency or I n s t i t u t i o n .71 2.53 . Programme Planning .50 .02 Instruction/Techniques .21 .33 Adult Education D i s c i p l i n e 1.00 .47 Theoretical/Conceptual .31 .85 A r c h i v a l / H i s t o r i c a l 2.28 .51 Empirical/Hard Data .29 .0 Admonitional/Prescriptive 4.29 .005 1.39 Methodological .85 3.22 Process No. of direct c i t a t i o n s .00 .00 No. of authors cited .01 .01 Deductive .34 .78 Inductive 6.94 .0002 7.96 State of the Research 1.25 4.63 Cumulative Literature 1.34 2.57 Novelty of Research 1.11 1.39 Research Design .60 .01 Data Collection .63 .63 Instrumentation .16 .01 Instrument R e l i a b i l i t y pa pa Instrument V a l i d i t y 1.00 .25 Sample or Population .14 .58 .49 .71 2.54 .78 2.59 1.10 Type of Analysis Results Conclusions Research Implications .76 .62 .92 .61 Theoretical Implications 1.28 Implications for the F i e l d Compositional .94 No. of words i n the t i t l e No. of words i n the abstract Original Form Attachments Presentation Voice Jargon Funding Source Flow of the Argument .0 .0 .17 .0 .98 .98 .28 .75 1.59 .0 .0 .0 .0 .01 .02 .88 .51 .03 .007 .04 65 The SPSS subprogram R e l i a b i l i t y , which performs a repeated measures design analysis of variance, was used to examine interjudge agreement concerning the nine abstracts coded by the f i v e judges. As shown i n Table 3, the s i g n i f i c a n c e of differences between the mean codes of the f i v e judges was generally n e g l i g i b l e . Considering that 39 calcu l a t i o n s were involved, i t i s possible that the two s i g n i f i c a n t F-values for the interjudge ratings occurred because of Type I errors. This, coupled with the 37 i n s i g n i f i c a n t F r a t i o s suggested that judges made consistent ratings of the nine abstracts. Despite t h i s conclusion, caution demanded that the s t a t i s t i c s f o r admoni-t i o n a l / p r e s c r i p t i v e (F = 4.29) and inductive (F = 6.94) be examined further. An examination of means and S.D.'s for each researcher on the v a r i a b l e inductive showed the following: Judge 1 (the researcher) —X=1.00, S.D.=.00; Judge 2 —X=2.56, S.D.=1.01; Judge 3 —X=1.78; S.D.=.67; Judge 4 —X=1.78, S.D.=.67; and Judge 5 —X=1.22, S.D.=.67. An examination of the v a r i a b l e admonitional/prescriptive showed the following means and S.D.'s. Judge 1 (the researcher) —X=1.00, S.D.=.00; Judge 2—X=1.00, S.D.=.00; Judge 3 — X = l . l l , S.D.=.33; Judge 4 —X=1.00, S.D.=.00; and Judge 5 —X=1.44, S.D.=.53. These findings suggest that the disagreement amongst the judges was not caused by the researcher. The researcher-judges r e l i a b i l i t y index was also computed using the SPSS subprogram R e l i a b i l i t y . Table 3 shows that on only two of the 39 va r i a b l e s , inductive (F=7.69, p < .007) and state of the research' (F=4.63, p < .04) were the researcher and judges i n s i g n i f i c a n t disagree-ment. The researcher's inductive' mean was 1.00 (S.D.=0.00): the mean for the other judges was X=1.83 (S.D.=.88). The inductive r e s u l t s suggest that the large discrepancy between the mean of Judge 2 and those of the other judges influenced the degree of agreement between the r e -searcher and the four judges. The v a r i a b l e admonitional/prescriptive yielded no s i g n i f i c a n t disagreement between the researcher and the other judges. Though state of the research showed that the researcher did not agree with the other judges, there was no s i g n i f i c a n t disagreement between the f i v e judges. Thus, although the findings yielded two variables on which the f i v e judges varied s i g n i f i c a n t l y , the disagreement did not stem from the r e -searcher. On the other 37 v a r i a b l e s , the f i v e judges agreed on t h e i r abstract codings. The primary purpose of the interjudge r e l i a b i l i t y s t r a t e -gies was to test the researcher's a b i l i t y to do the coding r e l i a b l y . It was concluded that the researcher's coding decisions were congruent with those of the other judges. V a l i d i t y Coding schedule v a l i d i t y was examined by considering the completeness of the l i s t of i n t e r n a l v a r i a b l e s r e l a t e d to acceptance/rejection. Two panels of judges c r i t i q u e d the research instrument. The f i r s t consisted of two professors and two doctoral students of adult education who p a r t i c i p a t e d i n the interjudge agreement study. Suggestions made by t h i s panel were i n -corporated i n the f i n a l draft of the instrument. The second panel con-s i s t e d of the 1981 four-member A.E.R.C. Steering: Committee. . J u s t p r i o r to the s e l e c t i o n of abstracts for the 1981 conference, each committee member reviewed a copy of the f i n a l coding schedule. Each member was asked to 67 i d e n t i f y any missing i n t e r n a l v a r i a b l e s which they considered would i n -fluence the judging process, and to c r i t i q u e each v a r i a b l e ' s coding cate-gories. No changes or additions were recommended by the second panel. 68 CHAPTER 5 PROCEDURES The following chapter describes the data c o l l e c t i o n procedures i n -volved i n t h i s study. P r i o r to describing the coding process i t i s necessary to discuss considerations that led to the adoption of coding procedures. The second part of t h i s chapter discusses the abstract coding process; the t h i r d concerns the s t a b i l i t y 'over time,, of the instrument. The f i n a l section discusses data analysis procedures. Preliminary Considerations The success of t h i s study rested on the need to avoid instrumentation bias (Campbell & Stanley, 1963) i n the abstract coding process. Considering the nature of the task i t appeared that the greatest threats to i n t e r n a l v a l i d i t y could stem from fatigue, boredom, and changes i n v a r i a b l e c r i t e r i a during the time of the coding process. These factors would be p a r t i c u l a r l y detrimental i f abstracts were coded on a year by year basis. Poorly written abstracts posed another threat i n that " i n v a l i d " codes might be assigned where abstracts were ambiguous. For example, i n a poorly written abstract, i t might be d i f f i c u l t to determine i f the abstract's t h e o r e t i c a l or con-ceptual basis was deductive or inductive. Another p o t e n t i a l problem was the p o s s i b i l i t y of bias associated with knowing that an abstract was accepted or re j e c t e d . It was also necessary to keep the data as close as possible to the coding categories. It was f e l t that coding error could be reduced by coding each abstract onto a schedule rather than d i r e c t l y onto an IBM form. In view of the above considerations, the following steps were follow-ed to code each abstract. 69 Coding 1. Abstracts were c o l l e c t e d from the 1978, 1979, and 1980 A.E.R.C. conferences. These years were studied because abstracts were known to be a v a i l a b l e . The decision was not based on anything other than t h i s p r a c t i c a l consideration. The t o t a l number of abstracts across the years was 329 (1978 n=77; 1979 n=126; 1980 n=126). Most abstracts were submitted on the standard form (see Appendix 2) issued with the A.E.R.C. C a l l for Papers. A few were submitted on regular typing paper. Abstracts contained no informa-t i o n concerning authorship or other external v a r i a b l e s . 2. Abstracts were c l a s s i f i e d as accepted/rejected. As described i n the previous chapter, abstracts "published" i n conference programmes or pro-ceedings were considered accepted; the remaining were c l a s s i f i e d as rejected. 3. A s e r i a l number and codes i n d i c a t i n g the "year" and whether the abstract was accepted or rejected were assigned to each abstract. 4. The s e r i a l number, codes, and the t i t l e of the abstract were con-cealed by f o l d i n g over the top portion of the page. 5. The abstracts were then shu f f l e d and mixed together across years to avoid coding on a year by year basis. As w e l l , during the coding pro-cess, abstracts were randomly selected from the p i l e to further ensure ran-domness. 6. The 329 abstracts were read and coded during a five-week period. Approximately f i f t e e n to twenty abstracts were processed d a i l y , although there was no attempt to code a set number each day. The minimum number of abstracts read i n a day was 6; the maximum was 26. 70 7. Each abstract was coded d i r e c t l y onto a coding schedule (see Appendix 1). 8. After each abstract had been coded, information pertaining to abstract number, acceptance, year the abstract was presented, and number of words i n the t i t l e was noted on the coding schedule. S t a b i l i t y Over Time During the coding process i t was necessary to take steps to measure the extent to which the procedures remained stable. The instrument's s t a b i l i t y across time was established as follows: 1. Five weeks a f t e r coding was completed, a random sample of 97 abstracts was recoded. A table of random numbers was used to sel e c t a pre-determined number of abstracts for each day of coding (approximately 25 percent, i . e . , i f f i f t e e n abstracts were coded on a s i n g l e day, four were randomly selected f o r recoding; i f eleven abstracts were coded then three were randomly selected f o r recoding). 2. Using a table of random numbers, abstracts were assigned to f i v e d i f f e r e n t coding days. Abstracts were recoded on f i v e consecutive days. Information concerning the t i t l e and acceptance were kept hidden. A f t e r coding, four abstracts were randomly selected and re-assigned to the follow-ing day. Thus, twenty abstracts were recoded three times. Table 4 shows s t a b i l i t y - a c r o s s - t i m e r e l i a b i l i t y c o e f f i c i e n t s (Pearson product-moment cor r e l a t i o n s ) f o r each v a r i a b l e . These were calculated using the SPSS program for Pearson product-moment c o r r e l a t i o n s . 71 Table 4 S t a b i l i t y Across Time Indices for Variables Related to Acceptance of A.E.R.C. Abstracts Variable Content Foundations Characteristics/Adult Learning Agency or Institution Programme Planning Instruction/Techniques Adult Education D i s c i p l i n e Theoretical/Conceptual A r c h i v a l / H i s t o r i c a l Empirical/Hard Data Admonitional/Prescriptive Methodological Process No. of direct c i t a t i o n s No. of authors cited Deductive Inductive State of the Research Cumulative Literature Novelty of Research Research Design Data Collection Instrumentation Instrument R e l i a b i l i t y Instrument V a l i d i t y Sample or Population Type of Analysis Results Conclusions Research Implications • Theoretical Implications Implications for the F i e l d Compositional No. of words i n t i t l e No. of words i n abstract Original Form Attachments Presentation Voice Jargon Funding Source Flow of the Argument T :-RT 2 n=97 - R T i .66 .75 .54 .78 .64 .69 .66 .71 .67 .54 .66 .84 .97 .29 .28 .63 .37 .42 .42 .73 .82 1.0 1.0 .80 .84 .95 .69 .59 .33 .52 .99 .99 .95 .81 .80 .71 .64 .62 .62 n=20 . 72 .85 .41' 1.0 .71 .82 .77 1.0 .78 1.0 .72 .96 1.0 .42* .58 .66 .64 .71 .71 .85 .97 1.0 1.0 .65 .79 .86 .62 .63 1.0 .46 1.0 .99 1.0 1.0 .82 .78 .88 1.0 .80 RT 2-RT 3 n=20 .72 .81 .72 .83 .86 .66 .72 1.0 .93 1.0 1.0 .96 1.0 .92 .88 .87 .82 .66 .54 .89 .96 1.0 1.0 .89 1.0 .91 1.0 1.0 1.0 .78 1.0 1.0 1.0 1.0 .89 .86 .89 .72 .78 • 0 5<P <-°7 (two-tailed test) Tj-RT 2 = Time One by Recode Time Two T 1 " R T 3 = T ime One by Recode Time Three RT 2-RT 3 - Recode Time Two by Recode Time Three A l l 117 c o r r e l a t i o n s i n Table 4 are s i g n i f i c a n t at the .05 l e v e l (one-t a i l e d t e s t ) . For a one-tailed t e s t , values greater than .16 i n column 1 ( i . e . , Time 1 by recode Time 2), .37 i n column 2 ( i . e . , Time 1 by recode Time 3) and .37 i n column 3 ( i . e . , recode Time 2 by recode Time 3) are s i g n i f i c a n t at the .05 l e v e l . For a two-tailed t e s t , 115 of the c o r r e l a -tions are s i g n i f i c a n t at the .05 l e v e l . The two remaining co r r e l a t i o n s (on agency or i n s t i t u t i o n and deductive i n the T^-RT^ column) were s i g n i -f i c a n t at the .07 but not the .05 l e v e l (.05 < p < .07). Having regard to the r e s u l t s of t h i s procedure, i t was concluded that across time, the instrument and coding process remained r e l a t i v e l y stable. Data Preparation and Analysis Following the coding of abstracts, the data were transcribed to key-punch forms. The data were then key-punched and v e r i f i e d (100 percent) by keypunch s t a f f at the University of B r i t i s h Columbia Computing Centre. The s t a t i s t i c a l analyses were completed i n two steps. F i r s t , the accepted and rejected abstracts were compared on each of the v a r i a b l e s separately for each year. The v a r i a b l e means of accepted and rejected abstracts were calculated by using the SPSS subprogram for t - t e s t s using a separate v a r i -ance estimate. Secondly, discriminant function analyses were employed to obtain combinations of variables which would d i s t i n g u i s h between accepted and rejected abstracts. Discriminant function analysis. Discriminant function analysis i s designed to predict group membership (accept/reject). The data consisted of " d i s c r i m i n a t i n g " (independent) variables which measured the c h a r a c t e r i s -t i c s on which the groups were expected to d i f f e r . Based on these v a r i a b l e s , 73 the discriminant function analysis would determine i f the groups d i f f e r e d and "weight and l i n e a r l y combine the discriminating v a r i a b l e s " to force groups to be as s t a t i s t i c a l l y d i s t i n c t as possible (Nie et a l . , 1975). Thus, va r i a b l e s were simultaneously analyzed to determine which configura-t i o n or combination best distinguished accepted from rejected abstracts. Based on previous t - t e s t a nalysis, those variables which distinguished between accepted and rejected abstracts were entered into the discriminant function equation for each year — 1978, 1979, and 1980. In preparation f o r entry into the 1978 discriminant function equation, the F-value to enter or e x i t from the equation was set at 2.77 which correspondend to a .01 l e v e l of s i g n i f i c a n c e . For 1979 and 1980 equations, the corresponding F-values were set at 2.75. As noted below, not a l l the variables that attained a s i g n i f i c a n t t-value met t h i s c r i t e r i a . Results a r i s i n g from the above data analysis procedures were derived from routines contained i n the S t a t i s t i c a l Package for the S o c i a l Sciences (SPSS). A l l analyses were done on the AMDAHL computer at the University of B r i t i s h Columbia Computing Centre. 74 CHAPTER 6 RESULTS This study was primarily designed to predict acceptance of abstracts submitted to the A.E.R.C. The development of an instrument to code A.E.R.C. abstracts, factors pertaining to coding, and procedures associated with the study were described i n the previous two chapters. This chapter i d e n t i f i e s v a r i a b l e s where the mean scores between accepted and rejected abstracts were s i g n i f i c a n t l y d i f f e r e n t , reports r e s u l t s showing the extent to which abstracts d i f f e r e d (during the three years encompassed by the study), describes c o r r e l a t i o n s between acceptance and the independent v a r i a b l e s , and f i n a l l y , presents r e s u l t s stemming from discriminant function equations used to predict acceptance i n 1978, 1979, and 1980. COMPARISON OF ACCEPTED AND REJECTED ABSTRACTS: BIVARIATE ANALYSIS Differences between accepted and rejected abstracts on each of the i n -t e r n a l v a r i a b l e s were examined separately for each of the three years. The r e s u l t s of the associated t - t e s t s are . presented i n Tables 5, 6, and 7 and are discussed below beginning with 1978. 1978 Table 5 contains the means and S.D.'s for accepted and rejected abstracts for 1978. As shown, accepted abstracts had s i g n i f i c a n t l y d i f f e r e n t (at the .05 l e v e l or greater) means on two content, f i v e process, and two compositional v a r i a b l e s . Table 5 75 Means and S.D.'s of Variables Associated with Accepted and Rejected A.E.R.C. Abstracts for 1978 Accept Accept Reject Reject t - 2 - t a i l Variable X S.D. X S.D. value prob. d.f Content Foundations 1.02 .45 1.06 .24 .76 53. 35 Characteristics/Adult Learning 1.37 .49 1.24 .43 -1.30 74. 09 Agency or Institution 1.07 .26 1.26 .45 2.26 .02 49. 87 Programme Planning 1.44 .50 1.47 .51 .25 70. 69 Instruction/Techniques 1.21 .41 1.15 .36 - .71 74. 22 Adult Education D i s c i p l i n e 1.14 .35 1.12 .33 - .28 72. 92 Theoretical/Conceptual 1.28 .25 1.24 .43 - .43 72. 49 Ar c h i v a l / H i s t o r i c a l 1.00 .0 1.06 .24 1.44 33. 00 Empirical/Hard Data 1.70 .47 1.56 .50 -1.24 68. 10 Admonitional/Prescriptive 1.02 .15 1.06 .24 .76 53. 35 Methodological 1.19 .39 1.00 .0 -3.10 .003 42. 00 Process No. of direct c i t a t i o n s .30 .56 .57 1.36 1.16 41. 90 No. of authors cited .37 .79 .59 1.08 .28 58. 60 Deductive 1.45 .77 1.56 .82 .51 68. 47 Inductive 2.16 .75 1.88 .95 -1.41 62. 14 State of the Research 4.42 1.69 3.62 1.76 -2.02 .05 69. 70 Cumulative Literature 3.16 .72 3.29 .52 .92 74. 52 Novelty of Research 1.47 .51 1.21 .41 -2.49 .01 74. 93 Research Design 1.86 .74 1.82 .63 - .24 74. 66 Data Collection 2.37 1.22 2.18 1.06 - .75 74. ,27 Instrumentation 2.16 1.31 1.56 .86 ^2.44 .01 72. ,78 Instrument R e l i a b i l i t y 1.21 .77 1.06 .34 -1.44 60. ,72 Instrument V a l i d i t y 1.14 .64 1.00 .0 -1.43 42. 00 • ••.Sample or Population 1.95 1.02 1.78 .95 - .71 73 .10 Type of Analysis 1.91 1.46 1.71 1.36 .62 72 .96 Results 2.67 1.39 2.21 1.34 -1.50 72 .04 Conclusions 1.53 1.08 1.65 1.07 .46 71 .17 Research Implications 1.47 .88 1.09 .38 -2.52 .01 59 .73 Theoretical Implications 1.05 .31 1.06 .34 .16 66 .68 Implications for the F i e l d 2.02 1.12 1.35 .65 -3.29 .002 69 .10 Compositional No. of words in the t i t l e 11.77 4.57 11.97 5.08 .18 67 .08 No. of words i n the abstract 290.19 133.84 260.47 80.16 -1.21 70 .34 Original Form 1.95 .21 1.85 .36 -1.44 50 .80 Attachments 1.02 .15 1.06 .24 .76 53 .35 Presentation 3.42 .73 3.12 .88 -1.60 63, .93 Voice 1.65 .48 1.38 .49 -2.40 .01 70, .22 Jargon 3.23 .84 3.06 .92 - .86 67, .78 Funding Source 1.05 .21 1.06 .24 .24 66, .82 Flow of the Argument 3.51 .74 2.91 .86 -3.22 .002 64, .85 The two content v a r i a b l e s upon which accepted and rejected abstracts d i f f e r e d s i g n i f i c a n t l y were methodological (t=-3.10, p < .003) and agency or i n s t i t u t i o n (t=2.26, p < .02). Accepted abstracts focused more on methodological research and were les s l i k e l y to concern agencies than were rejected abstracts. The f i r s t process v a r i a b l e upon which accepted and rejected abstracts d i f f e r e d was implications for the f i e l d (t=-3.29, p < .002). Thus, accepted abstracts were more l i k e l y to contain c l e a r l y i d e n t i f i e d statements concerning implications for the f i e l d than were rejected studies. Accepted abstracts were also more l i k e l y to include implications f o r further research (t=-2.52, p < .01) than were rejected studies. Other s i g n i f i c a n t process v a r i a b l e s were instrumentation (t=-2.44, p < .01), novelty of research (t=-2.49, p < .01), and state of the research (t=-2.02, p < .05). Accepted abstracts were more l i k e l y to c l e a r l y describe the instrumentation used, to break new ground or present new ideas, or to summarize studies i n advanced phases of the research process than were rejected studies. Two compositional v a r i a b l e s , voice (t=-2.40, p < .01) and flow of the argument (t=-3.22, p < .002), distinguished between accepted and rejected abstracts. Accepted abstracts contained a l o g i c a l argument and were l a r g e l y written i n an ac t i v e voice. Thus, i n 1978 accepted and rejected abstracts d i f f e r e d s i g n i f i c a n t l y on nine of the 39 v a r i a b l e s . It i s recognized that there i s no d e f i n i t i v e authority that s p e c i f i e s what should and should not be included i n an abstract. Moreover, i t i s also apparent that there are constraints associated with w r i t i n g a 250-word abstract. Nevertheless, i n view of the fac t adult education i s often deemed to be an emerging f i e l d of un i v e r s i t y study (Jensen et al.,1964), i t would be i n s t r u c t i v e to examine the extent to which A.E.R.C. abstracts (accepted and rejected) exemplify the q u a l i t i e s of "good" research. V a r i -able means reveal the extent to which authors i d e n t i f i e d the nature of th e i r population, instrumentation, data analysis, r e s u l t s , conclusions, and implications. In general, accepted abstracts ( i n 1978) had higher mean scores on the nine s i g n i f i c a n t v a r i a b l e s than did rejected work. However, on variables l a r g e l y coded on a four-point scale, even accepted abstracts scored low. These data suggest that even accepted work f a i l e d to describe c r u c i a l e l e -ments of the research process or content. Abstract C h a r a c t e r i s t i c s Content v a r i a b l e s . Content variables were coded dichotomously and a given abstract could be coded "yes" on more than one var i a b l e within each of the two content v a r i a b l e classes (adult education focus and methodologi-c a l o r i e n t a t i o n ) . Of the 77 abstracts submitted to A.E.R.C. i n 1978, 49 were em p i r i c a l l y oriented, 20 had a theoretical/conceptual focus, eight focused on methodological research, three were admonitional/prescriptive, and two had an a r c h i v a l / h i s t o r i c a l focus. Moreover, 35 abstracts focused on aspects of programme planning, 24 concerned c h a r a c t e r i s t i c s / a d u l t l e a r n -ing topics, 14 researched problems i n instruction/techniques, 12 focused on agency or i n s t i t u t i o n a l sponsorship, 10 concerned aspects of the d i s c i -p l i n e , and three focused on foundations and concepts of adult education. Thus, most abstracts submitted i n 1978 focused on the gathering and analysis 78 of data and concerned problems r e l a t e d to programme planning and/or the adult learner. There was a lack of work on basic concepts, foundations, or meta-research. Process v a r i a b l e s . Most process variables were coded on a four-point scale (l=Not i d e n t i f i e d , 2=Barely i d e n t i f i e d , 3=Partially i d e n t i f i e d , 4 = E x p l i c i t l y i d e n t i f i e d ) ; most accepted and rejected abstracts had low mean scores on these sc a l e s . For example, mean scores on instrumentation (which yielded a s i g n i f i c a n t d i f f e r e n c e between accepted and rejected abstracts) were low f o r both accepted (X=2.16) and rejected (X=1.56) abstracts. The t y p i c a l accepted abstract only barely i d e n t i f i e d the instrumentation used while the t y p i c a l rejected study did not describe the instrumentation. "Implications for the f i e l d " showed s i m i l a r r e s u l t s (accepted X=2.02; rejected X=1.35). Even though there were s i g n i f i c a n t differences between accepted and rejected abstracts, those accepted barely i d e n t i f i e d implica-tions for the f i e l d while rejected studies usually contained no implications statement. Other process variables such as data c o l l e c t i o n , instrument r e l i a b i l i t y and v a l i d i t y , r e s u l t s , and conclusions had low mean scores, i r r e s p e c t i v e of whether or not the abstract was accepted. An examination of the v a r i a b l e state of the research can o f f e r a possible explanation f o r these low mean scores. Accepted studies were more l i k e l y to be i n the " a n a l y t i c a l phase" (X=4.42, S.D.=1.69) while rejected abstracts were i n the "operational" stage of the research (X=3.62, S.D.=1.76). The v a r i a b l e cumulative l i t e r a t u r e i n d i c a t e s that both rejected (X=3.29, S.D.=.52) and accepted abstracts (X=3.16, S.D.=.72) were anchored i n the l i t e r a t u r e and both groups of abstracts tended to elaborate old ideas rather than present novel approaches (accepted X=1.47, S.D.=.51; rejected X=1.21, S.D.=.41). 79 Composition v a r i a b l e s . Most 1978 abstracts were well composed. The mean number of words for accepted (X=290.19) and rejected (X=260.47) abstracts, did not d i f f e r s i g n i f i c a n t l y . However, both means are higher than the 250-word maximum set by the A.E.R.C. Steering Committee. Ab-str a c t s were neatly presented (accepted X=3.42; S.D.=.73; rejected X=3.12, S.D.=.88) and used a minimum of dysfunctional jargon (accepted X=3.23, S.D.=.84; rejected X=3.06, S.D.=.92). Accepted abstracts were more l i k e l y to be written i n an a c t i v e voice (X=1.65, S.D.=.48) while rejected studies tended to be i n a passive voice (X=1.38, S.D.=.49). Accepted abstracts presented a l o g i c a l argument (X=3.51) while rejected studies were les s clear (X=2.91). As previously discussed, the l a t t e r two variables d i s -tinguished between accepted and rejected abstracts (p < .01). 1979 Table 6 summarizes the means, S.D.'s, and t - t e s t values f o r accepted and rejected abstracts for 1979. Accepted abstracts had s i g n i f i c a n t l y d i f f e r e n t means on three content and one process v a r i a b l e s . There were no s i g n i f i c a n t differences on compositional v a r i a b l e s . These r e s u l t s d i f f e r from those of 1978. The content variables which showed s i g n i f i c a n t differences between accepted and rejected abstracts were methodological (t=-2.13, p < .03), agency or i n s t i t u t i o n (t=2.05,p < .04), and programme planning (t=-1.98, p < .05). As i n 1978, accepted studies were more l i k e l y to focus on methodological research and les s l i k e l y to concern agencies than were r e -jected abstracts. In 1979 accepted work was also more l i k e l y to focus on programme planning than were rejected abstracts. 80 Table 6 Means and S.D.'s of Variables Associated with Accepted and Rejected A.E.R.C. Abstracts for 1979 Variable Accept X Accept S.D. Reject X. Reject S.D. t-value 2 - t a i l prob. d.f. Content Foundations 1.11 .31 1.05 .23 - .95 50.29 Characteristics/Adult Learning 1.22 .42 1.31 .46 1.04 70.83 Agency or Institution 1.11 .32 1.26 .43 2.05 .04 88.21 Programme Planning 1.58 .50 1.39 .49 -1.98 .05 63.40 Instruction/Techniques 1.11 .31 1.16 • .38 .84 75.35 Adult Education Dis c i p l i n e 1.08 .28 1.12 .32 .67 75.30 Theoretical/Conceptual 1.28 .45 1.32 .47 .49 66.58 Archiv a l / H i s t o r i c a l 1.03 .17 1.02 .15 - .17 58.35 Empirical/Hard Data 1.58 .50 1.56 .50 - .28 64.50 Admonitional/Prescriptive 1.02 .17 1.04 .21 .47 79.69 Methodological 1.19 .40 1.04 .21 -2.13 .03 42.67 Process No. of direct citations .72 1.60 .60 1.39 - .40 57.32 No. of authors cited .77 1.59 .62 1.43 - .51 58.85 Deductive 1.50 .76 1.43 .75 - .44 62.72 Inductive 2.11 .85 1.80 .72 -1.93 .059 56.05 State of the Research 3.66 1.83 3.62 1.77 - .12 62.30 Cumulative Literature 2.80 .75 3.12 .64 2.25 .02 54.85 Novelty of Research 1.44 .50 1.26 .44 -1.97 .054 57.35 Research Design 1.86 .68 1.67 .61 -1.48 59.22 Data Collection 2.25 1.18 2.12 1.06 - .56 59.70 Instrumentation 1.83 1.13 1.66 1.05 - .81 60.39 Instrument R e l i a b i l i t y 1.28 .81 1.04 .25 -1.69 37.79 Instrument V a l i d i t y Sample or Population Type of Analysis Results Conclusions Research Implications Theoretical Implications Implications for the F i e l d Compositional 1.17 1.97 1.83 1.94 1.61 1.33 1.08 1.72 .70 1.08 1.47 1.21 .99 .63 .73 .91 1.01 1.64 1.74 2.11 1.47 1.26 1.04 1.69 .10 .78 1.34 1.33 .96 .63 .20 .90 -1.33 -1.65 - .35 .67 - .69 - .53 - .60 - .19 35.64 50.35 60.03 70.41 62.73 66.18 44.14 64.14 No. of words in the t i t l e No. of words i n the abstract Original Form Attachments Presentation Voice Jargon Funding Source Flow of ythe Argument 10.94 259.0 1.94 1.05 3.33 1.53 3.14 1.06 3.61 4.45 50.82 .23 .23 .79 .51 .90 .23 .65 11.51 255.71 1.89 1.00 3.14 1.58 3.11 1.06 3.38 4.38 76.55 .32 1.00 .83 .50 .93 .23 .75 .65 - .28 -1.09 -1.43 -1.19 .50 - .16 .0 -1.91 63.46 96.18 87.20 35.00 67.23 63.43 66.52 64.05 74.95 81 The process v a r i a b l e upon which accepted and rejected abstracts d i f f e r e d s i g n i f i c a n t l y was cumulative l i t e r a t u r e (t=2.25, p < .02). In-ductive (£=-1.93, p < .059) and novelty of research (t=-1.97, p < .054) barely f a i l e d to a t t a i n s i g n i f i c a n c e at the .05 l e v e l . However, they are noted here and were subsequently entered i n the 1979 discriminant function equation because, when working c o n j o i n t l y with other v a r i a b l e s ( i n i n t e r -a c t i o n ) , they could have a more powerful e f f e c t . (The v a l i d i t y of t h i s reasoning was subsequently reinforced when inductive entered during the second step of the 1979 discriminant function equation — see p. 97). Thus, accepted abstracts had a more c l e a r l y defined inductive t h e o r e t i c a l development than rejected abstracts. As well, they were less anchored i n the l i t e r a t u r e than were rejected abstracts and more l i k e l y to present a novel approach than to elaborate on old ideas. Thus, of the 39 v a r i a b l e s , s i x s i g n i f i c a n t l y d i f f e r e n t i a t e d (at the .059 l e v e l ) between accepted and rejected abstracts. As f o r 1978, v a r i a b l e means of accepted abstracts were generally low. Abstract C h a r a c t e r i s t i c s Content v a r i a b l e s . Content variables focused on methodological orien t a t i o n s and adult education processes. Frequency counts f or 1979 show that of the 126 abstracts submitted, 71 had an empirical research focus. Thirty-nine abstracts had a theoretical/conceptual focus, 11 were oriented towards methodological research, f i v e had an admonitional or p r e s c r i p t i v e tone, and two were p r i m a r i l y a r c h i v a l / h i s t o r i c a l research. With regard to the adult education focus, a frequency count revealed that 56 abstracts focused on issues related to programme planning, while 36 described research concerning adult learners and learning. Twenty-seven abstracts focused on 82 agencies, 19 dealt with i n s t r u c t i o n , 14 concerned the d i s c i p l i n e , and nine were r e l a t e d to foundations. As i n 1978, most abstracts were emp i r i c a l l y oriented and concerned problems stemming from programme planning and the nature of the adult learner. Process v a r i a b l e s . Only three process variables s i g n i f i c a n t l y d i s -tinguished between accepted and rejected abstracts. Accepted abstracts (X=2.80, S.D.=.75) were only s l i g h t l y less anchored i n the l i t e r a t u r e than were rejected studies (X=3.12, S.D.=.64). Accepted abstracts were also more l i k e l y to present novel approaches to research (accepted X=1.44, S.D.=.50; rejected X=1.26, S.D.=.44). A frequency count f o r 1979 abstracts shows that of the 126 abstracts submitted, 39 were considered to break new ground while 87 were an elaboration of old ideas. Further examination re-veals low mean scores on most process v a r i a b l e s . With the exception of data c o l l e c t i o n (accepted X=2.25, S.D.=1.18, rejected X=2.12, S.D.=1.06) which shows that abstracts barely i d e n t i f i e d these procedures, other v a r i -able mean scores l a r g e l y f e l l within the not i d e n t i f i e d category. For example, instrumentation (accepted X=1.83, S.D.=1.13; rejected X=1.66, S.D.=1.05), sample or population (accepted X=1.97, S.D.=1.08, rejected X=1.64, S.D.=.78), and conclusions (accepted X=1.61, S.D.=.99; rejected X=1.47, S.D.=.96) were not i d e n t i f i e d i n the average abstract submitted i n 1979. As indicated by state of the research, accepted (X=3.66, S.D.=1.83) and rejected (X=3.62, S.D.=1.77) abstracts were p r i m a r i l y i n the operational phase of the research process. This could account for the incomplete des-c r i p t i o n of data a n a l y s i s , r e s u l t s , conclusions, or implications. However, var i a b l e s related to the operational phase such as data c o l l e c t i o n , i n s t r u -mentation, and sample or population were also incompletely described. 83 Compositional v a r i a b l e s . For 1979, no compositional v a r i a b l e s s i g n i -f i c a n t l y distinguished between accepted and rejected abstracts. Most abstracts were well composed. The t y p i c a l abstract (accepted X=259; r e -jected X=255) generally approximated the 250 words l i m i t set by the Steering Committee. As well, abstracts were neatly presented (accepted X=3.33, S.D.=.79; rejected X=3.11, S.D.=.93) and had a l o g i c a l flow to the argument (accepted X=3.61, S.D.=.65; rejected X=3.38, S.D.=.75). 1980 Table 7 shows accepted and rejected means and S.D.'s for 1980 abstracts. Accepted abstracts had s i g n i f i c a n t l y d i f f e r e n t means on three content, f i v e process, and one compositional v a r i a b l e . These r e s u l t s show a d i f f e r e n t pattern from those of the previous two years. The accepted and rejected means on three content v a r i a b l e s : foundations (t=-2.87, p < .006), instruction/techniques (t=2.4l, p < .03), and charac-t e r i s t i c s / a d u l t learning (t=-2.09, p < .04) were s i g n i f i c a n t l y d i f f e r e n t . For 1980, accepted abstracts were more l i k e l y to focus on research r e l a t e d to adult c h a r a c t e r i s t i c s , learning and foundations than rejected work. Re-jected abstracts were more l i k e l y to focus on problems of i n s t r u c t i o n than were accepted studies. The process v a r i a b l e , data c o l l e c t i o n , s i g n i f i c a n t l y distinguished between accepted and rejected abstracts (t=-3.19, p < .002). Accepted ab-s t r a c t s were more l i k e l y to report data c o l l e c t i o n procedures than were rejected studies. Other s i g n i f i c a n t process variables were type of analy-s i s (t=-2.55, p < .01), instrumentation (t=-2.23, p < .02), sample or population (t=-2.13, p < .03), and inductive (t=-2.03, p < .04). Thus, accepted abstracts were those which used higher-order data analysis (e.g., Table 7 Means and S.D.'s of Variables Associated with Accepted and Rejected A.E.R.C. Abstracts for 1980 84 Accept Accept Reject Reject t - 2 - t a i l V a r i a b l e , X S.D. X S.D. v a l u e p r ° b - d - f -Content Foundations 1.23 .43 1.04 .19 -2.87 .006 50. 56 Characteristics/Adult Learning 1.44 .50 1.25 .44 -2.09 .04 75. 49 Agency or I n s t i t u t i o n 1.16 .37 1.20 .41 .58 91. 58 Programme Planning 1.40 .50 1.42 .50 .28 85. 41 Instruction/Techniques 1.12 .32 1.27 .44 2.41 .03 110. 03 Adult Education Dis c i p l i n e 1.14 .35 1.17 .38 .43 90. 66 Theoretical/Conceptual 1.19 .39 1.30 .46 1.47 97. 72 A r c h i v a l / H i s t o r i c a l 1.09 .29 1.02 .15 -1.44 54. 28 Empirical/Hard Data 1.72 .45 1.60 .49 -1.35 91. 42 Admonitional/Prescriptive 1.05 .21 1.05 .21 .04 85. 94 Methodological 1.12 .32 1.03 .18 -1.50 56. 97 Process j No. of direct c i t a t i o n s .74 1.24 .56 1.20 - .83 83. 03 No. of authors cited 1.16 2.50 .59 1.56 -1.37 59. 50 Deductive 1.67 .97 1.48 .71 -1.15 65. 64 Inductive 2.11 .82 1.82 .68 -2.03 .04 72. 64 State of the Research 4.05 1.38 3.78 1.70 - .94 101. ,67 Cumulative Literature 3.23 .75 3.27 .68 .24 78. .28 Novelty of Research 1.33 .47 1.20 .41 -1.42 74, .47 Research Design 1.98 .56 1.84 .65 -1.20 97 .89 Data Collection 2.56 .96 1.84 .65 -3.19 .002 88 .29 Instrumentation 2.12 1.22 1.64 .97 -2.23 .02 70 .20 Instrument R e l i a b i l i t y 1.07 .34 1.07 . .38 . .04 93 .41 Instrument V a l i d i t y 1.23 .72 1.07 .35 -1.37 54 .18 Sample or Population 1.98 .74 1.67 .68 -2.13 .03 80 .79 Type of Analysis 2.30 1.68 1.57 1.21 -2.55 .01 65 .16 Results 2.40 1.28 2.17 1.38 - .92 91 .10 Conclusions 1.63 1.09 1.60 1.10 - .12 85 .92 Research Implications 1.26 .54 1.25 .54 - .03 84 .93 Theoretical Implications 1.12 .39 1.08 .38 - .44 84 .73 Implications for the F i e l d 1.72 .85 1.71 .94 - .06 92 .90 Compositional No. of words i n the t i t l e 11.05 3.63 11.94 5.02 1.14 110 .70 No. of words i n the abstract 294.42 83.58 269.82 80.67 -1.59 82 .49 Original Form 1.98 .15 1.90 .29 -1.83 124 .00 Attachments 1.0 .0 1.0 .0 .0 .0 Presentation 3.09 .90 3.18 .91 .52 86 .58 Voice 1.47 .51 1.54 .50 .81 84 .58 Jargon 2.70 1.17 3.24 .98 2.61 .01 73 .51 Funding Source 1.09 .29 1.06 .24 - .63 71 .50 Flow of the Argument 3.13 .77 3.20 .92 .42 98 .92 85 m u l t i v a r i a t e ) , c l e a r l y i d e n t i f i e d the instrumentation, sample/population, and had an inductive theory. The s i n g l e compositional v a r i a b l e which d i f f e r e n t i a t e d between accepted and rejected abstracts was jargon (t=2.61, p < .01). S u r p r i s i n g l y , accepted abstracts used s i g n i f i c a n t l y more dysfunctional jargon than did rejected studies. For 1980, only nine of the 39 variables s i g n i f i c a n t l y d i f f e r e n t i a t e d accepted from rejected abstracts. An examination of means and S.D.'s for a l l v a r i a b l e s shows a pattern s i m i l a r to the previous years; accepted and rejected abstracts generally have low mean scores. Abstract C h a r a c t e r i s t i c s Content v a r i a b l e s . The three content v a r i a b l e s with s i g n i f i c a n t l y d i f f e r e n t "accept" and " r e j e c t " means were rel a t e d to aspects of adult edu-cation. Of the 126 abstracts submitted, 40 focused on c h a r a c t e r i s t i c s of the adult learner, 52 concerned programme planning, 27 studied i n s t r u c t i o n , 24 were devoted to agencies, 20 with the d i s c i p l i n e , while 13 concerned foundations. As i n previous years, the primary methodological o r i e n t a t i o n of abstracts was towards empirical research. Eighty-one abstracts focused on the gathering and analysis of data, while 33 were t h e o r e t i c a l l y / c o n -ceptually oriented, eight focused on methodological research, s i x were admonitional or p r e s c r i p t i v e , and s i x reported a r c h i v a l or h i s t o r i c a l re-search. Thus, most abstracts submitted i n 1980 were empirical and r e l a t e d to programme planning, c h a r a c t e r i s t i c s of adult learners, and learning. i 86 Process v a r i a b l e s . An examination of process v a r i a b l e means showed that the t y p i c a l accepted (X=2.56, S.D.=.96) and rejected abstract (X=1.84, S.D.=.65) did not completely i d e n t i f y data c o l l e c t i o n procedures. The same r e s u l t applied to instrumentation (accepted X=2.12, S.D.=1.22; rejected X=1.64, S.D.=.97). Accepted abstracts mentioned the use of data analysis, though the type was usually unclear (accepted X=2.30, S.D.=1.68); rejected studies generally f a i l e d to describe analysis procedures. Both accepted (X=3.23, S.D.=.75) and rejected abstracts (X=3.27, S.D.=.68) were l a r g e l y anchored i n the l i t e r a t u r e and tended to elaborate o l d ideas. Only 31 of 126 abstracts submitted i n 1980 employed novel approaches to the research problem. Abstracts were neither c l e a r l y deductive (accepted X=1.67, S.D.=.97; rejected X=l.48, S.D.=.71) nor inductive (accepted X=2.11, S.D.=.82; rejected X=1.82, S.D.=.68). As i n previous years, v a r i a b l e s r e -lated to research processes were incompletely i d e n t i f i e d (e.g., sample or population accepted X=1.98, S.D.=.74; rejected X=1.67, S.D.=.68), even though most studies were i n the analysis (accepted X=4.05, S.D.=1.38) or operational phases (rejected X=3.78, S.D.=1.70). Compositional v a r i a b l e s . As noted, jargon was the compositional v a r i -able which s i g n i f i c a n t l y d i f f e r e n t i a t e d between accepted and rejected ab-s t r a c t s . But the d i r e c t i o n of the r e s u l t was s u r p r i s i n g . This v a r i a b l e was coded as l=Extensive, 2=Moderate, 3=Rare, 4=None. Accepted abstracts (X=2.70, S.D.=1.17) used moderate amounts of jargon, while rejected studies (X=3.24, S.D.=.98) used s i g n i f i c a n t l y l e s s (p < .05). On other composition-a l v a r i a b l e s , however, 1980 abstracts r e f l e c t e d normal patterns. Accepted and rejected abstracts were neatly presented (accepted X=3.09, S.D.=.90; rejected X=3.18, S.D.=.91) and had a moderately clear argument and l o g i c a l 87 flow (accepted X=3.13, S.D.=.77; rejected X=3.20, S.D.=.92). Note, how-ever, that although differences were minimal, rejected abstracts were coded s l i g h t l y higher than accepted studies on these v a r i a b l e s . With regard to the number of words, accepted (X=292.42) and rejected abstracts (X=269.82) both contained more than the suggested 250 words. Consistency Over Three Years Results presented above described variables upon which accepted and rejected abstracts had s i g n i f i c a n t l y d i f f e r e n t mean scores. It i s useful to examine the extent to which the independent variables c o n s i s t e n t l y d i f f e r -entiated between accepted and rejected across the three years. Table 8 l i s t s v a r i a b l e s that s i g n i f i c a n t l y d i f f e r e n t i a t e d between accepted and re-jected abstracts during at l e a s t one of the years. Only 19 of the 39 v a r i -ables appear i n the table. Of the 39, none d i f f e r e n t i a t e d between accepted and rejected abstracts i n a l l years of the study — 1978, 1979, and 1980. Of the 19 v a r i a b l e s , only f i v e s i g n i f i c a n t l y d i f f e r e n t i a t e d between accepted and rejected abstracts f o r more than one year. The two content v a r i a b l e s were methodological (1978, 1979) and agency or i n s t i t u t i o n (1978, 1979). Inductive (1979, 1980), novelty of research (1978, 1979), and instrumenta-t i o n (1978, 1980) were process v a r i a b l e s . A l l compositional and the remain-ing process and content variables d i f f e r e n t i a t e d between accepted and r e -jected abstracts i n only one of the three years. 88 Table 8 Extent to Which the Mean Content, Process, and Compositional Scores of Accepted and Rejected A.E.R.C. Abstracts were S i g n i f i c a n t l y D i f f e r e n t i n 1978, 1979, 1980* Year Variable 1978 1979 1980 1 Content Foundations Characteristics/Adult Learning Agency or I n s t i t u t i o n Programme Planning Instruction/Techniques Methodological Process Inductive State of the Research Cumulative L i t e r a t u r e Novelty of Research Data C o l l e c t i o n Instrumentation Sample or Population Type of Analysis Research Implications Implications for the F i e l d Compositional Voice Jargon Flow of the Argument *p<.059 The inconsistent effects of the variables suggests either that the characteristics of abstracts submitted i n different years vary greatly or judges regard the variables with varying degrees of importance. The extent to which differences between means of accepted and rejected abstracts stem from actual differences i n the characteristics of abstracts submitted each year was examined by calculating and comparing variable means for a l l abstracts. Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes 89 C h a r a c t e r i s t i c s of A l l Abstracts by Year The combined v a r i a b l e means for a l l abstracts submitted i n 1978, 1979, and 1980 are reported i n Table 9 together with F-values and associated p r o b a b i l i t y l e v e l s computed to test for differences among the years. It i s c l e a r that abstracts submitted during the three years of the study had e s s e n t i a l l y the same c h a r a c t e r i s t i c s . The one-way analyses of variance shown i n Table 9 produced only two s i g n i f i c a n t values f o r the variables cumulative l i t e r a t u r e and flow of the argument. Thus, the f a c t v a r i a b l e s l i k e methodological were associated with acceptance i n 1978 and 1979 but not 1980 apparently did not happen because of the d i f f e r e n c e i n the q u a l i t i e s of abstracts across the three years. It i s l i k e l y that judges assigned a d i f f e r e n t weight to t h i s and other v a r i -ables i n the years examined i n the present study. As noted i n Chapter 7, t h i s has implications for the A.E.R.C. judging process. COMPARISON OF ACCEPTED AND REJECTED ABSTRACTS: DISCRIMINANT FUNCTION ANALYSIS The previous discussion demonstrated that, for some v a r i a b l e s , the means of accepted and rejected abstracts d i f f e r e d from one another within each of the years encompassed by the study. But, as noted i n the previous section, the c h a r a c t e r i s t i c s of the abstracts (accepted and rejected) were la r g e l y the same across the three years. 9Q Table 9 Variable Means for a l l Abstracts Submitted to A.E.R.C. i n 1978, 1979, 1980 Variable 1978 X 1979 X 1980 X F. Sig. Content Foundations Characteristics/Adult Learning Agency or I n s t i t u t i o n Programme Planning Instruction/Techniques Adult Education D i s c i p l i n e Theoretical/Conceptual A r c h i v a l / H i s t o r i c a l Empirical/Hard Data Admonitional/Prescriptive Methodological Process No. of direct c i t a t i o n s No. of authors cited Deductive Inductive State of the Research Cumulative Literature Novelty of Research Research Design Data Collection Instrumentation Instrument R e l i a b i l i t y Instrument V a l i d i t y Sample or Population Type of Analysis Results Conclusions Research Implications • Theoretical Implications Implications for the F i e l d Compositional No. of words i n the t i t l e No. of words in the abstract Original Form Attachments Presentation Voice Jargon Funding Source Flow of the Argument 1.04 1.31 1.16 1.45 1.18 1.13 1.26 1.03 1.64 1.04 1.10 .43 .47 1.51 2.04 4.06 3.22 1.35 1.84 2.29 1.90 1.14 1.08 1.88 1.82 2.47 1.58 1.30 1.05 1.73 11.86 277.06 1.91 1.04 3.29 1.53 3.16 1.05 3.25 1.07 1.29 1.21 1.44 1.15 1.11 1.31 1.02 1.56 1.04 1.09 .63 .67 1.45 1.89 3.63 3.03 1.31 1.72 2.16 1.71 1.11 1.06 1.74 1.76 2.06 1.52 1.29 1.06 1.70 11.35 256.65 1.90 1.02 3.20 1.56 3.12 1.06 3.43 1.10 1.32 1.19 1.41 1.21 1.16 1.26 1.05 1.64 1.05 1.06 .62 .79 1.55 1.92 3.87 3.25 1.25 1.89 2.17 1.80 1.07 1.13 1.79 1.82 2.25 1.61 1.25 1.10 1.71 11.63 278.21 1.93 1.00 3.15 1.52 3.06 1.07 3.18 2.38 .12 .11 .17 .98 .63 .33 .67 .73 .06 .42 .77 1.27 .43 .18 1.03 4.51 .96 1.92 .04 .18 .35 .94 .16 .07 1.7 .25 .12 .65 .02 .50 1.87 .40 2.15 .40 .31 .45 .22 3.79 .09 .84 .37 .53 .71 .51 .47 .94 .65 .46 .28 .64 .82 .35 .01* .38 .14 .96 .83 .70 .39 .84 .92 .18 .77 .88 .52 .97 .60 .15 .67 .11 .60 .72 .63 .80 .02* P<.05 91 These preliminary analyses suggested that v a r i a b l e s associated with acceptance did not have a consistent e f f e c t over several years. To further c l a r i f y t h i s s i t u a t i o n , a discriminant function analysis was performed for each year. The following sections of t h i s chapter discuss the r e s u l t s of thi s analysis separately for 1978, 1979, and 1980. The c o r r e l a t i o n matrix among variables (for each year) i s discussed f i r s t , followed by the presenta-t i o n of r e s u l t s derived from the discriminant function equation. 1978 Corr e l a t i o n matrix. The c o r r e l a t i o n matrix for 1978 i s presented i n Appendix 3. As expected (given the r e s u l t s reported i n Table 5), nine of the 39 variables were s i g n i f i c a n t l y correlated with acceptance. Abstracts concerning methodologically oriented research were more l i k e l y to be accepted than other types (r=.30, p < .01); abstracts focusing on agencies or i n s t i t u t i o n s were more l i k e l y to be rejected than abstracts on other adult education topics (r=-.27, p < .05); research i n advanced states was more l i k e l y to be accepted than research i n preliminary states (r=.23, p < .05); novel research was more l i k e l y to be accepted than "old-hat" research (r=.27, p < .05); abstracts with c l e a r l y defined instrumentation were more l i k e l y to be accepted than those with l i t t l e or no d e s c r i p t i o n of instruments used (r=.26, p < .05); abstracts which contained statements con-cerning implications for research and/or the f i e l d of p r a c t i c e were more l i k e l y to be accepted than those which did not (r=.26, p < .05; r=.34, p < .01); abstracts written i n an a c t i v e rather than a passive voice were more l i k e l y to be accepted (r=.27, p < .05); and abstracts making an argument that flowed l o g i c a l l y were more l i k e l y to be accepted than i l l o g i c a l work (r=.35, p < .01). Of the 39 v a r i a b l e s , nine were s i g n i f i c a n t l y correlated with accep-tance. The strongest a s s o c i a t i o n was between flow of the argument and acceptance (r=.35). However, many variables were correlated with others i n the matrix. For example, state of the research was s i g n i f i c a n t l y cor-r e l a t e d (p < .05) with 11 other v a r i a b l e s . 4 The v a r i a b l e s with the greatest number of i n t e r - c o r r e l a t i o n s were state of the research (with 11 s i g n i f i c a n t c o r r e l a t i o n s ) ; empirical/hard data (with 10 s i g n i f i c a n t c o r r e l a t i o n s ) ; accept/reject (with nine s i g n i -f i c a n t c o r r e l a t i o n s ) ; theoretical/conceptual (with eight s i g n i f i c a n t cor-r e l a t i o n s ) ; and jargon and inductive (each with seven s i g n i f i c a n t c o r r e l a -tions) . Discriminant function a n a l y s i s . Table 5 shows that rejected and accepted abstracts d i f f e r e d s i g n i f i c a n t l y on two of the eleven content, f i v e of the 19 process, and two of the nine compositional v a r i a b l e s . These nine variables were: agency or i n s t i t u t i o n , methodological, state of the research, novelty of research, instrumentation, research implications, implications for the f i e l d , voice and flow of the argument. These were the v a r i a b l e s considered i n the discriminant function a n a l y s i s . As noted i n Table 10, only s i x emerged i n the equation r e s u l t i n g from the discriminant function a n a l y s i s . Thus, three of the v a r i a b l e s which yielded s i g n i f i c a n t t-values between accepted and rejected abstracts were not accepted. This i s a t t r i b u t e d to the fact that, a f t e r p a r t i a l l i n g out that proportion of This f i g u r e was determined by counting only c o r r e l a t i o n s on the underside of the matrix diagonal. Thus, a c o r r e l a t i o n , at .22 or above, between state of the research and sample or i t s r e c i p r o c a l , sample and state of the research was counted only once. This pro-cedure also excluded c o r r e l a t i o n s i n the diagonal. Their value was set at .22 (p < .05) for the 1978 c o r r e l a t i o n matrix while the r value for 1979 and 1980 matrices was set at .17 (p < .05). 93 t h e i r variances accounted for by the entered v a r i a b l e s , much of t h e i r i n -d i v i d u a l power was l o s t . Though there were s i g n i f i c a n t differences between accepted and rejected abstracts on state of the research, novelty of r e -search, and research implications, t h e i r i n d i v i d u a l differences were diminished when combined with other variables i n the discriminant function an a l y s i s . This i s supported by the v a r i a b l e i n t e r - c o r r e l a t i o n s discussed on page 90 and shown i n Appendix 3. State of the research was s i g n i f i c a n t l y correlated with 14 other v a r i a b l e s ; novelty of research was s i g n i f i c a n t l y correlated with three other v a r i a b l e s (one of which was agency or i n s t i t u -t i o n (r=-.24); and research implications were s i g n i f i c a n t l y correlated with two other v a r i a b l e s (one of which was flow of the argument (r=.26)). Table 10 l i s t s the variables i n t h e i r order of entry into the equation, the i n i t i a l F-value of the v a r i a b l e before entry into the equation, and the standardized discriminant function c o e f f i c i e n t r e s u l t i n g from the analysis. These c o e f f i c i e n t s are comparable to beta weights i n a regression equation and indicate the extent to which each v a r i a b l e has an e f f e c t (when "work-ing " with the other v a r i a b l e s ) . The larger the c o e f f i c i e n t , the more power-f u l the e f f e c t . The f i r s t v a r i a b l e to enter the equation was flow of the argument followed by voice; both concerned compositional elements of the abstract. Agency or i n s t i t u t i o n and methodological, which entered the equation at steps three and f i v e , r e s p e c t i v e l y , concerned abstract content. Two process v a r i -ables, implications for the f i e l d and instrumentation, entered at steps four and s i x . As indicated by the discriminant function c o e f f i c i e n t , voice (.53) was the most powerful v a r i a b l e when working together with the other f i v e . The strong negative c o e f f i c i e n t (-.44) for agency indicates that A.E.R.C. judges reacted unfavourably towards abstracts concerned with agencies. However, judges responded p o s i t i v e l y toward studies which focused on methodological research (.40). The va r i a b l e s flow of the argument (.38), implications for the f i e l d (.38), and instrumentation (.35) also strongly contributed to abstract acceptance. Thus, abstracts which presented a l o g i c a l argument, c l e a r l y described implications f o r the f i e l d of prac t i c e , or which i d e n t i f i e d research instrumentation were also favoured by the 1978 judges. TABLE 10 INTERACTIVE EFFECTS OF VARIABLES ASSOCIATED WITH ACCEPTANCE OF A.E.R.C. ABSTRACTS FOR 1978 Variable Step Entered Wilk's Lambda I n i t i a l Univariate F-value Standardized Discriminant Function C o e f f i c i e n t Flow of the Argument 1 .87 10.79 .38 Voice 2 .80 5.78 .53 Agency or I n s t i t u t i o n 3 .73 5.75 -.44 Implications for the F i e l d 4 .69 9.58 .38 Methodological 5 .65 7.57 .40 Instrumentation 6 .62 5.39 .35 Canonical C o r r e l a t i o n = .61 Based on the combined or i n t e r a c t i v e effects of the above v a r i a b l e s , i t i s possible to construct a b r i e f p r o f i l e of abstracts accepted f or the 1978 conference. 95 Accepted abstracts Rejected abstracts - were c l e a r l y and l o g i c a l l y written - were not c l e a r l y and l o g i -c a l l y written - were p r i m a r i l y written i n an ac t i v e voice - did not focus on agency or i n s t i -t u t i o n a l sponsorship - contained statements concerning implications for the f i e l d - were pr i m a r i l y written i n a passive voice - were focused on agency or i n s t i t u t i o n a l sponsorship - did not contain statements concerning implications for the f i e l d were oriented towards the use of a p a r t i c u l a r research methodology were not methodologically oriented had c l e a r l y i d e n t i f i e d i n s t r u -mentation did not have c l e a r l y i d e n t i -f i e d instrumentation As indicated by the canonical c o r r e l a t i o n (.61), the v a r i a b l e configu-r a t i o n described above accounted for approximately 60 percent of the v a r i -ance i n acceptance i n 1978. The discriminant function was able to c o r r e c t l y c l a s s i f y 81.8 percent of the studies submitted (Table 11). TABLE 11 PERCENTAGE OF 1978 A.E.R.C. ABSTRACTS CORRECTLY ASSIGNED TO ACCEPT AND REJECT GROUPS BY DISCRIMINANT FUNCTION ANALYSIS Group No. of Cases Predicted Group Membership Rej ect Accept Rej ect 34 30 4 88.2% 11.8% Accept 43 10 33 23.3% 76.7% Percent of "grouped" cases c o r r e c t l y c l a s s i f i e d : 81.8% 96 Of the 43 accepted abstracts, the equation c o r r e c t l y c l a s s i f i e d 33 (76.6 percent); 10 (23.3 percent) were i n c o r r e c t l y c l a s s i f i e d . That year, A.E.R.C. judges rejected 34 abstracts; the equation c o r r e c t l y assigned 30 (88.2 percent) to t h i s category, while four (11.8 percent) were i n c o r r e c t l y c l a s s i f i e d . 1979 Corr e l a t i o n matrix. The c o r r e l a t i o n matrix f o r 1979 i s presented i n Appendix 4. Nine of the 39 va r i a b l e s were s i g n i f i c a n t l y correlated (p < .05) with acceptance i n 1979. Abstracts which focused on programme planning concerns were more l i k e l y to be accepted than any other adult edu-cation topic (r=.18, p < .05); methodologically oriented research was more l i k e l y to be accepted than other types (r=.24, p < .01); inductive research was more i n c l i n e d to be accepted than research not thus characterized (r=.18, p < .05); abstracts which were les s well anchored i n the l i t e r a t u r e were more l i k e l y to be accepted than those well-anchored (r=.21, p < .05); abstracts presenting novel approaches to research were more l i k e l y to be accepted than those elaborating on o l d ideas (r=.18, p < .05); abstracts which described instrument r e l i a b i l i t y and/or v a l i d i t y procedures were more l i k e l y to be accepted than those which contained no such information (r=.21, p < .05); r=.18, p < .05); abstracts which c l e a r l y described the sample or population were more l i k e l y to be accepted than those which d i d not (r=.17, p < .05); and abstracts with attachments were more l i k e l y to be accepted than those on a s i n g l e sheet (r=.20, p < .05). As i n the previous year, many variables were s i g n i f i c a n t l y correlated with others i n the matrix (|r| > .17, p < .05). Variables with the great-est number of i n t e r - c o r r e l a t i o n s included: empirical/hard data (with 14 s i g n i f i c a n t c o r r e l a t i o n s ) ; jargon and theoretical/conceptual (with 10 s i g n i f i c a n t c o r r e l a t i o n s ) ; accept/reject and number of words i n the ab-st r a c t (with nine s i g n i f i c a n t c o r r e l a t i o n s ) ; and state of the research and instrumentation (with eight s i g n i f i c a n t c o r r e l a t i o n s ) . Twenty-eight were s i g n i f i c a n t l y correlated with between one and s i x v a r i a b l e s . Discriminant function analysis. As shown i n Table 6, four v a r i a b l e s : programme planning, agency or i n s t i t u t i o n , methodological, and cumulative l i t e r a t u r e d i f f e r e n t i a t e d between accepted and rejected abstracts. How-ever, (as noted on p. 81) i t was decided to also include the variables i n -ductive and novelty of research. Table 12 shows that f i v e of these v a r i -ables entered the discriminant function equation for 1979. TABLE 12 INTERACTIVE EFFECTS OF VARIABLES ASSOCIATED WITH ACCEPTANCE OF A.E.R.C. ABSTRACTS FOR 1979 Variable Step Entered Wilk's Lambda I n i t i a l Univariate F-value Standardized Discriminant Function C o e f f i c i e n t Methodological 1 .94 7.58 .50 Inductive 2 .90 4.29 .36 Cumulative L i t e r a t u r e 3 .87 6.00 -.54 Programme Planning 4 .84 4.00 .48 Agency or I n s t i t u t i o n 5 .82 3.22 -.38 Canonical C o r r e l a t i o n = .42 98 Methological, a content v a r i a b l e , entered the equation at step one. Two process v a r i a b l e s , inductive and cumulative l i t e r a t u r e followed at steps two and three. Programme planning and agency, which entered at steps four and f i v e , also focused on abstract content. No compositional variables appeared i n t h i s equation. Of these v a r i a b l e s , cumulative l i t e r a t u r e had the most powerful separate e f f e c t . The discriminant function c o e f f i c i e n t was negative (-.54) and indicated that judges were i n c l i n e d to r e j e c t rather than accept ab-s t r a c t s which were well anchored, i n the l i t e r a t u r e . "Agency or i n s t i t u t i o n " also had a negative e f f e c t (-.38). Thus, judges reacted unfavourably t o -wards abstracts focusing on agency or i n s t i t u t i o n a l sponsorship. The r e -maining variables — m e t h o d o l o g i c a l (.50), programme planning (.48), and i n -ductive (.36) -—were p o s i t i v e l y related to acceptance. Abstracts which focused on methodological research or aspects of programme planning and those employing inductive theory were more i n c l i n e d to be accepted than those which did not manifest these q u a l i t i e s . A discriminant function pro-f i l e of 1979 accepted and rejected abstracts shows the following: Accepted abstracts Rejected abstracts - were oriented towards the use of a - were not methodologically p a r t i c u l a r research methodology oriented - had a c l e a r l y defined inductive - did not have or had a vague t h e o r e t i c a l development inductive t h e o r e t i c a l development - had a l i t e r a t u r e base which was - had a l i t e r a t u r e base which "not at a l l " or " s l i g h t l y " was "moderately" or "ex-cumulative tremely" cumulative - focused on programme planning - did not focus on programme issues or topics planning - did not focus on agency or i n s t i - - focused on agency or i n s t i -t u t i o n a l sponsors of programmes t u t i o n a l sponsors 99 As indicated by the canonical c o r r e l a t i o n (.42), these f i v e v a r i a b l e s accounted f o r 42 percent of the variance. Based on th i s v a r i a b l e con-f i g u r a t i o n , the discriminant function equation was able to c o r r e c t l y c l a s s i f y 71.43 percent of the abstracts (Table 13). TABLE 13 PERCENTAGE OF 1979 A.E.R.C. ABSTRACTS CORRECTLY ASSIGNED TO ACCEPT AND REJECT GROUPS BY DISCRIMINANT FUNCTION ANALYSIS Group No. of Cases Predicted Group Membership Reject Accept Rej ect 90 68 22 75.6% 24.4% Accept 36 14 22 38.9% 61.6% Percent of "grouped" cases c o r r e c t l y c l a s s i f i e d : 71.4% In 1979, A.E.R.C. judges accepted 36 and rejected 90 abstracts. The d i s -criminant function equation c o r r e c t l y c l a s s i f i e d 68 (75.6 percent) and mis-c l a s s i f i e d 22 (24.4 percent) of rejected abstracts. It c o r r e c t l y c l a s s i -f i e d 22 (61.1 percent) and f a l s e l y c l a s s i f i e d 14 (38.9 percent) accepted abstracts. 1980 C o r r e l a t i o n matrix. The c o r r e l a t i o n matrix for 1980 appears i n Appen-dix 5. As i n the previous two years, nine variables were s i g n i f i c a n t l y cor-r e l a t e d with acceptance. Abstracts focusing on foundations and character-i s t i c s of adult learners or learning were more l i k e l y to be accepted than 100 those concerned with other adult education topics (r=.31, p < .01; r=.19, p < .05); research concerned with instruction or techniques was more l i k e l y to be rejected than accepted (r=-.17, p < .05); inductive research was more inclined to be accepted than research not thus charac-terized (r=.19, p < .05); i f data c o l l e c t i o n procedures were c l e a r l y described, abstracts were more l i k e l y to be accepted than i f procedures were not c l e a r l y specified (r=.27, p < .01); abstracts that described procedures pertaining to instrumentation (r=.21, p < .05), the nature of the sample or population (r=.19, p < .05), and the type of analysis (r=.25, p < .01) were more l i k e l y to be accepted than those which omitted t h i s information; abstracts which used dysfunctional jargon were more inclined to be accepted than those which used no dysfunctional jargon (r=-.24, p < .01). Many variables were s i g n i f i c a n t l y correlated with others i n the matrix (|r| > .17, p < .05). Variables with the greatest number of i n t e r -correlations included: empirical/hard data (with 12 s i g n i f i c a n t c o r r e l a -t i o n s ) ; theoretical/conceptual (with 10 s i g n i f i c a n t correlations); accept/ reject and state of the research (with nine s i g n i f i c a n t correlations); jargon, a r c h i v a l / h i s t o r i c a l , and data c o l l e c t i o n (with eight s i g n i f i c a n t correlations); and number of words i n the abstract, flow of the agrument and research design (with seven s i g n i f i c a n t correlations). 101 Discriminant function analysis. The pattern of va r i a b l e s associated with acceptance i n 1980 was d i f f e r e n t from the previous years (Table 14). As previously described, the means on nine v a r i a b l e s : foundations, charac-t e r i s t i c s / a d u l t learning, instruction/techniques, inductive, data c o l l e c -t i o n , instrument r e l i a b i l i t y , type of analysis, r e s u l t s , and jargon were s i g n i f i c a n t l y d i f f e r e n t when accepted and rejected abstracts were compared (Table 7). Of these v a r i a b l e s , only f i v e were retained i n the discriminant function equation. TABLE 14 INTERACTIVE EFFECTS OF VARIABLES ASSOCIATED WITH ACCEPTANCE OF A.E.R.C. ABSTRACTS FOR 1980 Variable Step Entered Wilk1' s Lambda I n i t i a l Univariate F-value Standardized Discriminant Function C o e f f i c i e n t Foundations 1 .91 12.82 .58 Cha r a c t e r i s t i c s / A d u l t Learning 2 .85 4.76 .35 Jargon 3 .80 7.61 -.60 Type of Analysis 4 .78 7.95 .38 Data C o l l e c t i o n 5 .74 9.88 .33 Canonical C o r r e l a t i o n = .51 Foundations and c h a r a c t e r i s t i c s / a d u l t learning, the f i r s t two va r i a b l e s to enter the equation, focused on abstract content. A compositional v a r i a b l e , jargon, entered at step three followed by type of analysis and data c o l l e c -t i o n which rel a t e d to research processes. 102 Examination of the discriminant function c o e f f i c i e n t shows that jargon (-.60) had the s i n g l e most powerful e f f e c t on acceptance. The e f f e c t was negative; accepted abstracts used unnecessary dysfunctional jargon. The v a r i a b l e with the second most powerful c o e f f i c i e n t was foundations (.58); thus, accepted abstracts were more i n c l i n e d to focus on issues re l a t e d to foundations of adult education. The other three v a r i a b l e s , c h a r a c t e r i s -t i c s / a d u l t learning (.35), type of analysis (.38), and data c o l l e c t i o n (.33) were also p o s i t i v e l y associated with acceptance. A discriminant function p r o f i l e of 1980 accepted and rejected abstracts shows the follow-ing: Accepted Abstracts - focused on foundations of adult education - focused on issues r e l a t e d to c h a r a c t e r i s t i c s and adult learning - used unnecessary or dysfunctional jargon - used higher order (e.g., m u l t i -v a r i a t e ) analysis - c l e a r l y i d e n t i f i e d data c o l l e c t i o n procedures Rejected Abstracts did not focus on foundations - did not focus on issues r e -lated to c h a r a c t e r i s t i c s and adult learning - used only minimal or no jargon used lower order (e.g., u n i -v a r i a t e , unclear) or no ana-l y s i s did not c l e a r l y i d e n t i f y data c o l l e c t i o n procedures As indicated by the canonical c o r r e l a t i o n (.51), these f i v e v a r i a b l e s accounted for 51 percent of the variance i n acceptance. Based on the v a r i -able configuration described above, the discriminant function equation was able to c o r r e c t l y c l a s s i f y 78.6 percent of the 1980 abstracts (Table 15). 103 TABLE 15 PERCENTAGE OF 1980 A.E.R.C. ABSTRACTS CORRECTLY ASSIGNED TO ACCEPT AND REJECT GROUPS BY DISCRIMINANT FUNCTION ANALYSIS Group No. of Cases Predicted Group Membership Reject Accept Reject 83 67 16 80.7% 19.3% Accept 43 11 32 25.6% 74.4% Percent of "grouped" cases c o r r e c t l y c l a s s i f i e d : 78.6% A.E.R.C. judges accepted 43 and rejected 83 abstracts. The discriminant function equation c o r r e c t l y c l a s s i f i e d 67 (80.7 percent) and m i s - c l a s s i f i e d 16 (19.3 percent) of rejected abstracts. It c o r r e c t l y c l a s s i f i e d 32 (74.4 percent) and f a l s e l y c l a s s i f i e d 11 (25.6 percent) of accepted abstracts. Summary As shown i n Tables 10, 12, and 14, d i f f e r e n t v a r i a b l e s combined i n th e i r a s s o c i a t i o n with acceptance. Thus, i n 1978 instrumentation and i m p l i -cations f o r the f i e l d were important but i n 1979 and 1980 they did not enter the equations. In 1979 compositional v a r i a b l e s were not associated with acceptance but i n 1978 and 1980 at least one of them entered the equation. These findings are summarized i n Table 16. In t h i s table a "yes" means the va r i a b l e s served to discriminate between accepted and rejected abstracts. 104 Table 16 Content, Process, and Compositional Variables Which Entered Discriminant Function Equations i n 1978, 1979, and 1980 Variable 1978 Year 1979 1980 Content Foundations Yes Characteristics/Adult Learning Yes Agency or I n s t i t u t i o n Yes Yes Programme Planning Yes Methodological Yes Yes Process Inductive Yes Cumulative L i t e r a t u r e Yes Data C o l l e c t i o n Yes Instrumentation Yes Type of Analysis Yes Implications for the F i e l d Yes Compositional Voice Yes Jargon Yes Flow of the Argument Yes Fourteen of the o r i g i n a l 39 variables were s u f f i c i e n t l y associated with acceptance to be entered into at least one of the discriminant function equations. Only two variables, methodological and agency or i n s t i t u t i o n appeared i n more than one equation. Thus, judges i n 1978 and 1979 favour-ably regarded abstracts focusing on methodological research and were more inclined to reject than accept studies concerning agencies. The remaining variables: inductive, cumulative l i t e r a t u r e , type of- analysis, data c o l l e c -t i o n , instrumentation, implications for the f i e l d , programme planning, foundations, characteristics/adult learning, jargon, voice, and flow of the argument s i g n i f i c a n t l y influenced the judging process i n only one year. Of 105 the above variables, s i x focused on research processes, f i v e concerned abstract content, and three were related to composition. These types of variables also had different effects across the years. For example, i n 1979 compositional variables did not enter the discriminant function equa-t i o n , while i n 1980 jargon had the most powerful ind i v i d u a l effect (-.60). In 1978 and 1980 process, content, and compositional variables manifested a similar association with acceptance. In 1979, however, content variables had the most influence. 106 CHAPTER 7 SUMMARY, CONCLUSIONS, IMPLICATIONS, AND LIMITATIONS P r i o r to the 1981 " C a l l f o r Papers", the only guidelines provided to researchers requested that abstracts should be approximately 250 words i n length. Given t h i s constraint, there i s a l i m i t to the amount of informa-ti o n that can be "squeezed" into an abstract. An abstract i s defined i n the Oxford Dictionary (Sykes, 1976) as -a -"summary". - A.s sucfi, it.should describe a l l the e s s e n t i a l elements of the research being reported. Research text-books l i k e Kerlinger (1973) l i s t sources from whence abstracts can be ob-tained but provide l i t t l e or no information concerning the q u a l i t i e s of a good abstract. Information storage and r e t r i e v a l systems l i k e E.R.I.C. employ abstracts (and abstract w r i t e r s ) . The Faculty of Graduate Studies at the University of B r i t i s h Columbia (and other u n i v e r s i t i e s ) requires thesis and d i s s e r t a t i o n committee chairpersons to sign an abstract to s i g -n i f y that i t f a i r l y represents the work reported. Most journals carry abstracts before t h e i r a r t i c l e s . S o c i o l o g i c a l Abstracts, Psychological  Abstracts, and other services p r i n t large compendiums of abstracts. A l l of the above r e i n f o r c e the fac t abstracts are c r u c i a l to the dissemination of s c i e n t i f i c information and must, therefore, parsimoniously describe key elements of the research being reported. An abstract i s a summary which usually consists of statements concerning the theory which supports or ari s e s from the research, a problem statement, a des c r i p t i o n of the metho-dology, instrumentation, a n a l y s i s , r e s u l t s , and conclusions. 107 SUMMARY The purposes of this study were twofold: to examine c h a r a c t e r i s t i c s of abstracts submitted for A.E.R.C.'s held i n 1978, 1979, 1980 and, to i d e n t i f y v a r i a b l e s associated with the acceptance/rejection of abstracts submitted to the A.E.R.C. i n each of the three years. Based on s o c i a l science l i t e r a t u r e focusing on variables associated with the acceptance/rejection of manuscripts submitted for p u b l i c a t i o n , a 41-item instrument was developed to assess the c h a r a c t e r i s t i c s of A.E.R.C. abstracts. As A.E.R.C. abstracts were judged b l i n d ( i . e . , authors are un-known to the judges), the study focused on variables i n t e r n a l to the manu-s c r i p t . The variables concerned the content (adult education focus and methodological orientation) of the research, the processes employed, and the composition of the abstract. To ensure that the instrument and coding system were r e l i a b l e and v a l i d , a series of p i l o t studies were ca r r i e d out. Two groups of expert judges attested to the content v a l i d i t y of the instrument. The f i r s t con-s i s t e d of two professors and two doctoral students of adult education; the second was the 1981 A.E.R.C. Steering Committee. Two aspects of interjudge r e l i a b i l i t y were considered. An interjudge r e l i a b i l i t y process showed the extent to which f i v e judges agreed among themselves while a researcher-judges procedure showed the extent to which the researcher agreed with the other judges. Based on the coding of nine abstracts, a repeated measures design analysis of variance (SPSS subprogram R e l i a b i l i t y ) showed that the judges made consistent coding decisions on 37 of the 39 v a r i a b l e s . This suggested that each of the "anchor" points i n scales used to quantify the variables were s u f f i c i e n t l y clear f or judges to make responses which 108 resembled those of t h e i r colleagues when the e n t i r e judging group worked alone. They made inconsistent judgements concerning the extent to which abstracts were admonitional/prescriptive (F = 4.29, p < .005) and inductive (F = 6.94, p < .001). An examination of each judge's codes on these v a r i -ables as well as the r e s u l t s of a researcher-judges r e l i a b i l i t y a n a lysis, showed that the u n r e l i a b i l i t y on these two v a r i a b l e s stemmed from the coding decisions of a judge, not the researcher. In other words, the researcher's codes more c l o s e l y resembled those of the three "conforming" judges than did those of a "non-conforming" judge. During a second r e l i a b i l i t y procedure, the codes of the researcher were compared, through analysis of variance, with the combined codes of the judges. On a l l but two v a r i a b l e s , the re-searcher's coding decisions were consistent with those of the judges. This instrument was used to code 329 (1978 n=77; 1979 n=126; 1980 n=126) accepted and rejected abstracts on 39 v a r i a b l e s . Information per-t a i n i n g to abstract number, acceptance, year the abstract was presented, and number of words i n the t i t l e was noted on the coding schedule a f t e r the abstract had been coded on the other v a r i a b l e s . Five weeks a f t e r coding was completed, a random sample of 97 abstracts were recoded while an a d d i t i o n a l 20 were recoded a t h i r d time. These steps were taken to ensure that the instrument and coding process was s t a b l e across time. Of 117 cor-r e l a t i o n s , 115 were associated at the .05 l e v e l of s i g n i f i c a n c e . The r e -maining two attained s t a t i s t i c a l s i g n i f i c a n c e (at the .07 l e v e l f o r a two-t a i l e d t e s t , or the .05 l e v e l f o r a one-tailed t e s t ) . It was concluded that across time, the instrument and coding process remained stable. 109 The data were subjected to both b l - and m u l t i v a r i a t e analyses. The f i r s t step involved an analysis of differences between the means of accepted and rejected abstracts on each of the i n t e r n a l v a r i a b l e s for each year. When using t - t e s t s , the item means of abstracts accepted and rejected i n 1978 d i f f e r e d s i g n i f i c a n t l y on two content, f i v e process, and two com-p o s i t i o n a l v a r i a b l e s : agency or i n s t i t u t i o n , methodological, implications for research and the f i e l d , instrumentation, novelty of research, state of the research, voice, and flow of the argument. In 1979, accepted abstracts had s i g n i f i c a n t l y d i f f e r e n t means on three content and three process v a r i -ables: agency or i n s t i t u t i o n , programme planning, methodological, inductive, novelty of research, and cumulative l i t e r a t u r e . The means of three content, f i v e process, and one compositional v a r i a b l e d i f f e r e d s i g n i f i c a n t l y f o r accepted and rejected abstracts i n 1980: foundations, i n s t r u c t i o n / t e c h -niques, c h a r a c t e r i s t i c s / a d u l t learning, data c o l l e c t i o n , type of analysis, instrumentation, sample or population, inductive, and jargon. For each of the three years, d i f f e r e n t v a r i a b l e s s i g n i f i c a n t l y d i f f e r -entiated between accepted and rejected abstracts. Of the f i v e which had s i g n i f i c a n t l y higher means for more than one year, two were content and three were process v a r i a b l e s . The inconsistent e f f e c t s of v a r i a b l e s suggested that abstracts submitted i n d i f f e r e n t years varied greatly or that A.E.R.C. Steering Committees regarded variables with varying degrees of importance. The f i r s t p o s s i b i l i t y was tested by c a l c u l a t i n g and comparing the v a r i a b l e means across accepted and rejected abstracts among the three years. The one-way analysis of variance produced only two s i g n i f i c a n t values — f o r the v a r i a b l e cumulative l i t e r a t u r e (F = 4.51, p < .01) and 110 and flow of the agrument (F = 3.79, p < .02) (Table 9). It i s possible, indeed probable, that the two (out of 39 possible) s i g n i f i c a n t F's resulted from Type I errors. The lack of s i g n i f i c a n t differences among the mean scores strongly suggests that abstracts submitted i n each of the three years were e s s e n t i a l l y the same. The v a r i a b l e s d i f f e r e n t i a t i n g accepted from rejected abstracts were entered into discriminant function equations for 1978, 1979, and 1980. P r o f i l e s for accepted abstracts d i f f e r e d by year. In 1978, accepted ab-s t r a c t s were pr i m a r i l y w r i t t e n i n an a c t i v e voice, presented a cle a r and l o g i c a l argument, were oriented towards use of a p a r t i c u l a r research metho-dology, had c l e a r l y i d e n t i f i e d instrumentation and implications for the f i e l d , and did not focus on agency sponsorship of adult education pro-grammes. In 1979, accepted abstracts were methodologically oriented, focused on programme planning issues but not agencies, had a c l e a r l y de-fined inductive t h e o r e t i c a l development, and were not well anchored i n the l i t e r a t u r e . The 1980 accepted p r o f i l e contained abstracts which focused on foundations of adult education or c h a r a c t e r i s t i c s of adults and learning, had c l e a r l y i d e n t i f i e d data c o l l e c t i o n procedures, used higher-order (e.g., multivariate) data analysis, and moderate amounts of pompous or dysfunctional jargon. Separate discriminant function equations for each year s u c c e s s f u l l y c l a s s i f i e d 81 percent of abstracts i n 1978, 71 percent i n 1979, and 78 per-cent i n 1980. Of great s i g n i f i c a n c e was the fact that, i n general, v a r i -ables associated with acceptance did not have the same, or even a s i m i l a r , e f f e c t i n each of the years studied. Judges appeared to weight variables d i f f e r e n t l y by year. I l l Several major conclusions were derived from the data analysis re-ported above. F i r s t , most abstracts (77 percent) were s u c c e s s f u l l y c l a s s i f i e d when using the i n t e r n a l v a r i a b l e s employed i n the study. Second, the extent to which variables were associated with acceptance varied from year to year. In t h i s regard, the 1978 equation only s l i g h t l y resembled the one for 1979 or 1980, and v i c e versa. A t h i r d conclusion was that, i n general, most A.E.R.C. abstracts (accepted and rejected) f a i l e d to include necessary information concerning content and research processes. These conclusions merit further discussion and give r i s e to implications that might be considered by organizers of future A.E.R.C.'s. CONCLUSIONS Acceptance of Abstracts With regard to the major purpose of the study, the most important con-c l u s i o n a r i s e s from the fact approximately 7 7 percent of a l l abstracts were c o r r e c t l y c l a s s i f i e d into the accept and r e j e c t groups. The b l i n d A.E.R.C. reviewing process has l e d ' to a s i t u a t i o n where abstracts are supposedly selected on the basi s of i n t e r n a l c h a r a c t e r i s t i c s . A s c i e n t i f i c report (or summary thereof) should contain c e r t a i n minimal information. It i s the presence of this information — concerning theory, instrumentation, data c o l l e c t i o n , a n a l y s i s , r e s u l t s , conclusions — that determines the acceptance of work for d i f f u s i o n through a conference l i k e the A.E.R.C. Yet the present study demonstrated that d i f f e r e n t variables had d i f f e r e n t e f f e c t s i n d i f f e r e n t years. Is t h i s a s a t i s f a c t o r y s i t u a t i o n and why does i t occur? One possible explanation to account for v a r i a b l e differences i n d i f f e r e n t years would be that the abstracts d i f f e r e d on a 112 year-to-year basis. Each year there i s a d i f f e r e n t crop of work; thus the i n f l u e n t i a l v a r i a b l e s would change. This explanation was rejected; a comparison of the v a r i a b l e means of a l l abstracts submitted i n each year revealed e s s e n t i a l l y no differences (Table 9). Thus, a more r e a l i s t i c i n t e r p r e t a t i o n of t h i s f i n d i n g concerned the judging process. Although i t has been shown that judges l a r g e l y attended to i n t e r n a l v a r i a b l e s i t appeared that the weight assigned to each v a r i a b l e varied from year to year. Judges "Weighting" Variables A further observation concerned the r e l a t i v e influence of the content, process, and compositional v a r i a b l e s over the three years. In 1978 and 1980 approximately two content, two process, and one ( i n 1980) to two ( i n 1978) compositional variables entered the discriminant function equations. But i n 1979, during the s e l e c t i o n of papers for the Ann Arbor conference, no compositional v a r i a b l e met the c r i t e r i a f o r entry into the equation. Despite t h i s anomaly, i t was concluded that process, content, and composi-t i o n a l v a r i a b l e s were a l l associated with acceptance. If future s e l e c t i o n behaviour i s to resemble that occurring i n the past, authors should attend to a l l three elements of the abstract. P r o f i l e s f o r accepted research i n each year — 1978, 1979, and 1980 — were d i f f e r e n t . As indicated by Table 16, variables for accepted and re-jected abstracts showed su b s t a n t i a l year by year differences. Two v a r i -ables, methodological, and agency or i n s t i t u t i o n , appeared i n two d i s c r i m -inant function equations (1978, 1979) and s u c c e s s f u l l y distinguished between accepted and rejected abstracts. The remaining twelve v a r i a b l e s appeared i n only one discriminant function equation.. No v a r i a b l e appeared i n a l l three 113 equations. The p r o f i l e of accepted abstracts varied from year to year. Furthermore, the r e l a t i v e contribution of each v a r i a b l e to the equation (as indicated by the discriminant function c o e f f i c i e n t ) varied as well. For example, i n 1979, methodological made a powerful contribution to ab-s t r a c t acceptance (discriminant function c o e f f i c i e n t .50); i t s influence i n 1978, however, was s l i g h t l y l e s s (.40). Voice also made a varying con-t r i b u t i o n to abstract acceptance. In 1978, voice made the most powerful contribution to abstract acceptance (.53), but i t had no influence i n 1979 or 1980. This f i n d i n g r a i s e s questions concerning the v a l i d i t y of the A.E.R.C. judging process. Is i t possible f o r A.E.R.C. judges to hold methodological research i n high esteem one year while i n the previous year i t was les s favourably regarded? Is i t possible that for one year, voice i s highly regarded, while i n following years i t s influence i s non-existent? For one year, A.E.R.C. judges regard voice favourably, yet i n the following year, two of the same plus two new judges did not react to i t s presence. Authors are i n the hands of judges whose preferences and knowledge of r e -search vary greatly. As noted below, t h i s has implications f o r the se l e c -tions of the A.E.R.C. Steering Committee. One explanation f o r d i f f e r e n t v a r i a b l e e f f e c t s i n the three years may stem from the possible influence of external variables when the f i r s t cut (in the s e l e c t i o n process) f a i l e d to y i e l d enough papers to f i l l the pro-gramme. Conversations with judges revealed that during the s e l e c t i o n process for 1981 only 22 abstracts were selected on the f i r s t round. A s i m i l a r f i g u r e was c i t e d f o r 1980, one of the years studied herein. Important v a r i -ables concerning analysis, design, and other research processes would have had consistent e f f e c t s when separating f i r s t round selections from those 114 "promoted" l a t e r . But when faced with a need to make up a programme :of about 40 papers, external variables such as "recognition of a colleague's work" or "judge's p r e d i s p o s i t i o n towards c e r t a i n types of research" could have influenced the s e l e c t i o n process. Although the A.E.R.C. i s now 22 years old i t i s s t i l l not possible to obtain 40 papers exemplary i n every respect. But as adult education research matures, and judges are con-fronted with better papers, i t i s l i k e l y that an e n t i r e programme: w i l l be obtained on the f i r s t cut. Authors wanting to have papers accepted would thus be well advised to keep the canons of s o c i a l science research i n mind and include a l l relevant d e t a i l . Summarizing the findings concerning acceptance, i t appears that i n the face of judges who are l i k e l y to assign an unknown weight to abstract v a r i a b l e s , future authors should s t r i v e to create a summary containing c l e a r l y i d e n t i f i e d descriptions of the c r u c i a l content and processes em-bodied i n the research. Because of the year-to-year d i f f e r e n c e s , i t was not possible to present a formula or l i s t of c h a r a c t e r i s t i c s that, i f em-bodied i n the abstract, would enhance the l i k e l i h o o d of acceptance. Thus, i n some ways, the present research has contributed l i t t l e except that i t i s now known that the judges have considerable influence. C h a r a c t e r i s t i c s of Abstracts As t h i s was the f i r s t study of any aspect of the A.E.R.C. i t was not possible to compare present with past findings. Moreover, as noted pre-v i o u s l y , there i s no absolute authority concerning abstracts which the present data can be measured against. Thus, the following conclusions mere-l y describe what appeared to be the nature of A.E.R.C. abstracts when viewed 115 i n the l i g h t of the widely accepted canons of s o c i a l science research. Conclusions concern the nature of the content, the processes reported, and the composition of the abstract. Content of abstracts. I t was concluded that most abstracts des-cribed research a r i s i n g from the f i e l d of p r a c t i c e with l i t t l e research devoted to d i s c i p l i n e b u i l d i n g . Content v a r i a b l e s were concerned with areas of adult education and the methodological o r i e n t a t i o n with which the abstract was concerned. Of the s i x va r i a b l e s r e l a t e d to areas of adult education, most abstracts (43 percent) focused on issues r e l a t e d to pro-gramme planning. The second most researched area concerned c h a r a c t e r i s t i c s of adult learners or learning processes (30 percent), followed by studies focusing on agency or i n s t i t u t i o n a l sponsorship of adult education pro-grammes (19 percent), and issues related to the design and management of i n s t r u c t i o n (19 percent). These categories were not mutually exclusive as abstracts could be coded "yes" on more than one primary focus. These re-s u l t s suggest that of adult educators submitting abstracts to the A.E.R.C, most were concerned with problems r e l a t e d to the i n i t i a t i o n and maintenance of educational a c t i v i t i e s . T y p i c a l studies were those on p a r t i c i p a t i o n and administration. Many researchers also focused on the sponsorship of adult education a c t i v i t i e s or the i n s t r u c t i o n of adults. These problems and issues arose primarily from the f i e l d of pra c t i c e as many adult educators were involved i n the organization and administration of programmes or i n the actual d e l i v e r y of i n s t r u c t i o n within an educational environment. Basic to these concerns was the concept of the "adult learner". Thus, many research-ers submitting abstracts to A.E.R.C. studied issues related to the charac-t e r i s t i c s of adult learners and aspects of t h e i r learning processes. 1 1 6 Few researchers submitted abstracts pertaining to the development of adult education as a d i s c i p l i n e . Few studies focused on meta-research and the t r a i n i n g of adult educators (eight percent), while even fewer reported research r e l a t e d to function, philosophies, or i n t e r n a t i o n a l per-spectives i n adult education (three percent). Of the f i v e methodological orie n t a t i o n s , most abstracts reported e m p i r i c a l l y oriented research (approximately 61 percent of submitted studies). The second most popular type was theoretical/conceptual (28 percent) followed by methodological (eight percent). Very few researchers, only three percent, submitted h i s t o r i c a l research. There are two possible reasons for the l a t t e r r e s u l t . F i r s t , the A.E.R.C. i s a North American based research conference and thus r e f l e c t s a c u l t u r a l t r a d i t i o n of empirically-oriented research. Adult education i n B r i t a i n i s heavily h i s t o r i c a l . Secondly, adult education i s often des-cribed as an applied d i s c i p l i n e (Jensen, 1964). Thus, much research i s devoted to solving immediate problems, p a r t i c u l a r l y those a r i s i n g from the f i e l d of p r a c t i c e . H i s t o r i c a l research does not have an immediate a p p l i c a -t i o n . In the pragmatic North American m i l i e u , i t receives minimal con-s i d e r a t i o n . In the t r a d i t i o n of an applied d i s c i p l i n e , adult educators have borrowed methodologies from other d i s c i p l i n e s and applied them to t h e i r research problems. There has been l i t t l e need for researchers to develop methodologies unique to adult education. Casual observation also suggests that, i n l i n e with t h i s t r a d i t i o n , most adult educators including those i n the professoriate, are concerned with meeting research needs expressed by the f i e l d . Much research i s f i e l d - o r i e n t e d . Few i n d i v i d u a l s do cumulative research or develop measurement techniques, methodological procedures, or data analysis strategies unique to adult education. 11(7 Research processes. Examination of the mean scores and frequencies of process v a r i a b l e s suggested that many abstracts submitted to the A.E.R.C. were defective. Abstracts tended to be longer than the suggested 250 words (accepted X=282; rejected X=262) but contained only a minimal or no d e s c r i p -t i o n of research processes followed by the researcher. For example, two key elements of the research process, instrumentation (accepted X=2.05; rejected X=1.63) and data c o l l e c t i o n (accepted X=2.36; rejected X=2.15) were barely i d e n t i f i e d , the second rank on a four-point scale. Most re-search seemed to be " i n progress", conceptually or procedurally, at the time researchers submitted abstracts. Thus, r e s u l t s were barely i d e n t i f i e d while conclusions and implications of the research were l a r g e l y ignored. It i s i n t e r e s t i n g to speculate about the extent to which completed ( i n con-t r a s t to " i n progress") studies were accepted on the f i r s t round. Un-fortunately, t h i s r e l a t i o n s h i p cannot be investigated i n the present study because the dependent v a r i a b l e (acceptance) makes no d i s t i n c t i o n between abstracts accepted i n the f i r s t , second, or subsequent rounds. Most studies tended to elaborate on previously conducted research and did not introduce new or novel approaches to the problem under i n v e s t i g a t i o n . Approximately 28 percent of the research submitted to the A.E.R.C. was con-sidered to break new ground while 72 percent was an elaboration of old ideas. Thus, most research problems were "established" i n the adult educa-t i o n l i t e r a t u r e (e.g., strategies for programme evaluation, l i f e - c y c l e or motivational o r i e n t a t i o n studies). The approach to these problems was t r a d i t i o n a l ; few explored new methodologies. Approximately 63 percent of the research conducted was ex post facto. Case studies, survey research, and one-group pretest-posttest designs (Campbell & Stanley, 1963) tended 1 1 8 to dominate. Few researchers manipulated treatments or implemented con-t r o l s f o r i n t e r n a l v a l i d i t y ; only eight percent of research was quasi-experimental or experimental. The paucity of information concerning empirical procedures could pose a p o t e n t i a l problem for the s e l e c t i o n committee. Discriminant function equations indicated that of 19 variables r e l a t e d to research processes, only s i x s i g n i f i c a n t l y d i f f e r e n t i a t e d between accepted and rejected ab-s t r a c t s (1978, 1979, and 1980). Accepted studies were those which outlined at l e a s t some empirical processes and included statements concerning the t h e o r e t i c a l o r i g i n s of the work, the data c o l l e c t i o n procedures, analysis, and instrumentation. Composition of abstracts. It was concluded that most abstracts were reasonably well composed. Compositional variables were concerned with the s t y l e and presentation of the abstract. Variables with the highest means included flow of the argument (accepted X=3.41; rejected X=3.22), presenta-ti o n (accepted X=3.28; rejected X=3.15), and jargon (accepted X=3.02; re-jected X=3.15). This suggested that most authors followed "good" composi-t i o n a l p r i n c i p l e s . Written i n either the a c t i v e (53 percent of a l l ab-strac t s ) or passive voice (47 percent), most were neatly laid-out and con-tained few typing errors. In most abstracts, the flow of the argument was moderately c l e a r while dysfunctional or pompous jargon r a r e l y appeared. The average length, however, was longer than the recommended 250 words. One anomalous r e s u l t involving a compositional v a r i a b l e concerned the strong a s s o c i a t i o n between jargon and acceptance i n 1980. Abstracts con-taini n g moderate (X=2.70) amounts of pompous or dysfunctional jargon were accepted whereas those where jargon was rare (X=3.24) were l a r g e l y rejected. 1 1 9 The discriminant function c o e f f i c i e n t of -.60 showed that t h i s was the most powerful v a r i a b l e associated with acceptance i n 1980. This was a curious r e s u l t which may have occurred for several reasons. It appears that when faced with jargon-ridden work that manifested other exemplary q u a l i t i e s judges were w i l l i n g to overlook jargon. Jargon was an i n t e r v a l v a r i a b l e that met the assumptions for Pearsonion c o r r e l a t i o n . The corre-l a t i o n matrix (Appendix 5) for 1980 gives clues which f a c i l i t a t e under-standing of the anomalous r e s u l t . Jargon was s i g n i f i c a n t l y correlated with: no. of words i n the abstract (r=-.20), theoretical/conceptual (r=-.28), empirical/hard data (r=.26), flow of the argument (r=.50), design (r=.26), sample or population (r=.19), state of the research (r=.27), admonitional/prescriptive (r=-.19), and i n s t r u c t i o n (r=.17). It appears that authors who used the greatest amounts of pompous or dysfunctional jargon also wrote abstracts which scored highest on flow of the argument (probably i n the moderately c l e a r or extremely clear categories). The c o r r e l a t i o n matrix suggests that jargon users were also those who reported empirical research, whose studies were i n advanced (e.g., r e s u l t s and con-clusions) stages at the time the abstract was written, and who tended to specify the nature of t h e i r research design and sample. Another possible explanation was that the coder used excessively i l l i b e r a l c r i t e r i a to : , judge whether jargon was pompous or dysfunctional. Although i t was possible that the coder became increasingly harsh as coding progressed t h i s explana-t i o n was discounted because during p i l o t t e s t i n g the coder's judgements concerning jargon were found to conform to those of the judges. The r e -searcher-judges r e l i a b i l i t y index (Table 3, p. 64) attested to t h i s f a c t . 120 Because of the numerous co r r e l a t i o n s between jargon and other variables i t was concluded that jargon per se was not a major a t t r i b u t e of accepted abstracts. Rather, i t appears that i f the abstract contained other d e s i r -able and needed information, moderate amounts of jargon, even that deemed to be pompous, was t o l e r a b l e . But note that i n 1978 and 1979 jargon did not even enter the discriminant function equations. DISCUSSION The data analysis and conclusions suggest that, as f a r as A.E.R.C. abstracts were concerned, i n t e r n a l variables were heavily associated with acceptance. However, i n 1978 19 percent, i n 1979 29 percent, and i n 1980 22 percent of abstracts were i n c o r r e c t l y c l a s s i f i e d . It i s i n t e r e s t i n g to speculate about v a r i a b l e s that, i f added to the present equations, would have resulted i n an even more accurate assignment to groups than was achieved here. Relative Impact The instrument was deemed to be content v a l i d . According to judges involved i n s e l e c t i n g abstracts for the 1981 conference, i t contained a l l the v a r i a b l e s known to be associated with acceptance, plus others. Why then was i t not possible to c o r r e c t l y c l a s s i f y a l l the abstracts? As judges made a f i r s t cut and then promoted f i r s t - r o u n d r e j e c t s into the accept p i l e , i t was possible that external variables had an influence. External v a r i a b l e s were not measured i n t h i s study and thus did not enter the discriminant function analyses. It would be reasonable to expect judges to recognize some of the work of colleagues even though abstracts did not contain names or other i d e n t i f y i n g marks. Was i t possible that when faced 121 with a need to promote r e j e c t s to f i l l out a programme judges hadr.to* employ external variables? After a l l , the i n t e r n a l c h a r a c t e r i s t i c s of the ab-s t r a c t s i n i t i a l l y caused them to r e j e c t the work. A more l i k e l y explanation for the f a i l u r e to c o r r e c t l y c l a s s i f y more than the 77 percent of the abstracts concerned an i n t e r n a l v a r i a b l e that was only p a r t l y measured i n this study. Conversation with former judges often revolved around the extent to which t h e i r " g u t - f e e l i n g " (or other colloquialisms) influenced acceptance. It appears that judges were some-times w i l l i n g to trade o f f desirable elements ( r e l i a b i l i t y , v a l i d i t y , and so on) i f the o v e r a l l impact of the abstract was appealing. For example, at the 1980 conference there was a paper concerned with Jean Paul Satre's novel "The F l i e s " (Knudson, 1980). The methodology, instrumentation, findings, and implications flowing from the study were either poorly des-cribed or not i d e n t i f i e d at a l l . Yet the abstract was accepted. The v a r i -able novelty was designed to measure t h i s elusive and somewhat tenuous c h a r a c t e r i s t i c of an abstract but, upon r e f l e c t i o n , had a narrower scope than the missing (gut-feeling) v a r i a b l e . It i s possible that during the f i r s t round of the s e l e c t i o n process t h i s g u t - f e e l i n g v a r i a b l e had l i t t l e influence; the conventional, usually empirically-oriented abstracts were accepted f i r s t . But on subsequent rounds, defective but i n t e r e s t i n g and unusual abstracts were a c t i v e l y considered and selected. The correct c l a s s i f i c a t i o n of approximately 77 percent of abstracts suggests that, i n general, abstracts are selected on the basis of i n t e r n a l v a r i a b l e s . Never-theless, there are implications for the future of the A.E.R.C. 122 POSSIBLE IMPLICATIONS FOR A.E.R.C. The need to modify some A.E.R.C. procedures has been noted above. Thus, most of the following suggestions are directed at future A.E.R.C. Steering Committees. The suggestions concern the c a l l for papers, the length of abstracts, the s e l e c t i o n process, and the possible development of guidelines. C a l l f or Papers U n t i l 1981, the only c r i t e r i o n noted was that abstracts should be no longer than 250 words i n length. Thus, authors were given no guidelines to a s s i s t t h e i r abstract w r i t i n g . Beginning with the.1981 conference, and continuing with 1982, the C a l l f o r Papers gave s p e c i f i c d e t a i l related to abstract content. Writers were asked to include information concerning objectives, perspectives or t h e o r e t i c a l frameworks, methods and/or tech-niques, data sources, r e s u l t s , conclusions, point of view, and the educa-t i o n a l or s c i e n t i f i c importance of the study. Given these guidelines, authors are i n a better p o s i t i o n to write acceptable abstracts i n terms of the provided framework. These c r i t e r i a were l a r g e l y adapted from those used by the A.E.R.A. (American Education Research Asso c i a t i o n ) . With more s p e c i f i c c r i t e r i a , however, abstract authors might be hard-pressed to stay within the 250-word guideline. One suggestion to benefit authors and judges would be to increase the recommended abstract length. Length of Abstracts The current suggested length i s 250 words, yet both accepted (X=282) and rejected (X=262) abstracts were longer. Longer abstracts would allow authors to more f u l l y o u t l i n e research processes and develop content. A v i a b l e length could be 500-600 words (the maximum length suggested by the 1 2 3 U n i v e r s i t y of B r i t i s h Columbia for d i s s e r t a t i o n a b s t r a c t s ) . A l t e r n a t i v e l y , a format s i m i l a r to that,of the A.E.R.A. could be considered. The 1981 A.E.R.A. Annual Meeting C a l l for Proposals stated that paper proposals must include a 2-3 page summary (single-spaced) as well as a 100-word nar r a t i v e abstract. Symposia proposals must include a s i m i l a r summary plus a 500-word abstract. Authors reporting well-designed and v a l i d studies would benefit from an increased word allowance; i t would allow them to more f u l l y describe the research problem, methodology, and r e s u l t s . However, authors summariz-ing questionable research would be le s s l i k e l y to benefit; a longer abstract would allow flaws to emerge. It i s l i k e l y judges would more e a s i l y d i f f e r e n -t i a t e between acceptable and inadequate research i f abstracts were longer. A second p o s s i b i l i t y i s to require authors to submit abstracts and completed papers. One development concerns the Graduate Student Research Award. A student entering t h i s competition must submit a completed paper (i n a form s u i t a b l e f o r i n c l u s i o n i n the Proceedings) as well as an abstract. One objection to submitting papers concerns the fa c t t h i s would place authors under an o b l i g a t i o n to complete work e a r l i e r ( i n the year) than at present. But i f students have to submit completed papers could not others do the same? Abstract Selection Process Committee members usually meet during the annual conference of the Adult Education Association (U.S.A.). Though judges are required to read and s e l e c t abstracts before the meeting, they are often hard-pressed to read a l l materials i n the time a v a i l a b l e . During the meeting, abstracts are discussed and judges must reach a decision concerning t h e i r acceptance. Between 40-45 abstracts must be chosen for the A.E.R.C. programme. Even i f 124 abstracts are read prior to the selection meeting the judges characteris-tically pool decisions verbally. Perhaps judges could use a rating scale and provide a written judgement to other judges? This study did not in-volve anything more than a casual inquiry into the group dynamics at work during the selection process. Perhaps the present informal system is superior. "But, in any event, i f the A.E.R.C. is to become a more serious scientific meeting, aspects of the selection process will probably need to be modified in the future. Abstract Guidelines Authors and judges would benefit from longer abstracts if both knew what to include in an abstract. As the results indicate, the quality of many A.E.R.C. abstracts is questionable. Many potential contributors cannot write a suitable abstract. As well, judges attribute varying im-portance to abstract variables. Perhaps the A.E.R.C. should sponsor a session at its annual conference which focuses on criteria for acceptable abstracts. The session would provide information on how to write a good abstract. The second suggestion is the publication of abstract guidelines. These would include information similar to that mentioned above. Of course, none of this guarantees that authors will do better research. Election of judges Another possible modification relates to the election of judges. At present, nominations are called for prior to and during the conference. It is usual for nominees to identify themselves just prior to the election. Otherwise, there is no "campaigning" and l i t t l e to identify the competencies (other than their reputation) of people seeking a position on the Steering Committee. The A.E.R.C. Steering Committee is an important gatekeeper of 125 adult education research. Their decisions can a f f e c t a person's career and influence the d i f f u s i o n of knowledge within the f i e l d . Although there i s l i t t l e time for campaigning at A.E.R.C, candidates could follow the example set i n other s c i e n t i f i c groups and professional associations and provide a b r i e f resume of q u a l i f i c a t i o n s that equip them to function as an A.E.R.C. judge. LIMITATIONS OF THE STUDY Limitations often r e l a t e to the r e l i a b i l i t y and v a l i d i t y of instruments. In t h i s study these a t t r i b u t e s of the instrumentation were c r u c i a l to the success of the study and thus explored i n a thorough fashion. However, there are other l i m i t a t i o n s which must be born i n mind when i n t e r p r e t i n g data r e -ported herein. 1. The study only encompassed three years. Since the San Antonio con-ference of 1978, Proceedings have been published. Match-up sessions were introduced i n 1980. In preparing for the 1981 conference the Steering Committee set new and stringent requirements for people presenting symposia. The A.E.R.C. i s changing so the findings reported herein could quickly become obsolete. Nevertheless, they provide a baseline which shows what conference abstracts were l i k e from 1978 to 1980. 2. With hindsight i t can now be seen that some of the coding c r i t e r i a could be improved. For example, with regard to the cumulative l i t e r a t u r e , i t was a mistake to include a l l s o c i a l science l i t e r a t u r e i n t h i s v a r i a b l e . One researcher, no matter how well read, cannot have s u f f i c i e n t knowledge to code a v a r i a b l e such as t h i s . It should have been r e s t r i c t e d to the "cumulativeness" of the adult education l i t e r a t u r e reported i n the abstract. 126 3. The unexpected negative influence of jargon i n 1980 may be re l a t e d to a l i m i t a t i o n concerning t h i s v a r i a b l e . It appears that jargon was used too broadly. Jargon should have been broken down into more precise v a r i -ables concerned with adult education, s o c i a l science, or empirical research jargon. Moreover, as coding progressed l i s t s of acceptable and unacceptable jargon should have been compiled. Although t h i s v a r i a b l e attained s a t i s f a c -tory r e l i a b i l i t i e s i t i s possible part of the 1980 r e s u l t was due to coding error. 4. It was not a l i m i t a t i o n to use acceptance as the dependent v a r i a b l e . However, having regard to the nature of the s e l e c t i o n process, i n p a r t i c u l a r the f i r s t cut and subsequent attempts to promote " r e j e c t s " into the "accept" p i l e , i t appears that the independent variables would have had even more explanatory power i f a d i f f e r e n t dependent v a r i a b l e was used. If i t had been possible to compare abstracts accepted i n the f i r s t cut with a l l other ab-s t r a c t s , more powerful equations might have resulted. 5. It was beyond the scope of t h i s study to examine completed A.E.R.C. papers. This study did not measure the q u a l i t y of abstracts or papers. A study concerned with q u a l i t y , and many other aspects of the work dif f u s e d through the A.E.R.C, remains to be done. 127 REFERENCE NOTES Copeland, H., & Long, H. Adult Education Research Conference — American  Research Association r e l a t i o n s h i p s : pro and con. Statement pre-pared at the request of the A.E.R.C. Steering Committee f or presenta-t i o n at the Adult Education Research Conference, Montreal, 1973. Copeland, H. Personal communication, November 3, 1980. Kidd, J.R. Letter to Dr. Lynn Davie, July 19, 1977. Fellenz, R. Memorandum to Roger Boshier, Ed. Simpson, and Frank Spikes. September 3, 1979. R o c k h i l l , K. Letter written to Dr. Lynn E. Davie. March 20, 1978. 128 REFERENCES Abramowitz, I., & Gomes, B. Publish or p o l i t i c : referee bias i n manu-s c r i p t review. Journal of Applied S o c i a l Science, 1975, _5, 187-200. American Educational Research Association. Adult education. Review of  Educational Research, 1950, 20, 161-249. Boshier, R.W. Factor analysts at large: a c r i t i c a l review of the moti-v a t i o n a l o r i e n t a t i o n l i t e r a t u r e . Adult Education, 1976, 27, 24-47. Boshier, R.W. Motivational orientations r e v i s i t e d : l i f e space motives and the Education P a r t i c i p a t i o n Scale. Adult Education, 1977, 27, 2, 89-115. Boshier, R.W. Program planning, motivation and i n s t r u c t i o n . Paper pre-sented to the Fed e r a l - P r o v i n c i a l Task Force on the Non-medical Use of Drugs, August, 1978. Boshier, R.W., & Pickard, L. C i t a t i o n patterns of a r t i c l e s published i n adult education, Adult Education, 1979, 30_> 34-51. Brunner, E. deS., Wilder, D.S., Kirchner, C., & Newberry, J.S. An over- view of adult education research. Chicago: Adult Education Asso-c i a t i o n of the U.S.A., 1959. Brunner, E. deS. Adult education and i t s research needs. Adult Education, 1960, 10, 218-227. Bryson, L. Adult education. New York: American Book Company, 1936. Campbell, D., & Stanley, J. Experimental and quasi-experimental designs  for research. Chicago: Rand McNally & Company, 1963. Chase, J . Normative c r i t e r i a for s c i e n t i f i c p u b l i c a t i o n . American Sociolo- g i s t , 1970,-5, 262-265. Clark, B. The marginality of adult education. Notes and Essays on Education for Adults, No. 20. Chicago: Centre for the Study of L i b e r a l Education for Adults, 1958. Coe, R.K., & Weinstock, I. E d i t o r i a l p o l i c i e s of major economic journals. Quarterly Review of Economics and Business, 1967, _7_, 34-43. Cotton, W.E. On behalf of adult education. Notes and Essays on Education for Adults, No. 56. Chicago: Center for the Study of L i b e r a l Education for Adults, 1968. Crane, D. The gatekeepers of science: some factors a f f e c t i n g the s e l e c t i o n of a r t i c l e s f o r s c i e n t i f i c journals. American S o c i o l o g i s t , 1967, 2_, 195-201. Dickinson, G., & Rusnell, D. A content analysis of adult education. Adult  Education, 1971, 22_, 177-185. Faure, E. (Chairman) Learning to be: the world of education today and  tomorrow. Paris: UNESCO and Harrap, 1972. Frantz, T.T. C r i t e r i a for publishable manuscripts. Personnel and Guidance  Journal, 1968, 47_, 384-386. G a r f i e l d , E. C i t a t i o n Indexing — i t s theory and a p p l i c a t i o n i n science, technology and humanities. New York: John Wiley & Sons, 1979. Gottfredson, S. Evaluating psychological research reports: dimensions, r e l i a b i l i t y and correlates of q u a l i t y judgments (with a reply by S. Scarr & B.L.R. Weber). American Psychologist, 1978, 33_, 920-935. H a l l , B.L. P a r t i c i p a t o r y research: an approach for change. Convergence, 1975, 8(2), 24-31. Hartung, A.B., & Latta, M.E. Professional p u b l i c a t i o n : the editor's side. The Clearinghouse, 1969, 104. Houle, CO. Ends and means i n adult education research. Adult Education, 1962, 12, 212-218. Houle, CO. The educators of adults. In R. Smith, G. Aker, & J.R. Kidd (Eds.), Handbook of adult education. New York: The Macmillan Company, 1970, 109-119. Jensen, G. How adult education borrows and reformulates knowledge of other d i s c i p l i n e s . In G. Jensen, A. L i v e r i g h t , & W. Hallenbeck (Eds.), Adult education: outlines of an emerging f i e l d of u n i v e r s i t y study., Washington, D.C: Adult Education Association of the U.S.A., 1964. Jensen, G., L i v e r i g h t , A.A., & Hallenbeck, W. (Eds.) Adult education: outlines of an emerging f i e l d of u n i v e r s i t y study. Washington, D.C: Adult Education A s s o c i a t i o n of the U.S.A., 1964. Kaplan, A. Research review. Research review. Research review. Research review. Research review. Research review. Adult Education, Adult Education. Adult Education. Adult Education, Adult Education. Adult Education, 1955a, _5, 114-127 1955b, 5, 240-246 1956, 6, 234-243. 1957, 7_, 196-207. 1958, 8_, 195-207. 1959, 9, 198-210. 130 Kerlinger, F.N. The influence of educational research on p r a c t i c e . The  Educational Researcher, 1977, 6_(8), 5-12. Kerlinger, F.N. Foundations of behavioural research (2nd ed.). New York: Holt, Rinehart and Winston, Inc., 1973. Kliebard, H.M. Structure of the d i s c i p l i n e s as an education slogan. Teachers College Record, 1965, 66, 598-603. Knox, A.B. Current research needs re l a t e d to systematic learning by adults. Occasional Paper No. 4. Urbana: Uni v e r s i t y of I l l i n o i s , O f f i c e for the Study of Continuing Ero;fess i o n a l Education, 1977. Knudson, R. The e x i s t e n t i a l e t h i c a l value of The F l i e s for adult education. Abstract published i n Program for the Twenty-first Annual Adult Educa- t i o n Research Conference. Vancouver, 1980, 44. Kreitlow, B.W. Educating the adult educator: Part 2. Taxonomy of needed research. Theoretical paper No. 13, Madison: University of Wisconsin, Research and Development Center, May, 1968. (ERIC Document Reproduction Service No. ED 023 031). Kreitlow, B.W. Federal support to adult education: Boon or boondoggle? Adult Education, 1975, 27_. 231-237. Lee, J. The contributions of graduate research to the body of knowledge i n  adult education. Paper presented to the Graduate Student Section of the Adult Education Association (U.S.A.) Conference, Boston, November 1979. Lindeman, E.C. The meaning of adult education. Montreal: Harvest House, 1961. (A r e p r i n t of the 1926 edition.) Lindsey, J.K. P a r t i c i p a t o r y research: some comments. Convergence, 1976, 9(3), 47-49. Long, H. P u b l i c a t i o n a c t i v i t y of selected professors of adult education. Adult Education, 1977, 17_, 173-186. Long, H., & Agyekum, S. Adult education 1964-73: r e f l e c t i o n s of a changing d i s c i p l i n e . Adult Education, 1974, 24, 99-120. Long, H., & Hiemstra, R. (Eds.) Changing approaches to studying adult  education. San Francisco: Jossey-Bass Publishers, 1980. McCartney, J.L. Manuscript reviewing. S o c i o l o g i c a l Quarterly, 1973a, 14(3), 290, 440-444. 131 McClusky, H.Y., Course of the adult l i f e s p a n . In I. Lorge, G. Jensen, & W. Hallenbeck, Psychology of adults. Washington, D.C: Adult Edu-cation Association of the U.S.A., 1963. McReynolds, P. R e l i a b i l i t y of ratings of research papers. American  Psychologist, 1971, 2_6, 400-401. Marx, M.H. Theories i n contemporary psychology. New York: The Macmillan Company, 1963. Mezirow, J. Towards a theory of pr a c t i c e . Adult Education, 1971, 21, 135-147. Mi n i s t r y of Reconstruction. F i n a l report of the adult education committee. London: His Majesty's Stationery O f f i c e , 1919. Monette, M.L. The concept of educational need: an analysis of selected l i t e r a t u r e . Adult Education, 1977, 27_, 116-127. Nie, N.H., H u l l , CH., Jenkins, J.G., Steinbrenner, K., & Bent, D.H. S t a t i s -t i c a l package f or the s o c i a l sciences (2nd ed.). New York: McGraw-H i l l Company, 1975. Pole, T. A h i s t o r y of the o r i g i n and progress of adult schools ... B r i s t o l : McDowell, 1816. Rodman, H., & Mancini, J. Edit o r s , manuscripts and equal treatment. Re- search i n Higher Education, 1977, 7_, 369-374. Schroeder, W. Adult education defined and described. In R. Smith, G. Aker, & J. Kidd (Eds.) Handbook of adult education. New York: The Macmillan Company, 1970, 25-43. Scott, W. Interreferee agreement on some c h a r a c t e r i s t i c s of manuscripts sub-mitted to the jour n a l of personality and s o c i a l psychology. American  Psychologist, 1974, 29, 698-702. Silverman, R., & C o l l i n s , E. Publishing r e l a t i o n s h i p s i n higher education. Research i n Higher Education, 1975, 3_, 365-382. Sork, T. Meta-research i n adult education: an h i s t o r i c a l analysis and c r i t i c a l a p p r a i s a l . Paper presented at the Adult Education Research Conference, University of B r i t i s h Columbia, May 1980. Sykes, J.B. (Ed.) The concise Oxford dict i o n a r y of current English (6th ed.). Oxford: Clarendon Press, 1976. Tough, A. Major learning e f f o r t s : recent research and future d i r e c t i o n s . Adult Education, 1978, 28, 260-263. 132 UNESCO. Adult education i n the context of l i f e l o n g learning. C0NFEDAD/5 Working Paper issued for the Third International Conference on Adult Education, 1972. Verner, C. The l i t e r a t u r e of adult education. In M. Knowles (Ed.) Handbook of adult education. Washington: Adult Education Associa-t i o n of the U.S.A., 1960, 162-175. Verner, C. A conceptual scheme f or the i d e n t i f i c a t i o n and c l a s s i f i c a t i o n  of processes. Washington, D.C: Adult Education Association of the U.S.A., 1962. Weber, J.A. Economics journals: p o l i c i e s , trends and problems. Southern  Economic Journal, 1972, 38, 559-565. Wolff, W. A study of c r i t e r i a f o r jou r n a l manuscripts. American Psycholo- g i s t , 1970, 25, 636-639. Zuckerman, H., & Merton, R. Patterns of evaluation i n science: i n s t i t u t i o n -a l i z a t i o n , structure and functions of the referee system. Minerva, 1971, 9_, 66-100. Appendix 1: A.E.R.C. Coding Schedule AERG ABSTRACT STUDY Column No. 1 - 5 6 Abstract Number Rejected/Accepted ' Year abstract was submitted/ Five Digits Reject - 1 Accept - 2 (78, 79, 80) 9-10 11 - 13 Number of words In the t i t l e Number of words in the abstract Abstract was presented on the original form Attachments were added to the abstract The presentation of the abstract vas (00 • (00 • 99) 999) No - 1 Yes - 2 No - 1 Yes - 2 !i°££y. (e.g. gross typing errors; crossing-out; bad Justification on typing) Not very neat (e.g. some but not gross errors In typing and layout) Moderately neat (e.g. no typo's but spacing etc. not perfect) 17 18 20 21 -26 Blank The abstract was written In Passive voice — transitive verbs attribute the verbal action to the person or object (e.g. It Is contended that; Chi-square analysis was performed; A questionnaire was completed by participants.) Active voice — the subject performs the action represented by the verb (e.g. The author contends that; The researcher performed a chi-square analysis; Participants completed a questionnaire). The use of jargon in this abstract was Jargon refers to "non-standard" jargon. Unnecessary jargon Is designed to inflate or dress-up the abstract. "Standard" Jargon which is O.K. includes: •factor analysis •motivational orientations •group dynamics *the adnlt's margin •norm-referenced evaluation •higher-order needs Very neat (e.g. 100X error free; impecable neatness and layout) Primarily passive voice - 1 Primarily active voice - 2 Extensive (e.g. extensive use of unnecessary Jargon to "dress-up" the abstract — Involves use of non-standard jargon) • 1 Moderate (e.g. two or three useages of "unnecessary" overly pompous jargon) • 2 Rare (e.g. one or bare minimum use of "unnecessary" Jargon) - 3 None (e.g. abstract i s cleanly written i n "plain" language using only "standard" adult education or social science jargon) • 4 (00 - 99) Citations directly cited in the abstract (enter the actual count) Refers to the^number of direct references made to studies, articles, Instru For example, "previous participation studies..." is not a direct citation, someone must be named. If an author is cited more than once for different 1978) this counts as two citations. Self-citations are also included. lents (e.g. Boshier's E.P.S.). For a citation to be direct, :ontributions (e.g. Tough, 1974; The number of different authors cited (enter the actual count) (00 - 99) Refers to the individual authors cited. If there are co-authors, each is counted individually (e.g Johnston & Riyera counts as two authors.) If an author is cited more than once (e.g. Verner, 1962; Verner & Booth, 1964) this counts as only one citation for Verner. An institution is an author (e.g. UNESCO, 1972.) The funding source for the research Is No - 1 revealed Y e s = 2 1 The primary f o c u s / f o c i of the a b s t r a c t was: No - 1 Yes - 2 No - 1 Yes - 2 No » 1 Yes - 2 No - 1 Yes - 2 No - 1 Yes - 2 Not d e d u c t i v e • 1 P o s s i b l y d e d u c t i v e • 2 P r o b a b l y d e d u c t i v e • 3 Uef±nit«l)r d e d u c t i v e - 4 Deductive — the author began w i t h a p r e v i o u s l y deducted theory o r c o n c e p t u a l framework. Data was c o l l e c t e d i n l i g h t of the t h e o r y . Theory precedes d a t a c o l l e c t i o n i n d e d u c t i v e r e s e a r c h . I f i t appears t h a t the type of data c o l l e c t e d i s i n a p p r o p r i a t e to the theory, the development i s s t i l l c o n s i d e r e d as d e d u c t i v e . Where data c o l l e c t i o n was " g u i d e d " by theory i r r e s p e c t i v e of whether the " t h e o r y " was from a d u l t e d u c a t i o n o r another d i s c i p l i n e , the development i s c o n s i d e r e d d e d u c t i v e . 33 The c o n c e p t u a l o r t h e o r e t i c a l development was Not i n d u c t i v e - 1 P o s s i b l y i n d u c t i v e • 2 P r o b a b l y i n d u c t i v e - 3 D e f i n i t e l y i n d u c t i v e • 4 I n d u c t i v e — the d a t a speaks f o r i t s e l f . Summary statements of e m p i r i c a l r e l a t i o n s h i p s a r e accumulated to form g e n e r a l e x p l a n a t o r y p r i n c i p l e s . Data i s c o l l e c t e d , a n a l y s e d and t h e o r e t i c a l statements made a f t e r the data "have spoken." I f the " t h e o r y " a r i s i n g from the data i s not i n d i g e n o u s to a d u l t e d u c a t i o n , i t i s s t i l l i n d u c t i v e . 34 The argument and l o g i c a l flow of the a b s t r a c t was Not a t a l l c l e a r = 1 Only s l i g h t l y c l e a r «• 2 M o d e r a t e l y c l e a r • 3 E x t r e m e l y c l e a r - 4 C l a r i t y o f the argument o r f l o w of the a b s t r a c t i s independent o f the substance o r o v e r a l l c o n t e n t . For example, the a b s t r a c t may be c l e a r l y and l o g i c a l l y w r i t t e n yet v o i d of any s u b s t a n t i v e i n f o r m a t i o n i . e . c l e a r but n a i v e . 35 Blank 36 The s t a t e of the r e s e a r c h i n t h i s study was i n a C o n c e p t u a l phase — e v i d e n c e t h a t the n a t u r e of the problem and v a r i a b l e s have been c o n c e p t u a l i z e d , but no e v i d e n c e t h a t data g a t h e r i n g o p e r a t i o n s have been performed. • 1 P l a n n i n g phase — p l a n s f o r Implementation of r e s e a r c h procedures a r e r e v e a l e d but no e v i d e n c e of a c t u a l implementation i s r e v e a l e d . - 2 O p e r a t i o n a l phase — r e s e a r c h e r has implemented pr o c e d u r e s and gathered d a t a . There i s no evidence of d a t a a n a l y s i s . • 3 A n a l y t i c a l phase — d a t a was gathered and a n a l y z e d but no e x p l i c i t r e s u l t s were r e v e a l e d . «= 4 R e s u l t s and C o n c l u s i o n s phase — e x p l i c i t r e s u l t s and c o n c l u s i o n s a r e d e s c r i b e d . Here i l l u s i o n t o, o r s t a t e -ment s a y i n g t h a t t h e r e a r e r e s u l t s and c o n c l u s i o n s i s Inadequate. A c t u a l r e s u l t s and c o n c l u s i o n s must be d e s c r i b e d . = 5 I m p l i c a t i o n s phase — e x p l i c i t i m p l i c a t i o n s f o r theory, f u t u r e r e s e a r c h or p r a c t i c e a r e d e s c r i b e d . Mere i l l u s i o n t o , o r statement s a y i n g t h a t t h e r e a r e i m p l i c a t i o n s i s i n adequate. A c t u a l i m p l i c a t i o n s must be d e s c r i b e d . « 6 T h e o r e t i c a l / C o n c e p t u a l — theory was the primary f o c u s of the s t u d y . A r c h i v a l / H i s t o r i c a l — i n v o l v e s i n v e s t i g a t i n g , r e c o r d i n g , a n a l y s i n g and i n t e r p r e t i n g events of the past f o r the purpose of d i s c o v e r i n g g e n e r a l -i z a t i o n s f o r u n d e r s t a n d i n g past and p r e s e n t . E m p i r i c a l / H a r d Data — the primary focus was the g a t h e r i n g and a n a l y s i s of d a t a e.g. "number c r u n c h i n g . " A d r o o n i t i o n a l / P r e s c r i p t i v e — the a b s t r a c t e x h o r t s t h a t a p a r t i c u l a r s t a n c e be adopted e.g. i n f a v o u r of grounded theory o r , the use of m u l t i v a r i a t e s t a t i s t i c s . The tone of the e n t i r e a b s t r a c t , not j u s t the c o n c l u s i o n s , must be p r e s c r i p t i v e . M e t h o d o l o g i c a l — where the i n t e n t of the r e s e a r c h was c l e a r l y to i n v e s t i g a t e a use of a p a r t i c u l a r methodology. The use of i n n o v a t i v e methodology as an i n c i d e n t a l a d j u n c t to a l a r g e r problem i s not a m e t h o d o l o g i c a l f o c u s . Methodology must be the c e n t r a l f o c u s . The c o n c e p t u a l or t h e o r e t i c a l development was The problem i n v e s t i g a t e d stems from l i t e r a t u r e / r e s e a r c h which i s For the d i s c i p l i n e o f a d u l t e d u c a t i o n , t h i s r e s e a r c h Blank The d e s i g n of the study was The methodology of the study was NoC a t a l l c u m u l a t i v e — problem does not appear to stem from any "known" body of r e s e a r c h or approach to the problem. - 1 S l i R h t l y c u m u l a t i v e — r e f e r e n c e to not more than one " a n t e c e d e n t " p i e c e of l i t e r a t u r e o r r e c o g n i z a b l e i d e a / m o d e l / t h e o r e t i c a l o r i e n t a t i o n i n - a d u l t e d u c a t i o n o r another d i s c i p l i n e - - 2 M o d e r a t e l y c u m u l a t i v e — r e f e r e n c e to at l e a s t two " a n t e c e d e n t s ' o r to a modest body of knowledge known to e x i s t i n a d u l t e d u c a t i o n o r elsewhere (e.g. problem stems from...meta-research l i t e r a t u r e ; l i t e r a t u r e on margin; l i t e r a t u r e on c o n t i n g e n c y management i n a d u l t e d u c a t i o n ) . - 3 Extremely c u m u l a t i v e — r e f e r e n c e to t h r e e o r more ' a n t e c e d e n t s " o r to a s u b s t a n t i a l body o f knowledge known to e x i s t i n a d u l t e d u c a t i o n and elsewhere (e.g. from p a r t i c i p a t i o n l i t e r a t u r e ; experiments on group dynamics; m o t i v a t i o n ) . » 4 I s an e l a b o r a t i o n of o l d i d e a s - 1 Breaks new ground o r p r e s e n t s new i d e a s " 2 Hot I d e n t i f i e d -r e s e a r c h d e s i g n . t h e r e i s no i n d i c a t i o n of a 1 Ex post f a c t o ( i n c l u d i n g h i s t o r i c a l ) — the r e s e a r c h e r , r a t h e r than c r e a t i n g the treatment, examines the e f f e c t s of a n a t u r a l l s t i c a l l y o c c u r r i n g treatment a f t e r the t r e a t m e n t has o c c u r r e d . Most a d u l t e d u c a t i o n s u r v e y s , content a n a l y s i s , and s u c h l i k e a r e ex post f a c t o . I f t h e r e Is no treatment b e i n g m a n i p u l a t e d , the study i s ex p o s t f a c t o . - 2 Ou a s1-exp er ime n ca1 — these d e s i g n s a r e not f u l l y t r u e e x p e r i m e n t a l d e s i g n s , they c o n t r o l f o r some but not a l l s o u r c e s of i n t e r n a l v a l i d i t y - - 3 E x p e r i m e n t a l — these d e s i g n s p r o v i d e f o r complete c o n t r o l o f a l l s o u r c e s of I n t e r n a l v a l i d i t y . - 4 Not i d e n t i f i e d — t h e r e I s no mention o f the metho-dology (e.g. Data was g a t h e r e d ) . - 1 B a r e l y i d e n t i f i e d — m e t h o d o l o g i c a l " h i n t s " but i s v e r y d i f f i c u l t to p i n p o i n t because of skimpy i n f o r -mation (e.g. 100 i n t e r v i e w s were conducted). » 2 P a r t i a l l y i d e n t i f i e d — i n c l u d e s some but not a l l i n f o r m a t i o n c o n c e r n i n g p r o c e d u r e s employed f o r d a t a g a t h e r i n g . - 3 E x p l i c i t l y i d e n t i f i e d — kay and p r o b a b l y a l l procedures a re c l e a r l y e x p l a i n e d . The a u t h o r has answered q u e s t i o n s l i k e : what v a r i a b l e s were mani-p u l a t e d ? how were the i n s t r u m e n t s a d m i n i s t e r e d ? e t c . The i n s t rumen t a t i o n used i n t h i s study was Not i d e n t i f i e d — no mention o f s p e c i f i c i n s t r u m e n t a t i o n - 1 B a r e l y i d e n t i f i e d — o n l y a "bare-bones" d e s c r i p t i o n i s g i v e n (e.g. A q u e s t i o n n a i r e c o n c e r n i n g a t t i t u d e s towards c o n t i n u i n g p r o f e s s i o n a l e d u c a t i o n was admin-i s t e r e d . ) • 2 P a r t i a l l y i d e n t i f i e d — some content o f . t h e ^ instrument i s d e s c r i b e d (e.g. The f i r s t q u e s t i o n asked*respondents to rank o r d e r statements c o n c e r n i n g the need f o r c o n t i n u i n g p r o f e s s i o n a l e d u c a t i o n . ) » 3 E x p l i c i t l y I d e n t i f i e d — c o n c i s e c o n t e n t o f a t l e a s t one i n s t r u m e n t used i s g i v e n . The name or a u t h o r o f an instrument (e.g. 16 P.F., EyBenck P e r s o n a l i t y I n v e n t o r y ) ar e s u f f i c i e n t i f the instrument i s known and has an e s t a b l i s h e d r e p u t a t i o n . Where the experimentor has used two o r more i n s t r u m e n t s o n l y one need be " e x p l i c i t l y " I d e n t i f i e d . - 4 136 Information p e r t a i n i n g to the r e l i a b i l i t y of the instrumentation i n t h i s study was Not i d e n t i f i e d — no information given at a l l . * 1 Barely i d e n t i f i e d — reference to the f a c t Instrument i s r e l i a b l e but no evidence of having tested i t s r e l i a b i l i t y i n the present study (e.g. ...an instrument with known r e l i a b i l i t i e s was employed...). * 2 P a r t i a l l y i d e n t i f i e d -instrument r e l i a b i l i t y oblique reference to the f a c t procedures were employed i n the present study, r e s u l t s are probably a v a i l a b l e but are not revealed i n the abstract (e.g. A s i x week t e s t re-tes r e l i a b i l i t y procedure was employed...) *= 3 Information p e r t a i n i n g to the v a l i d i t of the instrumentation In t h i s study E x p l i c i t l y i d e n t i f i e d — a c t u a l type of r e l i a b i l i t y t e s t s and/or r e s u l t s are revealed (e.g. A s i x week te s t r e - t e s t procedure showed that the instrument was r e l i a b l e r •= .67 p .05) - h Not i d e n t i f i e d no information given at a l l . 1 Barely i d e n t i f i e d — reference to the f a c t the i n s t r u -ment i s v a l i d but no evidence of having tested the v a l i d i t y In the present study (e.g. An Instrument with known v a l i d i t y was employed.) • 2 P a r t i a l l y i d e n t i f i e d — oblique reference the f a c t instrument v a l i d i t y procedures were employed In the present study, r e s u l t s are probably a v a i l a b l e but not revealed i n the abstract (e.g. Instrument v a l i d i t y was determined.) - 3 E x p l i c i t l y i d e n t i f i e d — act u a l type of v a l i d i t y pro-cedures and/or r e s u l t s are revealed (e.g. Content v a l i d i t y was determined by submitting the instrument to a panel of Judges.) m U The nature of the sample or population i n t h i s study was Some form of data a n a l y s i s was mentioned i n t h i s study "Highest" type of data a n a l y s i s (do not accept l i t e r a l "masking" statements — act l i k e a judge!) Not I d e n t i f i e d — no d e s c r i p t i o n i s given. Barely i d e n t i f i e d — only a given (e.g. t o t a l s i z e only "bare-bones" d e s c r i p t i o n — 100 adm i n i s t r a t o r s ) - 2 P a r t i a l l y i d e n t i f i e d — the t o t a l number plus two other pieces of Information concerning the S's (e.g. 100 women (18-35 years)) = 3 E x p l i c i t l y I d e n t i f i e d — the number and three or more a d d i t i o n a l pieces of Information concerning S's or selection/sampling procedures (e.g. 100 female Baptist high school teachers were randomly selected.) » 4 No - 1 ~ I—Ye s - 2 ^ A n a l y z e d data but "type" unclear U n i v a r i a t e — frequencies only B i v a r i a t e — chi-square a n a l y s i s , c o r r e l a t i o n s , One-way ANOVA, t - t e s t . 1Z1 The r e s u l t s of the study were M u l t i v a r i a t e — r e g r e s s i o n , f a c t o r a n a l y s i s , d i s c r i m i n a n t f u n c t i o n a n a l y s i s , AID 3 - 5 Not l d e n t l f l e d — no r e s u l t s were given - 1 Barely i d e n t i f i e d — i f the word r e s u l t ( s ) appears, code "barely" because the researcher acknowledges t h i s element e x i s t s (e.g. Results of the study w i l l be discussed.) "2 P a r t i a l l y l d e n t l f i e d — a general " r e s u l t " statement or only one r e s u l t appears (e.g. Results i n d i c a t e d female teachers have a more negative a t t i t u d e towards continuing education than male teachers.) - 3 E x p l i c i t l y i d e n t i f i e d — a d e f i n i t e statement of two or more r e s u l t s appears (e.g. Results i n d i c a t e d a more negative a t t i t u d e towards formal c o n t i n u i n g education by female teachers than male teachers. Female teachers however spend a greater amount of time on i n d i v i d u a l l e a r n i n g p r o j e c t s . ) - 4 137 The conclusions derived from t h i s study were The i m p l i c a t i o n s of the study f o r future research were The i m p l i c a t i o n s of the study f o r future t h e o r i z i n g were The i m p l i c a t i o n s of the study fo c h e f i e l d of p r a c t i c e of adult education were Hot l d e n t l f l e d — no conclusions were given. = 1 Barely i d e n t i f i e d — If the word conclusion(s) appears, code " b a r e l y " because the researcher acknowledges t h i s element e x i s t s (e.g. Conclusions w i l l be discussed.) = 2 P a r t i a l l y i d e n t i f i e d — a general statement of a conclusion appears (e.g. I t can be concluded that a conference i s a s u c c e s s f u l means of disseminating information.) = 3 E x p l i c i t l y i d e n t i f i e d — a d e f i n i t e statement of two of more conclusions appears (e.g. I t can be concluded that the Semantic D i f f e r e n t i a l i s both a r e l i a b l e and v a l i d measure of the a t t i t u d e change which conference p a r t i c i p a n t s underwent.) = 4 Wot i d e n t i f i e d — no i m p l i c a t i o n s were mentioned. e 1 Barely i d e n t i f i e d — the researcher " b a r e l y " acknow-ledges i m p l i c a t i o n s a r i s e from the study (e.g. I m p l i -c a t i o n s f o r future research w i l l be considered) but doesn't s t a t e what they are. = 2 P a r t i a l l y i d e n t i f i e d — a general statement of I m p l i -c a t i o n s f o r research appears (e.g. Further studies must be conducted to determine the extent of I n d i v i d u a l l e a r n i n g p r o j e c t s . ) At l e a s t one a c t u a l i m p l i c a t i o n i s noted. « 3 E x p l i c i t l y i d e n t i f i e d — a d e f i n i t e statement of at l e a s t two i m p l i c a t i o n s f o r research appears (e.g. Further s t u d i e s , u t i l i z i n g more p r e c i s e c r i t e r i a than those i n t h i s study, must be conducted to determine the extent of I n d i v i d u a l l e a r n i n g p r o j e c t s . As w e l l , l e a r n e r s must...) = 4 Hot i d e n t i f i e d — no I m p l i c a t i o n s were mentioned. = 1 Barely i d e n t i f i e d — the researcher "barely" acknow-ledges i m p l i c a t i o n s a r i s e from the study (e.g. Implica-t i o n s for f u t u r e t h e o r i z i n g w i l l be discussed) but doesn't discu s s any. " 2 Partla11y l d e n t l f l e d -— a general statement of at l e a s t one i m p l i c a t i o n f o r t h e o r i z i n g appears (e.g. Force f i e l d a n a l y s i s w i l l be a v a l u a b l e t o o l i n understanding adult p a r t i c i p a t i o n - ) = 3 E x p l i c i t l y i d e n t i f i e d — a d e f i n i t e statement of at l e a s t two i m p l i c a t i o n s f o r t h e o r i z i n g appears (e.g. Force f i e l d a n a l y s i s a p p l i e d to a d u l t p a r t i c i p a t i o n suggests the need to re-evaluate t h i s concept. Furthermore...) = 4 Not i d e n t i f i e d — no i m p l i c a t i o n s were mentioned. » l Barely i d e n t i f i e d — the researcher " b a r e l y " acknow-ledges i m p l i c a t i o n s a r i s e from the study (e.g. Implica-t i o n s for the p r a c t i c e of a d u l t education w i l l be discussed) but doesn't a c t u a l l y s t a t e any. = 2 P a r t i a l l y i d e n t i f i e d — a general statement of at l e a s t one i m p l i c a t i o n - f o r the f i e l d of p r a c t i c e appears (e.g. This study i n d i c a t e s the need to develop a futures o r i e n t a t i o n i n a d u l t education.) - 3 E x p l i c i t l y i d e n t i f i e d — a d e f i n i t e statement of at l e a s t two I m p l i c a t i o n s f o r the f i e l d of p r a c t i c e appears (e.g. This study i n d i c a t e s the need f o r programme planners to g i v e greater c o n s i d e r a t i o n to macro-level (e.g. community, s o c i e t a l and g l o b a l ) needs-da t a , and develop a ...) - 4 • • • • 138 The "primary" area(s) of research discussed i n thi<= abstract i s / a r e : FOUNDATIONS OF ADULT EDUCATION No - 1 I 1 Y e s " 2 | I e.g. Functions of adult education Philosophy I n t e r n a t i o n a l perspectives L i f e l o n g education P u b l i c p o l i c y Basic concepts CHARACTERISTICS OF ADULT LEARNERS No - 1 ( 1 AND LEARNING Yes " 2 e.g. L i f e c y c l e development P h y s i o l o g i c a l / p s y c h o l o g i c a l determinants of behaviour Adult l e a r n i n g Motivation AGENCY OR INSTITUTIONAL SPONSORS No - 1 i 1 Yes - 2 I • PROGRAM PLANNING, PARTICIPATION, No - 1 ADMINISTRATION AND METHODS Yes •= 2 e.g. I n d i v i d u a l , group, community methods Needs and needs a n a l y s i s Program goals Budgetting, marketing of programs Program evaluation P a r t i c i p a t i o n , drop-out, perslstance DESIGN AND MANAGEMENT OF INSTRUCTION: No - 1 I I TECHNIQUES AND DEVICES Yes = 2 I | e.g. Objective s e t t i n g A n a l y s i s i n t o l e a r n i n g tasks and techniques Techniques and devices Evaluation of l e a r n i n g Evaluation of I n s t r u c t i o n ADULT EDUCATION AS A DISCIPLINE AND No - 1 I I FIELD OF STUDY Yes •= 2 \ | e.g. Meta-research Dissemination of knowledge about adult education Tr a i n i n g adult educators Appendix 2": A.E.R.C. Abstract Form 13.9 ; Paper T i t l e : Program Abstract (about 250 words) e l i t e type pi c a type REJACCEP WORDSTIT WORDSABS ORIGINAL REJACCEP 1 .00000 -0 .02129 0 .13072 0 . 17368 YEAR 99 .00000 99 00000 99 .00000 99 .00000 ,WORDSTIT -0 .02129 1 00000 0 . 10860 0 .23831 WORDSABS 0 .13072 0 10860 1 00000 0 .09983 ORIGINAL 0 . 17368 0 23831 0 .09983 1 .00000 ATTACH -0 .09128 -0 12139 0 24603 0 .06367 PRESENT 0 .18599 -0 05408 -0 06435 -0 05622 VOICE 0 .26753 -0 01 177 0 02060 -0 02469 :JARGON 0 .09929 0 06533 -0 23923 -0 04727 DIRECTCI -0 . 14399 -0 07861 0 21853 0 09 163 ,AUTHORCI -o .11667 -0 01447 0 20805 0 1 1 160 iFUNDING -0 02755 -0 06703 -0 00065 0 07402 iTHEOCONC 0 04957 0 14284 0 24523 0 08429 ARCHHIST -0 18364 0 05661 -0 03481 0 05164 1EMPIHARD 0 14333 -0 13102 -0 16530 -0 05122 ADMONPRE -o 09128 -o 06474 -0 04233 0 06367 METHO 0 30278 0 03721 0 04957 0 10768 ',DEDUCT -0 05940 -0 12394 0 05088 0 03146 'INDUCT 0 16495 0 21237 0 06279 0 17512 :ARGUFLOW 0 35468 0 08064 0 08230 0 14670 ' STATERES 0 12783 -0 1731 1 -0 12866 -0 01411 ' CUMULIT -0 10237 0 09218 0 00558 -0 03223 DISCIPRE 0 26977 -0 09845 0 17920 -0 05164 DESIGN 0 02678 -0 02286 -o 05664 0 05995 IDATACOLL 0 08536 -0 12728 0 01775 0 03970 INSTRU 0 25910 -0 08320 0 01 158 -0 14547 RELIAB 0 12087 -0 03293 -0 01930 0 07307 ' VALID 0. 14521 i -0 1 1568 -o 04205 0 05164 | SAMPLE o. 08076 -0 17417 0 00793 -0 2221 1 TYPEANAL 0. 07120 0 06839 0 00779 -o 16984 i RESULTS 0. 16945 -0 07356 -0 07569 -0 02393 : CONCLUS -0. 05249 0 16380 -0 02428 0 17416 RESIMP 0. 25924 ' -0 06345 -0 10287 -0 05689 1 THEOIMP -o. 01922 -0 13291 0 01 148 0 05164 | FIELDIMP 0. 33666 ; -0 22727 0 06032 0 04984 1 FOUNDAT -0. 09128 -0 05058 0 10215 0 06367 CHARAC 0. 14666 -0 05071 0 15957 -0 07980 AGENCY -o. 26688 -0 04749 0 07715 -0 1 1323 PROGPLAN -0. 02865 -0. 00550 0. 01010 0 10722 INSTRUC 0. 08014 0. 12789 -0. 10020 0 03194 DISCIP 0. 03233 -0. 09433 -0. 22986 -0 01222 ATTACH PRESENT VOICE JARGON DIRECTCI -0 .09128 0 .18599 0 . 26753 0 .09929 -0 14399 99 .00000 99 00000 99 .00000 99 .00000 99 .00000 -0 .12139 -0 05408 -0 .01177 0 06533 -0 07861 > 0 .24603 -0 06435 0 02060 -0 23923 0 21853 Xi i-r-f 0 .06367 -0 05622 -0 .02469 -0 C4727 0 09163 o 1 .00000 0 01 193 -0 08036 0 04 1 1 3 0 59318 3 0 01 193 1 00000 0 00926 0 10364 -o 23652 H--o 08036 0 00926 1 00000 -0 04163 -0 04148 0 04 1 13 0 10364 -0 04 163 1 GOOOO -0 12344 U) 0 59318 -0 23652 -0 04 148 -0 12344 1 00000 •• 0 55426 -o 18071 0 05181 -0 12365 0 88154 -0 047 13 0 06243 -0 13252 -0 04199 -0 10175 n 0 33991 0 12111 0 25826 -0 17442 0 28322 o i-i -0 03288 -0 05806 -0 01063 -0 02929 -0 07098 H -0 26636 -0 20160 -0 22136 0 04237 -0 10953 ; ro M -0 04054 0 17899 -0 08036 0 1 1837 -0 08752 I P> C rt -0 06856 0 24972 -0 02216 0 08588 -0 14800 ' -0 04451 -0 10612 0 04100 -0 42128 0 30747 r t O -0 00929 0 07931 -0 08008 0 22188 -0 22288 ST 3 0 02076 0 10722 -0 00362 0 46342 -o 12771 > S -0 16127 -0 17987 0 01994 -0 02380 -0 18215 fu M rt -0 06978 -0 07249 -0 00212 0 07862 0 09748 • i-( -0 00731 0 14516 0 03400 0 08746 -0 18139 pi H-• X 0 04581 -0 29663 -0 13719 0 19355 0 02198 n 0 06740 -0 18870 -0 15358 0 23080 0 07606 o -0 15587 -0 05186 -0 12908 0 27436 -0 08616 -0 04653 -0 00374 0 21653 0 15198 -o 05783 • c < cn CD rt i-i -0 03288 0 04355 -0 01063 0 15866 -0 07098 ' 0 09250 -0 18849 -0 05864 0 29594 0 17281 pu cu o cr -0 06961 -0 21896 0 02700 0 12983 -o 07513 -0 06856 -o 14462 -o 041 1 1 0 05867 -0 05208 rt h-1 cn fu 0 07886 -0 1 1968 -0 02357 -0 04244 -0 16491 cn -0 08331 0 16632 -0 00890 0 17424 -0 17985 1—1 -o 03288 0 14516 -0 01063 -0 12326 -o 07098 cn 0 05554 0 13078 0 1 1009 -0 07146 -0 05329 Co cn O -0 04054 -0 07159 -0 08036 -o 26785 -0 01945 O -0 13549 -o 02991 0 06860 -0 08843 -0 00813 H-Cu 0 09853 -o 15278 -o 02796 -0 03586 0 06744 rt 0 22056 0 19476 0 12356 -0 04 366 0 15872 ro Pu -0 09492 -0 12572 -0 16565 0 0317 1 -0 13660 -0 07779 -o. 13737 -0 02514 -0 02483 0 06717 i—• AUTHORCI FUNDING THEOCONC ARCHHIST EMPIHARD ADMONPRE METHO DEDUCT INDUCT ARGUFLOW REJACCEP -0. •11667 -0. 02755 0. 04957 -0. 18364 0. 14333 -0. 09128 0. 30278 -0. 05940 0. 16495 0. 35468 '. YEAR 99 . 00000 99 . 00000 99 . 00000 99 . 00000 99 . ooooo 99. OOOOO 99 . OOOOO 99 . OOOOO 99. OOOOO 99 . OOOOO j WORDSTIT -0. 01447 -0. 06703 0. 14284 0. 05661 -0. 13102 -0. 06474 0. 03721 -0. 12394 0. 21237 0. 08064 WORDSABS 0. 20805 -0. 00065 0. 24523 -0. 03481 -0. 16530 -0. 04233 0. 04957 0. 05088 0. 06279 0. 08230 ORIGINAL 0. 1 1 160 0. 07402 0. 08429 0. 05164 -0. 05122 0. 06367 0. 10768 0. 03146 0. 17512 0. 14670 !ATTACH 0. 55426 -0. 04713 0. 33991 -0. 03288 -0. 26636 -0. 04054 -0. 06856 -0. 04451 -0. 00929 0. 02076 1 PRESENT -0. 18071 >0. 06243 0. 12111 -0. 05806 -0. 20160 0. 17899 0. 24972 -0. 10612 0. 07931 0. 10722 !VOICE 0. 05181 -0. 13252 0. 25826 -0. 01063 -0. 22136 -0. 08036 -0. 02216 0. 04100 -0. 08008 -0. 00362 JARGON -0. 12365 -0. ,04199 -0. 17442 -0. 02929 0. 04237 0. 1 1837 0. 08588 -0. 42128 0. 22188 0. 46.342 DIRECTCI 0. 88154 -0. ,10175 0. 28322 -0. 07098 -0. 10953 -0. 08752 -0. 14800 0. 30747 -0. 22288 -0. 12771 AUTHORCI 1 . 00000 -0. ,11896 0. 31065 -0. 08299 -0. 08537 -0. 10232 -0. 17304 0. 33820 -0. 19066 -0. 13251 'FUNDING -0. 1 1896 1 . ,00000 0. 12826 -0. 03823 -0. 06636 0. 25530 -0. 07971 -o. 15136 0. 19715 -0. 06877 THEOCONC 0. 31065 0. , 12826 1. 00000 -0. .09673 -0. 66047 -o. 1 1927 0. 08950 0. , 14633 -o. 06242 -0. 17404 ARCHHIST -0. 08299 -0. .03823 -0. 09673 1 . , 00000 -0. 21602 -0. 03288 -0. O5560 -0. 10559 -0. 00754 0. 04924 EMPIHARD -0. 08537 -0. ,06636 -0. 66047 -0. 21602 1 . OOOOO -0. 26636 -0. , 18500 0. 00627 O. 09884 0. 09351 iADMONPRE -0. 10232 0. ,25530 -0. 1 1927 -0. 03288 -0. 26636 1 . OOOOO -0. ,06856 -0. 13019 0. 14970 -0. 05916 :METHO -0. 17304 -0, .07971 0. 08950 -o. ,05560 -0. 18500 -0. ,06856 1 . OOOOO -0. 1 1 149 0. 0851 1 0. 20403 DEDUCT 0. 33820 -0. .15136 0. 14633 -0. 10559 0. 00627 -0. 13019 -0. , 1 1 149 1 , .OOOOO -0. 501 15 -0. 09127 INDUCT -0. 19066 0, ,19715 -0. 06242 -0. ,00754 0. 09884 0. , 14970 0. .08511 -0. ,50115 1 . OOOOO 0. 27950 ARGUFLOW -0. 13251 -0. .06877 -0. 17404 0. 04924 0. 09351 -0, 05916 0. ,20403 -0. ,09127 0. 27950 1 . OOOOO STATERES -0. 17254 0. ,02481 -0. 42919 -0. 00607 0. 52298 -0. 31505 -0. . 13457 -0. , 17600 0. 14808 0. , 14851 'CUMULIT 0. 04547 -0. .17299 0. 16GG5 -0. .05660 -0. 20420 -0. 06978 0. ,21606 0. , 1 1425 0. ,08060 -0. 00473 DISCIPRE -0. 22553 -0. ,04937 0. ,06126 -0. ,12000 -0. 06686 0. 13335 0. ,19577 0. ,01128 0. 06280 0. , 14057 DESIGN 0. 03319 -0, .03218 -0. ,51380 0. ,03715 0. 61627 -0. ,15015 -0. .17108 -0. .16754 0. 36981 0. ,17972 DATACOLL 0. 03367 0. .04408 . -o. ,40899 0. , 17425 0. 47448 -0. 16851 -0. ,12289 -0. ,06036 0. ,23173 0. ,34748 jINSTRU -0. 08854 0, .02101 -0. 45857 -0. ,05588 0. 42194 -0. ,03991 0. .06733 -0. .07087 0. ,26993 0. .37371 RELIAB -0. 02610 -0. .05409 -0. 04106 -0, .03774 0. 08734 -0, ,04653 0 .12786 0. ,19922 -0. ,06043 0. ,15723 VALID -0. 08299 -0, .03823 -0. ,09673 -0, ,02667 0. 12344 -o. ,03288 0. .21199 0 .10288 -0. ,00754 0. 14646 SAMPLE 0. 07502 . 0_. .,0_2792_ -0. ,29202 0. .01948 0- 40572 -0. ,11296 -0. .08968 -0. . 19357 0. . 24096 0. ,25595 TTYPEANAL -0, ,05490 -0 .05311 -0 .32445 " -o" .09527 o~ .40250 -0 .06961 -0 .07725 -0 .17625 0 .24729 o" .13732 RESULTS 0, ,00174 -0. .07971 -0 .41740 -0. .11507 0. .57199 -0. .21520 -0 .27093 -0 .08735 0 .23076 0 .22656 CONCLUS -0, ,16016 0 .03653 -o .04712 0 .06396 0. .03469 -0 .04764 -o .10731 -0 .01238 0 .16304 • 0 .20250 i RESIMP -0, ,21028 0 .06527 -0 .08099 0 . 15863 -0 .02380 0 .19560 0 .15390 -0 .10678 0 . 19399 0 .26400 THEOIMP -0 .08299 -0 .03823 0 .08947 -0 .02667 -0 .04629 -o .03288 -0 .05560 0 .20712 0 .08918 0 .04924 FIELDIMP -0. .03115 0 .18295 0 .01362 -0 .03754 -0. .07199 0 . 12342 0 .09392 0 .14482 -0 .03395 0 .25308 FOUNDAT -0. .02937 -0 .04713 0 .18685 -0 .03288 -0 .12684 -o .04054 -0 .06856 0 .38389 -0 .16828 -0 .05916 j CHARAC -0 .00673 -0 . 15752 0 .04900 -0 .10989 0 .10068 -0 .13549 0 .04654 0 .17339 0 .00216 -0 .09755 AGENCY 0. .13193 0 . 2221 1 -0 .00954 0 .38006 -0. .04737 -0 .08651 -0 .14630 -0 .04927 -0 .14706 0 .00166 PROGPLAN 0 .13145 0 . 13890 0 .17304 -0 .14907 -0 .23167 0 .22056 0 .03108 -0 .05751 0 . 14323 c .01129 INSTRUC -0 .12977 0 .04138 -0 .20245 -0 .07698 0 .28635 -0 .09492 -0 .05016 -0 .08988 -0 .06164 0 .06196 DISCIP. 0 .05563 -0 .09043 -0 .05264 -0 .06309 0 .05111 -0 .07779 0 .12168 -0 .00320 0 .02793 -0 .06751 H-1 STATERES CUMULIT DISCIPRE DESIGN 1REJACCEP 0 .22783 -0 .10237 0 . 26977 0 .02678 YEAR 99 .OOOOO 99 .OOOOO 99 OOOOO 99 .OOOOO iWORDSTIT -0 . 1731 1 0 09218 -0 09845 -0 .02286 WORDSABS -0 ."286G 0 00558 0 17920 -0 05664 ORIGINAL -o .01411 -0 03223 -0 05164 0 05995 ATTACH -0 16127 -0 06978 -0 00731 0 04581 PRESENT -0 17987 -0 07249 .0 14516 -0 29663 VOICE 0 01994 -0 00212 0 03400 -0 13719 JARGON -0 02380 0 07862 0 08746 0 19355 DIRECTCI -0 18215 0 09748 -0 18139 0 02 198 AUTHORCI -0 17254 0 04547 -0 22553 0 03319 • FUNDING 0 02481 -0 17299 -0 04937 -0 03218 :THEOCONC -o 42919 0 16665 0 06126 -o 51380 ARCHHIST -0 00607 -0 05660 -o 12000 0 03715 EMPIHARD 0 52298 -0 20420 -0 06686 0 61627 ADMONPRE -0 31505 -0 06978 0 13335 -0 15015 METHO -0 13457 0 21606 0 19577 -0 17 108 DEDUCT -0 17600 0 1 1425 0 01 128 -0 16754 INDUCT 0 14808 0 08060 0 06280 0 36981 ARGUFLOW 0 14851 -0 00473 0 14057 0 17972 i STATERES 1 OOOOO -0 24645 0 09739 0 37770 •CUMULIT -o 24645 1 OOOOO -0 12651 -0 18901 DISCIPRE 0 09739 -0 12651 1 OOOOO 0 00826 DESIGN 0 37770 -0 18901 0 00826 1 OOOOO DATACOLL 0 32405 -o 08702 -0 06492 0 59026 INSTRU 0 331 10 -0 03934 0 1 1298 0 48734 iRELIAB 0 02750 0 1 1778 0 09434 -0 00876 1 VALID 0 08747 0 07158 0 051 1 1 0 03715 'SAMPLE 0 31567 0 04134 0 03210 0 43718 :TYPEANAL 0 25943 0 03039 0 13408 0 52476 RESULTS 0 70249 -0 17740 0 0471 1 0 50551 CONCLUS 0 33004 -0 15245 -0 01999 0 12530 IRESIMP 0 1 1857 0 02604 0 1 1065 -0 06344 ITHEOIMP 0 04070 -0 05660 0. 051 1 1 -0 08204 FIELDIMP 0 26607 -0 17246 0 12012 -0 15863 iFOUNDAT -0 08438 0 03558 0. 13335 -0 15015 ICHARAC 0 1 1952 0 07488 0. 32814 -0 05156 AGENCY 0 00453 -0. 20512 -0. 24070 0. 09775 'PROGPLAN -0 22818 0. 0521 1 -0. 17889 -0. 17307 iINSTRUC 0. 27178 -0. 16338 -0. 06415 0. 20556 DISCIP 0 02989 -0. 07325 -0. 12197 0. 03150 DATACOLL INSTRU RELIAB VALID SAMPLE 0 .08536 0 25910 0 .12087 0 .14521 0 .08076 99 .OOOOO 99 OOOOO 99 .OOOOO 99 .OOOOO 99 .OOOOO -0 .12728 -0 08320 -0 .03293 -0 .11568 -0 .17417 0 .01775 0 01 158 -0 .01930 -0 .04205 0 .00793 0 03970 -0 14547 0 07307 0 .05164 -0 .22211 0 06740 -0 15587 -0 04653 -0 03288 0 .09250. -o 18870 -o 05186 -0 00374 0 04355 -0 .18849 -0 15358 -0 12908 0 21653 -0 01063 -0 .05864 0 23080 0 27436 0 15198 0 15866 0 29594 0 07606 -0 08616 -o 05783 -0 07098 0 17281 0 03367 -0 08854 -0 02610 -0 08299 0 07502 0 04408 0 02101 -0 O5409 -0 03823 0 02792 -o 40899 -0 45857 -o 04106 -o 09673 -0 29202 0 17425 -0 05588 -0 03774 -o 02667 0 01948 0 47448 0 42194 0 08734 0 12344 0 40572 -0 16851 -0 03991 -0 04653 -0 03288 -0 1 1296 -0 12289 0 06733 0 12786 0 21 199 -0 08968 -0 06036 -o 07087 0 19922 0 10288 -0 19357 0 23173 0 26993 -0 06043 -0 00754 0 24096 0 34748 0 37371 0 15723 0 14646 0 25595 0 32405 0 331 10 0 02750 0 08747 0 31567 -o 08702 -o 03934 O 1 1778 0 07 158 0 04 134 -0 06492 0 1 1298 0 09434 0 051 1 1 0 03210 0 59026 0 48734 -0 00876 0 03715 0 43718 1 OOOOO 0 54508 0 12658 0 10250 0 51905 0 54508 1 OOOOO 0 09334 0 22627 0 367 1 1 0 12658 0 09334 1 OOOOO 0 35849 -0 07962 0 10250 0 22627 0 35849 1 OOOOO 0 01948 0 51905 0 3671 1 -0 07962 0 01948 1 OOOOO 0 33363 : 0 54836 -0 OOOOO -0 09527 0 21 132 0 33840 ' 0 33290 -0 01748 0 12279 0 37842 0 08758 0. 03886 -o 12727 -0 08994 0 02822 0 02258 o. 14593 0 07898 0 04553 0 04935 -0 04100 -0. 12642 -0 03774 -0 02667 0. 01948 -0 05771 0. 15680 0 21247 0 04504 -0. 03290 -0 10953 -0. 09789 -0 04653 -o. 03288 -0. 18144 -0. 04576 o. 01 195 -0 15550 -0. 10989 0. 22332 0 14384 0. 00763 -0 09929 -0. 07017 -0. 05836 -0. 06876 -0. 14339 0 04219 0. 01491 -0. 18388 0 05918 0. 12958 0. 16340 0. 13472 0. 05622 -0. 02910 0. 03468 -0. 08927 -0. 06309 -0. 07219 TYPEANAL RESULTS CONCLUS RESIMP THEQIMP FIELDIMP FOUNDAT CHARAC AGENCY PROGPLAN REJACCEP 0. 07120 0. ,16945 -0. .05249 0. . 25924 -0. .01922 0. ,33666 -0 .09128 0 . 14666 -0 .26688 -0 .02865 YEAR 99 . 00000 99 .00000 99 . 00000 99 . 00000 99 . 00000 99 .00000 99 .00000 99 .00000 99 .00000 99 .ooooo WORDSTIT 0. .06839 -o. .07356 0. .16380 -0. .06345 -0. .13291 -0. . 22727 -0 .05058 -0. .05071 -0 .04749 -0 .0O550 WORDSABS 0. 00779 . -o. .07569 -0. .02428 -0. .10287 0. .01148 0. .06032 0 .10215 0 . 15957 0 .07715 0 .01010 ORIGINAL -o. .16984 -0. .02393 0. .17416 -.0. .05689 0. .05164 0. .04984 0 .06367 -0 .07980 -0 .11323 0 .10722 ATTACH -0. .06961 -0. .06856 0. .07886 -0. .08331 -0. .03288 0. .05554 -0 .04054 -0 .13549 0 .09853 0 .22056 PRESENT -0. .21896 -0. .14462 -0. .11968 0. . 16632 0. .14516 0. .13078 -0 .07159 -0 .02991 -0. .15278 0 . 19476 VOICE 0. .02700 -0 .041 1 1 -0. .02357 -0. .00890 -0 .01063 0 .11009 -0 .08036 0 .06860 -0. .02796 0 . 12356 JARGON 0. . 12983 0 .05867 -0. .04244 0. .17424 -0. .12326 -0 .07146 -0 .26785 -0 .08843 -0 .03586 -0 .04366 DIRECTCI -0. ,07513 -0 .05208 -0. .16491 -0. .17985 -0. .07098 -0 .05329 -0 .01945 -0 .00813 0 .06744 0 . 15872 AUTHORCI -0. .05490 0 .00174 -0 .16016 -0. .21028 -0. .08299 -0 .03115 -0 .02937 -0 .00673 0 .13193 0 . 13145 FUNDING • -o. .05311 -0 .07971 0 .03653 0. .06527 -o. ,03823 0 .18295 -0 .04713 -0. . 15752 0 . 222.1 1 0. . 13890 THEOCONC -0, , 32445 -o .41740 -o. .047 12 -0. .08099 0. .08947 0 .01362 0 .18685 0 .04900 -0 .00954 0 .17304 ARCHHIST -0. .09527 -0 .11507 0. .06396 0. .15863 -0. .02667 -0. .03754 -0 .03288 -0. . 10989 0 .38006 -0 .14907 EMPIHARD 0. .40250 o .57199 0 .03469 -0. .02380 -0 .04629 -0 .07199 -0 .12684 0 .10068 -0 .04737 -0. . 23167 ADMONPRE -0. .06961 -0 .21520 -0 .04764 0. . 19560 -0. .03288 0 .12342 -0 .04054 -0. . 13549 -0 .08651 0 .22056 METHO -0. .07725 -0 .27093 -0 .10731 0. . 15390 -0 .05560 0 .09392 -0 .06856 0 .04654 -0 . 14630 0. .03108 DEDUCT -0. . 17625 -0 .08735 -0 .01238 -0. .10678 0 ,20712 0 .14482 0 .38389 0 . 17339 -0 .04927 -0. .05751 INDUCT . 0. 24729 0 .23076 0 .16304 0. .19399 0 .08918 -0. .03395 -0 .16828 0 .00216 -o . 14706 0. . 14323 ARGUFLOW 0. . 13732 0 . 22656 0 .20250 0. .26400 0. .04924 0. .25308 -0 .05916 -0. .09755 0. .00166 0. .01129 STATERES 0. . 25943 0 .70249 0 .33004 0. .11857 0. .04070 0 .26607 -0 .08438 0 . 1 1952 0. .00453 -0. . 22818 CUMULIT 0. .03039 -0 .17740 -0 .15245 0 .02604 -0 .05660 -0 . 17246 0 .03558 0 .07488 -0. .20512 0. .05211 DISCIPRE 0 .13408 0 .0471 1 -0 .01999 0 .11065 0 .051 1 1 0. .12012 0 .13335 0 .32814 -0. .24070 -o. . 17889 DESIGN 0 .52476 0 .50551 0 . 12530 -0 .06344 -0. .08204 -0 . 15863 -0 .15015 -0 .05156 0. ,09775 -o, . 17307 DATACOLL 0 .33363 0 .33840 0 .08758 0 .02258 -0 .04 100 -0. .05771 -0 .10953 -0. .04576 0. .14384 -o. .06876 INSTRU 0 .54836 0 .33290 0 .03886 0 .14593 -0 . 12642 0 . 15680 -0 .09789 0 .01195 0 ,00763 -0. .14339 RELIAB -0 .00000 -0 .01748 -0 .12727 0 .07898 -0. .03774 0 .21247 -0. .04653 -0. . 15550 -0. .09929 0. .04219 VALID -0 .09527 0 . 12279 -0 .08994 0 .04553 -0 .02667 0 .04504 -0 .03288 -0. . 10989 -0. ,07017 0. .01491 SAMPLE__ 0 • 2.1 1_32 0 .37842 0 .02822 0 .04935 0 .01948 -0 .03290 -0 .18144 0 . 22332 -0. .05836 -0. 18388 TYPEANAL 1. 00000 6. .29367 0. .00159 0. 07929 -0. 09527 - 6 . 10132 -6. 1 1746 ' -6 . 09269 ' - 6: 081~23 " - 6 . "14201 RESULTS 0. 29367 1. .00000 0. .21358 -0. 04917 -0. 17453 0. 14176 -0. .21520 0. 15886 -0. 17238 -o. 10189 CONCLUS 0. 00159 0. ,21358 1, ,00000 0. 07728 0. 14090 0. 23856 0. 07885 -0. 10638 -0. 10167 0. 1 1 173 RESIMP 0. 07929 -0. .04917 0, .07728 1. 00000 0. 27174 0. 2051 1 0. 19559 -0. 08423 -0. 02899 -0. 08868 THEOIMP -0. 09527 -0, ,17453 0. .14090 0. 27174 1. 00000 0. 04504 0. 38907 0. 06639 -0. 07016 -o. 14907 FIELDIMP -0. 10132 0. .14176 0. .23856 0. 2051 1 0. .04504 1. 00000 0. 12342 -0. 06961 -0. 06255 0. 19905 FOUNDAT -0. .11746 -0 .21520 0. .07885 0. 19559 0. 38907 0. 12342 1. 00000 0. 0094 1 0. 09853 -0. 18380 CHARAC -0. .09269 0 ,15886 -Q. .10638 -0. 08423 0. .06639 -0. 06961 0. 0094 1 1. 00000 -0. 28913 -0. 44537 AGENCY 0. .08123 -0. .17238 -0. .10167 -0. 02899 -0. 07016 -0. 06255 0. 09853 -0. 289 13 1. 00000 -0. 03269 PROGPLAN -0. .14201 -0. .10189 0. .11173 -0. .08868 -0. 14907 0. 19905 -0. 18380 -0. 44537 -0. 03269 1. OOOOO INSTRUC 0. . 109 13 0 .30542 0. ,08943 0. 03816 -0. .07698 0. 1 3003 -o. 09492 -0. 17183 -0. 20255 -0. 36271 DISCIP 0. .10518 -0 .07527 -0. .21277 -0. 10634 -0. .06309 -0. 12788 -0. .07779 -0. 17657 0. 04703 0. 03527 I—' INSTRUC DISCIP REJACCEP 0. 08014 0 03233 YEAR 99 00000 99 00000 WORDSTIT 0 12789 -0 09433 WORDSABS . -0 10020 -0 22986 ORIGINAL 0 03194 -0 01222 ATTACH -0 09492 -0 07779 PRESENT -0 12572 -0 13737 VOICE -0 16565 -0 02514 JARGON 0 03171 -0 02483 DIRECTCI -0 13660 0 06717 AUTHORCI -0 12977 0 05563 FUNDING 0 04138 -0 09043 THEOCONC -0 20245 -0 05264 ARCHHIST -0 07698 -0 06309 EMPIHARD 0 28635 0 051 1 1 ADMONPRE -O 09492 -0 07779 METHO -0 05016 0 12168 DEDUCT -0 08988 -0 00320 INDUCT -0 06164 0 02793 ARGUFLOW 0 06196 -0 06751 STATERES 0 27178 0 02989 CUMULIT -0 16338 -o 07325 DISCIPRE -0 06415 -0 12197 .DESIGN 0 20556 0 03150 DATACOLL 0 05918 -0 02910 INSTRU 0 12958 0 03468 RELIAB 0 16340 -0 08927 VALID 0 13472 -0 06309 SAMPLE 0 05622 -0 07219 TYPEANAL 0 10913 0 10518 RESULTS 0 30542 -0 07527 CONCLUS 0 08943 -0 21277 RESIMP 0 03816 -0 10634 THEOIMP -o 07698 -0 06309 FIELDIMP 0 13003 -0 12788 FOUNDAT -o 09492 -0 07779 CHARAC -0 17183 -0 17657 AGENCY -0 20255 0 04703 PROGPLAN -0 36271 0 03527 INSTRUC 1 00000 -o 18212 DISCIP -0 18212 1 00000 REJACCEP WORDSTIT WORDSABS ORIGINAL ATTACH PRESENT VOICE JARGON DIRECTCI REJACCEP 1 . OOOOO -0 05855 0 02131 0 08550 0 20081 0 10448 -0 04554 0 01373 0 03835 YEAR 99. OOOOO 99 OOOOO 99 OOOOO 99 OOOOO 99 OOOOO 99 OOOOO 99 OOOOO 99 OOOOO 99 OOOOO WORDSTIT -0 05855 1 OOOOO 0 19688 0 13104 -o 28609 0 01393 0 03370 -0 06801 -0 00622 WORDSABS 0 02131 0 19688 1 OOOOO 0 12908 0 05529 -0 14140 0 12690 -0 21290 0 22560 ORIGINAL 0 08550 0 13104 0 12908 1 OOOOO -0 17512 -o 05360 -0 01298 -O 07608 0 14307 ATTACH 0 20081 -0 28609 0 05529 -0 17512 1 OOOOO 0 12465 0 1 1 178 0 12243 -0 05600 PRESENT 0 10448 0 01393 -0 14140 -0 05360 0 12465 1 OOOOO -0 09968 0 01089 -0 12738 VOICE -0 04554 0 03370 0 12690 -0 01298 0 1 1 178 -0 09968 1 OOOOO 0 21974 -0 04534 JARGON 0 01373 -0 06801 -0 21290 -0 07608 0 12243 0 01089 0 21974 1 OOOOO -0 25651 DIRECTCI 0 03835 -0 00622 0 22560 0 14307 -0 05600 -0 12738 -0 04534 -0 25651 1 OOOOO AUTHORCI 0 04801 0 00331 0 22597 0 14776 -0 05784 -0 09737 -0 05830 -0 27885 0 90624 FUNDING 0 OOOOO 0 13120 -0 01867 0 07869 -0 03080 0 02593 -0 06598 0 04424 -0 10695 THEOCONC • -o 04343 -0 05347 0 02675 0 04177 -o 08503 -0 01552 -O 06841 -0 23752 0 21746 ARCHHIST 0 01G46 0 04706 0 06949 0 05067 -0 01983 0 02580 0 03249 -0 02035 0 14807 EMPIHARD 0 02530 0 15082 0 08902 0 09605 -0 14430 0 01788 -0 00026 0 16720 -0 05645 ADMONPRE -0 03857 -0 01624 -0 06605 -0 07255 -0 02582 -0 04939 0 01496 0 01801 -0 00493 METHO O 24.006 -0 19190 0 06165 0 00456 0 18568 -0 00628 0 04544 -0 00952 -0 05827 DEDUCT 0 04006 0 12095 0 24639 0 15927 -0 07642 -0 10741 -0 06639 -0 19392 0 46785 INDUCT 0 18285 -0 1 1834 -0 09685 0 16415 0 01836 0 03512 -0 06477 0 10923 -0 07968 ARGUFLOW 0 15849 0 00534 -0 09943 0 07953 0 01245 0 057 18 0 18830 0 57912 -0 12324 STATERES 0 01 134 0 18251 0 17 139 0 1621 1 -0 18895 0 05008 -0 05493 -0 10065 -O 05849 CUMULIT -0 21483 0 18701 -0 01412 0 17787 -0 29213 -0 02618 -0 10223 -0 17576 0 23558 DISCIPRE 0 18458 -0 02992 0 13314 0 10026 0 05233 -0 05756 -0 06841 -o 08723 -0 12832 DESIGN 0 13770 0 21404 0 1 1989 0 1 1302 -0 04424 -0 04655 -0 00697 0 22010 -0 04992 DATACOLL 0 05241 0 01321 0 29308 0 14496 -0 01830 0 09734 -0 00392 0 06797 -0 00850 INSTRU - 0 07509 0 10339 0 19588 -0 01324 0 03487 -0 05142 -0 01722 0 08449 -0 00777 RELIAB 0 21444 -0 06237 0 004 14 -0 09167 -0 02871 0 04393 o 03617 -o 06479 0 06853 VALID 0 18285 -0 1 1545 0 00398 -0 16415 -0 01836 -0 06040 -0 03932 -0 04143 -0 06374 SAMPLE o 16760 0 12846 0 09555 0 14863 -0 10610 -0 15896 o 04701 0 21556 ..o 00594 TYPEANAL 0 03295 0 00725 0 06257 0 14085 0 16100 -0 00743 -6 10616 0 10497 :o 01589 'RESULTS -0 0581 1 0 13760 0 15843 0 07851 -0 05524 0 01810 0 15430 -0 02650 -0 02587 CONCLUS 0 06238 0 05697 0 08628 0 1 1734 0 06368 0 06143 0 00618 -0 10558 0 00988 RESIMP 0 04701 -0 11212 0 04 133 0 14469 0 14160 -0 00217 -0 13203 -0 08522 -0 06760 THEOIMP 0 06720 -0 07252 -o 02986 0 06895 0 45881 0 09706 -0 1 1902 0 00554 -0 03046 FIELDIMP 0 01669 0 14341 0 13867 0 13130 -0 09832 -0 04805 0 04280 -0 12012 -0 01756 FOUNDAT 0 09745 -0 04330 -o 03619 0 08998 -o 03522 -0 1051 1 -0 19085 -o 13730 0 11313 CHARAC -0 08889 -0 03846 0 19874 -0 03420 0 06024 -o 15365 0 06073 0 01373 0 14816 AGENCY -0 15903 0 04234 0 07920 -0 22591 0 08843 -0 00846 -0 04736 0 01663 -0 10939 PROGPLAN 0 17678 0 13313 -0 1 1535 0 12696 0 01420 0 03694 0 1 1093 -0 18644 0 06040 INSTRUC -0 07013 -0 05395 -0 17288 -o 08994 -0 05352 0 06056 -o 03159 0 09072 -0 07800 DISCIP -0 05590 -0 13798 0 02786 0 02868 0 15716 0 00687 -0 09618 0 09212 -0 10329 > fD 3 o o i-i m t—1 tu rt H- H-rt O 3 > , K rt ¥ H-X • O J-h > •< B> rt H n H-0) 05 o W ri- M en fD cn 1—1 VO :> cn cn o o M-(U fD AUTHORCI FUNDING THEOCONC REJACCEP 0 . 04801 0 . 0 0 0 0 0 - 0 . 0 4 3 4 3 YEAR 9 9 . 0 0 0 0 0 9 9 . 0 0 0 0 0 9 9 . 0 0 0 0 0 WORDSTIT 0 . 0 0 3 3 1 0 . 1 3 1 2 0 - 0 . 0 5 3 4 7 WORDSABS 0 . 2 2 5 9 7 - 0 . 0 1 8 6 7 0 . 0 2 6 7 5 ORIGINAL 0 . 1 4 7 7 6 0 . 0 7 8 6 9 0 . 0 4 1 7 7 ATTACH - 0 . 0 5 7 8 4 - 0 . 0 3 0 8 0 - 0 . 0 8 5 0 3 PRESENT - 0 . 0 9 7 3 7 0 . 0 2 5 9 3 - 0 . 0 1 5 5 2 ;VOICE - 0 . 0 5 8 3 0 - 0 . 0 6 5 9 8 - 0 . 0 6 8 4 1 'JARGON. - 0 . 2 7 8 8 5 0 . 0 4 4 2 4 -o. 2 3 7 5 2 DIRECTCI o: 9 0 6 2 4 - 0 . 1 0 6 9 5 0 . 2 1 7 4 6 AUTHORCI 1. 0 0 0 0 0 - 0 . 1 1 0 4 6 0 . 2 4 6 2 8 FUNDING -o. 1 1046. . 1 . 0 0 0 0 0 -6. 0 8 7 4 4 THEOCONC 0 . 2 4 6 2 8 - 0 . 0 8 7 4 4 1. 0 0 0 0 0 ARCHHIST 0 . 2 1 3 3 7 - 0 . 0 3 7 8 8 - 0 . 1 0 4 5 6 EMPIHARD . - o . 1 5 6 6 9 0 . 0 0 3 8 8 - 0 . 5 5 3 0 3 ADMONPRE o. 0 1 8 5 2 - 0 . 0 4 9 3 0 •. - 0 . 0 4 8 1 6 METHO -o. 0 0 6 4 0 . 0 . 0 4 7 7 3 -6'. 1 4 6 2 5 : DEDUCT 0 . 5 3 3 6 6 - 0 . 0 9 9 8 5 0 . 1 9 0 8 4 ! INDUCT -o. 1 5 2 8 2 -O. . 0 1 0 0 2 o. 0 5 2 1 1 ARGUFLOW - o . 1 1 9 0 9 0 . . 0 4 7 5 7 - 0 . . 1 5 8 2 4 STATERES -o. 0 9 5 9 2 0 . 0 1 0 8 7 - 0 . , 3 5 6 3 5 CUMULIT 6. . 1 8 1 8 3 - 0 . . 1 1 5 6 2 0 . . 0 7 1 2 0 DISCIPRE -o, , 0 4 6 9 1 . -o . 0 1 2 4 9 0 . . 2 2 0 1 6 DESIGN -o: . 1 0 7 6 5 --0-. ; 0 0 3 0 2 - 0 : . 4 3 5 0 7 DATACOLL -o . 0 3 1 1 8 -6 . 0 3 4 9 5 - 0 . 2 3 6 7 9 INSTRU -o . 0 5 2 3 8 -o . 0 3 0 5 9 - 0 . 3 9 4 0 2 RELIAB 0 . 0 2 9 4 1 0 . 1 5 6 6 4 - 0 . 0 4 6 5 7 VALID -o . 0 6 5 8 3 0 . 0 5 5 0 9 • • - 0 . 0 9 6 7 8 SAMPLE - 0 . 0 3 6 8 2 - 0 . 0 8 4 9 7 - 0 . 3 4 5 5 9 TYPEANAL - o . 0 5 1 4 1 0 . 0 6 7 3 9 - 0 . 3 4 7 0 2 RESULTS 7.6 . 0 4 7 4 3 . - 0 . 1 1 8 8 6 - 0 . 2 1 8 3 3 CONCLUS - o . 0 2 4 3 3 - 0 . 0 2 1 9 3 - 0 . 1 4 4 3 6 RESIMP - 0 ' . 0 5 9 2 4 0 . 0 5 4 0 8 0 . 0 4 9 7 7 'THEOIMP 0 . 0 6 9 1 3 . - 0 . 0 5 1 5 4 - 0 . 0 1 0 9 5 FIELDIMP -G . 0 2 8 0 5 - 0 . 0 3 4 1 4 - 0 . 0 9 9 6 8 FOUNDAT 0 . 2 3 1 5 7 - 0 . 0 6 7 2 7 6 . 0 8 0 9 4 CHARAC 0 . 1 0 8 0 1 - 0 . 0 7 6 7 0 0 . 0 7 0 5 7 AGENCY - 0 . 1 1 8 9 2 0 . 1 2 6 6 6 -o . 0 5 6 7 8 PROGPLAN 0 . 0 1 8 1 9 0 . 0 6 1 9 8 0 . 0 2 3 0 3 INSTRUC -o . 1 0 1 0 1 0 . 0 9 1 4 4 0 . 0 0 5 7 1 DISCIP - 0 . 0 0 5 7 5 - 0 . 0 8 5 7 5 - 0 . 0 1 8 2 1 ARCHHIST 0 99 O 0 0 - 0 0 0 - 0 o 0 - 0 - 0 1 -o - 0 01646 00000 04706 06949 05067 01983 02580 03249 02035 14807 21337 03788 .10456 .00000 .17744 .03.175 -0.04830 -0.02473 -0.04515 0:05105 -0.02659 -0:08562 O.12065 -0.09520 -0.02251 -0.10314 -0.03530 . 02258 04629 08679 12820 07830 01161 03319 12091 36095 09877 O. 17218 0. 13968 0.06581 0.05522 - 0 . 0 . -O. - 0 . 0 . 0 . ^O. - 0 . 0 . ^ 0 . EMPIHARD 0.02530 99.00000 0. 15082 0.08902 0.09605 .14430 .01788 .00026 16720 -0.05645 -0.15669 .00388 .55303 17744 . 00000 -0.23096 -0.29470 -0.06639 .18506 12240 .51432 18615 .24 148 .56998. .43199 .44660 .10128 -0.03933 0.42736 ADMONPRE METHO DEDUCT INDUCT ARGUFLOW -0. 0. -0. 0. 0 . - 0 . - 0 . 1 0. 0. 0. 0. -0. 0.  0'  0. 0. 0. 40740 0. 45074 0. 13877 6. 15701 0. 05781 0. 23793 0, ,19085 0. .02530 •o. ,16436 0. .04652 0 .05785 •0 .04526 - 0 . 0 3 8 5 7 0 . 2 4 0 0 6 0 . 0 4 0 0 6 0 . 1 8 2 8 5 0 . 1 5 8 4 9 99 . 0 0 0 0 0 9 9 . 0 0 0 0 0 99 . 0 0 0 0 0 9 9 . 0 0 0 0 0 9 9 . 0 0 0 0 0 - 0 . 0 1 6 2 4 - 0 . 1 9 1 9 0 0 . 12095 -o. 1 1834 0 . 0 0 5 3 4 - 0 . 0 6 6 0 5 0 . 0 6 1 6 5 0 . 2 4 6 3 9 - 0 . 0 9 6 8 5 - 0 . 0 9 9 4 3 - 0 . 0 7 2 5 5 0 . 0 0 4 5 6 0 . 15927 0 . 1 6 4 1 5 0 . 0 7 9 5 3 - 0 . 0 2 5 8 2 0 . 18568 - 0 . 0 7 6 4 2 0 . 0 1 8 3 6 0 . 0 1 2 4 5 - 0 . 0 4 9 3 9 - 0 . 0 0 6 2 8 - 0 . 1074 1 0 . 0 3 5 1 2 0 . 0 5 7 18 0 . 0 1 4 9 6 0 . 0 4 5 4 4 - 0 . 0 6 6 3 9 - 0 . 0 6 4 7 7 0 . 1 8 8 3 0 0 . 0 1 8 0 1 - 0 . 0 0 9 5 2 - 0 . 19392 0 . 1 0 9 2 3 0 . 5 7 9 1 2 - 0 . 0 0 4 9 3 - 0 . 0 5 8 2 7 0 . 4 6 7 8 5 - 0 . 0 7 9 6 8 - 0 . 1 2 3 2 4 0 . 0 1 8 5 2 - 0 . 0 0 6 4 0 0 . 5 3 3 6 6 - 0 . 1 5 2 8 2 - 0 . 1 1 9 0 9 - 0 . 0 4 9 3 0 0 . 0 4 7 7 3 - 0 . 0 9 9 8 5 - 0 . 0 1 0 0 2 0 . 0 4 7 5 7 - 0 . 0 4 8 1 6 - 0 . 1 4 6 2 5 0 . 19084 0 . C 5 2 1 1 - 0 . 1 5 8 2 4 - 0 . 0 3 1 7 5 - 0 . 0 4 8 3 0 - 0 . 0 2 4 7 3 - 0 . 0 4 5 1 5 0 . 0 5 1 0 5 - 0 . 2 3 0 9 6 - 0 . 2 9 4 7 0 - 0 . 0 6 6 3 9 0 . 1 8 5 0 6 0 . 1 2 2 4 0 1 . 0 0 0 0 0 0 . 2 2 5 1 8 0 . 0 3 9 9 2 - 0 . 2 3 5 0 8 0 . 0 4 7 8 4 : 0 . 2 2 5 1 8 1 . 0 0 0 0 0 - 0 . 0 3 6 5 1 - 0 . 0 2 8 4 5 0 . 0 4 9 6 3 0 . 0 3 9 9 2 . - 0 . 0 3 6 5 1 1 . 0 0 0 0 0 - 0 . 4,761 1 - 0 . 1 2 2 1 5 - 0 . 2 3 5 0 8 - 0 . 0 2 8 4 5 - 0 . 4761 1 1. 0 0 0 0 0 0 . 1 5 5 9 2 0 . . 0 4 7 8 4 0 . 0 4 9 6 3 - 0 . 1 2 2 1 5 0 . 1 5 5 9 2 1. 0 0 0 0 0 . 0 . . 0 1 8 9 5 - b . , 19026 0 . 0 1 6 7 5 0 . 0 8 6 8 0 0 . 0 0 4 3 9 0 . 0 5 1 3 6 - 0 , , 1 4 1 4 1 0 . . 1 4 5 7 0 . 0 . 0 3 7 9 0 - 0 . 0 6 0 7 7 - 0 . 0 4 8 1 6 0 . 0 9 7 0 2 0 . 0 5 3 8 3 0 . , 1 6 3 7 9 0 . . 0 3 0 3 0 0 . 0 2 4 7 8 - 0 . 1 2 9 7 7 - 0 . 0 8 5 5 0 0 . . 2 6 0 7 6 . " . 0 . 2 3 9 1 i - 0 - . '14004 0 . 0 0 6 4 8 . . -o . 0 2 9 2 1 0 . . 1 8 9 5 9 0 : 1 4 2 7 4 - 0 . 0 5 8 2 3 0 . 0 0 6 0 5 0 : 0 2 7 0 3 0 , . 1 0 5 1 2 . •. c . . 1 8 1 9 1 - 0 . 0 4 5 9 5 o . 2 7 3 2 8 0 . 0 5 7 2 7 - 0 . 0 0 9 3 4 0 . 0 6 6 5 0 - 0 . 0 2 9 3 8 0 . 4 6 7 3 9 - 0 . 0 8 6 9 8 0 . 1 5 5 2 3 0 . 1 1 3 4 0 - 0 : 0 3 1 7 7 : • -o . 0 9 9 2 6 0 . 0 3 4 9 9 0 . 1 4 4 1 3 0 . 2 8 5 3 9 _ - 0 . 1 1 2 9 6 -0 . 0 4 8 8 3 -o . 0 1 8 7 0 . 0 . 1 7 0 7 0 0 . 0 9 4 2 2 - 0 . 1 0 4 1 0 - 0 . 1 8 8 7 7 0 . 0 8 4 5 9 - 0 . 0 4 0 7 3 - 0 . 0 2 8 8 3 0 . 1 0 1 9 2 - 0 . 0 4 8 7 6 0 .061 18 - 0 . 0 5 1 0 9 0 . 0 3 5 4 6 0 . 0 3 6 2 6 0 . 2 5 7 0 5 - 0 . 1 0 3 5 8 0 . 1 7 7 2 9 0 . 0 9 4 7 5 - 0 . 0 4 3 2 0 . 0 . 14938 0 , 0 3 3 6 5 0 . 0 3 0 7 2 . . 0 . 0 0 0 0 0 - 0 . 1 5 7 3 8 - 0 . 1 1477 • - 0 . 0 2 1 1 8 0 . 0 8 9 0 2 0 . 1 1 2 1 5 0 . 1 0 1 4 8 0 . 0 2 3 3 9 - 0 . 0 0 2 9 3 - 0 . 0 4 0 0 9 - 0 . 1 2 0 8 7 0 . 14142 -o : 0 0 8 8 9 0 . 2 5 0 3 8 - 0 . 0 6 8 5 7 - 0 . 0 5 8 5 7 - 0 . 0 0 7 0 8 - 0 . 1 6 1 5 1 - 0 . 0 0 5 5 1 - 0 . 1 7 6 1 4 0 . 0 3 7 9 3 - 0 . 0 1 8 1 8 - 0 . 0 5 0 3 0 - 0 . 1 7 7 0 4 0 . 1 5 0 0 7 - 0 . 0 4 3 8 5 - 0 . 0 8 5 6 6 0 . 0 2 6 8 1 0 . 0 4 1 4 4 0 . 0 6 0 9 1 0 . 0 2 6 1 0 - 0 . 0 7 1 8 7 0 . 1 5 9 0 5 0 . 0 2 2 3 9 - 0 . 0 8 0 3 1 - 0 . 1 0 4 0 1 ON STATERES CUMULIT DISCIPRE DESIGN DATACOLL INSTRU RELIAB VALID SAMPLE REJACCEP 0 01 134 -0 21483 0 18458 0 13770 0 05241 0 07509 0 21444 0 18285 . . 0 16760 YEAR 99 00000 99 00000 99 00000 99 00000 99 00000 99 00000 99 00000 99 00000 99 00000 WORDSTIT 0 18251 0 18701 -0 02992 0 21404 0 01321 0 10339 -0 06237 -o 1 1545 6 12846 WORDSABS 0 17139 -0 01412 0 13314 0 1 1989 0 29308 0 19588 0 00414 0 00398 0 09555 ORIGINAL 0 1621 1 0 17787 0 10026 0 1 1302 0 14496 -0 01324 -0 09167 -0 16415 0 14863 ATTACH -0 18895 -0 29213 0 05233 -0 04424 -0 01830 0 03487 -0 02871 -0 01836 -0 10610 PRESENT 0 05008 -0 02618 -0 05756 -0 04655 0 09734 -0 05142 0 04393 -0 06040 -0 15896 VOICE -0 05493 -0 10223 -0 06841 -o 00697 -0 00392 -0 01722 0 03617 -o 03932 0 04701 JARGON -0 10065 -0 17576 -0 08723 0 22010 0 06797 0 08449 -0 06479 -0 04143 0 21556 DIRECTCI -0 05849 0 23558 -0 12832 -0 04992 -0 00850 -0 00777 0 06853 -0 06374 0 00594 ! AUTHORCI -o 09592 0 18183 -0 04691 -0 10765 -0 031 18 -0 05238 0 02941 -0 06583 -0 03682 I FUNDING 0 01087 -0 1 1562 -o 01249 -0 00302 -0 03495 -0 03059 0 15664 0 05509 -o 08497 1THEOCONC -0 35635 0 07120 0 22016 -0 43507 -0 23679 -0 39402 -0 04657 -0 09678 -0 34559 ,ARCHHIST -0 02659 -0 08562 0 12065 -0 09520 -0 02251 -0 10314 -0 03530 -0 02258 0 04629 EMPIHARD 0 51432 0 18615 -0 24148 0 56998 0 43199 0 44660 0. 10128 -0 03933 0 42736 ADMONPRE 0 01895 0 05136 -0 04816 0 02478 -0 14004 -0 05823 -0 04595 -0 02938 -0 03177 ;METHO -o 19026 -0 14141 0 09702 -0 12977 0 00648 0 00605 0 27328 0 46739 -o 09926 1 DEDUCT 0 01675 0 14570 0 05383 -0 08550 -0 02921 0 02703 0 05727 -0 08698 0 03499 1 INDUCT 0 08680 0 03790 0 16379 0 26076 0 18959 0 10512 -0 00934 0 15523 0 14413 IARGUFLOW 0 00439 -0 06077 0 03030 0 2391 1 0 14274 0 18191 0 06650 0 1 1340 0 28539 ISTATERES 1 00000 0 1 1077 -o 09463 0 47221 0 32675 0 33306 0 13776 0 09976 0 44610 |CUMULIT 0 1 1077 1 00000 -0 16082 0 03944 0 20956 0 03538 -0 03502 -0 13094 0 08158 lDISCIPRE -0 09463 -0 16082 1 00000 -o 11213 -0 03415 -0 05694 -0 08149 0 03722 0 02359 DESIGN 0 47221 0 03944 -o 11213 1 00000 0 42424 0 46204 0 07312 0 06294 0 54681 DATACOLL 0 32675 0 20956 -0 03415 0 42424 1 00000 0 53143 0 26063 0 05417 0 30373 INSTRU 0 33306 0 03538 -0 05694 0 46204 0 53143 1 00000 0 18283 0 19415 0 42253 i RELIAB 0 13776 -o 03502 -0 08149 0 07312 0 26063 0 18283 1 00000 0 63950 0 04873 1 VALID 0 09976 -0 13094 0 03722 0 06294 0 05417 0 19415 0 63950 1 00000 0 06622 ;SAMPLE 0 44610 0 08158 0 02359 0 54681 0 30373 0 42253 0 04873 0 06622 1 00000 1TYPEANAL 0 17339 0 04305 -0 09659 0 25104 0 34039 0 40153 0 03925 -0 02008 . 0 26300 RESULTS 0 73648 0 13565 -o 08582 0 32862 0 25436 0 23683 0 10108 -o 02302 0 40973 |CONCLUS 0 36076 0 04859 -0 00212 0 16822 0 14690 0 17743 0 01301 -0 03446 0 31653 RESIMP 0 07095 -0 09568 0 15696 0 01942 0 00321 0 13403 0 05040 0 19340 -0 00801 1THEOIMP -0 02476 -0 05572 0 05472 -0 05023 -o 03063 0 05835 -0 04804 -0 03072 -0 10881 1FIELDIMP 0 38313 -0 02370 -0 00453 0 14404 0 08014 0 15498 0 00398 0 07122 0 1 1001 i FOUNDAT 0 C3977 -0 05950 0 14760 -0 12077 -0 12392 -0 09673 -0 06269 -0 04009 0 01246 | CHARAC -0 01842 0 10176 -0 04343 0 02754 0 16408 0 23935 0 00000 -0 04571 0 08806 (AGENCY -0 08893 -0 1 1203 -0 01494 -0 07580 -0 21577 -0 20022 -0 1 1805 -0 07549 -0 08600 1 PROGPLAN 0 09419 0 02932 -o 08062 0 16413 -0 05640 -0 09790 -0 00722 -0 08773 0 04821 1 INSTRUC -0 01332 0 04652 0 00571 -0 09464 0 01982 0 07422 0 26570 0 28533 -0 05080 IDISCIP -0 05545 -o 13063 0 09105 -0 08357 -0 00510 -0 06821 -0 07992 -o 051 1 1 -o 06669 TYPEANAL RESULTS CONCLUS RESIMP THEOIMP FIELDIMP FOUNDAT CHARAC AGENCY PROGPLAN REJACCEP 0 .03295 -0 .05811 0 .06238 0 .04701 0 .06720 0 .01669 0 .09745 -0 .08889 -0 .15903 0 .17678 YEAR 99 .OOOOO 99 .OOOOO 99 .00000 99 .OOOOO 99 .OOOOO 99 .OOOOO 99 .OOOOO 99 .OOOOO 99 .OOOOO 99 .OOOOO WORDSTIT 0 .00725 0 .13760 0 .05697 -0 .11212 -0 .07252 0 . 1434 1 -0 .04330 -0 .03846 0 .04234 0 . 13313 WORDSABS 0 .06257 0 .15843 0 .08628 0 .04133 -0 .02986 0 . 13867 -0 .03619 0 . 19874 0 .07920 -o .11535 ORIGINAL 0 .14085 0 .07851 0 .11734 0 .14469 0 .06895 0 .13130 0 .08998 -0 .03420 -0 .22591 0 .12696 ATTACH 0 .16100 -0 .05524 0 .06368 0 .14 160 0 .45881 -0 .09832 -0 .03522 0 .06024 0 .08843 0 .01420 PRESENT -o .00743 0 .01810 0 .06143 -0 .00217 0 .09706 -0 .04805 -0 .10511 -0 .15365 -0 .00846 0 .03694 VOICE -0 .10616 0 .15430 0 .00618 -0 .13203 -0 .11902 0 .04280 -0 .19085 0 .06073 -o .04736 0 .11093 JARGON 0 .10497 -0 .02650 -0 .10558 -0 .08522 0 .00554 -0 .12012 -0 .13730 0 .01373 0 .01663 -0 . 18644 DIRECTCI -0 .01589 -0 .02587 0 .00988 -0 .06760 -0 .03046 -0 .01756 0 .11313 0 . 148 16 -0 .10939 0 .06040 AUTHORCI -0 .05141 -0 .04743 -0 .02433 -0 .05924 0 .06913 -0 .02805 0 .23157 0 .10801 -0 .11892 0 .01819 FUNDING 0 .06739 -0 .11886 -0 .02193 0 .05408 -0 .05154 -0 .034 14 -0 .06727 -0 .07670 0 .12666 0 .06198 THEOCONC -0 .34702 -0. .21833 -0 . 14436 0 .04977 -0 .01095 -0 .09968 0 .08094 0 .07057 -0 .05678 0 .02303 ARCHHIST -0 .08679 -0 .12820 0 .07830 0 .01161 -0 .03319 -0 .12091 0 . 36095 -0 .09877 0. . 17218 • -0 . 13968 EMPIHARD 0 .40740 0. .45074 0 . 13877 -0. .15701 -0 .05781 0 .23793 -0 .19085 0 .02530 -0. .16436 0 .04652 ADMONPRE -0 .11296 -0. .10410 0 .10192 0 .03626 -0 .04320 -0. ,15738 0 .10148 0 .14 142 -0. ,00708 -0 .01818 METHO -0 .04883 -0. .18877 -0. .04876 0. ,25705 0. . 14938 -0. , 1 1477 0 .02339 -0 .00889 -0. ,16151 -0. .05030 DEDUCT -o. .01870 0. .08459 0. .06118 -0. .10358 0. .03365 -0. 021 18 -0 .00293 0 ,25038 -.0. 00551 -o. ,17704 INDUCT 0. .17070 -0. .04073 -0, ,05109 0. ,17729 0. .03072 0. 08902 -0 .04009 -o .06857 -0. 17614 0. ,15007 ARGUFLOW 0. .09422 -0. 02883 0. .03546 0. 09475 0. .OOOOO 0. 11215 -0 .12087 -0. ,05857 0. 03793 -o. 04385 STATERES 0. ,17339 0. 73648 0. ,36076 0. 07095 -o. 02476 0. 38313 0. .03977 -0. ,01842 -0. 08893 0. 09419 CUMULIT 0. .04305 0. 13565 0, ,04859 -0. 09568 -0. 05572 -0. 02370 -0. .05950 0. 10176 -o. 1 1203 0. 02932 DISCIPRE -0, .09659 -0. 08582 -o. 002 1 2 0. 15696 0. 05472 -0. 00453 0. , 14760 -0. 04343 -0. 01494 -0. 08062 DESIGN 0. .25104 0. 32862 0. 16822 0. 01942 -0. 05023 0. 14404 -0. 12077 0. 02754 -0. 07580 0. 16413 DATACOLL 0. 34039 0. 25436 0. 14690 0. 00321 -0. 03063 0. 08014 -0. 12392 0. 16408 -0. 21577 -0. 05640 INSTRU 0. 40153 0. 23683 0. 17743 0. 13403 0. 05835 0. 15498 -0. 09673 0. 23935 -0. 20022 -o. 09790 RELIAB 0. 03925 0. 10108 0. 01301 0. 05040 -0. 04804 0. 00398 -0. 06269 0. OOOOO -0. 1 1805 -0. 00722 VALID -0. 02008 -0. 02302 -0. 03446 0. 19340 -0. 03072 0. 07122 -0. 04009 -0. 0457 1 -0. 07549 -0. 08773 SAMPLE 0, 263.00 , o. 40973 0. 31653 -0. 00801 .. -Q_. 10881 0. 1 1001 0. 01246 . 0. 08806 -0. 08600 0. 04821 TYPEANAL 1. .OOOOO 0. 14702 0. 27263 :0. 05809 6. 25833 -0. 02597 -0. 10917 -o. 031 12 -o. 07860 0. 00388 RESULTS 0. .14702 1. OOOOO 0. 32276 0. 02595 0. 13018 0. 29479 0. 01019 0. 05037 -0. 18982 0. 04246 CONCLUS 0. 27263 0. 32276 1. OOOOO 0. 06965 0. 0751 1 0. 17861 0. 17098 -0. 02859 -0. 01860 0. 06801 RESIMP -0. 05809 0. 02595 0. 06965 1. OOOOO 0. 23694 0. 09417 -0. 02749 0. 12928 -0. 20271 -0. 09972 THEOIMP o. 25833 0. 13018 0. 0751 1 0. 23694 1. OOOOO -0. 02991 0. 05894 -0. 06720 -o. 1 1098 -o. 12898 FIELDIMP -0. 02597 0. 29479 0. 17861 0. 09417 -o. 02991 1. OOOOO -0. 00976 -0. 15858 -o. 01838 0. 19279 FOUNDAT -o. 10917 0. 01019 0. 17098 -0. 02749 0. 05894- -0. 00976 1. OOOOO -0. 17541 -0. 06974 -0. 12403 CHARAC -o. 031 12 0. 05037 -o. 02859 0. 12928 -0. 06720 -o. 15858 -o. 17541 1. OOOOO -o. 20184 -o. 38891 AGENCY -0. 07860 -0. 18982 -0. 01860 -0. 20271 -0. 1 1098 -0. 0-1838 -0. 06974 -o. 20184 1. OOOOO 0. 1 1678 PROGPLAN 0. 00388 0. 04246 0. 06801 -0. 09972 -0. 12898 0. 19279 -0. 12403 -0. 38891 0. 1 1678 1. OOOOO INSTRUC 0. 04083 -o. 02065 -0. 08732 0. 01978 -0. 00471 -0. 15414 -0. 03076 -0. 02104 -0. 16601 -0. 33227 DISCIP 0. 07982 0. 06064 -o. 03197 0. 07884 0. 31 127 0. 00622 -o. OOOOO -0. 16770 -0. 18464 -0. 31623 -ft-cc-INSTRUC DISCIP REJACCEP YEAR WORDSTIT WORDSABS ORIGINAL ATTACH PRESENT VOICE JARGON i DIRECTCI • AUTHORCI FUNDING . THEOCONC ARCHHIST EMPIHARD ADMONPRE METHO DEDUCT INDUCT ARGUFLOW | STATERES ' CUMULIT ;DISCIPRE 1 DESIGN 'DATACOLL t INSTRU !RELIAB I VALID 1 SAMPLE TYPEANAL RESULTS CONCLUS ! RESIMP THEOIMP : FIELDIMP '• FOUNDAT CHARAC AGENCY PROGPLAN INSTRUC DISCIP -0.07013 99.OOOOO -0.05395 -0.17288 -0.08994 -0.05352 0.06056 -0.03159 0.09072 -0.07800 -0.10101 0.09144 0.00571 -0.06581 0.05785 -0.08566 0.02681 0.04144 0.06091 0.02610 -0.01332 0.04652 0.00571 -0.09464 0.01982 0.07422 0.26570 0.28533 ^ 0 . 0 5 0 8 0 0.04083 -0.02065 -0.08732 0.01978 -0.00471 -O.15414 -0.03076 -0.02104 -0.16601 -0.33227 1.OOOOO ' -0.14898 -0.05590 99.OOOOO -0.13798 0.02786 0.02868 0. 15716 0.00687 -0.09618 0.09212 -0.10329 -0.00575 -0.08575 -0.01821 -0.05522 -0.04526 -0.07187 0. 15905 0.02239 -0.08031 -0.10401 -0.05545 -0.13063 0.09105 -0.08357 -0.00510 -0.06821 -0.07992 -0.05111 -0.06669_ "0.07982 0.06064 -0.03197 0.07884 0.31127 0.00622 -O.OOOOO -O.16770 -0.18464 -O.31623 -0.14898 1.OOOOO REJACCEP WORDSTIT WORDSABS ORIGINAL ATTACH PRESENT VOICE JARGON DIRECTCI REJACCEP 1 00000 -0 .09250 0 . 14249 0 . 13463 99 .00000 -0 .04619 -0 .07310 -0 . 24045 0 .07459 YEAR 99 00000 99 .00000 99 .00000 99 .00000 99 .00000 99 .00000 99 .00000 99 .00000 99 .00000 WORDSTIT -0 09250 1 .00000 0 .28805 -0 .04904 99 .00000 -o .05018 0 .08925 -0 .08486 -0 .06535 WORDSABS 0 14249 0 .28805 1 00000 -0 .03993 99 .00000 -o .14066 -0 .10398 -o .20703 0 .23441 ORIGINAL 0 13463 i -o 04904 -0 03993 1 00000 99 .00000 0 .01222 -0 .08369 -0 12944 0 .04010 ATTACH 99 00000 99 00000 99 00000 99 00000 1 .00000 99 .00000 99 00000 99 00000 99 00000 PRESENT -0 04619 -0 05018 -0 14066 0 01222 99 .00000 1 .00000 0 03878 0 1641 1 0 03824 VOICE -o 07310 0 08925 -0 10398 -0 08369 99 .00000 0 03878 1 00000 0 10953 -0 01628 JARGON -0 24045 -0 08486 -0 20703 -0 12944 99 .00000 0 1641 1 0 10953 1 00000 -0 1 186 1 DIRECTCI 0 07459 -0 0S535 0 23441 0 04010 99 .00000 0 03824 -0 01628 -0 1 186 1 1 00000 AUTHORCI 0 14035 ' -o 12357 0 10759 0 04894 99 .00000 0 1 1884 0 00763 0 00575 0 70258 FUNDING 6 06035 -0 02500 -0 01089 -0 04274 99 00000 0 02200 0 08369 0 01438 • 0 01094 THEOCONC -0 12419 0 01202 -0 07852 0 09512 99 00000 0 10072 -0 00086 -0 28363 0 06834 ARCHHIST 0 15346 I -o 05543 0 05132 0 06202 99 00000 -0 07884 -0 08168 -0 081 17 -o 1 1464 EMPIHARD 0 1 1728 0 08166 0 02441 -0 01378 99 00000 -o 04074 0 00710 0 2551 1 0 03919 ADMONPRE -0 00374 ! -o 07171 -0 04839 -0 08269 99 00000 -0 07884 -0 15625 -o 18553 -0 02205 METHO 0 15581 i -o 0361 1 -0 00585 0 07222 99 00000 0 02869 -0 13852 -0 07426 -0 07958 DEDUCT 0 1 1368 -0 09027 0 04423 -0 00274 99 00000 0 01752 0 00801 -0 07228 0 30525 INDUCT 0 19001 ! 0 09438 0 06897 0 13662 99 00000 0 02983 -0 06088 -0 07441 -0 04265 ARGUFLOW -0 03569 ' -o 06515 -0 06561 -o 15480 99 00000 0 36109 0 07571 0 50153 0 06639 STATERES 0 07861 -0 07403 -0 09586 0 09422 99 00000 0 01894 0 10251 0 26530 -o 03349 CUMULIT -0 02199 0 02148 0 00002 0 10052 99 00000 -0 03554 0 10180 0 07635 0 23627 DISCIPRE 0 13294 -0 08324 0 02237 0 08688 99 00000 0 00666 -0 11033 -o 04682 0 08866 DESIGN 0 10198 0 05560 0 03690 -0 04970 99 00000 0 00158 -0 09676 0 26019 0 02827 DATACOLL 0 27161 0 01881 0 30092 -0 07363 99 00000 -0 06347 -0 16171 0 10043 0 18353 INSTRU 0 21039 -0 09681 0 19430 0 06338 99 00000 0 06362 -0 04578 0 06461 0 25932 RELIAB -0 00332 -0 00344 0 23199 -0 1 1607 99 00000 0 1 1359 0 01574 0 01028 0 06255 VALID 0 14635 0 01950 0 27798 -o 05087 99 00000 0 04393 -0 09953 0 01585 -o 01 145 r SAMPLE TYPEANAL olo' 19112 24542 0 00034 0 15159 0 04590 99 00000 -o 08490 0 02050 0 19050 0 03393 olo' d 04706 0 13888 0 13776 99 00000 " -o 02189 0 08783 0" 15252 0 1 1 203 RESULTS o 08036 0 09762 0 07426 0 12015 99 00000 -0 05718 0 10696 0 14002 -0 01568 CONCLUS 0 01 108 -0 16349 -0 00822 -0 04237 99 00000 -o 03726 0 17874 0 01 170 0 01406 RESIMP THEOIMP FIELDIMP 0 00249 -0 10174 0 02710 0 01650 99 00000 0 05245 0 01465 -0 12187 0 12553 0 03915 -o 06999 0 09159 0 06828 99 00000 0 1 1827 -0 00782 -0 05107 0 23058 0 00527 -0 19134 0 07243 0 1 1646 99 00000 0 04303 0 01000 -0 04083 0 01656 FOUNDAT CHARAC 0 3061 5 0 01565 -o. 07547 0. 09407 99 00000 0 001 15 0 01533 -o 04194 -0 02264 0 19236 ' -0. 07223 0 21214 0 05675 99 00000 -0 05741 -0 02166 0 01238 0 24340 AGENCY -0 05075 0. 13138 0. 00935 0. 13453 99 00000 0 09835 0 06548 -o 13835 -0 16499 PROGPLAN INSTRUC DISCIP -0 02536 0. 17599 0. 06831 0. 0447 1 99 00000 -0 03297 0 10241 -0 05852 -0 09600 -0 17192 -0. 20338 -0. 1 1266 -0. 08047 99 00000 0 04143 0 00276 0 17152 0. 06865 -0 0378 1 0. 09630 0. 08005 -0. 04819 99 00000 -0 04863 -0 10071 -0 06307 -0. 09678 XI XI fD Cu o o ft ft fD 1—1 CD rt H -rt O 0 > s CD rt H H -X • O l-h > tr < cn CDrt i-i ft H> DJ CD n c r rt cn fD cn h-1 VD > 0 0 cn o cn O n H> CD rt fD Cu On O AUTHORCI FUNDING THEOCONC ARCHHIST EMPIHARD ADMONPRE METHO DEDUCT INDUCT ARGUFLOW REJACCEP 0 14035 0 06035 -0 12419 0 15346 0 1 1728 -0 00374 0 15581 0 1 1368 0 19001 -0 03569 YEAR 99 OOOOO 99 OOOOO 99 OOOOO 99 OOOOO 99 OOOOO 99 OOOOO 99 OOOOO 99 00000 99 OOOOO 99 OOOOO WORDSTIT -0 12357 -0 02500 0 01202 -0 05543 0 08166 -0 07171 -0 03611 -0 09027 0 09438 -0 06515 WORDSABS 0 10759 -0 01089 -0 07852 0 05132 0 02441 -0 04839 -0 00585 0 04423 0 06897 -0 06561 ORIGINAL 0 04894 -0 04274 0 09512 0 06202 -o 01378 -0 08269 0 07222 -o 00274 0 13662 -o 15480 ATTACH 99 OOOOO 99 OOOOO 99 OOOOO 99 OOOOO 99 OOOOO 99 OOOOO 99 OOOOO 99 OOOOO 99 OOOOO 99 OOOOO PRESENT 0 1 1884 0 02200 0 10072 -0 07884 -0 04074 -o 07884 0 02869 0 01752 0 02983 0 36109 VOICE 0 00763 0 08369 -0 00086 -0 08168 0 00710 -0 15625 -0 13852 0 00801 -0 06088 0 07571 • JARGON 0 00575 0 01438 -0 28363 -0 081 17 0 2551 1 -0 18553 -0 07426 -o 07228 -0 07441 0 50153 DIRECTCI 0 70258 0 01094 0 06834 -0 1 1464 0 03919 -0 02205 -0 07958 0 30525 -0 04265 0 06639 AUTHORCI 1 OOOOO -0 01707 0 14068 -0 09085 0 01 162 0 06332 -0 02164 0 34651 -0 05617 0 16526 FUNDING -0 01707 1 OOOOO -0 09512 -0 06202 0 01378 -0 062O2 0 05416 0 00274 -O 13662 -0 12942 . THEOCONC 0 14068 -0 09512 1 OOOOO 0 03633 -0 61084 0 03633 -0 15510 0 20075 -0 00928 -0 14617 ARCHHIST -0 09085 -0 06202 0 03633 1 OOOOO -0 30000 -0 05000 0 24744 -0 15252 0 22509 -0 00409 EMPIHARD 0 01 162 0 01378 -0 61084 -0 30000 1 OOOOO -0 30000 -0 34933 -0 04863 O 05428 0 13776 ADMONPRE 0 06332 -0 06202 0 03633 -0 05000 -0 30000 1 OOOOO 0 09461 -0 05968 -0 17720 -0 04706 METHO -0 02164 0 05416 -o 15510 0 24744 -0 34933 0 09461 1 OOOOO -0 09652 0 02788 0 05777 ; DEDUCT ' INDUCT 0 34651 0 00274 0 20075 -0 15252 -0 04863 -0 05968 -0 09652 1 OOOOO -0 54053 0 05020 -0 05617 -0 13662 -o 00928 0 22509 0 05428 -0 17720 0 02788 -0 54053 1 OOOOO 0 04723 ARGUFLOW 0 16526 -0 12942 -0 14617 -0 00409 0 13776 -0 04706 0 05777 0 05020 0 04723 1 OOOOO STATERES -0 01 144 -0 03603 -0 27055 0 15863 0 30535 -0 26365 -0 061 14 0 02963 0 07233 0 24145 CUMULIT o 25099 0 07539 0 04171 -0 08104 -0 15533 0 13169 0 04497 0 23243 -0 20570 0 04124 DISCIPRE 0 17764 -0 15843 0 12074 0 13185 -0 07417 0 04532 -0 07317 -0 02241 0 18549 0 09223 DESIGN 0 03309 -0 09940 -0 41727 -0 02003 0 50754 -0 26045 -0 00583 -0 02126 0 15352 •0 27381 DATACOLL 0 13210 0 04331 -0 33322 -0 03841 0 33990 -o 14842 ' 0 08336 0 08707 0 13430 0 22491 INSTRU 0 10919 -0 12063 -0 35969 -0 16649 0 47804 -0 16649 -0 16364 0 07980 0 17920 0 23426 RELIAB ' o 01058 -0 05498 -0 1 1809 -0 04433 0 14776 -0 04433 -0 05162 0 00196 0 05095 0 18683 VALID -0 03613 -0 06783 -0 1 1092 -0 05469 0 15039 -0 05469 -0 06368 -0 05258 0 19119 0 08951 SAMPLE 0 041 15 0 03977 -0 44985 0 01480 0 44566 -0 19241 -0 01293 0 06576 0 087 19 0 26617 TYPEANAL 0 02041 -0 1 1609 -0 31704 -0 10233 0 40517 -0 12854 -0 14968 0 05975 o 1 3686 0 2 1360 RESULTS -0 08088 -0 12015 -0 23106 0 09819 0 29812 -0 15260 -0 14524 0 05132 0 07575 0 15969 CONCLUS -0 03574 0 12712 0 01379 0 01 139 -0 05314 0 04555 -0 05636 -0 02870 -0 02836 0 05825 RESIMP 0 09889 -0 07425 0 15626 0 03326 -0 20399 0 03326 0 12006 0 06440 -0 04938 -0 04872 THEOIMP 0 26066 0 09104 0 17998 0 04129 0 01223 -0 05505 -0 06410 0 11316 0 08173 0 09010 FIELDIMP 0 07819 -0 08249 0 04832 -0 05281 -0 08867 0 07042 -0 06150 0 04047 0 06070 0 10661 FOUNDAT 0 14552 0 00724 0 15401 0 16919 -0 01945 -0 07584 0 01868 0 09362 0 14194 0 01886 CHARAC 0 22545 0 07566 -0 09602 -0 07244 0 25925 -0 15250 -0 10765 0 25687 0 09604 0 05304 AGENCY -0 13438 0 10090 0 17074 0 17626 -0 22898 -0 10846 -0 04342 -0 07912 0 05195 -0 14868 PROGPLAN -0 08217 0 01788 -0 09603 -0 18744 0 05287 0 1 1535 -0 08605 -o 17020 0 06802 -0 00914 INSTRUC -0 04215 0 00537 -0 091 13 -0 1 1677 0 06632 -0 02595 0 .02266 0 05335 -0 17896 0 06849 DISCIP -0 .09787 -0 .03614 -0 01 176 0 20882 -0 12950 0 10684 0 24317 -o 21513 -0 01209 0 00874 i H Ui STATERES CUMULIT DISCIPRE DESIGN DATACOLL INSTRU RELIAB VALID SAMPLE REJACCEP 0 07861 -0 02199 0 13294 0 10198 0 27161 0 21039 -0 00332 0 14635 0 19112 YEAR 99 00000 99 00000 99 00000 99 00000 99 00000 99 00000 99 00000 99 00000 99 00000 WORDSTIT -0 07403 0 02148 -0 08324 0 05560 0 01881 -0 09681 -0 00344 0 01950 0 00034 WORDSABS -0 09586 0 00002 0 02237 0 03690 0 30092 0 19430 0 23199 0 27798 0 15159 ORIGINAL 0 09422 0 10052 0 08688 -0 04970 -0 07363 0 06338 -0 1 1607 -0 05087 0 C4590 ATTACH 99 00000 99 00000 99 00000 99 00000 99 00000 99 00000 99 00000 99 00000 99 00000 PRESENT 0 01894 -0 03554 0 00666 0 00158 -0 06347 0 06362 0 1 1359 0 04393 -0 08490 VOICE 0 10251 0 10180 -0 1 1033 -0 09676 -0 16171 -0 04578 0 01574 -0 09953 0 C2050 JARGON 0 26530 0 07635 -0 04682 0 26019 0 10043 0 06461 0 01028 0 01585 0 19050 DIRECTCI -0 03349 0 23627 0 08866 0 02827 0 18353 0 25932 0 06255 -0 01 145 0 03393 AUTHORCI -0 01 144 0 25099 0 17764 0 03309 0 13210 0 10919 0 01058 -0 03613 0 04 1 15 FUNDING -0 03603 0 07539 -0 15843 -0 09940 0 04331 -0 12063 -0 05498 -0 06783 0 03977 THEOCONC -0 27055 0 04171 0 12074 -0 41727 -0 33322 -0 35969 -0 1 1809 -0 1 1092 -0 44985 1 ARCHHIST 0 15863 -0 08104 0 13185 -0 02003 -0 03841 -0 16649 -0 04433 -0 05469 0 01480 IEMPIHARD 0 30535 -0 15533 -0 07417 0 50754 0 33990 0 47804 0 14776 0 15039 0 44566 j ADMONPRE -0 26365 0 13169 0 04532 -0 26045 -0 14842 -0 16649 -0 04433 -0 05469 -0 19241 IMETHO -0 061 14 0 04497 -0 07317 -0 00583 0 08336 -o 16364 -0 05162 -o 06368 -0 01293 IDEDUCT 0 02963 0 23243 -0 02241 -0 02126 0 08707 0 07980 0 00196 -0 05258 0 06576 jINDUCT 0 07233 -0 20570 0 18549 0 15352 0 13430 0 17920 0 05095 0 19119 0 08719 , ARGUFLOW 0 14145 0 04124 0 09223 0 27381 0 22491 0 23426 0 18683 0 08951 0 26617 STATERES 1 00000 0 12878 0 01086 0 40465 0 25459 0 22194 0 05745 0 04842 0 351 19 CUMULIT 0 12878 1 00000 -0 25.962 -0 15424 0 00460 0 02471 -0 04041 0 06405 0 10795 DISCIPRE 0 01086 -0 25962 1 00000 0 01321 0 02878 -0 03165 -0 01096 -0 03324 -0 13722 DESIGN 0 40465 -0 15424 0 01321 1 00000 0 421 17 0 49009 0 17762 0 1 1778 0 35582 DATACOLL 0 25459 0 00460 0 02878 0 421 17 1 00000 0 53213 0 20434 0 16854 0 44 191 INSTRU 0 22194 0 02471 -o 03165 0 49009 0 53213 1 00000 0 15930 0 20125 0 33449 RELIAB 0 05745 -0 04041 -0 01096 0 17762 0 20434 0 15930 1 00000 0 67272 0 02843 VALID 0 04842 0 06405 -0 03324 0 1 1778 0 16854 0 20125 0 67272 1 00000 0 03035 SAMPLE 0 351 19 0 10795 -0 13722 0 35582 0 44191 0 33449 0 02843 0 03035 1 00000 TYPEANAL 0 37268 0 22174 -0 09513 6' 18401 0 "36799" 0 40661 0 19584 0 30012 0 45825 RESULTS 0 72820 o 12810 -0 06375 0 29136 0 24865 0 23785 -0 00353 0 10359 0 32700 CONCLUS 0 48444 0 19149 -0 04974 0 07692 0 08272 0 02891 -0 01010 0 05916 0 04551 RESIMP 0 03805 0 01853 0 03892 -0 01066 -0 02323 -0 03662 0 07077 -0 00182 -0 16835 THEOIMP 0 13591 0 1 1571 -0 04537 0 01 103 0 24030 0 12159 0 17895 0 09784 0 04481 FIELDIMP 0 21712 0 08917 0 05803 -0 02822 0 04549 -0 03366 0 15955 0 07702 -0 02084 FOUNDAT 0 17493 -0 04846 0 10914 -0 02338 0 19844 -0 03443 -0 06724 -0 08295 -o odTfi CHARAC -0 02061 0. 04480 0 16463 0 14971 0 20157 0 29990 0 00676 0 06307 0 22684 AGENCY 0 11511 -0 17580 0 00447 -0 17384 -0 02367 -0 17343 -0 09616 -o 00185 -0 16455 PROGPLAN 0 05686 0 13328 -0 14199 -0 03178 -0 00126 0 07962 0 01278 0 01232 0 00320 INSTRUC -0 05566 0 00394 -o 1 1869 0 15597 -0 03262 0 07828 0 1 1 120 0 13304 -o 03265 DISCIP -0 1 1567 -0 00246 0 05443 -0 09730 -0 13873 -0 16203 -o 0861 1 -0 10623 -0 05175 TYPEANAL RESULTS CONCLUS RESIMP THEOIMP FIELDIMP FOUNDAT CHARAC AGENCY PROGPLAN REJACCEP 0 . 24542 ... 0 .08036 0 .01108 . . . 0 .00249 0 .03915 0 .00527 0 .30615 0 .19236 YEAR 99 .00000 99 .00000 99 .00000 99 .OOOOO 99 .OOOOO 99 .00.000 99 .00000 99 .OOOOO WORDSTIT 0 .04706 0 .09762 -b .16349 -0 .10174 -0 .06999 -0 . 19134 . 0 .01565 -0 .07223 WORDSABS 0 .13888 0 .07426 -0 .00822 0 .02710 0 .09 159 0 .07243 -0 .07547 0 .21214 ORIGINAL .. 0 .13776 0 .12015 -0 .04237 0 .01650 0 .06828 0 . 1 1646 0 .09407 0 .05675 ATTACH 99 .obooo 99 .00000 99 .OOOOO 99 .OOOOO 99 .OOOOO 99 .OOOOO 99 .OOOOO 99 .OOOOO PRESENT -0 .02189 -0 .05718 -0 .03726 0 .05245 0 .11827 0 .04303 0 .00115 -0 .05741 VOICE • .0 .08783 • - -0 .10696 ... 0 .17874 0 .01465 -o .00782 0 .01000 0 .01533 -0 .02166 JARGON 0 .15252 0 . 14002' 0 .01170 -6 . 12187 • -0 .05107 . -o 04083 -0 .04194 0 .01238 DIRECTCI 0 1 1 203 -0 01568 0. .01406 0 .12553 . 0 .23058 0 01656 -0 02264 0 .24340 AUTHORCI 0 0204 1 -0 08088 -0 .03574 0 .09889 0 .26066 0 07819 0 14552 0 22545 FUNDING -o 1 1609 -0 12015 0 .12712 -0 07425 0 .09104 -o 08249 0 00724 0 07566 THEOCONC -0 31704 -0 23106 0 01379 0 15626 0 1 7998 0 04832 . 0 15401 -0 09602 ARCHHIST -0 10233 0 09819 0 01 139 0 03326 0 04129 -0 05281 . 0 16919 -o 07244 EMPIHARD 0 40517 0 29812 -0 05314 . -o 20399 0 01223 -0 08867 -0 01945 0 25925 ADMONPRE -0 12854 -o. 15260. . 0 04555 • • . 0 03326 . -0 05505 0 07042 -0 07584 -0 15250 METHO -o 14968 -0 14524 -0 05636 ':'•• 0 12006 -0 06410: .. ^0 06150 ... 0 01868 -o 10765 DEDUCT 0 05975 0 05192 -o 02870 0 06440 0 1 1316 0 04047 0 09362 0 25687 INDUCT 0 13686 0 07575 -o 02836 -o 04938 0 08173 0 06070. 0 14194 0 09604 ARGUFLOW 0 21360 0 15969 0 05825 -b 04872 0 09010 ' 0 10661 0 0.1886 0 05304 STATERES 0 37268 0 72820 o 48444 •' ' b 03805 0 13591 0 2 1712 0 17493 -0 0206 1 CUMULIT 0 22174 0 12810 0 19149 0 01853 0 1 1571 0 08917 -0 04846 0 04480 DISCIPRE -0 09513 -0 06375 -0 04974 0 03892 -0 04537 0 05803 0 10914 0 16463 DESIGN b 18401 0 29136 0 07692 -0 01066 0 01 103 -0 02822 -0 02338 0 14971 DATACOLL b 36799 0 24865 0 08272 -0 02323 0 24030 0 04549 0 19844 0 20157 INSTRU 0 40661 0 23785 0 0289t -0 03662 0 12159 -0 03366 -0 03443 0 29990 RELIAB 0 19584 -0 00353 . ,-o. 01010 0 07077 0 17895 0 15955 . -0 06724 0 00676 VALID 0 3001 2 • b 10359 .0 05916 -0 00182 0 09784 0 07702 -o 08295 0 06307 SAMPLE 0 45825 '.' 0 32700 - o 04551 -0 16835 0 0448 1 -o 02084 -o 00777 0 22684 TYPEANAL 1. 00000 0. 45761 0. 08726 -0. 05396 o 06045 0. 08261 0. 08023 0. 24341 RESULTS 0. 45761 1. 00000 b. 34845 0. 07926 0. 12345 0. 08410 0. 07416 0. 06577 CONCLUS 0. 08726 0. 34845 1. OOOOO 0. 07423 0. 12537 0. 14434 -o. 02259 -o. 05383 RESIMP -0. 05396 0. 07926 0. 07423 1. OOOOO 0. 15196 0. 11710. -0. 06364 -0. 06898 THEOIMP 0. 06045 0. 12345 0. 12537 0. 15196 1. OOOOO 0. 16798 0. 18627 -0. 03568 FIELDIMP' 0. 08261 0. 08410 0. 14434 0. 1 1710 0. 16798 1. OOOOO 0. 04930 -0. 14229 FOUNDAT 0. 08023 0. 07416 -0. 02259 ' -o. 06364 0. 18627 ' 0. 04930 1. OOOOO -0. 1 1922 CHARAC 0. 24341 0. 06577 -0. 05383 -0. 06898 -0. 03568 -0. 14229 -0. 1 1922 1. OOOOO AGENCY -0. 12250 0. 10723 0. 06176 -0. 07937 0. 08956 0. 06365 0. 30058 . -0. 33082 PROGPLAN 0. 08493 -0. 00957 0. 15106 0. 02398 -o. 03969 0. 17515 -0. 23 133 -0. 12149 INSTRUC -0. 02817 -0. 03822 -0. 13298 -0: 10357 -0. 02857 -0. 04873 -0. 1 1354 -0. 02375 DISCIP -0. 15806 -o. 07991 -0. 04424 0. 1 1888 -o. 05079 -0. 12653 -0. 14733 -0. 24958 -o 99 0 O 0 99 0 0 -0 -0 -0 0 o o -o -o -0 -b o. -o. o. -0. o. -o. -o. -o. -0. -o. J<>--6. 0. o. .05075 .OOOOO .13138 .00935 .13453 .OOOOO .09835 .06548 . 13835 . 16499 13438 10090 17074 17626 22898 10846 04342 07912. 05195 14868 1151 1 17580 00447 17384 02367 17343 09616 00185 16455 12250 10723 06 176 0 07937 0.08956 0.06365 0.30058 O.33082 1.00000 0.03715 O.15481 O.15539 -0.02536 99.OOOOO 17599 .06831 .04471 99.OOOOO -0.03297 0. 1024 1 -0.05852 .09600 .08217 .01788 -0.09603 -0.18744 0.05287 1 1535 .08605 17020 .06802 ,00914 .05686 13328 14 199 -0.03178 -0.00126 .07962 .01278 .01232 .00320 .08493 .00957 15106 .02398 .03969 17515 .23133 12 149 .03715 .OOOOO 39849 .27589 0. 0. 0. -0. -0. 0. 0. -0. -o, ' 0. -0, 0. 0.  0. 0. 0. 0. b. -0. o. o. -0. o. -0. -o. -o. 1 -o. -0. Ln INSTRUC DISCIP REJACCEP -0 17192 -0 03781 YEAR 99 OOOOO 99. OOOOO WORDSTIT -0 20338 0 09630 WORDSABS -0 1 1266 0 08005 ORIGINAL -0 08047 -0 04819 ATTACH 99 OOOOO 99 OOOOO PRESENT 0 04143 -0 04863 VOICE 0 00276 -0 10071 JARGON 0 '7152 -0 06307 DIRECTCI 0 06865 -0 09678 AUTHORCI -0 04215 -0 09787 FUNDING O 00537 -0 03614 THEOCONC -0 091 13 -0 01 176 ARCHHIST -0 1 1677 0 20882 EMPIHARD 0 06632 -0 12950 ADMONPRE -0 02595 0 10684 METHO 0 02266 0 24317 DEDUCT 0 05335 -0 21513 INDUCT -0 17896 -0 01209 ARGUFLOW 0 06849 0 00874 STATERES -0 05566 -0 1 1567 CUMULIT 0 00394 -0 00246 I DISCIPRE -0 1 1869 0 05443 DESIGN 0 15597 -0 09730 j DATACOLL -0 03262 -0 13873 INSTRU 0 07828 -0 16203 ! RELIAB o 1 1 120 -0 0861 1 I VALID 0 13304 -o 10623 ! SAMPLE -0 03265 -o 05175 . TYPEANAL -0 02817 -0 15806 ! RESULTS -0 03822 -0 07991 CONCLUS -0 13298 -0 04424 : RESIMP • -0 10357 0 1 1888 \ THEOIMP -0 02857 -0 05079 ! FIELDIMP -0 04873 -0 12653 ' FOUNDAT -0 11354 -0 14733 CHARAC -0 02375 -0 24958 AGENCY -0 15481 -0 15539 PROGPLAN -0 .39849 -0 .27589 INSTRUC 1 .OOOOO -0 .12098 DISCIP -0 .12098 1 .OOOOO 

Cite

Citation Scheme:

        

Citations by CSL (citeproc-js)

Usage Statistics

Share

Embed

Customize your widget with the following options, then copy and paste the code below into the HTML of your page to embed this item in your website.
                        
                            <div id="ubcOpenCollectionsWidgetDisplay">
                            <script id="ubcOpenCollectionsWidget"
                            src="{[{embed.src}]}"
                            data-item="{[{embed.item}]}"
                            data-collection="{[{embed.collection}]}"
                            data-metadata="{[{embed.showMetadata}]}"
                            data-width="{[{embed.width}]}"
                            async >
                            </script>
                            </div>
                        
                    
IIIF logo Our image viewer uses the IIIF 2.0 standard. To load this item in other compatible viewers, use this url:
http://iiif.library.ubc.ca/presentation/dsp.831.1-0055836/manifest

Comment

Related Items