UBC Theses and Dissertations

UBC Theses Logo

UBC Theses and Dissertations

A citation analysis of "Adult education quarterly" 1971-1986 1987

You don't seem to have a PDF reader installed, try download the pdf

Item Metadata

Download

Media
UBC_1987_A8 K38_2.pdf
UBC_1987_A8 K38_2.pdf
UBC_1987_A8 K38_2.pdf [ 3.65MB ]
UBC_1987_A8 K38_2.pdf
Metadata
JSON: 1.0055771.json
JSON-LD: 1.0055771+ld.json
RDF/XML (Pretty): 1.0055771.xml
RDF/JSON: 1.0055771+rdf.json
Turtle: 1.0055771+rdf-turtle.txt
N-Triples: 1.0055771+rdf-ntriples.txt
Citation
1.0055771.ris

Full Text

A C I T A T I O N A N A L Y S I S OF " A D U L T E D U C A T I O N Q U A R T E R L Y " 1971-1986 by RICHARD O W E N K A V A N A G H B .S .N . , University of Victoria, 1983 A THESIS S U B M I T T E D I N P A R T I A L F U L F I L M E N T OF T H E R E Q U I R E M E N T S F O R T H E D E G R E E OF M A S T E R OF ARTS in T H E F A C U L T Y OF G R A D U A T E STUDIES Department of Administrative, Adult and Higher Education We accept this thesis as conforming to the required standard T H E U N I V E R S I T Y OF BRITISH C O L U M B I A Apri l 1987 ® Richard Owen Kavanagh, 1987 3 2 In presenting this thesis in partial fulfilment of the requirements for an advanced degree at The University of British Columbia, I agree that the Library shall make it freely available for reference and study. I further agree that permission for extensive copying of this thesis for scholar^ purposes may be granted by the Head of my Department or by his or her representatives. It is understood that copying or publication of this thesis for financial gain shall not be allowed without my written permission. Department of Administrative, Adult and Higher Education The University of British Columbia 2075 Wesbrook Place Vancouver, Canada V6T 1W5 Date: Apri l 1987 A B S T R A C T Adult education has long been described as an emerging discipline, but there has been little empirical study of its emergence. This study examined 'emergence' by monitoring that particular knowledge base which is unique to adult education. Studies concerned with the theory and practice of adult education are a quantifiable indicator of unique knowledge about adult education. Evidence that researchers in adult education increasingly cite the work of other researchers in adult education would support the contention that the body of knowledge in adult education is growing. The articles published in Adult Education Quarterly between 1971 and 1986 were analyzed using citation analysis methodology. The frequency of citation to previous adult education studies (primary literature) as opposed to citation of studies peripheral to an adult education context (secondary literature) was determined. Distinguishing between citation categories was carried out by analyzing each title cited. The phenomenon of concern in the cited article was interpreted from the words used in the title, and coded dichotomously as 'primary literature' or 'secondary literature'. Each coded item was then recorded under named authors; thus, the cited author was credited for total frequency cited along with the coded category of writing (author of primary literature or author of secondary literature). Reliability measures performed for intra-judge consistency (recoding data), and inter-judge agreement (independent coding of data) resulted in differences in coding of less than four percent for the former, and nine percent with the latter. Validity of the procedures used in coding cited authors was ii tested by comparing results obtained to a 'standard'. 'Independent experts' were asked to identify from a list of the twenty most cited authors from each four volume period, those who were "primarily known for their adult education activities." The study's coding outcome of these authors compared with the expert's 'standard' resulted in greater than 75 percent agreement. With 4700 citations classified, it was found that a rising percentage of citations were to the "authors of primary literature"; from 41 percent of all citations in the first half of the study period (1971-1978), to 46 percent in the last half (1979-1986). A further breakdown showed the percentage of citations to "primary literature journals" also increasing; from 31 percent of all journals cited in '1971-1978' to 39 percent in '1979-1986'. As the scope of literature analyzed was exclusively from one North American journal, results need to be regarded with this limitation in mind. However, the empirical evidence of an increasing 'primary literature' base in adult education research suggests emergence of the field. Implications for future research are discussed in light of this and previous studies. iii T A B L E O F C O N T E N T S A B S T R A C T ii LIST OF T A B L E S v LIST OF F I G U R E S vi C H A P T E R 1: P R O B L E M 1 Operationalizing Emergence 1 Purposes 5 Scope of the Study 6 Significance 8 Thesis Structure 9 C H A P T E R 2: R E V I E W OF T H E L I T E R A T U R E 10 Knowledge About Adult Education 10 Content Analysis 12 Citation Analysis 16 Rationale for Study 18 Hypotheses 22 C H A P T E R 3: M E T H O D O L O G Y 24 Data Collection 24 Data Analysis 31 Reliability and Validity 32 Reliability: Test Retest 33 Reliability: Reproducibility 34 Validity: Expert Standard 34 C H A P T E R 4: R E S U L T S 37 Reliability and Validity 37 Study Population 41 Cited Authors 42 Cited Journals 46 C H A P T E R 5: DISCUSSION, L IMITATIONS & IMPLICATIONS 51 Discussion 51 Limitations 58 Implications 59 R E F E R E N C E S 61 A P P E N D I X A 65 A P P E N D I X B 74 A P P E N D I X C 75 A P P E N D I X D 76 iv L I S T O F T A B L E S Table 1: Study Procedures Compared with Boshier and Pickard (1979) 18 Table 2: Steps Used to Code Eligible Citations 26 Table 3: Major Components of the Study 27 Table 4: Example of Author Coding Sheet 30 Table 5: Example of Journal Coding Sheet 30 Table 6: Procedures for Establishing Reliability and Validity 33 Table 7: Discrepancies in Intra-Judge Recoding 38 Table 8: Discrepancies Found by Validity Test 40 Table 9: Percent Citations to Authors of Primary Literature Volume 21-36 44 Table 10: Most Frequently Cited Authors: Four Periods 45 Table 11: Top Twenty Authors Ranked by Citation Value 46 Table 12: Percent Journal Citations to Primary Literature Journals 48 Table 13: Most Frequently Cited Journals 50 Table 14: Citations in the Adult Education Quarterly to Its Own Articles 52 v L I S T O F F I G U R E S Figure 1: Hypothesized Proportions of Citations as a Field Emerges 21 Figure 2: Proportions of Citation to Authors of Primary Literature 42 Figure 3: Proportion Cited Journals to Primary Literature Journals 47 Figure 4: Primary and Secondary Literature Found in Four Periods 57 vi CHAPTER 1 PROBLEM Adult education has long been described as an emerging discipline, but there has been little empirical study of its emergence. In part, this is due to a controversy within the 'community' of adult educators as to whether adult education can be a discipline unto itself. Proponents point to the establishment of adult education within universities, its rising professionalism, and the production of sophisticated research as evidence of emergence. Opponents claim that since adult education is without theory of its own, questions concerning emergence warrant no further investigation. This thesis was an attempt to quantitatively assess emergence of the field by studying the changing nature of citations found in articles appearing in Adult Education Quarterly (formerly Adult Education), the major journal of research and theory in North American adult education. Operationalizing Emergence A discipline is best represented by the body of knowledge used to guide practice. According to Verner et al. (1970), " A n academic discipline is defined as a body of systematic knowledge founded in theory and research" (p. 7). Knowledge is the quintessential element in all disciplines. Adult education is a relatively new field of study and has had to rely on knowledge obtained from 1 2 more established disciplines. However, the extent to which adult education has shucked this fledgling nature, won over practitioners from their original intellectual alignments, and created unique knowledge of its own is not clear. Unique knowledge distinguishes one discipline from another. In the past, concepts developed outside the field dominated knowledge which guided adult education practice. Recent speculation holds that this situation may be changing because of the increase in the numbers and advancement in the skills of adult education researchers. A community of researchers forms in order to share and control knowledge, knowledge developed through testing ideas in the field of practice. It is this evolution that Peters (1980) spoke of when he said, "Ultimately, we need strong theories to guide our research efforts, but first we need concepts, and clusters of concepts, to frame our thinking and to underpin our practice" (p. 121). There is a symbiotic relationship between the discipline and the field. The discipline studies problems in the field, and the field uses knowledge developed by the discipline (e.g., Boshier, 1980; Brunner & Verner, 1968). According to Verner (1960): The growing maturity of a field is represented by the changing character of its literature and by the attention it pays to it . . . . [Adult education] is reaching a point where research is beginning to be structured on prior research so that a consistent body of knowledge is being accumulated, (p. 171) The "consistent body of knowledge" that Verner refers to is research-based knowledge from an adult education context. The extent to which studies concerned with phenomena of the field are referenced in the research is an indicator of the awareness of adult education researchers of the unique 3 knowledge available. Evidence that increased use is being made by these researchers of unique knowledge would support an argument concerning 'emergence' in adult education. Establishing a quantitative measurement of the degree to which adult education research is, as Verner said, "structured on prior research" was the focus of this thesis. Research conducted by scholars in a field is based on questions they hold to be important, and pursued in manners they believe to be promising. Behavior of researchers in a field can be most readily observed through their research reports. These reports describe the problems, significance, approaches, and resources used in the research process. A n essential resource in each study is the review of previous studies related to the research question. The act of citing in research literature is the most common means of recognizing the contribution, or relevance, of the work of others. According to Garfield (1979) citations are "nothing more, or less, than a reflection of that community's work and interest" (p. 247). Thus, citing behavior reveals common connections in conceptualization of the field. Citation analysis has been used for many investigations of trends in and characteristics of literature. Researchers have attempted to identify patterns in the citing behavior of groups of authors, journals, and disciplines as a whole. Citation analysis as a research method has been used to address a variety of concerns: the eminence of scholars by the frequency with which they are cited (e.g., Aaronson, 1975; Garfield, 1979); the impact of articles by their connection through citation to the work of others (Garfield, 1972); the contribution of other 4 fields as determined by investigating where cited materials were originally developed (Miller, 1982); and interjournal citing behaviors (e.g., Myers, 1970; Smith & Caulley, 1981; Xhignesse & Osgood, 1967). Miller (1982) states, "The analysis of citations is valuable in identifying the core research literature in a particular field or discipline" (p. 798). Citation analysis is a research method capable of providing empirical answers to questions concerning the origin of the concepts used in a field's literature. The origins of the concepts used in scholarly literature are normally identified by the bibliography or reference section of an article. Citations have three components: the name(s) of the author(s), the title of the work, and publication information or a description of the context in which the material was produced (e.g., speech at conference with dates, personal communications). In this study all three of these components were used to establish whether a citation was considered "primary literature" or "secondary literature." In this section it has been argued that emergence of a discipline is linked to possession of unique knowledge, and that knowledge is unique because of the context in which it was produced. Thus, to find that studies produced in an adult education context represent an increasing proportion of citations found in an important journal would suggest that the field is indeed emerging. 5 Purposes This study examined emergence in adult education by identifying changes in the citing behavior of authors publishing in Adult Education Quarterly. Two categories of cited studies were designated, those by authors whose work is grounded in an adult education context, deemed 'primary literature', and those by authors who are not identified with an adult education context, deemed 'secondary literature'. For the purposes of this thesis these terms were defined as follows: 'Primary Literature' is any publication whose title explicitly identifies it as being concerned with concepts, processes, or data from an adult education context or whose publication information (e.g., name of journal, name of publisher, name of editor or sponsor) is explicitly identified with an adult education context or whose author(s) has previously been classified as writing primary literature (as above). 'Secondary Literature' is any publication whose title does not manifestly identify it as being concerned with an adult education context and whose publication information is not explicitly identified with an adult education context and whose author(s) has not previously been classified as writing 'primary literature'. 'Primary literature' contains studies authored by someone whose work is grounded in an adult education context. The result of these studies is knowledge which is unique to adult education, because the author(s) is writing from an adult education context. 'Secondary literature' contains studies by authors not manifestly connected to an adult education context, designed to relate to the problems of another field. Results meaningful to adult educators are only a by-product of this 6 research, and require reformulation to become relevant in an adult education context. These studies are deemed to be 'secondary literature' because their Findings require the additional step of reformulation before any understanding, control, or prediction which their concepts may offer can be of benefit in an adult education context. In this attempt to quantify 'emergence' in adult education a citation analysis of Adult Education Quarterly for the years 1971-1986 was performed for the purposes of: 1. Determining proportions of citations to "authors of primary literature" and "authors of secondary literature", 2. Determining proportions of citations to "authors of primary literature" and "authors of secondary literature" who are most frequently cited, 3. Determining proportions in a subset of the data collected, that of the journals deemed "primary literature journals" and "secondary literature journals." Scope of the Study It is widely assumed that a relationship exists between the doing and reporting of research and the development of the discipline. In this study it was necessary to investigate adult education research, but decisions had to be made on where to find representative research, and how much to analyze. Allcorn (1985) was confronted with a similar situation and recognized 7 that any effort to analyze the information published on adult education must be restricted to a manageable size (p. 12). Three general categories of adult education literature were identified by Grabowski (1974): published material in the form of books and pamphlets; fugitive documents; and periodicals (p. 3). To find literature containing research reports from adult education over time required a collection that was both archival (reports past research) and current (up to date). Journals and conference proceedings met both of these criteria. Nelson (1972), in his study on the fate of conference papers, found that, "The two most important media of scientific communication, national meetings and journal articles, are closely related" (p. 4). The results of his study showed that 33 percent of conference papers were published in journals within two years of presentation at an American Educational Research Association meeting (another seven percent were accepted but published later, bringing the total to 40 percent). It appeared that journals were a good medium for monitoring the forefront of research reported at meetings. Adult education's oldest, most highly circulated, and most prominent research and theory periodical in North America is said to be Adult Education Quarterly (e.g., Brookfield, 1982; Griffith & Roberts, 1981; Long, 1977; Verner, 1960). The American Association for Adult Education began publishing the Journal of Adult Education in February, 1929. Although the Journal has gone through a number of name changes, it has been a consistent communication channel for adult educators for fifty-seven years. Presently this journal's coverpage states, "Adult Education Quarterly is a refereed journal committed to 8 the dissemination of research and theory in adult and continuing education." For the purposes of this study the articles contained in Adult Education Quarterly (formerly Adult Education, The Journal of Adult Education and Adult Education Journal) were deemed to be representative of the typical research literature of the field in North America, both past and present. Studies cited in another's work are an observable indicator of the influence one author's work has on the work of others. Specifically, the questions posed were: 1. To what extent were references cited in articles published in Adult Education Quarterly attributable to "authors of primary literature"? 2. What changes, if any, occur in the proportion of authorships to 'primary literature' and 'secondary literature' cited over time. The study population consisted of citations from the articles published in the Adult Education Quarterly. The most recent sixteen year period (1971-1986) was studied because of a shift in the Journal's editorial policy in 1966 when the 'research and theory' emphasis was proclaimed (Griffith, 1966). Data collection commenced with Volume 21 (1971) and continued through Volume 36 (1986) of the Journal. Significance The results of this study help clarify questions concerning creation of knowledge in adult education and the influence of 'primary literature' and 9 'secondary literature' on this development. The empirical evidence this study sought to find was that of a pattern in citations from Adult Education Quarterly's articles over time (one of change or stability would be equally significant). The ability to identify a pattern in adult education research with empirical data (rather than by intuition) allows for analysis, interpretation, and greater understanding of questions concerning emergence of the adult education field. As Liveright (1964) emphasized, "More important . . . than whether or not adult education is a true discipline is an examination of the kind of discipline it is likely to become as it moves toward professional status" (p. 89). Thesis Structure Chapter 2 contains a review of studies and methodologies previously used to analyze the knowledge base in adult education. The findings of these investigations were used to form hypotheses for this study. In Chapter 3 the procedures for data collection and analysis are discussed, along with measures taken to examine the reliability and validity of procedures used. Chapter 4 describes the findings and Chapter 5 presents discussion concerning results, limitations, and implications for future research. CHAPTER 2 REVIEW OF T H E LITERATURE Characterizing the knowledge that lies behind adult education practice has been attempted by investigating the literature of the field. One route has been to analyze the research literature available: a general review, qualitative review, interpretive review, or content analysis. Another more indirect approach has been through studying the citations found in the research literature. Earlier studies concerned with knowledge about adult education are reviewed in this chapter. Knowledge About Adult Education Adult education researchers have tried to describe, define, and organize a body of knowledge in order to guide practitioners. Sork (1982) in his review of research literature defines meta-research as ". . . systematic study of the processes and products of inquiry which characterize a discipline or field of study, or, more simply, research on research" (p. 1). Basing his typology on an analysis of publications about adult education research, Sork suggested there were six types of meta-research: 1. Inventories/registers of research, 2. General reviews, 3. Interpretive reviews on specific topics, 4. Research agendas, 5. Focused critiques of methodology, 6. Frameworks or Paradigms for Understanding and Improving Research. 10 11 These categories offer a framework for reviewing knowledge about adult education. Each characterizes the knowledge base in a particular way and provides clues as to adult education's identity. The focus of this study was on the relative use of 'primary literature' and 'secondary literature' as a resource in adult education research. Thus, studies from categories 2 and 6 were most relevant. Sork's category 2, "general reviews of adult education literature", contains studies which attempt to provide insights into the direction researchers in the field are taking, the progress being made, and the needs for further inquiry. However, adult education literature reflects the diversity of the field of practice. It does not exist in a central place, is not disseminated through predictable channels, and has become too vast to be gathered together for a comprehensible summary (e.g., Allcorn, 1985; Verner, 1960). Beals and Brody (1941) attempted to summarize the literature over 45 years ago, and noted then that they had "been thwarted by the range in importance of the topics treated, the. extent and character of the relevant literature, and the limitations of our knowledge and judgement" (p. xvi). To analyze the adult education field is too massive a task for any single study. Long (1983a) observed that: The quantit}' of research concerning the education of adults is expanding at such a rapid rate that not since 1959 [Brunner et al.] has an effort been made to provide an extensive and intensive organized review of the broad field, (p. 19) Comprehensive reviews of adult education research were rarely available, and when reviews were published they tended to look at aspects of the knowledge base such as research methods (e.g., Merriam & Simpson, 1984), or consisted of 12 more superficial over-views of the field (e.g., Draper, 1985; Verner, 1968). Category 6 studies, "frameworks or paradigms for understanding and improving research", focuses on the task of reviewing what is known about the character of the body of knowledge in adult education. Two research methods within this category that have been used in adult education to characterize the knowledge base are content analysis and citation analysis. The Adult Education Quarterly has been the subject of four content analyses, and one citation analysis. Content Analysis One of the first content analyses of adult education literature was done by Dickinson and Rusnell (1971). They recognized that professional journals were at the forefront of an area of study. They chose to do an analysis of articles in the first twenty volumes of Adult Education, "to ascertain trends and patterns in the contents . . . as indications of the development of the discipline of adult education" (p. 177). Articles with named authors were reviewed for "the use of space, type and subject of articles, citation practices, and authorship as well as a detailed analysis of research articles . . . " (p. 178). They recognized that their study covered a transition period in the Journal with the last few years being under an editor who emphasized research and theory. They reported that research methods were becoming more sophisticated and that more complex statistical techniques were being used. They found an increasing concern for fundamental principles (foundations of practice), with a corresponding lesser emphasis on current forms (e.g., program description) within the subject matter of 13 the Journal. They felt the results of their study supported the notion of adult education emerging as a distinct field of study. Several years later, Long and Agyekum (1974) approached Adult Education with slightly different questions from those above. Their study was not intended as a replication of Dickinson and Rusnell, but examined the possible relationship between different editors and content, research design, article length and topics, and articles based on the author's dissertation. They also examined the relationship between the author's university and the number of articles published. The nine year period of their study overlapped with the period covered by Dickinson and Rusnell, so three further volumes were analyzed. In their results they asserted that the Journal was responding to the discipline's needs. They arrived at this conclusion by noting changes in the subjects studied (e.g., the importance placed on adult learning in the 60's was being overshadowed by an emphasis on program planning in the 70's), and the changes in university affiliation of authors, which Long and Agykum speculated reflected the changing fortunes of various university programs. They confirmed Dickinson and RusnelPs conclusions concerning the increasing sophistication of research designs and analytical techniques. Peters and Banks' (1982) period of analysis dovetailed with the end of Long and Agyekum's study, and continued on to the end of 1980 in Adult Education Quarterly. They compared their findings with Dickinson and Rusnell's results on content trends and methods. They concluded that the research designs they found were following the same trends as had been reported in the earlier 14 studies (increasing sophistication of research methods with more experimental designs). As well, they reported the content trends were continuing to emphasize the foundations of the field (formal philosophy of adult education), while declining proportions of articles were about personal beliefs and opinions of the authors. The latest content analysis completed on adult education literature was by Allcorn (1985). Adult Education Quarterly, Lifelong Learning, and 18 articles discussing possible futures for adult education (collected from various journals) were analyzed to find: . . . content trends that indicate the extent and direction of the development of adult education's knowledge base, [and] an effort is made to discern whether trends represent a purposeful development of knowledge for the field, (p. 12) Allcorn's effort to discern trends used Knowles' typology of "sequential research needs" as a comparison device. Knowles (1973) suggested that adult educators could better understand the research needs of their field if they contrasted them with those of other fields. Using his own memories from his days as a social worker Knowles contrasted the research efforts of the two fields he knew. He outlined six phases he thought would correspond with the "sequential research needs" of fields at different levels of development: 1. Definition of the field 2. Differentiation of the field 3. Standard-setting 4. Technological refinement 5. Respectability and justification 6. Understanding of the dynamics of the field 15 Allcorn (1985) made use of Knowles' phases to evaluate selected adult education literature. He combined the findings of Dickinson and Rusnell (1971); Long and Agyekum (1974); Long (1977); Boshier and Pickard (1979); and his own content analysis, to compare 'trends' with Knowles' phases. The period under study in Adult Education Quarterly was not stated, although the content categories were developed from the first 20 volumes of that journal (1950-70). Lifelong Learning, was analyzed for the period 1977-1983, and the 18 articles on adult education futures were dated from 1952 to 1982. Allcorn concluded that the content trends from previous analyses and his own, 'fit' with Knowles' six phases. He stated that findings demonstrated research was beginning to appear that was identified with Knowles' phases 4, 5, and 6 with a concurrent disappearance of research aimed at definition of the field. He concludes, "The findings . . . indicate adult education is developing a unique field of knowledge; however, some gaps in the knowledge base still exist" (p. 12). Content analyses of selected adult education literature have identified the subject matter and methodological trends in the field. Results have listed the themes studied, compared their strength as demonstrated by frequency of occurrence, and monitored changes over time. Research studies were surveyed for their use of methods and techniques, and changes indicating use of more sophisticated research designs were noted. Comparison was attempted between the type of questioning made in research activities of adult education and those thought to denote the level of questioning of emerging disciplines, with the claim that adult education is increasingly asking more sophisticated research questions. 16 Citation Analysis Citation analysis is used to identify formal links between already existing knowledge (cited reports) and the extension of knowledge (the citing document). More to the point, analysis of citing behavior allows meaning to be attributed to the patterns of linkage found. Dickinson and Rusnell (1971) counted the number of citations made by authors publishing in Adult Education, finding a trend in citing that rose sharply from 0.7 per article in Volumes 1 to 5, to 13.2 citations per article in Volumes 16 to 20. The total number of citations to other articles in Adult Education, however, did not increase in proportion to the increase in the total number of citations referenced. These results were limited to the above mentioned figures and no further interpretation of citation patterns was reported. The single citation analysis that was attempted on adult education materials was by Boshier and Pickard (1979) who claimed that: The citation count is a quantitative measure employed to establish the extent to which researchers utilize indigenous (primary literature) concepts and processes, as opposed to those of other disciplines (secondary literature), (p. 35) They sought answers to three questions concerning (1) the number and percentage of primary literature citations, (2) the impact of individual scholars, and (3) the impact of individual items, by most frequent citation. They hypothesized that, "as the discipline emerges, researchers will place increasing reliance on concepts and processes developed by other scholars in the discipline" (p. 35). This pattern would be affirmed by an increasing ratio of citations to 17 primary adult education literature, for which they provided the following definition: Primary Adult Education Literature describes concepts, processes and data clearly identified with adult education. It is usually published in adult education journals, books or monographs and is produced by people for whom adult education is their primary professional concern, (p. 36) Boshier and Pickard (p. 36) analyzed, "citations made by authors of all research, theory and 'forum' articles published in Adult Education during the ten-year period from 1968-1977." 'Primary' literature was that deemed so by seven of nine judges for each citation. Ninety percent of the citations (2047 out of 2247) were successfully classified for the volumes under study, which they found, "reveals a general increase in the percentage of primary literature citations. . . ." They listed the twelve most frequently cited authors, and eleven most frequently cited individual items concluding that most were authored by well known adult educators. Their results led them to conclude that: . researchers publishing in Adult Education wrote articles which showed an increasing tendency to cite primary adult education literature, (p. 47) in view of the major role of Adult Education, it is contended that this study has demonstrated that adult education is creating its own body of knowledge, (p. 48) In the study by Dickinson and Rusnell (1971), only citations to Adult Education as a percentage of total citations were presented. Analysis showed, through four 5-year periods that this percentage was not rising. Boshier and Pickard (1979), in their classification of citations for the nine volumes studied, found a general increase in the proportion of citations to "primary adult 18 education literature." Although this increase was never stated in empirical terms in the discussion or conclusions of the study, the data shows the increase averages 4.8 percent per year for the nine year period. Along with their claim of an "even development" (in the increase) this would suggest a substantive change occurring in the structure of adult education research literature. However, using the data reported in Table 1 of their article it was possible to aggregate data into two 4-year intervals (Volumes 20-23 and Volumes 24-27). Analysis of this recombined data results in an increase in the proportion of the 'primary literature' category of 6.2 percent between the first and second 4-year intervals. The result of the single citation analysis published concerning adult education materials reveals an increasing trend toward citing "primary adult education literature." The magnitude and the rate of this change as suggested by the Boshier and Pickard study, however, are limited to the nine years (Volumes 19-27) investigated in the Journal. Rationale for Study This thesis, in some respects, replicated the earlier work of Boshier and Pickard (1979); a comparison of the studies is presented in Table 1. 19 Table 1 Study Procedures Compared with Boshier and Pickard (1979) Procedures Boshier and Pickard Study Methodology Citation Analysis Citation Analysis Data Set Adult Education Quarterly Adult Education Quarterly (Volumes 19-27) (Volumes 21-36) Questions Frequency of Primary/Secondary Items Cited Computation of Primary/Secondary Citation Values (cited authors times authorships to them) Most Frequently Cited Individual Authors Proportions Primary/Secondary of Most Cited Authors Most Frequently Cited Individual Items Proportions Primary/Secondary of Most Cited Journals Coding Reliability Nine Judges Independent Consistency Check of Coded Item Researcher 1) Recoding Consistency Check of Coded Authors 2) Independent Judge Agrees on Code Validity Majority of Judges Agree on Coding Item (seven of nine) Independent Experts Agree with Code of Author (ten of eleven) 20 As can be seen in Table 1, the differences are not so much in methodology or data set but in the research questions asked. While the first concern of Boshier and Pickard was the basis for the intial hj'pothesis of the present investigation, the focus on the individual's impact, whether that be of the editor's influence, the most frequently cited authors, or most frequently cited items, was excluded as a research focus. Significant differences between these studies involve who codes the data, but more importantly, by what process citations were coded. In Boshier and Pickard (1979) each item was assessed individually for 'primary' content, whereas in this study, once an item was designated 'primary literature' its author was deemed an "author of primary literature" and all citations to that name were automatically coded 'primary literature'. If Boshier and Pickard's claim that "researchers publishing in Adult Education showed an increasing tendency to cite primary adult education literature" holds true in this investigation, the proportions of citations should be rising for those to "authors of primary literature" with a concurrent decrease in those to. "authors of secondary literature" as time passes. The rationales for this study's hypotheses were based on the studies reviewed within this chapter. Hypothesis 1 stems from Boshier and Pickard's claim that citations to "primary adult education literature" increased with time. Hypothesis 2 had as its basis their finding that the most frequently cited authors were predominantly well known adult educators. Hypotheses 3 and 4 propose the possibility that a subset of the total citations (those citations to journals) follow a similar pattern to the 'primary literature' citation proportion when journals are coded separately. These last hypotheses also serve to extend 21 the analysis of citations reported by Boshier and Pickard to include an investigation of journal usage: thus, it facilitates understanding of citing behavior in Adult Education Quarterly's articles. Figure 1 displays graphically the logical extension of these rationales in relation to the predicted results of this study. Sequential Periods Time 1 Time 2 Time 3 Time 4 Time 5 Time 6 Proportion of Citations to Primary Literature 1.00 •••• •••• •••• •••• •••• •••• •••• •••• •••• •••• 1.00 Proportion of Citations to Secondary Literature Figure 1. Histogram of hypothesized proportions of citations to "authors of primary literature" and "authors of secondary literature" as a field emerges. 22 Figure 1 is an heuristic device to foster understanding of the changes occurring in fields. In 1929 when the Journal first began publishing, authors did not identify themselves with the labels 'adult educator/education'. This situation is portrayed diagrammatically on the left side, of the histogram where secondary literature dominates the knowledge used to guide research concerning the education of adults. In more recent times speculation holds that authors are increasingly identifying themselves with the field, and with other practitioners in the field. This situation is described on the right side of the histogram where 'primary literature' increasingly influences the authors of adult education research. Hypotheses Through the review of previous studies on knowledge about adult education it was possible to hypothesize that: 1) The percentage of citations to "authors of primary literature" as opposed to "authors of secondary literature" in Adult Education Quarterly would rise between the first half of the period under study (1971-1978) and the second half (1979-1986). 2) The most frequently cited authors would increasingly be coded as "authors of primary literature." 3) The proportion of citations to "primary literature journals" as opposed to "secondary literature journals" would rise between the first half of the period under study (1971-1978) and the second 23 half (1979-1986). 4) The most frequently cited journals would icreasingly be coded as "primary literature journals." The research studies reviewed in this section suggest that adult education is building a unique body of knowledge through its increasing reliance on studies grounded in an adult education context. This thesis examined citations taken from articles in the journal Adult Education Quarterly (1971-1986), and attempted to document this increasing reliance through the methodology of citation analysis. The procedures of this study were designed to measure the proportion of 'primary literature' citations by tabulating the citations attributed to "authors of primary literature" rather than simply coding each item on its manifest content. The rationale advanced was that authors of adult education materials are from an adult education context; thus, neither titles of subsequent materials nor the publication information should alter the 'primary literature' designation of these authors. The next chapter contains a description of the research procedures used in this study. CHAPTER 3 METHODOLOGY This study proposed to measure the relative presence of 'primary literature' and 'secondary literature' cited in articles published in Adult Education Quarterly. Two questions were asked: to what extent were "authors of primary literature" cited in these articles, and what trends can be identified over time? Four hypotheses were advanced. In this chapter the procedures for data collection and analysis are described. Validity and reliability measures employed to examine the coding procedures are also detailed. Data Collection The data collection included all citations from articles, and the 'forum' section (excluding book reviews, critiques, or response sections) in Adult Education Quarterly Volumes 21 through 36 inclusive (1971-1986). Excluded from classification were citations in languages other than French/English, citations involving personal communications or other non-retrievable sources, multiple citations to a work within the same article, citations without named author, and self-citations. Because it is a citation category's influence upon research of 'another author's study' which was the focus of this thesis, self-citations (an author's reference to 'own' previous work) were excluded. This follows the exhortations of Arl in (1978); Buss (1976); Jones (1980); Myers (1970); and Roche and Smith (1978). 24 25 The procedure used to designate a citation as 'primary literature' or 'secondary literature' was as follows. First, if the citation was not excluded for the reasons above, its title was read and the words employed noted. These words were interpreted for the meaning they lend the title, irrespective of the actual words employed. As it was adult education phenomena that was the focus in this study, the essential meaning to note was 'adult education'. To assist the researcher in recognizing terminology referring to adult education concepts, processes and data, a list of synonymous terms was compiled. Darkenwald and Merriam (1982:12-14) offer some synonyms and related terms to which were added terminology found in the titles of articles from the Adult Education Quarterly. The list was constructed as follows: Adult; Andragogy; Basic; Community; Continuing; Correction; Cooperative; Disadvantaged; Extension; Further; Human Resource; Immigrant; Inservice; Lifelong; Literacy; Night School; Noncredit; Nontraditional; Nonformal; Recurrent; Reentry; Self-Directed; Seniors; Training/Retraining; Voluntary; Women's; Worker's, And" Education; Development; Learning; And: Projects; Programs; Chautauqua; Folk High School; Frontier College; Highlander School; Mechanic's Institute; Open University; Residential Workshops; Study or Listening -Circle/Forum/Group. If one or more of the terms used in the title of the reference cited appeared on the list or was a variant of a listed term (e.g., continuing, continuous, continued) the 'primary literature' designation was automatic. If the citation could not be designated from its title, a further analysis of the publication information (e.g., journal, publisher, editor, association) was made to determine if an adult education context was explicitly referenced. One procedure used was to search for journal titles in Ulrich's International Periodicals Directory to see if the particular 26 journal cited was considered an adult education journal. If a citation was still not designated 'primary literature' a comparison with all previously cited materials was made to determine if the author(s) had previously been designated as an "author of primary literature." If not designated 'primary literature' by this point the citation was designated as 'secondary literature' (see Table 2). Table 2 Steps Used to Code Citations Step Procedure Outcome 1. Examine Title Compare Words used to Adult Education Terms (e.g., andragogy, learning projects) Primary or go to Step 2. 2. Examine Publication Information Compare Words used to Adult Education Terms or Title to Ulrich's Directory or Editor to Authors List Primary or go to Step 3. 3. Examine Authors List Check for Prior Designation of Author(s) Primary or Secondary Once a citation was classified 'primary literature' or 'secondary literature' that designation was recorded beside the author(s) named for the cited material. In the event that an author had previously been coded as one category and in subsequent citations was classified differently, the 'primary literature' designation took precedence. This was justified because the researcher intended to identify "authors of primary literature." It was assumed that an author who wrote from an adult education context was an "author of primary literature" irrespective of the title of any other works cited in Adult Education Quarterly. 27 Some citations had multiple authors; therefore, a procedure was established to recognize the relative presence of each individual author in the cited literature. This procedure involved attaching a value to each citation of four points ("citation value"), so that a single author received a full four points each time cited, co-authors received two points each, and three or more authors named in a citation received one point each. By linking the classification process to coding the authors named, the accumulated number of authorships and co-authorships within the literature was tabulated. In this way the most frequently cited authors were assessed more accurately, while 'primary literature' and 'secondary literature' proportions would reflect the presence of "authors of primary literature" and "authors of secondary literature" in the materials being studied. In summary, citations were judged for 1) author variability in coding and "citation value", and 2) journal variability in coding and frequency cited (see Table 3). Table 3 Major Components of the Study Item Variable Weighting Results Authors Authors of Primary or Secondary Literature Times Citation Value (author = 4 points) (2 authors = 2 each) (3+ authors = 1 each) Frequency Cited Sum and Percentage Journals Journals of Primary or Secondary Literature Frequency Cited Sum and Percentage 28 Journals were coded using a slightly different process. Only journals referenced in eligible citations were considered. As above, the list of synonyms of adult education was compared to the cited journal's title to discover matches between the words used in the title and the terms listed as synonyms and related terms. If not classified as a "primary literature journal" a further step involved consulting Ulrich's International Periodicals Directory, Twenty-Fourth Edition (1985), to see if the journal was listed in the 103 international periodicals under the adult education section (pp. 532-34). If the journal was not designated as a "primary literature journal" by this point, it was classified as a "secondary literature journal." Journals which had changed their names were recorded only under their latest title for tabulation purposes. A complete list of eligible journals and their frequency cited can be found in Appendix A . Here is an example of the classification process using citations from an article by author J . Philips in volume 37 of an imaginary journal. R E F E R E N C E S 1. Broski, R. & Sole, B . (1979). Continuing Education Needs. Journal of Nursing, _4(2), 24-28. 2. Branhan, L . (1977). Ilees Pinyin Chu Ladme. Chang-chu, China: Peking Press. 3. Collins, J . (1980). Educational Research. Harvard, Mass.: Harvard University Press. 4. Commission of Professors. (1969). Adult Education in U.S . Boston: A A R C Press. 5. Essert, P. (1980). The Need for Women's Retraining. Continuing Education in Nursing, 33(2), 21-22. 6. Manning, P.B. (1983). Alone Again. Proceedings of the Adult Education Research Conference, Montreal, Canada, Apri l . 29 7. Philips, J . (1983). Adult Education. Adult Education Quarterly, 33(3), 8-12. 8. Sole, B. (1980). Nursing Futures. Journal of Nursing, 5(3), 5-7. These eight citations were classified as follows. Three were excluded, No. 2 because it was in a foreign language, No. 4 because there was no named author, and No. 7 because it was a self-citation. Four were classified as 'primary literature', No. 1 and 5 because of the title, No. 6 because of the publication information, and No. 8 because this author had been previously classified as an "author of primary literature" from No. 1. Number 3 was classified as 'secondary literature' because it did not meet criteria of 'primary literature'. The classification was recorded under 'author' rather than 'item', both for ease in record keeping and ability to identify the most frequently cited authors. To account for co- and multiple authors a "citation value" was assigned to each reference cited (four points). Thus, co-authors received two points each and three or more authors cited received one point each. In the example above Broski received two points as a co-author, while Sole received two from the co-authorship shared with Broski, and an additional four points for his other citation. Thus, Sole achieves a "citation value" of six in this example. The data on cited journals was less complex to record as each eligible citation represented a frequency of one citation to that journal. In our example journals are cited four times (No. 1, 5, 7 and 8). However, No. 7 was excluded 30 as it is from a self-citation. Numbers 1 and 8 both reference Journal of Nursing; thus, this journal was recorded, classified according to its title ~ after a check of Ulrich's International Periodicals Directory - as a "secondary literature journal", and credited with two citations. Number 5 was also recorded as it was from an eligible citation. This title ~ Continuing Education in Nursing -- was classified as a "primary literature journal" because it meets the criteria of words in the title which match terms used in the list of adult education synonyms and related terms (i.e., continuing education). The coding sheets for the example above would appear as seen in Tables 4 and 5 below. Table 4 Example of Author Coding Sheet (Volume 37) Author Citation Name Classification Value Broski, R. Primary Literature 2 Collins, J . Secondary Literature 4 Essert, P. Primary Literature 4 Manning, P.B. Primary Literature 4 Sole, B- Primary Literature 2 + 4 = 6 Table 5 Example of Journal Coding Sheet (Volume 37) Journal Frequency Name Classification Cited Journal of Nursing Secondary Literature 1+1 = 2 Continuing Education in Nursing Primary Literature 1 In this example the coding process would continue as above until all the eligible citations from Volume 37 were coded. The outcome of this process would be two 31 alphabetical lists, one of authors and the other of journals cited in this volume. The classification of the author along with the accumulated values of the authorships and co-authorships being cited would be on one list. The second list was names of journals cited, their classification and the tabulation of their frequency cited. These lists served as the raw data for the study. Data Analysis Lists of "authors of primary literature" and "authors of secondary literature" cited in articles from Adult Education Quarterly and the total value of citations attributed to them by volume throughout the sixteen volumes were compiled. Through this process the proportion of authorships from 'primary literature' as opposed to 'secondary literature' could be established. In preparing the list of most frequently cited authors (twenty in each consecutive 4, and 8-volume period under study), the authors were judged on their total "citation value", providing they were cited in more than one article. This was necessary because of the distortion in findings caused by historical/biographical reviews which may cite a prolific author's collected works (e.g., see Dickinson 1979: citing Verner). In the most frequently cited journal list it was necessary to eliminate those journals cited in fewer than three volumes of Adult Education Quarterly. The distortion caused by multiple citations from a small number of articles dealing with specific subject matter was deemed harmful to interpretation of citation patterns (e.g., China Quarterly, China Reconstructs cited twelve times in a single article). 32 In reviewing the literature for this study it became apparent that transitory influences on citation behavior (such as editor's mood, fads, and external historical events) could be partially controlled by using longer periods for analysis and comparison. It was decided that time periods to be used for presentation of data would not be less than that covered in four volumes (four years) of the journal (approximately 60 articles). The study period covered sixteen volumes, therefore, data were aggregated into four sequential 4-volume periods and two sequential 8-volume periods. Reliability and Validity Reliability means that data and judgments are consistent in their application across cases and between observers.. Two measures of reliability were assessed: the test-retest stability-over-time (intra-judge consistency); and the reproducibility (inter-judge agreement). In their study Boshier and Pickard (1979) made use of a panel of judges to assess the extent to which citations were reliably classified as 'primary literature' or 'secondary literature' (seven of nine judges had to agree on each item's classification or the citation was excluded from the study). In the present study when a citation was not classified as 'primary literature', the data analysis procedures forced the 'secondary literature' classification on the author of the item. Through this process the author cited always attained a code. Thus, reliability and validity measures are necessarj' to assess the appropriateness and success of these collection and analysis procedures (see Table 6). 33 Table 6 Procedures for Establishing Reliability and Validity Employed in this Study Test Procedure Errors Detected Reliability: Test-Retest Researcher Recodes Data (Intra-Judge Stability-over-time) Inconsistency Reliability: Reproducibility Independent Judge Codes Data (Inter-Judge Agreement) Disagreement Validity: Expert's Standard Compare Results to Independent Expert Judgements Inaccuracy Reliability: Test Retest In this study reliability was handled in two ways. First, intra-judge stability-over-time was verified by the researcher recoding and analyzing a random sample of the data. One volume of the journal (17 articles with 327 citations) was chosen at random and citations coded without reference to the original data set. The results were compared with the first analysis of these materials and differences noted (see Table 7). 34 Reliability: Reproducibility The second measure of reliability involved having a randomly drawn 'issue' of the Journal (Volume 34, Issue No. 3 with 126 citations) coded by an independent judge using only the rules as written by the researcher. The judge's results were then compared to the researcher's findings. Andren (1981) notes that when employing semantical content analysis techniques, differences between judges are not, on the face of it, detractors from the confidence levels of the research, but rather require analysis and interpretation. He says: Semantic content analysis is . . . an activity which often demands extensive knowledge . . . . This means that it may be futile to demand that the task must be such that "regardless of who does the analysis or when it is done, the same data should be secured under similar conditions." (p. 65) The findings of this procedure are reported in the Results Chapter. Validity: Expert Standard Validity concerns the extent to which an instrument measures what it purports to measure. Krippendorf (1980) said: Semantic validity assesses the degree to which a method is sensitive to the symbolic meanings that are relevant within a given context, (p. 157) Validity concerned the extent to which the results obtained using the rule-based procedures for distinguishing "authors of primary literature" from "authors of secondary literature" compared to a 'standard' established by 'experts' in the 35 adult education field. This researcher had assumed that authors using adult education terminology in the titles of their works, or publishing within adult education literature, were writing about adult education phenomena and therefore, were directly associated with the field of adult education. To test these measures an 'expert' panel of adult educators (see Appendix B) was consulted to provide a 'standard' for comparison. The criteria for inclusion on this panel of 'experts' were: Professor of Adult or Continuing Education Current faculty position at North American Universitj^ Doctoral preparation Not presently faculty at University of British Columbia Sixteen letters and checklists (see Appendix C) were mailed in the first week of September 1986 and eleven replies were received within four calendar weeks (68.8 percent return rate). A l l checklists returned were usable. A list of the twenty most frequently cited authors in each of the four 4-volume periods covered by the study was sent to this panel, without reference to the designation these most cited authors received by the study procedures. The panel was asked to identify from the names listed those "primarily known for their adult education activities." Eleven 'experts' responded to this request. If at least ten of the experts (greater than 90 percent of the respondents) agreed that the named author was "primarily known for their adult education activities" this became the 'standard' with which the research results were compared (see Table 8). 36 In this chapter the procedures used for data collection and analysis have been described. In the following chapter the results of the analysis are presented. CHAPTER 4 RESULTS The findings of this study are organized as follows. First, results pertaining to reliability and validity are discussed, then the characteristics of the population, the author variability, and the journal variability are presented. Reliability and Validity The test retest results of intra-judge consistency (recoding Volume 28: 17 articles; 369 citations) compared the proportions of citations to "authors of primary literature" and "authors of secondary literature" found on two separate occasions (July and September 1986). In the original coding 321 authors were coded, whereas, during recoding only 316 authors were. Five authors were eliminated during recoding because it was discovered that the two citations involved were to unpublished materials, making them ineligible citations (this fact had been overlooked in the original analysis). Comparison between coding of authors on two occassions resulted in 306 agreements (about 97 percent). Disagreement in author classification between the two codings were to ten names on the author coding sheet (about 3 percent). This difference is to some extent accounted for b}' the fact that authors in the original collection may have been coded "authors of primary literature" because they appeared on the author list from a prior designation, however, this 37 38 information was not available in the process of re-analysis (e.g., Paulo Freire could not be designated from the titles cited in this volume as an "author of primary literature" but holds this designation from an earlier volume). The disputed authors and explanation of the discrepant coding are displayed in Table 7. Table 7 Discrepancies in Intra-Judge Recoding with Reasons Original Reason for Name Coding Re-coded Error Result Result in Original Abbott G. Secondary Primary Mistake Biddle L . Secondary Primary Mistake Biddle W. Secondary Primary Mistake Bradford L . Primary Secondary Earlier Designation Borowsky G. Primary Secondary Mistake Caylor F . Primary Secondary Mistake Faure E . Primary Secondary Earlier Designation Freire P. Primary Secondary Earlier Designation Gross R. Primary Secondary Mistake Schwartz S.H. Primary Secondary Earlier Designation Four of the errors in coding were because prior designations were not available in the recoding process, meaning that only six authors were mistakenly coded in the original research. Along with the mistaken inclusion of the two citations to non-published material, this indicated that errors were due to mistakes made by the researcher in the original data collection process rather than to inadequacies of procedures used. This result tends to favor the view that the data collection and analysis procedures of this study were followed consistently and that a high degree of confidence in findings is warranted. The reproducibility test was that of having an independent judge recode a 39 portion of the sample (Volume 34, Issue No. 3, 126 citations) and by comparing the results with the researcher's original coding outcomes. Of the 135 authors coded, agreement was achieved in 123 cases (about 91 percent agreement). The twelve disagreements (about nine percent) can be explained as differences of opinion about the meaning of terms in the list of synonyms of adult education. The independent judge included eleven citations as 'primary literature' which refer to university courses, higher education, and college programs in a prison setting. These had not been so classified by the researcher because education in a prison setting had not been viewed as a variant of 'correction education'. These results appear to challenge the study's potential reproducibility because of a nine percent difference between the judge and the researcher in coding outcomes. However, differences of this magnitude (with these explanations) could be predicted to occur as a 'constant discrepancy' throughout the analysis of the judge and the researcher. Resulting proportions (formed by comparing groups of coded authors internally in each case) would predictably yield findings and lead to conclusions about rates of change of a similar nature. The expert's standard was used to estimate the validity (accuracy) of the study procedures. The validity of the rule-based classification system used in this study was examined by consulting a panel of 'experts'. The 'experts' established the 'standard' by which accuracy in coding the citations could be judged. A checklist was provided to these 'experts' on which the names of the twenty most 40 frequently cited authors from each 4-volume period within the study were recorded. Findings of 75.5 percent agreement (40 out of 53 authors) resulted when the judgments of at least ten of the eleven 'experts' (90 percent agreement) were compared with the rule-based classifications made during the research process. The complete results of the 'expert' panel's designation of the' twenty most frequently cited authors from four 4-volume periods are listed in Appendix D. Table 8 shows these authors with disputed codes and the level of agreement of the experts with the rule-based classification system. Table 8 Discrepancies in Author Classification Found by Validity Test Name Rule-Based Code Number Ageeing of Experts with Code Vincent, John H . primary literature 1/11 Havighurst, R . J . primary literature 2/11 Sheffield, Sherman primary literature 3/11 Benne, Kenneth primary literature 5/11 Douglah, M . primary literature 5/11 Cross, K . Patricia primary literature 6/11 Litchfield, Anne primary literature 6/11 Rivera, Ramon J . primary literature 6/11 Cartwright, Morse A . primary literature 7/11 Johnstone, John W.C. primary literature 7/11 Miller, Harry H . primary literature 7/11 Monette, Maurice primary literature 8/11 Penland, Patrick R. primary literature 9/11 The validity of the rule-based procedures used to differentiate "authors of primary literature' from "authors of secondary literature" seemed to be supported by the judgments of the panel of experts, especially when it is realized that the 41 levels of disagreement on the disputed codes were generally high (average of six experts in agreement on the disputed code). The following sections summarize the data obtained through the procedures described in Chapter 3. Study Population A total of 5,413 citations were contained in the 249 articles from sixteen volumes of Adult Education Quarterly (Volumes 21 to 36) that make up the study population. Of these citations, 713 were excluded! which meant that 4,700 citations were eligible for classification. These citations represented the work of 3,381 individual authors; however, only 269 (7.9 percent) of these authors are cited an equivalent of twice ("citation value" of eight points) in the sixteen volumes studied. Data related to the sample were: Number of Eligible Articles with Citations = 249 Number of Citations Present = 5,413 Total Excluded Citations = 713 Total Classified Citations = 4,700 Total Authors Classified = 3,381 Number of Authors With Citation Value of Eight = 269 t Excluded were: all self-citation; any without a named author; personal communication or non-retrievable sources (e.g., speech); foreign language (any other than French/English); citation greater than once for same material within an article 42 Number of Journals Cited = 381 Number of Journals Cited Three Times or More = 73 Journals Designated Primary Literature = 31 Total Classified Citations to Journals = 1,534 Percentage of A l l Classified Citations to Journals = 32.6 percent Cited Authors The proportion of citations to "authors of primary literature" accounted for almost one third of the citations in Volume 21 and grew to account for almost two thirds of the citations by Volume 36 (see Figure 2). Volume 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 Proportion .70 .65 • .60 • .55 • .50 • • • .45 • • .40 • • • .35 • • • .30 • • Figure 2. Proportion of citations made to "authors of primary literature" in each of the sixteen volumes under study. The proportion of citations to "author of primary literature" can be seen to follow an increasing (though inconsistent) rise throughout the period under study. 43 Consolidating data into a longer time frame reveals trends. Where Figure 2 clearly reveals the peaks and troughs in the citations to "authors of primary literature" during this period, Table 9 summarizes data by 4-volume accumulations in the table and 8-volume tabulation at the bottom of the table. In Table 9 each volume is described first by the "citation value" of the designated "authors of primary literature", second by the total "citation values" of the volume, and finally, by the computation of the percentage of "citation values" attributable to "authors of primary literature" (column on far right). This last column provides the figures which are used for comparison of changes in citing behavior over time. Minimally it can be said that the percent of citations to "authors of primary literature" has increased 4.8 percent between the first half (1971-1978) and the second half (1979-1986) of the study period. Any author whose accumulation of "citation value" was eight points or more during the study period (excluding those cited in only one article) was added to the list of most frequently cited authors. Two hundred and sixty-nine authors qualified for this listing (about 8 percent of authors), accounting for 39.5 percent of all citations classified. "Authors of primary literature" as a proportion of most cited authors listed generally rises through the period under study (see Table 10). 44 Table 9 Percent Citations to Authors of Primary Literature Volume 21-36 Primaryt Total Primary Citation Citation Citation Volume Values Values Percentage 21 232 765 30.3 22 440 1001 44.0 23 250 756 33.0 24 412 1255 32.8 21-24 1334 3777 35.3 25 289 705 41.0 26 409 976 41.9 27 562 1006 55.9 28 632 1292 48.9 25-28 1892 3979 47.5 29 405 1103 36.7 30 528 1202 43.9 31 448 1114 40.2 32 395 1187 33.3 29-32 1776 4606 38.6 33 821 1700 48.3 34 696 1424 48.9 35 804 1401 57.4 36 869 1394 62.3 33-36 3109 5919 53.9 Primary Total Primary Citation Citation Citation Years Values Values Percentage 1971-1978 3226 7756 41.6 1979-1986 4885 10525 46.4 tAccumulated Citation Values of "Authors of Primary Literature" 45 Table 10 Percentage of Most Frequently Cited Authors who are Authors of Primary Literature in Four Periods and Their Percentage of Total Citation Values Number and Percent Percentage of Total Time Volumes Primary Authors Citation Values T. 21-24 12 of 20 (60.0%) 66.5% 2. 25-28 16 of 23 (69.7%) 81.5% 3. 29-32 13 of 20 (65.0%) 78.8% 4. 33-36 18 of 21 (85.7%) 92.3% The data on the twenty most cited authors in 8-volume periods (see Table 11) demonstrate a similar rise in both the proportion of the "authors of primary literature" and in the percentage of citations made by this 'primary literature' group. In the first 8-volume period (Volume 21-28) "authors of primary literature" constituted seventeen of the twenty-one most cited authors (81 percent), accounting for 1116 of the 1452 "citation values", or 76.9 percent of all citations. Whereas, in the second 8-volume period (Volume 29-36) "authors of primary literature" were eighteen of the twenty-one (85.7 percent) most cited authors and now accounted for 1400 out of 1550 (90.3 percent) of all "citation values" in this group. The number in brackets on Table 11 is the sum of "citation values" for each author in this group: four for a full authorship; two for a co-authorship; and one for being one of three or more authors. Due to ties in "citation values" there are twenty-one authors listed. 46 Table 11 Top Twenty Authors Ranked by Citation Value: Volumes 21-28 and 29-36t Volume 21 to 28 Volume 29 to 36 Author Citation Value Author Citation Value •Houle, Cyril 0 . (130) •Knowles, Malcolm (226) •Verner, Coolie (130) •Tough, Allen (144) •Knowles, Malcolm (118) •Houle, Cyri l (140) •Freire, Paulo (80) •Knox, Alan B. (105) Maslow, A . H . (56) •Lindeman, Eduard (98) •Knox, Alan B . (49) •Boshier, Roger (96) •Lindeman, Eduard (40) •Cross, K . Patricia (74) •Douglah, M . (38) •Freire, Paulo (64) •Boshier, Roger (36) •McClusky, Howard (62) •Sheffield, Sherman (36) •Darkenwald, Gordon (60) •Ohliger, John (34) •Havighurst, Robert (59) •London, Jack (32) •Hiemstra, Roger (56) •Bergevin, Paul (32) •Verner, Coolie (54) Illich, Ivan (30) •Kidd , J.R. (48) Gagne, R . M . (30) •London, Jack (45) •Dickinson, Gary (30) Erikson, Erik (40) •Johnstone, J .W.C. (30) •Monette, Maurice (40) •Rivera, Ramon (30) Kerlinger, Fred (36) •Havighurst, Robert (29) •Mezirow, Jack (35) Dewey, John (28) Sticht, Thomas (34) •Litchfield, Anne (28) •Grabowski, S .M. (34) Cited Journals Citations to journals that were deemed "primary literature journals" by study procedures accounted for a low of less than one-tenth of journal citations in Volume 21 to a high of over one-half of citations in Volume 36. As with the citations to "authors of primary literature", "primary literature journal" citations were found to be an increasing proportion of all journal citations (see Figure 3). t The • indicates a designated "author of primary literature" 47 Volume 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 Proportion .65 .60 .55 • .50 • .45 • • .40 • • • • .35 • .30 • • • .25 • • .20 • .15 .10 .05 • Figure 3. Proportion of journal citations made which are to "primary literature journals" by volume studied. While from Figure 3 inconsistency of the pattern on a volume-to-volume comparison is apparent, when consecutive 8-volume periods are compared an increase of 7.7 percent in the "primary literature journal" category can be seen (see Table 12). Table 12 48 Journals Volume 21-36 Primary Total Percentage Volume Journals Journals to Primary Cited Cited Journals 21 6 94 6.3 22 35 87 40.2 23 14 58 24.1 24 47 129 36.4 21-24 102 368 27.7 25 12 52 23.0 26 24 78 30.7 27 47 110 42.7 28 37 93 39.7 25-28 120 333 36.0 29 20 89 22.4 30 36 124 29.0 31 27 69 39.1 32 28 70 40.0 29-32 111 352 31.5 33 75 143 52.4 34 40 133 30.0 35 49 106 46.2 36 53 99 53.5 33-36 217 481 45.1 Primary Total Percentage Journal Journal to Primary Years Citations Citations Journals 1971-1978 1979-1986 222 328 701 833 31.7 39.4 49 In Table 12 the left hand column identifies the volume studied, next the frequency of citation to "primary literature journals" is shown, then the total frequency of citation to journals, and finally the far right hand column is the computed percentage of "primary literature journal" citations as a proportion of journal citations. Running down the fourth column are the figures used to monitor change in citing behavior toward journal categories over time. A t the foot of Table 12 the figures of the frequency of citing to "primary literature journals" and total journals are shown, along with a computation of the percentage of citing to "primary literature journals" in the first half of the study (1971-1978: 31.7%) and those citings during the second half of the study (1979-1986: 39.4%). This rise of 7.7 percent in the proportion of citations to "primary literature journals" parallels the findings concerning "authors of primary literature" shown earlier. The most frequently cited journal list was defined as all journals cited in a minimum of three separate volumes with the highest accumulated frequency of citations. In Table 13 the first 8-volume period (Volume 21-28) shows three of the nine most cited journals are "primary literature journals" accounting for 192 of 271 citations (70.8 percent). In the second 8-volumes, five of the nine most cited journals are "primary literature journals" and they account for 287 of the 338 citations (84.9 percent). Twelve journals make-up those most frequently cited from the study's 381 cited journals in both 8-volume periods. 50 Table 13 Nine Most Frequently Cited Journals: Ranked for Volume 21-28 and 29-36t Volumes 21-28 Volumes 29-36 Citations Journal Citations Journal •Adult Education Quarterly (149) •Adult Education Quarterly (196) •Lifelong Learning (33) •Lifelong Learning (57) Journal of Gerontology (17) A m . Sociological Review (17) Journal of Ed. Psychology (16) Harvard Educational Review (14) American Sociological Review • (15) •Convergence (12) Journal of Ed. Research ( ID •Studies in Adult Education (11) •Convergence (10) •Educational Gerontology (11) Journal of Reading (10) Journal of Gerontology (10) Develomental Psychology (10) Journal of Ed. Psychology (10) The purpose of this chapter was to present Findings relevant to the research questions which guided this study. First, data supporting the validity and reliability of the data analysis procedures was presented. Second, data produced by the procedures used to measure proportions of the citations made to "authors of primary literature" and "authors of secondary literature" were reported. This same data was also examined to show the proportions of the items by the authors most frequently cited in this literature. Finally, the data from procedures used to determine the proportions of "primary literature journals" and "secondary literature journals" found in the study were presented. The next and final chapter contains discussion of the findings presented, implications related to hypotheses made, limitations relevant to the study procedures, and suggestions for future research employing citation analysis of the type used in this study. tThe • before the title refers "Primary Literature Journals" CHAPTER 5 DISCUSSION, LIMITATIONS & IMPLICATIONS This Chapter contains a discussion of findings, an analysis and interpretation of results related to the study's hypotheses, and a discussion of limitations that are relevant to these interpretations. The Chapter concludes with a discussion of the implications of the study for future investigation of the knowledge base in adult education Discussion This study provided empirical evidence of the changing nature of citations in adult education research through an analysis of the articles published in Adult Education Quarterly. Results suggest that authors using Adult Education Quarterly as a communication channel for their research reports are paying increased attention to previous adult education studies, and, therefore, are building their own studies on a foundation of unique and tested knowledge about the field. 'Primary literature' as a classification of citations experienced a rise of 4.8 percent between the period covered by Volumes 21-28 and the period covered by Volumes 29-36; however, the possible reasons for this rise have not fully been considered. It would not be unreasonable to suggest that editors of a journal may choose to publish articles that cite the Journal more often than articles that do not. Similarly, it could be suggested that authors who publish in 51 52 a particular journal may have something in common that is not readily observable, and may cite each other's work as a matter of familiarity, thereby driving up citation rates. To assess the impact of the above arguments on the results obtained in this investigation, an analysis was made of the proportion of citations to articles from the Journal itself over the last 35 years. The first 20 years of data comes from Dickinson and Rusnell (1971) while the last 15 years of data were from this study's findings. In order to determine these proportions, the cumulative frequency of all citations for each five-year interval was divided by the cumulative frequency of citations to articles published in the Journal in the same five year interval. By plotting this proportion for each time interval it is possible to determine whether an increasing citation level was due to 'favoritism' toward citing articles published in Adult Education Quarterly (see Table 14). Table 14 Citations in the Adult Education Quarterly to Its Own Articles: Volumes 1-35 0 Adult Education Quarterly Total Proportion of Volume Articles Cited : in Citations Citing to Its Adult Education Quarterly Own Articles 1-5 8 111 .07 6-10 22 462 .04 11-15 50 790 .06 16-20 36 1030 .03 21-25 90 1333 .06 26-30 93 1727 .05 31-35 127 1974 .06 53 As can be seen from Table 14, the proportion of citations to Adult Education Quarterly has not increased over time, lending support to an argument that any rise in citing behavior to adult education research is not simply a product of colleague familiarity on the part of the researchers publishing in the Journal. A second matter for discussion is related to the assumed values of this study. It was the contention of this researcher that the field of adult education must increase the attention it pays to its own research if it is to build theory and accumulate knowledge unique to adult education. This building is assumed to be part of the maturation of the field. Therefore, the increasing proportion of citation to 'primary literature' was argued to be a sign of 'emergence' and felt to be a positive course for adult education researchers to be pursuing. There is, of course, another side to this. Is adult education becoming narrow in its viewpoint, looking inward for answers to questions concerning problems of the field when the answers may already be available in other disciplines? At what point does drawing from a field's research change from being a sign of strength to a sign of 'incestuousness'? The answers to these concerns have not been addressed by the research design of this thesis. These are qualitative questions requiring a kind of evaluative answer which the data collection and analysis in this thesis do not confront. Further mention of this is made in the implication section below. When Boshier and Pickard (1979) concluded that their data supported the presence of a growing unique knowledge base in adult education, they also 54 cautioned: It is difficult to know whether the increasing citation of primary literature was due to editorial policies, the continued emergence of adult education as a discipline with a distinctive body of knowledge, or the interests and activities of authors who have made extensive use of Adult Education as a publication outlet over the most recent part of the ten-year period, (p. 39) The period of analysis covered by this thesis was selected to limit the influence of overt changes in editorial policy at Adult Education Quarterly in order to more confidently suggest the results obtained were due to changes in the practice of adult education research. Data presented in Table 14 suggest that the rising proportions of citation to 'primary literature' was not an artifact of a particular group of researchers citing each other out of 'familiarity'. The fact that the 4,700 citations analyzed represented 3,381 different authors was evidence that overall citation patterns were not unduly influenced by the fact that a few well known adult education authors are being cited frequently in the literature. A l l of the hypotheses presented in Chapter 2 are supported by the data described in the previous chapter. However, as was illustrated throughout the study the inconsistent nature of some data were a weakness when forming conclusions about patterns at times (volumes 24, 29, 31 and 32 were not consistent with a pattern of increasing 'primary literature' citation). Hypothesis 1 - that the percentage of citations to "authors of primary literature" would rise, was supported in the data found in Table 9. In the first 8-volume period 41.6 percent of the citations were to "authors of primary literature". In the second 8-volume period 46.4 percent of the citations were to 55 these authors. This represents an increase of 4.8 percent from the first to the second period. Hypothesis 2 - that the most frequently cited authors would be increasingly identified as "authors of primary literature" was supported by data presented in Table 11 where of the twenty most frequently cited authors in the initial 8-volume period 81.0 percent were "authors of primary literature" whereas, in the subsequent 8-volume period 85.7 percent were "authors of primary literature", a 4.7 percent rise in "authors of primary literature." Although the evidence supports this hypothesis, as the increase was only one author between two 8-volume periods, this hypothesis is tentatively accepted until further research is undertaken. Hypothesis 3 - that the percentage of citations to "primary literature journals" would rise was supported by data in Table 12, where the percentage in Volumes 21-28 was 31.7 percent and in Volumes 29-36 rose to 39.4 percent, a 7.7 percent rise. Finally, Hypothesis 4 which predicted that the most frequently cited journals would increasingly be "primary literature journals" was supported by data in Table 13 which identified three of nine most cited journals in the first 8-volumes as "primary literature journals" rising to five of nine in the following 8-volume period. Further analysis was necessary to determine if this increase in "primary literature journals" among the top cited was not a artifact of having too low a threshold on most cited journals included. Three of the top nine 56 journals most frequently cited in Volumes 21-28 were "primary literature journals" and accounted for 70.8 percent of most frequently cited journal citations. Five of the top nine most cited journals in Volumes 29-36 were "primary literature journals" and accounted for 84.9 percent of most frequently cited journal citations. This was an increase of two "primary literature journals" In Figure 4 the results of this study's procedures in measuring the proportions of citations to "authors of primary literature" and "authors of secondary literature" are shown in a histogram representing the four 4-volume intervals in the sixteen years under study (1971-1986). The trend of increasing proportions of citation to "authors of primary literature" as was hypothetically predicted in Figure 1 was confirmed through study findings as Figure 4 illustrates. This result also confirms the conclusions of earlier studies such as Boshier and Pickard (1979); and Allcorn (1985), that adult education is emerging as a field of study and practice. 57 Volumes 21-24 25-28 29-32 33-36 Proportion of Citations to Primary Literature .70 .65 .60 .55 .50 .539 .45 •••• .40 .475 •••• .35 •••• .386 •••• .30 .353 •••• •••• •••• .25 •••• •••• •••• •••• .20 •••• •••• •••• •••• .10 •••• •••• •••• •••• .05 •••• •••• . •••• •••• .00 •••• •••• •••• •••• .05 •••• •••• •••• •••• .10 •••• •••• •••• •••• .15 •••• •••• •••• •••• .20 •••• •••• •••• •••• .25 •••• •••• •••• •••• .30 •••• •••• •••• •••• .35 •••• •••• •••• •••• .40 •••• •••• •••• •••• .45 •••• •••• •••• .461 .50 •••• .525 •••• .55 •••• •••• .60 •••• .614 .65 .647 .70 Proportion of Citations to Secondary Literature Figure 4. Histogram of proportions of citations to "authors of primary literature" and "authors of secondary literature" in four intervals during the study period. 58 Limitations There are a number of limitations to take into account when reviewing this study. The first is the matter of the use of a citation's title to judge the material as 'primary literature'. The rationale used was one of face validity. If a title identifies it as being concerned with adult education, then the article is most likely about adult education phenomena. A second and related limitation has to do with the matter of attributing all of an author's work to 'primary literature' if any work is designated so. The justification for this procedure was that authors writing directly about adult education phenomena are more than likely affiliated with the field, and their subsequent writings cited in Adult Education Quarterly even when not entitled manifestly adult education are still 'primary literature'. Thirdly, the fact, that only journal articles, and moreover, only one journal's articles were examined, is a strong limiting factor. As was noted in a Baath University Library (1979) publication that discussed the common limitations of citation analysis, "Usually only journals have been used as sources . . . there is good reason for believing that references in journals may be different from those in other sources" (p. 5). This limitation is a very serious one when interpreting the findings of this study. The generalizability of these results can be taken no farther than the Journal from which they were obtained until confirming research using citation analysis is attempted on other forms of literature in adult education. 59 A fourth limitation arises from the fact that coding rules were fundamental. In short, the rule was applied even when common sense dictated otherwise. Thus, Havighurst was categorized as an "author of primary literature" because of terms used in the title of a single article, although the titles of a plethora of other writings by him were obviously 'secondary literature'. The consistent use of absolute rules in a study of this kind is acceptable when results are derived by comparing consecutive periods internally. Thus, the impact of this type of error is minimized when it is a constant error. Implications Further study is needed on the broader research efforts in adult education. As pointed out in the limitation section above, other forms of research reports need to be analyzed in order to confirm the citation trends demonstrated in Adult Education Quarterly. Future studies could well focus on the 'meeting' literature which is accessible through 'Proceedings' of research conferences. A second line of inquiry that is a natural extension of this study would be investigation of the impact, duration and significance of particular research trends in the field. As was mentioned in the discussion section qualitative questions need to be addressed on the matters of the value of increasing intra-discipline citing and the 'incestuous' risks inherent in such citing behavior. Further, the value of adult education research trends or lines of pursuit should be investigated. 60 Finally, the implication of the Boshier and Pickard (1979) study along with the present study's confirming results suggest a need to analyze in more depth the cited materials in adult education research. Future research should attempt to expand on the dichotomous categories used in these past efforts in order to identify more precisely the nature of the value researchers are finding in 'primary literature'. This thesis has contributed empirical findings relevant when questioning creation of knowledge in adult education. The knowledge base of adult education now has one more empirically based characterization by which greater understanding of the field may be achieved. R E F E R E N C E S Aaronson, Steve. (1975). 'The Footnotes of Science.' Mosaic, 6(2), 22-7. Allcorn, Seth. (1985). 'The Knowledge Gap of Adult Education.' Lifelong Learning: An Omnibus of Practice and Research, 8(5), 12-16. Andren, Gunnar. (1981). 'Reliability and Content Analysis. ' In Kar l Erik Rosengren (Ed.), Advances in Content Analysis. Sage Annual Review of Communication Research, Volume 9. Beverly Hills, C A : Sage Publications, 58-65. Arl in , Marshall. (1978). 'Quantity and Impact of Scholarly Journal Publication in Canadian Faculties of Education.' Canadian Journal of Education, 3(1), 1-18. Baath University Library. (1979). The Structure of Social Science Literature as Shown by Citations Design of Information Systems in the Social Sciences. (Research Reports Series A No.3). Bath, U . K . : Bath University. Beals, Ralph A . & Brody, Leon. (1941). The Literature of Adult Education. New York: American Association for Adult Education. Boshier, Roger (Ed.). (1980). Towards a Learning Society. Vancouver: Learningpress. Boshier, Roger & Pickard, Lynette. (1979). 'Citation Patterns of Articles Published in Adult Education 1968-1977.' Adult Education, 30(1), 34-51. Brookfield, Stephen. (1982). 'Adult Education Research: A Comparison of North American and British Theory and Practice.' International Journal of Lifelong Education, 1(2), 152-67. Brunner, Edmund deS. & Verner, Coolie. (1968). 'Adult Education.' in David L . Sills (Ed.), International Encyclopedia of the Social Sciences, Volume 1. N.P. : The Macmillan Company & The Free Press, 100-06. Brunner, Edmund deS.; Wilder, David S.; Kirchner, Corinne & Newberry, John S. Jr . (1959). A n Overview of Adult Education Research. Washington, D.C.: Adult Education Association Buss, Allan R. (1976). 'Evaluation of Canadian Psychology Departments Based upon Citation and Publication Counts.' Canadian Psychological Review, 17(2), 143-50. Darkenwald, Gordon G. & Merriam, Sharan B. (1982). Adult Education Foundations of Practice. New York: Harper & Row. Dickinson, Gary. (1979). Contributions to a Discipline of Adult Education A Review and Analysis of the Publications of Coolie Verner. Occasional 61 62 Papers in Continuing Education Number 18. Vancouver: Centre for Continuing Education, University of British Columbia. Dickinson, Gary & Rusnell, Dale. (1971). ' A Content Analysis of Adult Education.' Adult Education Journal, 21(3), 177-85. Draper, James A. (1985). 'Adult Education.' in The Canadian Encyclopedia, Volume 1. Edmonton: Hurtig Publishing, 10-11. Garfield, Eugene. (1979). Citation Indexing Its Theory and Application in Science, Technology, and Humanities. Toronto: John Wiley & Sons. Garfield, Eugene. (1972). 'Citation Analysis as a Tool in Journal Evaluation.' Science, 178(4060), 471-78. Grabowski, Stanley M . (1974). 'The Literature of Adult Education: A Review Report.' Paper presented to the Commission of Professors of Adult Education, Adult Education Association of the U .S .A . (Miami, Florida, November 3rd, 1974). ED 102 413 Griffith, William S. (1966). 'Guidelines for Writers Preparing Manuscripts for Adult Education.' Adult Education, 16(2), 85-91. Griffith, William S. & Roberts, Pearl J . (1981). 'Adult Education.' in H . M . Andrews & W. T. Rogers (Eds.), Canadian Research in Education: A State of the Ar t Review. Report prepared for the Social Science and Humanities Research Council of Canada, Vancouver: University of British Columbia, 3-11. Jams, M . (1965). 'The Problem of Validating Content Analysis. ' In Harold D. Laswell, Nathan Leites and Associates (Eds.), Language of Politics: Studies in Quantitative Semantics. Cambridge: MIT Press, p.5 7. Jensen, Gale; Liveright, A . A . & Hallenbeck, Wilbur. (1964). Adult Education Outlines of an Emerging Field of University Study. N.P. : Adult Education Association of the United States, forward. Jones, Lyle V . (1980). 'The Assessment of Scholarship.' in Edward H . Loveland (Ed.), Measuring the Hard-to-Measure , (New Directions for Program Evaluation, 6). San Francisco: Jossey-Bass, 1-20. Knowles, Malcolm S. (1973). 'Sequential Research Needs in Evolving Disciplines of Social Practice.' Adult Education, 23(4), 298-303. Knowles, Malcolm S. (1960). Handbook of Adult Education in the United States. Chicago: Adult Education Association of the U.S .A. Krippendorf, Klaus. (1980). Content Analysis An Introduction to Its Methodology. (Volume 5 The Sage CommText Series). Beverly Hills: S A G E . 63 Liveright, A . A . (1964). 'The Nature and Aims of Adult Education as a Field of Graduate Education.' in Gale Jensen; A . A . Liveright & Wilbur Hallenbeck (Eds.), Adult Education Outlines of an Emerging Field of University Study. N.P. : Adult Education Association in the United States, 85-102. Long, Huey B. (1983a). Adult Learning Research and Practice. New York: Cambridge. Long, Huey B. (1983b). 'Characteristics of Adult Education Research Reported at the Adult Education Research Conference, 1971-1980.' Adult Education, 33(2), 79-96. Long, Huey B. (1977). 'Publication Activity of Selected Professors of Adult Education.' Adult Education, 27(3), 173-86. Long, Huey B. & Agyekum, Stephen K . (1974). 'Adult Education 1964-1973: Reflections of a Changing Discipline.' Adult Education, 24(2), 99-120. Merriam, Sharan B. & Simpson, Edwin L . (1984). A Guide to Research for Educators and Trainers of Adults. Malabar, F l . : Kreiger Publishing. Miller, Max D. (1982). 'Citation Patterns of Articles Published in the Journal of Medical Education, 1970-1980.' Journal of Medical Education, 57, 797-99. Myers, C. Roger. (1970). 'Journal Citations and Scientific Eminence in Contemporary Psychology.' American Psychologist, 25(7), 1041-48. Nelson, Carnot E . (1972). 'Journal Publication of Material Presented at an Annual A E R A Meeting.' Educational Researcher, 1(8), 4-6. Peters, John M . (1980). ' A Systems Approach to Examining Adult Education.' in John M . Peters & Associates (Eds.) Building an Effective Adult Education Enterprise. San Francisco: Jossey-Bass, 115-57. Peters, John M . & Banks, Betty B. (1982). 'Adult Education.' in Harold Mitzel (Editor in Chief), Encyclopedia of Educational Research, Fifth Edition. New York: The Free Press, 83-87. Price, Derek J . de Solla. (1970). 'Citation Measures of Hard Science, Soft Science, Technology, and Nonscience.' in Carnot E . Nelson & Donald K . Pollock (Eds.), Communication Among Scientists and Engineers. Lexington, Mass.: Heath Lexington Books, 3-22. Roche, Thomas & Smith, David Lewis. (1978). 'Frequency of Citation as Criterion for Ranking of Departments, Journals, and Individuals.' Sociological Inquiry, 48(1), 49-57. Smith, Nick L . & Caulley, Darrel N . (1981). 'The Evaluation of Educational Journals Through the Study of Citations.' Educational Researcher, 10(5), pp.11-12, 22-24. 64 Smith, Robert M . ; Aker, George F. & Kidd, J.R. (Eds.). (1970). Handbook of Adult Education. New York: Macmillan Publishing. Sork, Thomas J . (1982). 'Meta-Research in Adult Education Through 1981: A n Historical Analysis and Critical Appraisal. ' Unpublished Paper. Adult Education Research Centre, University of British Columbia. Ulrich's International Periodicals Directory, 24th edition . (1985). A Bowker Serials Bibliography. New York: Bowker, 532-534. Verner, Coolie. (1960). 'The Literature of Adult Education.' in Malcolm S. Knowles (Ed), Handbook of adult education in the United States. Chicago: American Adult Education Association of the U.S .A. , 162-75. Verner, Coolie; Dickinson, Gary; Leirman, Walter & Niskala, Helen. (1970). The Preparation of Adult Educators. N.P . : ERIC Clearinghouse on Adult Education. Xhignesse, Louis V . & Osgood, Charles E . (1967). 'Bibliographical Citation Characteristics of the Psychological Journal Network in 1950 and in I960.' American Psychologist, 22 (7), 778-91. APPENDIX A Journals Cited in Adult Education Quarterly f c A D U L T E D U C A T I O N I N F I N L A N D 2 o A D U L T E D U C A T I O N (UK) 13 © A D U L T E D U C A T I O N Q U A R T E R L Y 330 (J. AD.ED. , A D . E D . J . , AD.ED.) o A D U L T L I T E R A C Y A N D BASIC E D U C A T I O N 1 © A U S T R A L I A N J O U R N A L OF A D U L T E D U C A T I O N 2 A M E R I C A N SOCIOLOGICAL R E V I E W 32 A M E R I C A N E D U C A T I O N A L R E S E A R C H J O U R N A L 12 A M E R I C A N J O U R N A L OF SOCIOLOGY 13 A M E R I C A N SOCIOLOGIST 6 A M E R I C A N J O U R N A L OF P S Y C H O L O G Y 4 A M E R I C A N PSYCHOLOGIST 9 A M E R I C A N E D U C A T I O N 1 A M E R I C A N P O L I T I C A L S C I E N C E R E V I E W 2 A M E R I C A N SCIENTIST 1 A M E R I C A N J O U R N A L OF P S Y C H I A T R Y 3 A M E R I C A N H E R I T A G E 1 A M E R I C A N ANTHROPOLOGIST 3 A M E R I C A N L I B R A R Y ASSOCIATION B U L L E T I N 1 A M E R I C A N J O U R N A L OF P H A R M A C Y E D U C A T I O N 4 A M E R I C A N B E H A V I O R SCIENCE 1 A M E R I C A N J O U R N A L OF O C C U P A T I O N A L T H E R A P Y 1 A M E R I C A N C O U N C I L O N E D U C A T I O N 1 A M E R I C A N A C A D E M Y OF P O L I T I C A L & SOCIAL SCIENCE A N N A L S 2 A M E R I C A N Q U A R T E R L Y 1 A M E R I C A N J O U R N A L OF P H Y S I C S 1 A M E R I C A N P O L I T I C A L SCIENCE R E V I E W 1 A M E R I C A N R E V I E W OF P S Y C H O A N A L Y S I S 1 A M E R I C A N J O U R N A L OF O R T H O P S Y C H I A T R Y 1 A M E R I C A N B A R F O U N D A T I O N R E S E A R C H J O U R N A L 1 A M E R I C A N J O U R N A L OF E D U C A T I O N 1 A D M I N I S T R A T I V E SCIENCE Q U A R T E R L Y 9 A D M I N I S R A T I V E M A N A G E M E N T 1 A R C H I V E S OF P S Y C H O L O G Y 2 A U D I O V I S U A L C O M M U N I C A T I O N R E V I E W 3 A N N U A L R E V I E W OF SOCIOLOGY 2 A N N U A L R E V I E W OF P S Y C H O L O G Y 2 A L B E R T A J O U R N A L OF E D U C A T I O N A L R E S E A R C H 1 A G I N G A N D H U M A N D E V E L O P M E N T 2 ADMINISTRATOR'S N O T E B O O K 1 A U D I O V I S U A L INSTRUCTOR 5 A A U P B U L L E T I N 1 t- Of 381 Journals, 31 were designated ' • ' "primary literature journals", number following the journal title indicates frequency of citation 65 A U S T E N A M E R I C A N S T A T E S M A N 1 ACROSS T H E B O A R D 1 A C C O U N T I N G R E V I E W 1 A N N A L S OF I N T E R N A L M E D I C I N E 1 A R C H I V E S OF I N T E R N A L M E D I C I N E 1 A E D S J O U R N A L 1 A Z T L A N 1 BRITISH J O U R N A L OF E D U C A T I O N A L P S Y C H O L O G Y 3 BRITISH J O U R N A L OF SOCIOLOGY 2 BRITISH J O U R N A L OF P S Y C H O L O G Y 3 BRITISH J O U R N A L OF STATISTICAL P S Y C H O L O G Y 2 B E H A V I O R SCIENCE 3 BUSINESS E D U C A T I O N F O R U M 1 B U L L E T I N OF ATOMIC S C I E N C E 1 • C O M M U N I T Y E D U C A T I O N J O U R N A L 1 • C A N A D I A N J O U R N A L OF U N I V E R S I T Y CONTINUING E D U C A T I O N C A N A D I A N J O U R N A L OF HIGHER E D U C A T I O N 1 C A N A D I A N J O U R N A L O F P S Y C H O L O G Y 1 C E N T U R Y 1 • C H A U T A Q U A N 2 • C O N T I N U I N G E D U C A T I O N 1 C H R O N I C A L OF H I G H E R E D U C A T I O N 5 • C O N V E R G E N C E 22 • C O M M U N I T Y D E V E L O P M E N T J O U R N A L 1 C O L L E G E & U N I V E R S I T Y 3 C A N A D I A N J O U R N A L OF E D U C A T I O N 1 C A N A D I A N P S Y C H O L O G I C A L R E V I E W 1 C O U N S E L I N G E D U C A T I O N & SUPERVISION 3 C O U N S E L I N G P S Y C H O L O G I S T 8 C A L T F O R N I A N M A N A G E M E N T R E V I E W 3 C H A N G E 3 C A N A D I A N J O U R N A L OF B E H A V I O R S C I E N C E 1 CHILD S T U D Y J O U R N A L 1 C O M M U N I T Y & JUNIOR C O L L E G E J O U R N A L 4 C H I N A Q U A R T E R L Y 2 C H I N A RECONSTRUCTS lOt C A R E E R D E V E L O P M E N T A C T I V I T Y 1 C O M M U N I C A T I O N E D U C A T I O N 1 C A N A D I A N J O U R N A L OF P H I L O S O P H Y 1 • C O N T I N U U M 1 C O L L E G E & U N I V E R S I T Y BUSINESS 1 C O N T E M P O R A R Y E D U C A T I O N A L P S Y C H O L O G Y 3 C O M M U N I T Y C O L L E G E F R O N T 3 CORRECTION T O D A Y 1 C O M M U N I T Y M E N T A L H E A L T H J O U R N A L 1 C O M M U N I T Y C O L L E G E R E V I E W 1 C O N T E M P O R A R Y E D U C A T I O N 1 C A N A D I A N R E V I E W OF SOCIOLOGY & A N T H R O P O L O G Y 3 tFrom a single article C O M M U N I T Y T E A M 1 C H I L D D E V E L O P M E N T 4 C O L L E G E COMPOSITION & C O M M U N I C A T I O N 1 C O L L E G E B O A R D R E V I E W 2 C H I L D R E N T O D A Y 1 C O M M U N I C A T I O N R E V I E W 1 C A N A D I A N E D U C A T I O N & R E S E A R C H DIGEST 1 C O M P A R A T I V E E D U C A T I O N A L R E V I E W 1 D E V E L O P M E N T A L P S Y C H O L O G Y 14 D A E D A L U S 4 DIRECTOR & B O A R D 1 E D U C A T I O N A L L E A D E R S H I P 3 E D U C A T I O N A L CONSIDERATION 1 • E D U C A T I O N A L G E R O N T O L O G Y 14 E D U C A T I O N & C U L T U R E 2 E D U C A T I O N & P S Y C H O L O G I C A L M E A S U R E M E N T 3 ERISTICS 1 E D U C A T I O N A L R E S E A R C H 6 E N G L I S H E D U C A T I O N 1 E N G L I S H J O U R N A L 1 E D U C A T I O N A L F O R U M 1 E X C E P T I O N A L CHILD 1 E D U C A T I O N 1 E D U C A T I O N A L R E V I E W 3 E D U C A T I O N A L T E C H N O L O G Y 4 E D U C A T I O N A L R E S E A R C H B U L L E T I N 1 E D U C A T I O N B R O A D C A S T I N G I N T E R N A T I O N A L 2 E D U C A T I O N A L STUDIES 1 E D U C A T I O N A L RECORD 1 E D U C A T I O N A L ADMINISTRATOR Q U A R T E R L Y 1 E D U C A T I O N A L T H E O R Y 3 E D U C A T I O N F O R SOCIAL W O R K 1 E V A L U A T I O N 1 E V A L U A T I O N C O M M E N T 3 E V A L U A T I O N & H E A L T H P R O F E S S I O N A L S 2 • E X T E N S I O N INSIGHTS 1 FINDINGS 3 F U T U R I S T 4 F A R E A S T S U R V E Y 1 F A M I L Y COORDINATOR 1 F R E E M A N 1 F O R U M 1 GERONTOLOGIST 9 GENETIC P S Y C H O L O G I C A L M O N O G R A P H S 1 GERIATRICS 1 H I G H E R E D U C A T I O N 1 H I G H SCHOOL B E H A V I O R S C I E N C E 1 H U M A N ORGANIZATION 3 H A R V A R D BUSINESS R E V I E W 7 H A R V A R D E D U C A T I O N A L R E V I E W 21 H U M A N R E L A T I O N S 5 H E A L T H SERV. & M E N T A L H E A L T H A D M I N . H E A L T H REPORTS 1 H I G H S C H O O L J O U R N A L 1 H U M A N D E V E L O P M E N T 5 HOSPITALS 1 H U M A N R E S O U R C E M A N A G E M E N T 1 H O S P I T A L A N D C O M M U N I T Y P S Y C H I A T R Y 1 I N D U S T R I A L R E L A T I O N S 3 I N D U S T R I A L G E R O N T O L O G Y 1 I N D U S T R I A L PIONEER 1 I N T E R N A T I O N A L SOCIAL SCIENCE J O U R N A L 1 I M P R O V I N G C O L L E G E & U N I V E R S I T Y T E A C H I N G 4 I M M I G R A T I O N I N A M E R I C A R E V I E W 2 I N T E R N A T I O N A L J O U R N A L OF P S Y C H O A N A L Y S I S 1 I N T E R N A T I O N A L J O U R N A L OF A G I N G & H U M A N D E V E L O P M E N T 5 I N T E R N A T I O N A L J O U R N A L OF P S Y C H O L O G Y 2 I N T E R N A T I O N A L M A N A G E M E N T 1 I N T E R N A T I O N A L R E V I E W OF APPIED P S Y C H O L O G Y 1 I N T E L L E C T U A L DIGEST 1 INSTRUCTOR 1 I N S T R U C T I O N A L T E C H N O L O G Y 1 I N T E L L E C T 1 ILLINOIS E D U C A T I O N A L R E V I E W 1 INSTRUCTOR OF SCIENCE 1 I N N O V A T I O N S I N H I G H E R E D U C A T I O N 1 I N D I A N E D U C A T O R 1 J O U R N A L OF E D U C A T I O N A L P S Y C H O L O G Y 26 J O U R N A L O F M E D I C A L E D U C A T I O N 3 J O U R N A L O F A P P L I E D P S Y C H O L O G Y 12 J O U R N A L O F SOCIAL P S Y C H O L O G Y 6 J O U R N A L OF P S Y C H O L O G Y 2 J O U R N A L O F HIGHER E D U C A T I O N 4 • J O U R N A L OF CONTINUING E D U C A T I O N & TRAINING 2 • J O U R N A L OF E X T E N S I O N 4 (J. C O O P E R A T I V E EXTENSION) J O U R N A L O F E D U C A T I O N FOR SOCIAL W O R K 1 • J O U R N A L OF A M E R I C A N SOCIETY FOR T R A I N I N G 1 J O U R N A L O F R E A D I N G B E H A V I O R 2 J O U R N A L O F R E S E A R C H & D E V E L O P M E N T I N E D U C A T I O N 6 • J O U R N A L OF T H E C O M M U N I T Y D E V E L O P M E N T SOCIETY 2 J O U R N A L O F G E R O N T O L O G Y 27 J O U R N A L O F R E A D I N G 17 J O U R N A L O F H U M A N I S T P S Y C H O L O G Y 5 J O U R N A L O F SOCIAL ISSUES 2 J O U R N A L O F HISTORY OF P H I L O S O P H Y 1 • J O U R N A L OF C O R R E C T I O N A L E D U C A T I O N 2 J O U R N A L O F A B N O R M A L & SOCIAL P S Y C H O L O G Y 15 J O U R N A L O F A B N O R M A L SOCIAL P S Y C H O L O G Y 5t J O U R N A L O F C L I N I C A L P S Y C H O L O G Y 1 tFrom a single article J O U R N A L O F P H Y S I C A L T H E R A P Y 1 J O U R N A L O F A P P L I E D B E H A V I O R A N A L Y S I S l i t J O U R N A L OF A P P L I E D B E H A V I O R SCIENCE 10 J O U R N A L OF E X P E R I M E N T A L A N A L Y S I S OF B E H A V I O R 1 J O U R N A L OF N A T . ASSOC. OF W O M A N D E A N S & C O U N S E L O R S 6 J O U R N A L O F M A R R I A G E & T H E F A M I L Y 1 J O U R N A L O F H O M E ECONOMICS 1 J O U R N A L OF R E S E A R C H I N SCIENCE T E A C H I N G 5 J O U R N A L OF M A R K E T R E S E A R C H 1 J O U R N A L OF C O M M U N I C A T I O N 6 J O U R N A L O F E X P E R I M E N T A L E D U C A T I O N 1 J O U R N A L OF A P P L I E D SOCIOLOGY 1 J O U R N A L O F T E A C H E R E D U C A T I O N 2 J O U R N A L OF B L A C K STUDIES 1 J O U R N A L O F I N T E R D I S C I P L I N A R Y HISTORY 1 J O U R N A L OF BRITISH SOCIETY OF P H E N O M E N O L O G Y 1 • J O U R N A L OF C O N T I N U I N G E D U C A T I O N I N N U R S I N G 3 J O U R N A L OF R E S E A R C H I N T E A C H I N G 1 J O U R N A L OF A M E R I C A N I N S T R U C T I O N A L P L A N N I N G 1 J O U R N A L O F G E R O N T O L O G I C A L SOCIAL WORK 1 J O U R N A L OF A L L I E D H E A L T H 1 J O U R N A L OF M E D I C A L ASSOCIATION OF GEORGIA 1 J O U R N A L OF C R I M I N A L L A W & C R I M O N O L O G Y 1 J O U R N A L OF D E N T A L E D U C A T I O N 1 J O U R N A L O F E D U C A T I O N A L SOCIOLOGY 1 J O U R N A L O F A M E R I C A N D E N T A L ASSOCIATION 1 J O U R N A L O F P H I L O S O P H Y OF E D U C A T I O N 1 J O U R N A L OF K E N T U C K Y M E D I C A L ASSOCIATION 1 J O U R N A L OF H E A L T H & SOCIAL B E H A V I O R 1 J O U R N A L OF E X P E R I M E N T A L L E A R N I N G & S I M U L A T I O N 1 J O U R N A L O F M E D I C A L E D U C A T I O N 4 J O U R N A L OF E D U C A T I O N OF T E C H N O L O G I C A L S Y S T E M S 1 J O U R N A L OF P E R S O N A L I T Y & SOCIAL P S Y C H O L O G Y 2 J O U R N A L OF G E N E T I C P S Y C H O L O G Y 1 J O U R N A L OF I N S T R U C T I O N A L D E V E L O P M E N T 1 J O U R N A L OF P S Y C H O S O M A T I C R E S E A R C H 2 J O U R N A L O F SOCIOLOGY 1 J O U R N A L OF CROSS C U L T U R A L P S Y C H O L O G Y 1 J O U R N A L OF C O N S U L T I N G & C L I N I C A L P S Y C H O L O G Y 1 J O U R N A L OF C O U N S E L I N G P S Y C H O L O G Y 4 J O U R N A L OF Y O U T H A N D A D O L E S C E N C E 1 J O U R N A L O F C O L L E G E S T U D E N T P E R S O N N E L 8 J O U R N A L O F I N D U S T R I A L T E A C H E R E D U C A T I O N 1 J O U R N A L OF A M E R I C A N P H A R M A C Y E D U C A T I O N 2 J O U R N A L O F SOCIAL S C I E N C E 1 J O U R N A L OF E D U C A T I O N 2 J O U R N A L OF N A T I O N A L INSTITUTE OF P E R S O N N E L R E S E A R C H 1 J O U R N A L O F P R O G R A M M E D L E A R N I N G 1 tTen from single article J O U R N A L OF L E A R N I N G DISABILITY 1 J O U R N A L OF C O N S U L T I N G P S Y C H O L O G Y 2 J O U R N A L OF H U M A N R E S O U R C E 3 J O U R N A L OF D E V E L O P M E N T I N R E A D I N G 5 J O U R N A L OF E D U C A T I O N A L M E A S U R E M E N T 2 J O U R N A L OF E D U C A T I O N A L R E S E A R C H 13 J O U R N A L OF D E V E L O P M E N T A R E A S 2 J O U R N A L OF A M E R I C A N M E D I C A L ASSOCIATION 2 J O U R N A L OF M A R K E T I N G l i t J O U R N A L OF H E A L T H , P H Y S I C A L E D U C A T I O N & R E C R E A T I O N J O U R N A L OF E X P E R I M E N T A L P S Y C H O L O G Y 1 J O U R N A L OF R E G I O N A L S C I E N C E 1 J O U R N A L OF A P P L I E D SOCIAL P S Y C H O L O G Y 1 J O U R N A L OF P E R S O N N E L & SOCIAL P S Y C H O L O G Y 1 J O U R N A L OF P S Y C H O S O M A T I C R E S E A R C H 2 J O U R N A L OF ETHNIC STUDIES 1 J O U R N A L I S M Q U A R T E R L Y 6 J U N I O R C O L L E G E J O U R N A L 1 K E Y REPORTER 1 L I B R A R Y TRENDS 1 • L I F E L O N G L E A R N I N G 90 (ADULT LEADERSHIP) • L I T E R A C Y DISCUSSION 1 • L E A R N I N G 2 (CONTINUOUS L E A R N I N G ) MIDWEST SOCIOLOGY 1 M A N A G E M E N T R E V I E W 1 M c G I L L J O U R N A L OF E D U C A T I O N 1 M A N A G E M E N T 1 M B A 1 M O N T H L Y L A B O R REVIEW 2 M A R K E T I N G F O R U M 1 M A N A G E M E N T SCIENCE 1 M E D I C A L & B I O L O G I C A L I L L U S T R A T I O N 1 M E A S U R E M E N T & E V A L U A T I O N IN G U I D A N C E 1 M E R R I L L - P A L M E R Q U A R T E R L Y 2 M U L T I V A R I A T E B E H A V I O R R E S E A R C H 4 M I C H I G A N S T A T E U N I V E R S T Y BUSINESS TOPICS 1 N E A U Q U A R T E R L Y 2 N A S S P B U L L E T I N 1 NOTES O N E D U C A T I O N 1 N U R S I N G R E S E A R C H 2 N U R S I N G O U T L O O K 4 N U R S I N G F O R U M 1 • NOTES & E S S A Y S O N A D U L T E D U C A T I O N 1 N E W E N G L A N D J O U R N A L OF M E D I C I N E 1 N A T I O N ' S SCHOOLS 2 N E W Y O R K U N . E D U C A T I O N Q. 2 N E W POLITICS 1 N E W O U T L O O K FOR T H E B L I N D 1 tTen from single article 71 N E W R E P U B L I C 2 O R G A N I Z A T I O N A L D Y N A M I C S 2 Q U A R T E R L Y J O U R N A L OF E X P E R I M E N T A L P S Y C H O L O G Y 1 Q U A R T E R L Y J O U R N A L OF ECONOMICS 1 PEDIATRICS 1 P E R C E P T I O N & MOTOR S K I L L S 6t P S Y C H O L O G I C A L REPORTS 7 P S Y C H O A N A L Y T I C Q U A R T E R L Y 1 P S Y C H O L O G Y B U L L E T I N 14 P S Y C H O L O G Y T O D A Y 8 P S Y C H O M E T R I K A 5 P S Y C H O L O G I C A L M O N O G R A P H S 7 P S Y C H O L O G I C A L R E V I E W 6 P S Y C H O L O G I C A L ISSUES 3 P S Y C H O T H E R A P Y T H E O R Y R E S E A R C H & P R A C T I C E 1 PHI D E L T A K A P P A N 6 P S Y C H O M A T I C SCIENCE 1 P S Y C H O L O G Y I N T H E SCHOOLS 2 P E R S O N N E L P S Y C H O L O G Y 2 PROGRESSIVE E D U C A T I O N 1 PROGRESSIVES 2 P E R S O N N E L & G U I D A N C E J O U R N A L 3 P U B L I C OPINION Q U A R T E R L Y 3 P R O G R A M M E D L E A R N I N G & E D U C A T I O N A L T E C H N O L O G Y 1 P E R S P E C T I V E S O N BIOLOGY & M E D I C I N E 1 P U B L I C A D M I N I S T R A T I O N R E V I E W 2 P U B L I U S 1 P H Y S I C A L T H E R A P Y R E V I E W 2 P U B L I C H E A L T H REPORTS 1 P O P U L A T I O N STUDIES 1 P E K I N G R E V I E W 2 PROSPECTS 1 P E D A G O G I C A E U R O P E A 1 P U B L I C R E L A T I O N S J O U R N A L 1 P E R S O N N E L A D M I N I S T R A T I O N 1 P H I L O S O P H I C A L STUDIES 1 P H Y S I C A L T H E R A P Y 1 R E S E A R C H I N E D U C A T I O N 1 R E S E A R C H I N HIGHER E D U C A T I O N 1 R E S E A R C H O N A G I N G 1 R E S E A R C H O N M A N A G E M E N T 1 R E S E A R C H I N T E A C H I N G OF E N G L I S H 1 RELIGIOUS E D U C A T I O N 3 R E W L E Y H O U S E P A P E R S 1 R E A D I N G R E S E A R C H Q U A R T E R L Y 6 R U R A L SOCIOLOGY 6 R E V I E W OF E D U C A T I O N A L R E S E A R C H 13 R E A D I N G T E A C H E R 1 tFrom single article R E V I E W OF R A D I C A L P O L I T I C A L E C O N O M I C S 1 R E V I E W OF E X I S T E N T I A L P S Y C H O L O G Y & P S Y C H I A T R Y 2 SOCIOLOGIE E T SOCIETE 1 SETTING T H E P A C E 1 S P E E C H M O N G R A P H S 1 SOCIOLOGY R E V I E W 2 S E M I N A R IN P S Y C H A T R Y 1 S C A N D I N A V I A N J O U R N A L OF P S Y C H O L O G Y 3 SCIENTIFIC M O N T H L Y 1 SCHOOL R E V I E W 10 SOCIOLOGY OF E D U C A T I O N 3 SOCIAL B I O L O G Y 1 SOCIAL P O L I C Y 1 • STUDIES I N A D U L T E D U C A T I O N 14 SCIENCE E D U C A T I O N 1 SOCIAL FORCES 5 SOCIAL R E S E A R C H 1 SOCIAL R E V I E W 1 SOCIOMETRY 7 SCHOOL S C I E N C E & M A T H 1 SOCIAL E D U C A T I O N 1 SOCIAL P R O B L E M 4 SCIENCE 3 SCIENTIFIC A M E R I C A N 1 S A T U R D A Y R E V I E W 2 SOCIAL SCIENCE Q U A R T E R L Y 2 SOCIOLOGY T O D A Y 1 SOCIAL S C I E N C E Q U A R T E R L Y 2 SOCIOLOGY Q U A R T E R L Y 1 SCIENCE & C H I L D R E N 1 SOCIOLOGY F O C U S 1 SCHOOL & SOCIETY 2 S C A N D I N A V I A N J O U R N A L OF E D U C A T I O N A L R E S E A R C H 1 SOCIAL W O R K 2 S U P E R V I S O R Y M A N A G E M E N T 1 S E C U R I T Y M A N A G E M E N T 1 S L O A N M A N A G E M E N T R E V I E W 1 • T R A I N I N G 1 T R A N S - A C T I O N 2 T E A C H E R C O L L E G E RECORD 12 • T R A I N I N G & D E V E L O P M E N T J O U R N A L 4 • TUTOR'S B U L L E T I N OF A D U L T E D U C A T I O N 1 • U N I V E R S I T Y E X T E N S I O N W O R L D 1 • U N I V E R S I T Y of N O R T H C A R O L I N A E X T E N S I O N B U L L E T I N 2 U N I V E R S I T Y RECORD 1 U R B A N L I F E 1 U R B A N A F F A I R S Q U A R T E R L Y 1 V O C A T I O N A L G U I D A N C E Q U A R T E R L Y 1 VIEWPOINTS 3 V I S I B L E L A N G U A G E 1 W O R K E R ' S E D U C A T I O N Q U A R T E R L Y J O U R N A L APPENDIX B Names of Experts Participating in Instrument Validation Procedures Dr. Roger Hiemstra - Syracuse U . , Syracuse, N . Y . Dr. Sharon B . Merriam - Northern Illinois U . , DeKalb Dr. Huey B. Long - U . of Georgia, Athens Dr. Paul J . Ilsley - Syracuse U . , Syracuse, N . Y . Dr. Jerold W. Apps - U . of Wisconsin, Madison Mr . David J . Littlet - University of Regina, Regina, Sask. Dr. Harold Beder - Rutgers U . , New Brunswick, N . J . Dr. Alan B. Knox - U . of Wisconsin, Madison Dr. Hayden Roberts - U . of Alberta, Edmonton Dr. Allen Tough - O.I.S.E., Toronto Dr. Stephen Brookfield - Teacher's College, Columbia University, N . Y . t M r . Little is currently a doctoral candidate at the University of British Columbia, and a faculty member at the University of Regina. 74 APPENDIX C Letter to Experts Requesting Their Participation in Instrument Validation Procedures Adult Education Research Centre 5760 Toronto Road, Vancouver, V6T 1L2, B.C. September 6th, 1986. Dear Professor: I am an M . A . student in the Adult Education Program at the University of British Columbia. While meeting with my thesis committee: Drs. Roger Boshier and Tom Sork, it was suggested that I write to you. M y reason for writing is to request your assistance with the validation procedures of thesis. The checklist on the reverse side of this letter contains names which have previously been categorized during my research effort. I am hoping to validate my research tool by having independent experts check off the professional adult educators from the names supplied. The process should take from five to ten minutes of your time. I have enclosed a stamped, self-addressed envelope for your convenience. I wish to thank you in advance for helping me to present a more complete research project to my committee. Yours Sincerely, Richard O. Kavanagh Graduate Student Administrative, Adult and Higher Education University of British Columbia 75 APPENDIX D Validation Procedure and Results INSTRUCTIONS: Please indicate with a check (/) all of the following who are primarily known for their adult education activities. Name Rule-Based Classification Experts Agreeing Knowles, Malcolm S. primary 11/11 Houle, Cyri l 0 . primary 11/11 Verner, Coolie primary 11/11 Tough, Allen primary 11/11 Knox, Alan B . primary 11/11 Freire, Paulo primary 10/11 Lindeman, Eduard C. primary 10/11 Boshier, Roger primary 11/11 Havighurst, R . J . primary 2/11 Maslow, A . H . secondary 11/11 London, Jack primary 10/11 Cross, K . Patricia primary 6/11 Darkenwald, Gordon G. primary 11/11 Bergevin, Paul primary 11/11 Johnstone, John W.C. primary 7/11 Rivera, Ramon J . primary 6/11 Dickinson, Gary primary 11/11 Hiemstra, Roger primary 11/11 Illich, Ivan secondary 10/11 Grabowski, Stanley M . primary 11/11 Kerlinger, Fred N . secondary 11/11 Erikson, Erik H . secondary 11/11 Kidd, J.R. primary 11/11 Dewey, John secondary 11/11 Ohliger, John primary 11/11 Sticht, Thomas secondary 10/11 Miller, Harry H . primary 7/11 Rogers, Carl secondary 10/11 Sheffield, Sherman primary 3/11 Griffith, William S. primary 10/11 Carlson, Robert A . primary 11/11 Douglah, M . primary 5/11 Litchfield, Anne primary 6/11 Monette, Maurice primary 8/11 Penland, Patrick R. primary 9/11 Gagne, R . M . secondary 11/11 Blakeley, Robert J . primary 10/11 Allport, Gordon W. secondary 11/11 76 Levinson, Daniel J . secondary 11/11 Neugarten, B . L . secondary 11/11 Rubenson, Kjell primary 10/11 Benne, Kenneth primary 5/11 Brockett, R. primary 10/11 Edwards, Allen L . secondary 11/11 Kotler, Philip secondary 11/11 Sheehy, Gail secondary 11/11 Skinner, B .F . secondary 11/11 Ausubel, David P. secondary 11/11 Cartwright, Morse A . primary 7/11 Rokeach, Milton secondary 11/11 Fishbein, M . secondary 11/11 Vincent, John H . primary 1/11 Corwin, Ronald G. secondary 11/11

Cite

Citation Scheme:

    

Usage Statistics

Country Views Downloads
Japan 24 0
United States 2 0
China 1 15
City Views Downloads
Tokyo 24 0
Unknown 1 4
Beijing 1 0
Ashburn 1 0

{[{ mDataHeader[type] }]} {[{ month[type] }]} {[{ tData[type] }]}

Share

Share to:

Comment

Related Items