UBC Theses and Dissertations

UBC Theses Logo

UBC Theses and Dissertations

A citation analysis of "Adult education quarterly" 1971-1986 Kavanagh, Richard Owen 1987-12-31

You don't seem to have a PDF reader installed, try download the pdf

Item Metadata

Download

Media
UBC_1987_A8 K38_2.pdf [ 3.65MB ]
Metadata
JSON: 1.0055771.json
JSON-LD: 1.0055771+ld.json
RDF/XML (Pretty): 1.0055771.xml
RDF/JSON: 1.0055771+rdf.json
Turtle: 1.0055771+rdf-turtle.txt
N-Triples: 1.0055771+rdf-ntriples.txt
Original Record: 1.0055771 +original-record.json
Full Text
1.0055771.txt
Citation
1.0055771.ris

Full Text

A CITATION  A N A L Y S I S OF " A D U L T E D U C A T I O N  Q U A R T E R L Y " 1971-1986  by RICHARD  OWEN  KAVANAGH  B . S . N . , University of Victoria,  A THESIS SUBMITTED  1983  IN PARTIAL FULFILMENT  THE REQUIREMENTS  FOR THE DEGREE  OF  OF  MASTER OF ARTS  in THE F A C U L T Y OF GRADUATE  STUDIES  Department of Administrative, Adult and Higher Education  We accept this thesis as to the required  THE UNIVERSITY  conforming  standard  OF BRITISH  April  COLUMBIA  1987  ® Richard Owen Kavanagh,  1987  32  In presenting this thesis in partial fulfilment of the requirements for an advanced degree at The University of British Columbia, I agree that the Library shall make it freely available for reference and study. I further agree that permission for extensive copying of this thesis for scholar^ purposes may be granted by the Head of my Department or by his or her representatives. It is understood that copying or publication of this thesis for financial gain shall not be allowed without my written permission.  Department of Administrative, Adult and Higher Education The University of British Columbia 2075 Wesbrook Place Vancouver, Canada V 6 T 1W5 Date: April 1987  ABSTRACT  Adult education has long been described as an emerging discipline, but there has been little empirical study of its emergence. This study examined 'emergence' monitoring Studies  that  particular  concerned  quantifiable  with  knowledge the  theory  indicator of unique  base  which  and  practice  knowledge  about  is unique of  to  adult  adult  adult  by  education.  education  are  a  education.  Evidence that  researchers in adult education increasingly cite the work of other  researchers in  adult education would support the contention that the body of knowledge in adult education is growing.  The articles published in Adult Education  Quarterly between  1971 and 1986 were  analyzed using citation analysis methodology. The frequency of citation to previous adult  education  peripheral  to  an  studies  (primary  adult  education  literature) context  as  opposed  (secondary  to  citation  literature)  of  was  studies  determined.  Distinguishing between citation categories was carried out by analyzing each title cited. The phenomenon of concern in the words  used  in  the  title,  and  coded  cited article was interpreted  dichotomously  as  'primary  'secondary literature'. Each coded item was then recorded under  from  literature'  named  the or  authors;  thus, the cited author was credited for total frequency cited along with the coded category  of  literature). data),  writing Reliability  and  (author  of  measures  inter-judge  primary performed  agreement  literature for  with  the  latter.  Validity  of the  procedures  ii  author  intra-judge  (independent  differences in coding of less than four percent  or  coding  of  secondary  consistency of  data)  resulted  for the former, and nine  used  in coding cited  (recoding in  percent  authors  was  tested by comparing results asked  obtained to a 'standard'.  to identify from a list of the  volume  period,  activities."  The  those  who  study's  were  coding  twenty  were  most cited authors from each  "primarily  outcome  'Independent experts'  known  of  these  for  their  authors  adult  compared  four  education with  the  expert's 'standard' resulted in greater than 75 percent agreement.  With  4700 citations classified, it was found that a rising percentage of citations  were to the the  first  "authors of primary literature"; from 41 percent  half of the  (1979-1986). A further literature  journals"  '1971-1978'  to  study  also  increasing;  39 percent  exclusively from one  with  this  from  in '1979-1986'.  base in adult education  field. Implications for future  31  percent  in the  last  half  As the  of  all journals  cited in  scope  of literature  analyzed  North American journal, results  limitation in mind. However, the literature'  to 46 percent  breakdown showed the percentage of citations to "primary  was  'primary  period (1971-1978),  of all citations in  research  need  to be  empirical evidence of an research  regarded increasing  suggests emergence  of the  are discussed in light of this and previous  studies.  iii  TABLE OF CONTENTS ABSTRACT  ii  LIST OF T A B L E S  v  LIST OF FIGURES  vi  C H A P T E R 1: P R O B L E M Operationalizing Emergence Purposes Scope of the Study Significance Thesis Structure  1 1 5 6 8 9  C H A P T E R 2: R E V I E W O F T H E L I T E R A T U R E Knowledge About Adult Education Content Analysis Citation Analysis Rationale for Study Hypotheses  10 10 12 16 18 22  C H A P T E R 3: M E T H O D O L O G Y Data Collection Data Analysis Reliability and Validity Reliability: Test Retest Reliability: Reproducibility Validity: Expert Standard  24 24 31 32 33 34 34  C H A P T E R 4: R E S U L T S Reliability and Validity Study Population Cited Authors Cited Journals  37 37 41 42 46  C H A P T E R 5: D I S C U S S I O N , L I M I T A T I O N S & I M P L I C A T I O N S Discussion Limitations Implications  51 51 58 59  REFERENCES  61  APPENDIX A  65  APPENDIX B  74  APPENDIX  C  75  APPENDIX D  76 iv  LIST OF T A B L E S  Table 1: Study Procedures  Compared with Boshier and Pickard (1979)  18  Table 2: Steps Used to Code Eligible Citations  26  Table 3: Major Components of the Study  27  Table 4: Example of Author Coding Sheet  30  Table 5: Example of Journal Coding Sheet  30  Table 6: Procedures  33  for Establishing Reliability and Validity  Table 7: Discrepancies in Intra-Judge Recoding  38  Table 8: Discrepancies Found by Validity Test  40  Table 9: Percent Citations to Authors of Primary Literature Volume 21-36  44  Table 10: Most Frequently Cited Authors: Four Periods  45  Table 11: Top Twenty Authors Ranked by Citation Value  46  Table 12: Percent Journal Citations to Primary Literature Journals  48  Table 13: Most Frequently Cited Journals  50  Table 14: Citations in the Adult Education Quarterly to Its Own Articles  52  v  LIST OF F I G U R E S  Figure 1: Hypothesized Proportions of Citations as a Field Emerges  21  Figure 2: Proportions of Citation to Authors of Primary Literature  42  Figure 3: Proportion Cited Journals to Primary Literature Journals  47  Figure 4: Primary and Secondary Literature Found in Four Periods  57  vi  CHAPTER 1  PROBLEM  Adult  education  has  long been  described as  an emerging discipline, but  there has been little empirical study of its emergence. controversy  within  the  'community'  of  adult  In part, this is due to a  educators  as  to  whether  adult  education can be a discipline unto itself. Proponents point to the establishment of adult education  within universities, its rising professionalism, and the  production  of sophisticated  research  that  adult  is  education  as  without  evidence theory  of emergence.  of its  own,  Opponents  questions  claim  concerning  since  emergence  warrant no further investigation.  This field  thesis  was  an  attempt  to  quantitatively  assess emergence  of  the  by studying the changing nature of citations found in articles appearing in  Adult  Education  Quarterly  (formerly  Adult  Education),  the  major  journal  of  research and theory in North American adult education.  Operationalizing Emergence  A discipline is best represented  by the body of knowledge used to guide  practice. According to Verner et al. (1970), " A n academic discipline is defined as a  body  of  systematic  Knowledge is the relatively new  field  knowledge  quintessential of study  founded  element  and has  in  theory  and  research"  in all disciplines. Adult  (p.  7).  education  is a  had to rely on knowledge obtained  from  1  2 more  established disciplines.  shucked  this  fledgling  However, the  nature,  won  extent  over  to which  adult education  practitioners  from  their  has  original  intellectual alignments, and created unique knowledge of its own is not clear.  Unique knowledge distinguishes one discipline from concepts  developed  outside  the  field  dominated  another.  In the  past,  which  guided  adult  knowledge  education practice. Recent speculation holds that this situation may be changing because of the increase in the numbers and advancement in the skills of adult education researchers. control  knowledge,  practice.  It  is  A community of researchers forms in order to share and  knowledge  this  developed  evolution  that  through  Peters  testing  (1980)  ideas  spoke  of  in  the  field  when  he  of  said,  "Ultimately, we need strong theories to guide our research efforts, but first we need concepts, and clusters of concepts, to frame our practice" (p. and  the  field.  knowledge  121). There  is a symbiotic relationship between  The discipline studies  developed by  the  our thinking and to underpin  problems in the  discipline  field,  (e.g., Boshier,  the  and the  discipline field  uses  1980; Brunner & Verner,  1968). According to Verner (1960):  The growing maturity of a field is represented by the changing character of its literature and by the attention it pays to it . . . . [Adult education] is reaching a point where research is beginning to be structured on prior research so that a consistent body of knowledge is being accumulated, (p. 171)  The research-based  "consistent  body  knowledge from  of  knowledge"  indicator  of  the  Verner  refers  an adult education context. The extent  studies concerned with phenomena of the field an  that  awareness  of  adult  to  is  to which  are referenced in the research is  education  researchers  of  the  unique  3 knowledge  available.  researchers  of  'emergence'  Evidence  unique  that  increased  knowledge  would  use  is  support  in adult education. Establishing a  being  an  made  by  argument  quantitative  these  concerning  measurement  of the  degree to which adult education research is, as Verner said, "structured on prior research" was the focus of this thesis.  Research hold  to  be  conducted by  important,  and  Behavior of researchers research  reports.  These  scholars in a  pursued  in a  field  reports  field  in manners  is based  they  on questions  believe to  be promising.  can be most readily observed through  describe  the  problems,  they  their  significance, approaches,  and resources used in the research process. A n essential resource in each study is the citing  review of previous studies in  research  contribution,  literature  is  related the  or relevance, of the  most  to the  research  common  work of others.  means  question. The act of of  According to  recognizing the Garfield  (1979)  citations are "nothing more, or less, than a reflection of that community's work and  interest"  (p.  247).  conceptualization of the  Thus,  citing  behavior  reveals  common  connections  in  field.  Citation analysis has been used for many investigations of trends in and characteristics the  of literature.  Researchers  have  attempted  to  identify  patterns in  citing behavior of groups of authors, journals, and disciplines as  Citation analysis as  a research  method has been used to address  concerns: the eminence of scholars by the frequency (e.g., Aaronson, 1975; Garfield,  a whole.  a variety of  with which they  are cited  1979); the impact of articles by their connection  through citation to the work of others (Garfield,  1972); the contribution of other  4 fields  as  determined  developed Smith  (Miller,  by  1982);  & Caulley,  investigating and  where  interjournal  cited  citing  materials  behaviors  (e.g.,  1981; Xhignesse & Osgood, 1967). Miller  analysis of citations is valuable in identifying  the  were  originally  Myers,  1970;  (1982) states, "The  core research  literature  in a  particular field or discipline" (p. 798).  Citation answers  to  analysis  questions  is  a  concerning the  literature. The origins of the identified three  by  the  research  method  origin  concepts  the  name(s)  of the  of providing empirical  concepts  used  used in scholarly literature  bibliography or reference  components:  capable  of the  section of an  author(s),  the  in  a  field's  are normally  article. Citations have  title  of  the  work,  and  publication information or a description of the context in which the material was produced (e.g., speech at conference with dates, personal communications). In this study  all three of these components  were used to establish whether  a citation  was considered "primary literature" or "secondary literature."  In  this  section  it  has  been  argued  that  emergence  of a  discipline  linked to possession of unique knowledge, and that knowledge is unique  is  because  of the context in which it was produced. Thus, to find that studies produced in an adult education context represent  an increasing proportion of citations found in  an important journal would suggest that the field is indeed emerging.  5 Purposes  This study examined emergence in adult education by identifying in  the  citing behavior of authors publishing in Adult  categories  of cited  studies  were  designated,  those  by  Education authors  changes  Quarterly. Two whose  work  is  grounded in an adult education context, deemed 'primary literature', and those by authors who are not identified with an adult education context, deemed 'secondary literature'. For the purposes of this thesis these terms were defined as follows:  'Primary Literature' is any publication whose title explicitly identifies it as being concerned with concepts, processes, or data from an adult education context or whose publication information (e.g., name of journal, name of publisher, name of editor or sponsor) is explicitly identified with an adult education context or whose author(s) has previously been classified as writing primary literature (as above). 'Secondary Literature' is any publication whose title does not manifestly identify it as being concerned with an adult education context and whose publication information is not explicitly identified with an adult education context and whose author(s) has not previously been classified as writing 'primary literature'. 'Primary literature' contains studies authored by someone whose work is grounded in an adult education context. The result of these studies is knowledge which is unique  to  adult  education,  because  the  author(s)  is  writing  from  an  adult  education context. 'Secondary literature' contains studies by authors not manifestly connected  to  an  adult education context, designed to relate  to the  problems of  another field. Results meaningful to adult educators are only a by-product of this  6 research,  and  require  reformulation  context.  These  Findings  require  control,  or prediction which  studies the  are  deemed  additional step their  to  become  to  be  relevant  'secondary  in  an  literature'  of reformulation before  concepts  may  adult  offer  can  in  adult  because  any be  education their  understanding,  of benefit  in an  adult education context.  In  this  attempt  to  quantify  'emergence'  analysis of Adult Education Quarterly for the years  education  a citation  1971-1986 was performed for  the purposes of: 1.  Determining proportions  of citations to  "authors of primary  literature"  "authors of primary  literature"  and "authors of secondary literature", 2.  Determining proportions  of citations to  and "authors of secondary literature" who are most frequently cited, 3.  Determining proportions in a subset of the journals deemed  data  "primary literature journals" and  collected, that of the "secondary  literature  journals."  Scope of the Study  It  is widely assumed  that a relationship exists between  the  doing and  reporting of research and the development of the discipline. In this study it was necessary  to investigate  adult education research, but decisions had to be  made  on where to find representative research, and how much to analyze.  Allcorn  (1985)  was  confronted  with  a  similar  situation  and recognized  7 that any effort to analyze the information published on adult education must be restricted  to  a  manageable  size  (p.  12).  Three  general  categories  of  adult  education literature were identified by Grabowski (1974): published material in the form of books and pamphlets; fugitive documents; and periodicals (p. 3). To find literature containing research reports  from  adult education over time required a  collection that was both archival (reports past research) and current (up to date). Journals and conference proceedings met both of these criteria.  Nelson (1972), in his study on the fate of conference papers, found that, "The two most important media of scientific communication, national meetings and journal articles, are closely related" (p. 4). The results of his study showed that 33 percent of conference presentation seven  at  percent  percent).  It  an  papers  American  were  accepted  appeared  that  were published in journals within two years of  Educational Research but  published  journals  were  a  Association  later, good  bringing medium  meeting the  for  (another  total  to  40  monitoring the  forefront of research reported at meetings.  Adult research  education's  oldest,  most  highly  circulated,  and  most  and theory periodical in North America is said to be Adult  Quarterly (e.g., Brookfield, 1960).  The  Journal  of Adult  through  a  channel  for  coverpage  American  Association  Education  number adult  states,  1982; Griffith for  Adult  in February,  of name educators  "Adult  & Roberts,  changes, for  Education  it  1981; Long,  Education  1929. Although has  fifty-seven  been years.  a  began the  prominent Education  1977; Verner, publishing  Journal  has  the gone  consistent communication Presently  this  journal's  Quarterly is a refereed journal committed to  8 the dissemination of research and theory in adult and continuing education." For the  purposes  of this  (formerly Adult Journal)  study  Education,  the  articles  The Journal  contained of Adult  were deemed to be representative  the field in North America, both past and  Studies  cited  in  another's  in Adult Education  of the  Education and  Adult  typical research  Quarterly Education  literature of  present.  work  are  an  observable  indicator  influence one author's work has on the work of others. Specifically, the  of  the  questions  posed were: 1.  2.  To what  extent  Education  Quarterly attributable to "authors of primary literature"?  What  changes,  were  if  references  any,  occur  cited  in  in  the  articles  proportion  published  of  in  Adult  authorships  to  'primary literature' and 'secondary literature' cited over time.  The study population consisted of citations from the articles published in the Adult  Education  Quarterly. The most recent  was studied because of a shift in the Journal's 'research  and  theory'  emphasis  was  sixteen year period (1971-1986) editorial policy in 1966 when the  proclaimed (Griffith,  1966). Data  collection  commenced with Volume 21 (1971) and continued through Volume 36 (1986) of the  Journal.  Significance  The knowledge  results in  adult  of this  study  education  and  help the  clarify  questions  influence  of  concerning creation of  'primary  literature'  and  9 'secondary sought  literature'  to  find  was  Quarterly's  articles  significant).  The  empirical data greater field.  on  this  that  over  ability (rather  of  time to  than  understanding  development. a  pattern  (one  of  identify a  The in  empirical citations  change  pattern  or  evidence  from  stability  in adult  this  Adult would  education  study  Education be  equally  research  with  by intuition) allows for analysis, interpretation,  of questions  concerning emergence  of the  adult  and  education  As Liveright (1964) emphasized, "More important . . . than whether or not  adult education is a true discipline is an examination of the kind of discipline it is likely to become as it moves toward professional status" (p. 89).  Thesis Structure  Chapter 2 contains a review of studies and methodologies previously used to  analyze  the  knowledge  investigations were procedures taken  to  describes  for  data  examine the  used  to form  collection the  findings  base  and  reliability and  in  adult  hypotheses analysis and  Chapter  limitations, and implications for future  education. for  5  this study.  are  validity  discussed, of  presents  research.  The  findings In  of  Chapter  along with  procedures discussion  used.  these 3  the  measures Chapter  concerning  4  results,  CHAPTER 2  REVIEW OF T H E L I T E R A T U R E  Characterizing the knowledge that lies behind adult education practice has been attempted by investigating the literature of the field. One route has been to analyze  the  research  literature  available: a  general  review, qualitative review,  interpretive review, or content analysis. Another more indirect approach has been through  studying the  citations found  in the  research  literature.  Earlier  studies  concerned with knowledge about adult education are reviewed in this chapter.  Knowledge About Adult Education  Adult education researchers have tried to describe, define, and organize a body of knowledge in order to guide practitioners. Sork (1982) in his review of research  literature  defines  meta-research  as  ".  .  .  systematic  study  of  the  processes and products of inquiry which characterize a discipline or field of study, or,  more  simply,  research  on  research"  (p.  1).  Basing  his  typology  on  an  analysis of publications about adult education research, Sork suggested there were six types of meta-research: 1. 2. 3. 4. 5. 6.  Inventories/registers of research, General reviews, Interpretive reviews on specific topics, Research agendas, Focused critiques of methodology, Frameworks or Paradigms for Understanding Improving Research.  10  and  11 These education.  categories  Each  offer  characterizes  provides clues as  a framework the  for reviewing knowledge about  knowledge  to adult education's  base  in  a  particular  adult  way  and  identity. The focus of this study was on  the relative use of 'primary literature' and 'secondary literature' as a resource in adult  education  research.  Thus,  studies  from  categories  2  and  6  were  most  relevant.  Sork's category 2, "general reviews of adult education literature", contains studies field  which  are  attempt to provide  taking, the  progress  insights  being made,  However, adult education literature reflects It  does  not  exist  in  a  into the  central  place,  is  and  the  (e.g., Allcorn,  to summarize the  needs for  the diversity of the not  disseminated  channels, and has become too vast to be gathered summary  direction researchers further field  over  45 years  the  inquiry.  of practice.  through  predictable  together for a comprehensible  1985; Verner, 1960). Beals and Brody (1941)  literature  in  attempted  ago, and noted then that they  had  "been thwarted by the range in importance of the topics treated, the. extent and character  of the  judgement"  relevant  literature,  (p. xvi). To analyze the  and  the  limitations of our  knowledge  and  adult education field is too massive a task  for any single study. Long (1983a) observed that:  The quantit}' of research concerning the education of adults is expanding at such a rapid rate that not since 1959 [Brunner et al.] has an effort been made to provide an extensive and intensive organized review of the broad field, (p. 19) Comprehensive  reviews  when reviews were  of  adult  education  published they  base such as research methods  tended  research to look at  were  rarely  available,  aspects of the  and  knowledge  (e.g., Merriam & Simpson, 1984), or consisted of  12 more superficial over-views of the field (e.g., Draper, 1985; Verner, 1968).  Category  6  studies,  "frameworks  or  paradigms  for  understanding  and  improving research", focuses on the task of reviewing what is known about the character  of the  body of knowledge in adult education. Two research  methods  within this category that have been used in adult education to characterize knowledge base are  content  analysis and citation analysis. The Adult  the  Education  Quarterly has been the subject of four content analyses, and one citation analysis.  Content Analysis  One of the first content analyses of adult education literature was done by Dickinson and Rusnell (1971). They recognized that professional journals were at the forefront of an area of study. They chose to do an analysis of articles in the first twenty volumes of Adult Education, the  contents  . . .  as  indications of the  "to ascertain trends and patterns in  development of the  education" (p. 177). Articles with named authors  discipline of adult  were reviewed for "the use of  space, type and subject of articles, citation practices, and authorship as well as a detailed analysis of research articles . . . " (p. 178). They recognized that their study covered a transition period in the Journal under  an  research statistical  editor methods  who were  techniques  fundamental  emphasized becoming  were  principles  being  research more  used.  (foundations  of  and  with the last few years being theory.  sophisticated They  found  practice),  They  and an  with  that  reported more  that  complex  increasing concern a  corresponding  for  lesser  emphasis on current forms (e.g., program description) within the subject matter of  13 the Journal.  They felt the  results  of their  study  supported  the  notion of adult  education emerging as a distinct field of study.  Several Education intended  years  later,  Long  and  Agyekum  (1974)  approached  Adult  with slightly different questions from those above. Their study was not as  a  replication of Dickinson  relationship between  different  editors  topics, and articles based  on the  the  relationship  author's  the  Rusnell,  and content,  and  between  and  but  examined  research  and  possible  design, article length  author's dissertation. university  the  They also examined  the  number  of  articles  published. The nine year period of their study overlapped with the period covered by  Dickinson  results They  and  Rusnell,  they asserted arrived at  so  three  further  that the Journal  volumes  was responding to the  this conclusion by noting changes  the importance placed on adult learning in the an  emphasis  affiliation  on program  of authors,  fortunes  planning in the  in the  concerning  the  increasing  analyzed.  In  their  discipline's needs.  subjects  studied  (e.g.,  60's was being overshadowed by  70's), and the  which Long and Agykum speculated  of various university programs.  conclusions  were  changes  in university  reflected the  changing  They confirmed Dickinson and RusnelPs sophistication  of  research  designs  and  analytical techniques.  Peters Long  and  Education  and Banks' (1982) period of analysis dovetailed with the  Agyekum's study, Quarterly.  and  They compared  continued their  results on content trends and methods.  on  to  findings  the with  end  of  Dickinson  1980  in  Adult  and Rusnell's  They concluded that the research  they found were following the same trends as had been reported  end of  in the  designs earlier  14 studies  (increasing  sophistication of  research  methods  with  more  experimental  designs). As well, they reported the content trends were continuing to emphasize the foundations of the field (formal philosophy of adult education), while declining proportions of articles were about personal beliefs and opinions of the authors.  The latest by  Allcorn  discussing  content  (1985). Adult possible  analysis completed on adult education literature Education  futures  for  Quarterly, Lifelong Learning,  adult  education  (collected from  and  was  18 articles  various journals)  were analyzed to find: . . . content trends that indicate the extent and direction of the development of adult education's knowledge base, [and] an effort is made to discern whether trends represent a purposeful development of knowledge for the field, (p. 12) Allcorn's effort to discern trends  used Knowles'  typology of "sequential research  needs" as a comparison device.  Knowles the  research  fields.  Using  (1973)  suggested  needs of their his  own  field  memories  that  adult  if they from  his  educators  contrasted days  as  could  them a  better  with  social  understand  those  of other  worker  Knowles  contrasted the research efforts of the two fields he knew. He outlined six phases he  thought  would  correspond with  the  "sequential research  different levels of development:  1. 2. 3. 4. 5. 6.  Definition of the field Differentiation of the field Standard-setting Technological refinement Respectability and justification Understanding of the dynamics of the field  needs"  of  fields  at  15 Allcorn education  (1985)  literature.  Long and  Agyekum  made  use  of Knowles'  He combined the (1974);  Long  phases to evaluate  findings of Dickinson  selected  and Rusnell  (1977); Boshier and Pickard  adult (1971);  (1979); and  his  own content analysis, to compare 'trends' with Knowles' phases. The period under study  in  categories  Adult  Education  Quarterly  were developed from the  Lifelong Learning, adult education the content  was  first  not  was  was analyzed for the period 1977-1983, and the futures  were  dated  from  1952 to  trends from previous analyses  identified  disappearance  although  with  of research  Knowles' aimed  at  the  20 volumes of that journal  content (1950-70).  18 articles on  1982. Allcorn concluded that  and his own, 'fit' with Knowles' six  phases. He stated that findings demonstrated that  stated,  research  phases  4,  5,  was  and  definition of the  beginning to  6  field.  with  a  appear  concurrent  He concludes,  "The  findings . . . indicate adult education is developing a unique field of knowledge; however, some gaps in the knowledge base still exist" (p. 12).  Content analyses of selected adult education literature have identified the subject  matter  themes  and  studied,  methodological trends  compared  their  in the  strength  as  field.  Results  demonstrated  have by  listed  the  frequency  of  occurrence, and monitored changes over time. Research studies were surveyed for their  use  of  methods  and  techniques,  and  changes  indicating  sophisticated research  designs were noted. Comparison was attempted  type  made  of questioning  in  research  activities  of  adult  use  education  of  more  between and  the  those  thought to denote the level of questioning of emerging disciplines, with the claim that adult education is increasingly asking more sophisticated research  questions.  16 Citation Analysis  Citation analysis is used to identify formal links between already existing knowledge  (cited reports)  and the  extension of knowledge (the  citing document).  More to the point, analysis of citing behavior allows meaning to be attributed to the patterns of linkage found. Dickinson and Rusnell (1971) counted the of citations made citing  that  rose  by authors sharply  publishing in Adult  from  0.7  per  article  Education, in  finding  Volumes  1 to  number  a trend in 5,  to  13.2  citations per article in Volumes 16 to 20. The total number of citations to other articles  in  Adult  Education,  however,  did  not  increase  in  proportion  to  the  increase in the total number of citations referenced. These results were limited to the above mentioned figures and no further interpretation of citation patterns was reported.  The  single  citation  analysis  that  was  attempted  on  adult  education  materials was by Boshier and Pickard (1979) who claimed that: The citation count is a quantitative measure employed to establish the extent to which researchers utilize indigenous (primary literature) concepts and processes, as opposed to those of other disciplines (secondary literature), (p. 35) They  sought  percentage and  (3)  answers  to  three  of primary literature the  hypothesized  impact that,  "as  of the  concerning  citations, (2) the  individual discipline  reliance on concepts and processes (p. 35). This pattern  questions  items, emerges,  by  (1)  the  number  impact of individual most  frequent  researchers  will  scholars,  citation. place  and  They  increasing  developed by other scholars in the discipline"  would be affirmed by  an  increasing ratio of citations to  17 primary  adult  education  literature,  for  which  they  provided  the  following  definition:  Primary Adult Education Literature describes concepts, processes and data clearly identified with adult education. It is usually published in adult education journals, books or monographs and is produced by people for whom adult education is their primary professional concern, (p. 36)  Boshier and Pickard (p. 36) analyzed, "citations made by authors of all research, theory  and  'forum'  period from  articles  published  in  Adult  1968-1977." 'Primary' literature  Education  during  was that deemed  the  so by  ten-year seven of  nine judges for each citation. Ninety percent of the citations (2047 out of 2247) were  successfully  classified  for  the  volumes  under  study,  which  they  found,  "reveals a general increase in the percentage of primary literature citations. . . ."  They  listed  frequently  the  cited  twelve  individual  most  items  frequently  cited  concluding that  authors,  most  and  were  eleven  authored  by  most well  known adult educators. Their results led them to conclude that:  . researchers publishing in Adult which showed an increasing tendency education literature, (p. 47)  Education wrote articles to cite primary adult  in view of the major role of Adult Education, it is contended that this study has demonstrated that adult education is creating its own body of knowledge, (p. 48)  In Education  the as  study a  by Dickinson  percentage  in  the  percentage  presented.  (1979), in their classification of citations for the increase  this  were  Pickard  general  that  citations  (1971), only citations to  four  a  periods  of total  Rusnell  through  found  5-year  and  proportion  of  was  not  Analysis  rising.  nine  citations  to  Adult  showed,  Boshier and  volumes studied, "primary  adult  18 education literature." Although this increase was never stated in empirical terms in  the  discussion  or  averages 4.8 percent of  an  "even  conclusions of the  study,  the  data  shows  the  increase  per year for the nine year period. Along with their claim  development"  (in  the  increase)  this  would  suggest  a  substantive  change occurring in the structure of adult education research literature. However, using the data reported in Table 1 of their article it was possible to aggregate data into two 4-year intervals (Volumes 20-23 and Volumes 24-27). Analysis of this recombined data  results  in an  increase  in the  proportion of the  'primary  literature' category of 6.2 percent between the first and second 4-year intervals.  The education  result  materials  of  the  reveals  single an  citation  increasing  analysis trend  published  toward  citing  concerning  adult  "primary  adult  education literature." The magnitude and the rate of this change as suggested by the Boshier and Pickard study, however, are limited to the nine years (Volumes 19-27) investigated in the  Journal.  Rationale for Study  This thesis, in some respects, replicated the earlier work of Boshier and Pickard (1979); a comparison of the studies is presented in Table 1.  19 Table 1  Study Procedures Compared with Boshier and Pickard (1979) Procedures  Boshier and Pickard  Study  Methodology  Citation Analysis  Citation Analysis  Data  Adult Education Quarterly (Volumes 19-27)  Adult Education Quarterly (Volumes 21-36)  Frequency of Primary/Secondary Items Cited  Computation of Primary/Secondary Citation Values (cited authors times authorships to them)  Most Frequently Cited Individual Authors  Proportions Primary/Secondary of Most Cited Authors  Most Frequently Cited Individual Items  Proportions Primary/Secondary of Most Cited Journals  Coding  Nine Judges  Researcher  Reliability  Independent Consistency Check of Coded Item  1) Recoding Consistency Check of Coded Authors 2) Independent Judge Agrees on Code  Validity  Majority of Judges Agree on Coding Item (seven of nine)  Independent Experts Agree with Code of Author (ten of eleven)  Set  Questions  20 As  can  be  seen  or  data  set  methodology  in  Table  but  1,  in the  concern of Boshier and Pickard  the  differences  research  was  the  are  questions  basis for the  not  so  much  asked. While  the  intial hj'pothesis  in first  of the  present investigation, the focus on the individual's impact, whether that be of the editor's items,  influence, the was  studies  excluded as  involve  citations  most  who  were  individually  a  research  codes  coded. In  for  frequently  the  data,  content,  designated  'primary  literature'  literature"  and  citations  literature'.  If Boshier and  Education  showed  an  focus.  to  more  Pickard  whereas  its  author  that  or  most  this  was  name  importantly, (1979)  in  Pickard's claim  increasing  authors,  frequently  cited  between  these  Significant differences  but  Boshier and  'primary'  all  cited  each  were  item  study,  deemed  once  an  to  automatically  cite  what  process  was  assessed  an  "author  that "researchers  tendency  by  item  of primary  coded  'primary  publishing in  primary  was  adult  Adult  education  literature" holds true in this investigation, the proportions of citations should be rising for those to "authors  of primary literature" with a concurrent decrease in  those to. "authors of secondary literature" as time passes.  The reviewed  rationales  within  this  for  this  chapter.  study's Hypothesis  hypotheses  were  based  1 stems  from  Boshier and  claim that citations to  "primary adult education literature"  Hypothesis  its  2  authors  were  propose  the  journals)  had  as  basis  predominantly possibility  follow  a  when journals are  that  similar  well a  their  known adult  subset  pattern  finding  of the  to the  coded separately.  that  the  educators. total  the  studies Pickard's  increased with most  frequently  Hypotheses  citations  'primary literature'  These last hypotheses  on  (those  3  time. cited and  4  citations  to  citation proportion  also serve to extend  21 the  analysis  of  citations  reported  by  Boshier  and  Pickard  to  include  an  investigation of journal usage: thus, it facilitates understanding of citing behavior in Adult Education Quarterly's articles.  Figure  1 displays graphically the logical extension of these rationales in  relation to the predicted results of this study.  Time 1  Sequential Periods Time 2 Time 3 Time 4  Time 5  Time 6  Proportion of Citations to Primary Literature 1.00  •••• •••• •••• •••• •••• •••• •••• •••• •••• ••••  1.00 Proportion of Citations to Secondary Literature  Figure  1. Histogram of hypothesized proportions of citations to "authors of primary literature" and "authors of secondary literature" as a field emerges.  22 Figure  1 is an  heuristic device to foster  occurring in fields. In 1929 when the Journal  understanding  first  of the  changes  began publishing, authors did  not identify themselves with the labels 'adult educator/education'.  This situation is  portrayed  diagrammatically on the  where  literature  dominates  education  of  the  adults.  In  knowledge more  recent  increasingly identifying themselves the  field.  left  side, of the  histogram  used  to  times  speculation  with the  guide  field,  This situation is described on the  research holds  secondary  concerning that  the  authors  are  and with other practitioners in  right side of the  histogram  where  'primary literature' increasingly influences the authors of adult education research.  Hypotheses  Through  the  review  of  previous  studies  on  knowledge  about  education it was possible to hypothesize that: 1) The percentage of citations to  "authors of primary  literature"  as opposed to "authors of secondary literature" in Adult  Education  Quarterly  would  rise  between  the  first  half  of the  period  under  study (1971-1978) and the second half (1979-1986).  2) The most frequently  cited authors would increasingly be coded  as "authors of primary literature."  3) The proportion of citations to opposed to "secondary first  half of the  "primary literature journals"  literature journals" would rise between  period under  study  (1971-1978)  and the  as the  second  adult  23 half (1979-1986).  4) The most frequently  cited journals  would icreasingly be coded  as "primary literature journals."  The research studies reviewed in this section suggest that adult education is building a unique body of knowledge through its increasing reliance on studies grounded in an adult education context. This thesis examined citations taken from articles  in the journal Adult  document The  this  procedures  Education  increasing reliance of  this  study  Quarterly  through were  the  designed  (1971-1986), and  methodology to  measure  attempted  to  of citation analysis. the  proportion  of  'primary literature' citations by tabulating the citations attributed to "authors of primary literature" rather than  simply coding each item on its manifest  The rationale advanced was that authors of adult education materials an  adult education context;  thus,  publication information should alter  neither the  content. are  titles of subsequent materials nor 'primary literature'  from the  designation of these  authors.  The next chapter in this study.  contains a description of the research  procedures  used  CHAPTER 3  METHODOLOGY  This  study  proposed  to  measure  the  relative  presence  of  literature'  and 'secondary literature' cited in articles published in Adult  Quarterly.  Two questions  literature"  cited in these articles, and what  Four hypotheses  were  asked: to what  extent  were  'primary Education  "authors of primary  trends can be identified over time?  were advanced. In this chapter the procedures for data collection  and analysis are described. Validity and reliability measures employed to examine the coding procedures are also detailed.  Data Collection  The data  collection included all citations from  section (excluding book reviews, critiques, or response Quarterly  Volumes  classification involving  were  personal  21  through  citations  in  36  inclusive  languages  communications  or  other  other  articles, and  the  sections) in Adult  (1971-1986). than  Education  Excluded  French/English,  non-retrievable  'forum'  sources,  from  citations multiple  citations to a work within the same article, citations without named author, self-citations.  Because  it  'another  author's  study'  author's  reference  to  is  a  which  'own'  citation was  previous  category's  the  focus  work)  influence  of this were  thesis,  excluded.  upon  research  self-citations This  follows  and of (an the  exhortations of Arlin (1978); Buss (1976); Jones (1980); Myers (1970); and Roche and Smith (1978).  24  25 The  procedure  used  to  designate  a  citation as  'primary  literature'  or  'secondary literature' was as follows. First, if the citation was not excluded for the  reasons  above,  its  words were interpreted  title  was  for the  read  and  meaning they  the  words  lend the  employed  noted.  These  title, irrespective of the  actual words employed. As it was adult education phenomena that was the focus in this study, the essential meaning to note was 'adult education'. To assist researcher  in  recognizing  processes and data, Merriam added  referring  to  adult  education  concepts,  a list of synonymous terms was compiled. Darkenwald and  (1982:12-14)  terminology  terminology  the  offer  found  some  in  the  synonyms titles  of  and  related  articles  from  terms  to  which  the  Adult  were  Education  Quarterly. The list was constructed as follows:  Adult; Andragogy; Basic; Community; Continuing; Correction; Cooperative; Disadvantaged; Extension; Further; Human Resource; Immigrant; Inservice; Lifelong; Literacy; Night School; Noncredit; Nontraditional; Nonformal; Recurrent; Reentry; Self-Directed; Seniors; Training/Retraining; Voluntary; Women's; Worker's, And" Education; Development; Learning; And: Projects; Programs; Chautauqua; Folk High School; Frontier College; Highlander School; Mechanic's Institute; Open University; Residential Workshops; Study or Listening -Circle/Forum/Group. If one or more of the terms used in the title of the reference cited appeared on the list or was a variant of a listed term (e.g., continuing, continuous, continued) the  'primary literature'  designated from journal,  designation was  its title, a further  publisher,  editor,  automatic.  analysis of the  association)  was  made  If the  citation could not be  publication information (e.g., to  determine  if  an  adult  education context was explicitly referenced. One procedure used was to search for journal titles in Ulrich's International Periodicals Directory to see if the particular  26 journal cited was considered an adult education journal. If a citation was still not designated  'primary literature'  was made  to determine  if the  a  comparison with author(s)  all previously cited  materials  had previously been designated  as  an  "author of primary literature." If not designated 'primary literature' by this point the citation was designated as 'secondary literature' (see Table 2).  Table 2  Steps Used to Code Citations Step  Procedure  Outcome  1. Examine Title  Compare Words used to Adult Education Terms (e.g., andragogy, learning projects)  Primary or go to Step 2.  2. Examine Publication Information  Compare Words used to Adult Education Terms or Title to Ulrich's Directory or Editor to Authors List  Primary or go to Step 3.  3. Examine Authors List  Check for Prior Designation of Author(s)  Primary or Secondary  Once a citation was classified 'primary literature' or 'secondary literature' that designation was recorded beside the author(s)  named for the cited material.  In the event that an author had previously been coded as one category and in subsequent citations was classified took precedence.  This  differently, the  was justified because  the  'primary literature' designation researcher  intended  to  identify  "authors of primary literature." It was assumed that an author who wrote from an adult education context was an "author  of primary literature" irrespective of  the title of any other works cited in Adult Education  Quarterly.  27 Some established  citations  had  to recognize the  multiple  authors;  relative presence  therefore,  of each  a  procedure  individual  author  was in  the  cited literature. This procedure involved attaching a value to each citation of four points ("citation value"), so that a single author received a full four points each time cited, co-authors received two points each, and three or more authors named in  a  citation received one  coding  the  authors  co-authorships  within  frequently cited authors and  'secondary  primary  studied. In  named, the  and  summary,  the  literature  proportions "authors  By linking  accumulated  were assessed  literature'  literature"  point each.  was  the  classification process  number  tabulated.  In  of this  authorships way  the  to and  most  more accurately, while 'primary literature' would  reflect the  of secondary  citations were judged  presence  literature" in the  for  1) author  of "authors  of  materials being  variability  in coding  and "citation value", and 2) journal variability in coding and frequency cited (see Table 3). Table 3  Major Components of the Study Item  Variable  Authors  Authors of Primary or Secondary Literature  Journals  Journals of Primary or Secondary Literature  Weighting  Results  Frequency Cited Times Citation Value (author = 4 points) (2 authors = 2 each) (3+ authors = 1 each)  Sum and Percentage  Frequency Cited  Sum and Percentage  28 Journals  were  coded  using  a  slightly different  process.  Only  journals  referenced in eligible citations were considered. As above, the list of synonyms of adult  education  between  the  related  terms.  involved Edition  was  words  used  to  in the  to  Ulrich's see  if  the title  If not classified as  consulting (1985),  compared  cited journal's and  the  step  was  listed  as  matches  a "primary literature journal" a further Periodicals  listed  discover  and  journal  terms  to  synonyms  International  the  title  Directory, in  the  Twenty-Fourth  103  international  periodicals under the adult education section (pp. 532-34). If the journal was not designated  as a "primary literature journal" by this point, it was classified as a  "secondary  literature  recorded only under  journal."  Journals  which  had  changed  their latest title for tabulation purposes.  their  names  were  A complete list of  eligible journals and their frequency cited can be found in Appendix A .  Here is an example of the classification process using citations from an article by author J . Philips in volume 37 of an imaginary journal. REFERENCES 1. Broski, R. & Sole, B . (1979). Continuing Education Needs. Journal of Nursing, _4(2), 24-28. 2. Branhan, L . (1977). Ilees Pinyin Chu Ladme. Chang-chu, China: Peking Press. 3.  Collins, J . (1980). Educational Research. Harvard, Mass.: Harvard University Press. 4. Commission of Professors. (1969). Adult Education in U . S . Boston: A A R C Press. 5. Essert, P. (1980). The Need for Women's Retraining. Continuing Education in Nursing, 33(2), 21-22. 6.  Manning, P . B . (1983). Alone Again. Proceedings Research Conference, Montreal, Canada, April.  of  the  Adult  Education  29 7. Philips, J . (1983). Adult Education. Adult Education Quarterly, 33(3), 8-12. 8. Sole, B. (1980). Nursing Futures. Journal of Nursing, 5(3), 5-7.  These  eight  because  it  author,  and  'primary  citations was  in  No.  a  7  literature',  publication  were  classified  foreign because  language, it  was  No. 1 and  information,  classified  as  an  classified  as  'secondary  and  "author  as  follows.  8  a  self-citation. of the  because  of primary  were  No. 4 because  5 because  No.  Three  this  literature"  literature' because  excluded, No. 2  there  Four  was  were  no  classified  title, No. 6 because author  from  had  No.  it did not meet  1.  named  been  as  of the  previously  Number  3  was  criteria of 'primary  literature'.  The classification was recorded under 'author' rather than 'item', both for ease in record keeping and ability to identify the most frequently cited authors. To account for co- and multiple authors a "citation value" was assigned to each reference  cited (four points). Thus, co-authors received two points each and three  or  authors  more  received  two  co-authorship  cited received one  points shared  as with  a  point each.  co-author,  Broski,  and  while an  In Sole  the  example  received  additional four  above  two  points  Broski  from  for  his  the other  citation. Thus, Sole achieves a "citation value" of six in this example.  The data citation represented  on cited journals was less complex to record as a frequency  each eligible  of one citation to that journal. In our example  journals are cited four times (No. 1, 5, 7 and 8). However, No. 7 was excluded  30 as  it  is  Nursing;  from  a  self-citation.  Numbers  1 and  8 both  reference  Journal  of  thus, this journal was recorded, classified according to its title ~ after a  check of Ulrich's International  Periodicals Directory -  as a "secondary  literature  journal", and credited with two citations. Number 5 was also recorded as it was from  an eligible citation. This title  ~ Continuing  Education  in Nursing  -- was  classified as a "primary literature journal" because it meets the criteria of words in the title which match terms used in the list of adult education synonyms and related terms (i.e., continuing education). The coding sheets for the example above would appear as seen in Tables 4 and 5 below. Table 4 Example of Author Coding Sheet (Volume 37) Name  Author Classification  Citation Value  Broski, R. Collins, J . Essert, P. Manning, P . B . Sole, B-  Primary Literature Secondary Literature Primary Literature Primary Literature Primary Literature  2 4 4 4 2+4= 6  Table 5 Example of Journal Coding Sheet (Volume 37) Name  Journal Classification  Frequency Cited  Journal of Nursing  Secondary Literature  1+1 = 2  Continuing Education in Nursing  Primary Literature  1  In this example the coding process would continue as above until all the eligible citations from Volume 37 were coded. The outcome of this process would be two  31 alphabetical lists, one of authors and the other of journals cited in this volume. The  classification  authorships was  of  the  author  and co-authorships  names  of journals  along  with  the  accumulated  values  of  the  being cited would be on one list. The second list  cited,  their  classification and  the  frequency cited. These lists served as the raw data for the  tabulation  of  their  study.  Data Analysis  Lists  of  "authors  of  primary  literature" cited in articles from Adult citations  attributed  compiled.  Through  literature'  as  the  list  8-volume  to this  them  by  process  frequently  period under  the  cited  study),  Education  volume  opposed to 'secondary  of most  literature"  proportion  authors authors  of  the  distortion  cite  a  which  may  citing  Verner). In  eliminate Quarterly.  prolific the  those journals The  in  most  findings  author's  cited in fewer  in each  were judged  from  in a single article).  (e.g., China  on their  by  4,  and  total "citation necessary  historical/biographical reviews (e.g., list  see it  three volumes  by multiple citations from  Quarterly,  were  'primary  consecutive  Dickinson was  a  China Reconstructs  1979:  necessary  of Adult  to  Education  small number  articles dealing with specific subject matter was deemed harmful to of citation patterns  secondary  volumes  one article. This was  cited journal than  sixteen  authorships  collected works  frequently  distortion caused  the  of  (twenty  caused  of  could be established. In preparing  value", providing they were cited in more than because  "authors  Quarterly and the total value of  throughout  literature'  the  and  of  interpretation  cited twelve  times  32 In transitory  reviewing influences  the on  literature citation  for  this  behavior  study  (such  it  as  became  editor's  apparent  mood,  that  fads,  and  external historical events) could be partially controlled by using longer periods for analysis  and  comparison.  It  was  decided  that  time  periods  to  be  used  for  presentation of data would not be less than that covered in four volumes (four years) of the journal volumes,  therefore,  (approximately 60 articles). The study period covered sixteen data  were  aggregated  into four  sequential  4-volume periods  and two sequential 8-volume periods.  Reliability and Validity  Reliability  means  that  data  and  judgments  are  consistent  application across cases and between observers.. Two measures assessed:  the  test-retest  reproducibility (inter-judge  In  their  study  stability-over-time  (intra-judge  in  their  of reliability were  consistency);  and  the  agreement).  Boshier  and  Pickard  (1979)  made  use  of a  panel of  judges to assess the extent to which citations were reliably classified as 'primary literature' or 'secondary literature' (seven of nine judges had to agree on each item's classification or the citation was excluded from the study). In the  present  study when a citation was not classified as 'primary literature', the data analysis procedures item.  forced  Through  the  this  'secondary process  the  reliability and validity measures  literature' author  classification  cited  are necessarj'  always  on  the  attained  author a  of  code.  to assess the appropriateness  success of these collection and analysis procedures (see Table 6).  the  Thus, and  33 Table 6  Procedures for Establishing Reliability and Validity Employed in this Study Test  Procedure  Errors Detected  Reliability: Test-Retest  Researcher Recodes Data (Intra-Judge Stability-over-time)  Inconsistency  Reliability: Reproducibility  Independent Judge Codes Data (Inter-Judge Agreement)  Disagreement  Validity: Expert's Standard  Compare Results to Independent Expert Judgements  Reliability:  In  Test Retest  this  study  stability-over-time random sample citations) original  Inaccuracy  was data  was of the  chosen set.  reliability verified data. at  was by  the  in  researcher  two  ways.  recoding  One volume of the journal  random  The results  handled  and  were  citations  compared  materials and differences noted (see Table 7).  coded with  First, and  intra-judge  analyzing  (17 articles with  without reference  the  first analysis  to  a  327 the  of these  34 Reliability:  Reproducibility  The  second  'issue' of the Journal  measure  of reliability  involved  having  (Volume 34, Issue No. 3 with  a  randomly  drawn  126 citations) coded by an  independent judge using only the rules as written by the researcher.  The judge's  results were then compared to the researcher's findings. Andren (1981) notes that when employing semantical content analysis techniques, differences between judges are not, on the face of it, detractors from the confidence levels of the research, but rather require analysis and interpretation. He says:  Semantic content analysis is . . . an activity which often demands extensive knowledge . . . . This means that it may be futile to demand that the task must be such that "regardless of who does the analysis or when it is done, the same data should be secured under similar conditions." (p. 65) The findings of this procedure are reported in the Results Chapter.  Validity: Expert Standard  Validity  concerns the  extent  to which  an instrument  measures  what it  purports to measure. Krippendorf (1980) said: Semantic validity assesses the degree to which a method is sensitive to the symbolic meanings that are relevant within a given context, (p. 157) Validity concerned the extent procedures secondary  for  to which the results obtained using the rule-based  distinguishing "authors  literature"  compared  to  a  of primary literature" 'standard'  established  by  from  "authors  'experts'  in  of the  35 adult  education  education  field.  This  terminology in the  researcher titles  had  assumed  of their  that  works, or  authors  using  adult  publishing within  adult  education literature, were writing about adult education phenomena and  therefore,  were directly associated with the field of adult education.  To  test  these  measures  an  'expert'  panel  Appendix B) was consulted to provide a 'standard'  of  adult  educators  (see  for comparison. The criteria  for inclusion on this panel of 'experts' were: Professor of Adult or Continuing Education Current faculty position at North American Universitj^ Doctoral preparation Not presently faculty at University of British Columbia Sixteen letters and checklists (see Appendix C) were mailed in the first week of September  1986  and  eleven  replies  were  received within  four  calendar  weeks  (68.8 percent return rate). A l l checklists returned were usable.  A  list of the twenty most frequently cited authors in each of the four  4-volume periods covered by the study was sent to this panel, without reference to the designation these most cited authors received by the study procedures. The panel was  asked to identify  from  the  names  listed  those  "primarily  known for  their adult education activities." Eleven 'experts' responded to this request. least ten of the experts (greater than 90 percent of the respondents)  If at  agreed that  the named author was "primarily known for their adult education activities" this became the 'standard' with which the research results were compared (see Table 8).  36 In this chapter the procedures used for data collection and analysis have been described. In the following chapter the results of the analysis are presented.  CHAPTER 4  RESULTS  The  findings  of  this  study  are  organized  as  follows.  First,  results  pertaining to reliability and validity are discussed, then the characteristics of the population, the author variability, and the journal variability are presented.  Reliability and Validity  The test retest results of intra-judge consistency (recoding Volume 28: 17 articles;  369  citations)  compared  primary literature" and "authors occasions (July coded,  and September  whereas,  eliminated  during  during  the  proportions  of  citations  to  "authors  of  of secondary literature" found on two separate 1986). In the  recoding  only  recoding because  it  original coding 321 authors  316  authors  were.  was  discovered  that  Five the  authors two  were were  citations  involved were to unpublished materials, making them ineligible citations (this fact had been overlooked in the original analysis).  Comparison between coding of authors  on two occassions resulted in 306  agreements (about 97 percent). Disagreement in author classification two codings were  to ten names  on the  author  between  coding sheet (about  the  3 percent).  This difference is to some extent accounted for b}' the fact that authors in the original collection may have been coded "authors they  appeared  on  the  author  list  from  37  a  of primary literature"  prior  designation,  because  however,  this  38 information  was  not  available in the  process  of re-analysis  (e.g.,  could not be designated  from the titles cited in this volume as  primary  holds  literature"  but  this  designation  from  an  Paulo Freire an "author of  earlier  volume).  The  disputed authors and explanation of the discrepant coding are displayed in Table 7. Table 7 Discrepancies in Intra-Judge Recoding with Reasons Name  Original Coding Result  Re-coded Result  Reason for Error in Original  Abbott G. Biddle L . Biddle W. Bradford L . Borowsky G. Caylor F . Faure E . Freire P. Gross R. Schwartz S . H .  Secondary Secondary Secondary Primary Primary Primary Primary Primary Primary Primary  Primary Primary Primary Secondary Secondary Secondary Secondary Secondary Secondary Secondary  Mistake Mistake Mistake Earlier Designation Mistake Mistake Earlier Designation Earlier Designation Mistake Earlier Designation  Four of the errors in coding were because prior designations  were not available  in the recoding process, meaning that only six authors were mistakenly coded in the original research.  Along with the mistaken inclusion of the  non-published material, this indicated that errors the researcher of procedures  two citations to  were due to mistakes made by  in the original data collection process rather than to  inadequacies  used. This result tends to favor the view that the data collection  and analysis procedures of this study were followed consistently and that a high degree of confidence in findings is warranted.  The reproducibility test was that of having an independent judge recode a  39 portion of the sample (Volume 34, Issue No. 3, 126 citations) and by comparing the results with the researcher's  Of the 91  percent  explained  135 authors  agreement).  as  coded, agreement  The  differences  original coding outcomes.  twelve  of opinion  was achieved in 123 cases  disagreements about  the  (about  nine  percent)  meaning of terms  (about can  in the  be  list of  synonyms of adult education. The independent judge included eleven citations as 'primary literature' which refer to university courses, higher education, and college programs in a prison setting. These had not been so classified by the because  education  in a  prison setting  had  not  been  viewed  challenge  the  study's  as  a  researcher variant of  'correction education'.  These  results  of a  nine percent  because  appear  to  difference  between  the judge  coding outcomes. However, differences of this magnitude  potential reproducibility and  the  researcher  in  (with these explanations)  could be predicted to occur as a 'constant discrepancy' throughout the analysis of the judge and the researcher. of  coded  authors  Resulting proportions (formed by comparing groups  internally in each  case)  would  predictably yield  lead to conclusions about rates of change of a similar  findings and  nature.  The expert's standard was used to estimate the validity (accuracy) of the study procedures. The validity of the rule-based classification system used in this study was examined by consulting a panel of 'experts'. The 'experts' established the  'standard'  by  which  accuracy  in coding the  citations  could  be judged.  A  checklist was provided to these 'experts' on which the names of the twenty most  40 frequently  cited  recorded. when  from  Findings of 75.5  the  judgments  agreement) research twenty  authors  were  of  percent at  compared  each  4-volume agreement  least  with  ten  the  frequently  within  the  (40 out  of 53  authors)  the  rule-based  process. The complete results most  of  period  cited authors  eleven  four  'experts'  (90  classifications made  of the 'expert' from  study  panel's  were  resulted percent  during  the  designation of the'  4-volume periods are  listed in  Appendix D.  Table  8  shows  these  authors  with  disputed  codes  and  the  level  of  agreement of the experts with the rule-based classification system. Table 8 Discrepancies in Author Classification Found by Validity Test Name Vincent, John H . Havighurst, R . J . Sheffield, Sherman Benne, Kenneth Douglah, M . Cross, K . Patricia Litchfield, Anne Rivera, Ramon J . Cartwright, Morse A . Johnstone, John W . C . Miller, H a r r y H . Monette, Maurice Penland, Patrick R.  Rule-Based Code primary primary primary primary primary primary primary primary primary primary primary primary primary  literature literature literature literature literature literature literature literature literature literature literature literature literature  Number of Experts Ageeing with Code 1/11 2/11 3/11 5/11 5/11 6/11 6/11 6/11 7/11 7/11 7/11 8/11 9/11  The validity of the rule-based procedures used to differentiate  "authors of  primary literature' from "authors of secondary literature" seemed to be supported by the judgments of the panel of experts, especially when it is realized that the  41 levels of disagreement  on the disputed codes were generally high (average of six  experts in agreement on the disputed code).  The  following  sections  summarize  the  data  obtained  through  the  articles  the  procedures described in Chapter 3.  Study Population  A  total  of  5,413  sixteen volumes of Adult the  study  citations Education  were  contained  in  249  from  Quarterly (Volumes 21 to 36) that make up  population. Of these citations, 713 were excluded! which  meant  that  4,700 citations were eligible for classification. These citations represented the work of 3,381 individual authors; however, only 269 (7.9 percent) of these authors cited  an  equivalent  of twice  ("citation  value"  of eight  points)  in  the  are  sixteen  volumes studied. Data related to the sample were:  Number of Eligible Articles with Citations = Number of Citations Present =  5,413  Total Excluded Citations =  713  Total Classified Citations =  4,700  Total Authors Classified  3,381  =  249  Number of Authors With Citation Value of Eight =  269  t Excluded were: all self-citation; any without a named author; personal communication or non-retrievable sources (e.g., speech); foreign language (any other than French/English); citation greater than once for same material within an article  42 Number of Journals Cited =  381  Number of Journals Cited Three Times or More = Journals Designated Primary Literature = Total Classified Citations to Journals  =  73  31 1,534  Percentage of A l l Classified Citations to Journals =  Cited  Authors  The proportion of citations to for  32.6 percent  almost one third of the  citations  "authors of primary literature" in Volume  21  and  grew  to  accounted  account  for  almost two thirds of the citations by Volume 36 (see Figure 2).  Volume 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 Proportion .70 .65 .60 .55 .50 .45 .40 .35 .30  • • •  • •  •  •  • •  • •  36  •  • • •  •  Figure 2. Proportion of citations made to "authors of primary literature" in each of the sixteen volumes under study. The  proportion  of citations  to  "author  of primary  literature"  can  be  seen  follow an increasing (though inconsistent) rise throughout the period under study.  to  43 Consolidating data into a longer time frame reveals trends. Where Figure 2 clearly reveals the peaks and troughs in the citations to "authors of primary literature"  during  this  period,  Table  9  summarizes  data  by  4-volume  accumulations in the table and 8-volume tabulation at the bottom of the table.  In Table 9 each volume is described first by the "citation value" of the designated of  the  "authors of primary literature",  volume, and  finally,  by the  second by the  computation  of the  total "citation values" percentage  of "citation  values" attributable to "authors of primary literature" (column on far right). This last column provides the  figures  which  are  used for comparison of changes  in  citing behavior over time. Minimally it can be said that the percent of citations to  "authors of primary literature"  has  increased  4.8  percent  between  the  first  half (1971-1978) and the second half (1979-1986) of the study period.  Any more  author  during the  whose accumulation of "citation value" was eight points or  study  period  added to the list of most  (excluding those  frequently  cited in only  cited authors.  one  Two hundred  article)  was  and sixty-nine  authors qualified for this listing (about 8 percent of authors), accounting for 39.5 percent of all citations classified. "Authors of primary literature" as a proportion of most cited authors listed generally rises through the period under study Table 10).  (see  44 Table 9  Percent Citations to Authors of Primary Literature Volume 21-36  Volume  Primaryt Citation Values  Total Citation Values  Primary Citation Percentage  21 22 23 24 21-24  232 440 250 412 1334  765 1001 756 1255 3777  30.3 44.0 33.0 32.8 35.3  25 26 27 28 25-28  289 409 562 632 1892  705 976 1006 1292 3979  41.0 41.9 55.9 48.9 47.5  29 30 31 32 29-32  405 528 448 395 1776  1103 1202 1114 1187 4606  36.7 43.9 40.2 33.3 38.6  33 34 35 36 33-36  821 696 804 869 3109  1700 1424 1401 1394 5919  48.3 48.9 57.4 62.3 53.9  Primary Citation Values  Total Citation Values  Primary Citation Percentage  3226 4885  7756 10525  41.6 46.4  Years 1971-1978 1979-1986  tAccumulated Citation Values of "Authors of Primary Literature"  45 Table  10  Percentage of Most Frequently Cited Authors who are Authors of Primary Literature in Four Periods and Their Percentage of Total Citation Values  Time  Volumes  Number and Percent Primary Authors  Percentage of Total Citation Values  T.  21-24  12 of 20 (60.0%)  66.5%  2.  25-28  16 of 23 (69.7%)  81.5%  3.  29-32  13 of 20 (65.0%)  78.8%  4.  33-36  18 of 21 (85.7%)  92.3%  The Table  data  on  the  11) demonstrate  primary  literature"  twenty  most  cited authors  in 8-volume  periods  a similar rise in both the proportion of the  and  in the  percentage  of citations  made  by  (see  "authors of  this  'primary  literature' group. In the first 8-volume period (Volume 21-28) "authors of primary literature" percent),  constituted  seventeen  accounting for  of  1116 of the  the  twenty-one  most  cited  1452 "citation values", or  authors  (81  76.9 percent of  all citations. Whereas, in the second 8-volume period (Volume 29-36) "authors of primary  literature"  were  eighteen  authors  and now accounted for  values"  in this  group.  The  of the  (85.7  percent)  1400 out of 1550 (90.3 percent)  number  "citation values" for each author  twenty-one  in brackets  on  Table  11  in this group: four for a full  most  cited  of all "citation is  the  sum of  authorship; two  for a co-authorship; and one for being one of three or more authors. Due to ties in "citation values" there are twenty-one authors listed.  46 Table 11 Top Twenty 29-36t  Authors  Volume 21 Author  Ranked  by  Citation  to 28 Citation Value  •Houle, Cyril 0 . •Verner, Coolie •Knowles, Malcolm •Freire, Paulo Maslow, A . H . •Knox, Alan B . •Lindeman, Eduard •Douglah, M . •Boshier, Roger •Sheffield, Sherman •Ohliger, John •London, Jack •Bergevin, Paul Illich, Ivan Gagne, R . M . •Dickinson, Gary •Johnstone, J . W . C . •Rivera, Ramon •Havighurst, Robert Dewey, John •Litchfield, Anne  (130) (130) (118) (80) (56) (49) (40) (38) (36) (36) (34) (32) (32) (30) (30) (30) (30) (30) (29) (28) (28)  Value:  Volumes  21-28  and  Volume 29 to 36 Author Citation Value •Knowles, Malcolm •Tough, Allen •Houle, Cyril •Knox, Alan B . •Lindeman, Eduard •Boshier, Roger •Cross, K . Patricia •Freire, Paulo •McClusky, Howard •Darkenwald, Gordon •Havighurst, Robert •Hiemstra, Roger •Verner, Coolie • K i d d , J.R. •London, Jack Erikson, Erik •Monette, Maurice Kerlinger, Fred •Mezirow, Jack Sticht, Thomas •Grabowski, S . M .  (226) (144) (140) (105) (98) (96) (74) (64) (62) (60) (59) (56) (54) (48) (45) (40) (40) (36) (35) (34) (34)  Cited Journals  Citations to journals study  procedures accounted  that were deemed  "primary  literature  journals"  for a low of less than one-tenth of journal  by  citations  in Volume 21 to a high of over one-half of citations in Volume 36. As with the citations  to "authors of primary  literature",  "primary literature  journal"  citations  were found to be an increasing proportion of all journal citations (see Figure 3). t The • indicates a designated "author of primary  literature"  47 Volume Proportion .65 .60 .55 .50 .45 .40 .35 .30 .25 .20 .15 .10 .05  Figure  While  21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36  • • •  •  •  •  •  • • •  •  comparison  Figure is  •  • •  •  3. Proportion of journal citations made literature journals" by volume studied.  from  •  3  apparent,  inconsistency when  of  consecutive  the  pattern  8-volume  which  on periods  are  a  to  "primary  volume-to-volume are  compared  an  increase of 7.7 percent in the "primary literature journal" category can be seen (see Table 12).  48 Table 12  Journals Volume 21-36 Volume  Primary Journals Cited  Total Journals Cited  Percentage to Primary Journals  21 22 23 24 21-24  6 35 14 47 102  94 87 58 129 368  6.3 40.2 24.1 36.4 27.7  25 26 27 28 25-28  12 24 47 37 120  52 78 110 93 333  23.0 30.7 42.7 39.7 36.0  29 30 31 32 29-32  20 36 27 28 111  89 124 69 70 352  22.4 29.0 39.1 40.0 31.5  33 34 35 36 33-36  75 40 49 53 217  143 133 106 99 481  52.4 30.0 46.2 53.5 45.1  Years  Primary Journal Citations  Total Journal Citations  Percentage to Primary Journals  1971-1978 1979-1986  222 328  701 833  31.7 39.4  49 In Table 12 the left hand column identifies the volume studied, next frequency  of citation to  frequency  of citation to journals,  computed  percentage of "primary literature journal" citations  journal  citations.  monitor  change  foot of Table journals"  and  "primary  Running in  down  total  figures  the  are  the  fourth  far  column  is shown, then right hand  are  frequency  shown,  of citing  along  with  as  the  toward journal categories  of the  journals  journals"  and finally  citing behavior  12 the  literature  to a  the  the total  column is  the  a proportion of figures  over  used  to  time. A t  the  "primary  literature  computation  of  the  percentage of citing to "primary literature journals" in the first half of the  study  (1971-1978:  31.7%)  and  study  (1979-1986:  39.4%).  This rise  those  citings of  7.7  during percent  the in the  second  half  of  the  proportion of citations  to  "primary literature journals" parallels the findings concerning "authors of primary literature" shown earlier.  The most frequently cited journal list was defined as all journals cited in a minimum of three separate volumes with the highest accumulated frequency of citations. In Table  13 the  first  8-volume period (Volume 21-28) shows three of  the nine most cited journals are "primary literature journals" accounting for of 271 citations (70.8 percent). cited journals are  In the  second 8-volumes, five of the  nine  192 most  "primary literature journals" and they account for 287 of the  338 citations (84.9 percent). Twelve journals make-up those most frequently cited from the study's 381 cited journals in both 8-volume periods.  50 Table 13  Nine Most 29-36t  Frequently  Cited  Journals:  Ranked  Volumes 21-28 Journal  Citations  research and  purpose  questions  reliability  of  of this  which the  of  reported.  This  items by the  primary  chapter  analysis  same authors  literature" data  was  and  Citations  •Adult Education Quarterly •Lifelong Learning A m . Sociological Review Harvard Educational Review •Convergence •Studies in Adult Education •Educational Gerontology Journal of Gerontology Journal of Ed. Psychology  was study.  and also  most frequently  to  present First,  procedures  produced by the procedures used to measure "authors  21-28  Journal  guided this data  Volume  Volumes 29-36  •Adult Education Quarterly (149) •Lifelong Learning (33) Journal of Gerontology (17) Journal of Ed. Psychology (16) American Sociological Review• (15) Journal of Ed. Research (ID •Convergence (10) Journal of Reading (10) Develomental Psychology (10)  The  for  data was  (196) (57) (17) (14) (12) (11) (11) (10) (10)  Findings relevant  to  supporting the presented.  the  validity  Second,  data  proportions of the citations made to  "authors  of  examined to  secondary show  the  cited in this literature.  literature" proportions Finally,  were of the  the  data  from procedures used to determine the proportions of "primary literature journals" and "secondary literature journals" found in the study were presented.  The next  and final chapter contains discussion of the findings presented, implications related to hypotheses made, limitations relevant to the study procedures, and  suggestions  for future research employing citation analysis of the type used in this study.  tThe • before the title refers "Primary Literature Journals"  CHAPTER 5  DISCUSSION, LIMITATIONS & IMPLICATIONS  This interpretation  Chapter  contains  a  of results  related  to  discussion the  of  study's  findings,  hypotheses,  limitations that are relevant to these interpretations. a  discussion  of the  implications of the  study  an and  analysis  and  a discussion of  The Chapter concludes with  for  future  investigation  of  the  knowledge base in adult education  Discussion  This study provided empirical evidence of the changing nature of citations in adult education research through an analysis of the articles published in Adult Education as  a  Quarterly. Results suggest that authors using Adult  communication  attention  to  previous  channel adult  for  their  education  research  studies,  reports  and,  are  therefore,  Education  Quarterly  paying  increased  are  building  their  own studies on a foundation of unique and tested knowledge about the field.  'Primary  literature'  as  a classification of citations experienced  a rise of  4.8 percent between the period covered by Volumes 21-28 and the period covered by  Volumes 29-36;  been journal  considered. may  It  choose  however, would to  the  not  possible be  reasons for  unreasonable  publish articles  that  to  cite the  this  rise have  suggest Journal  that  not  editors  more  often  fully of  a  than  articles that do not. Similarly, it could be suggested that authors who publish in  51  52 a  particular  observable,  journal  may  have  something  in  common  that  is  not  readily  and may cite each other's work as a matter of familiarity,  thereby  driving up citation rates.  To assess the impact of the above arguments  on the results obtained in  this investigation, an analysis was made of the proportion of citations to articles from the Journal  itself over the last 35 years. The first 20 years of data comes  from  Dickinson and Rusnell (1971) while the  this  study's  frequency  findings.  of  all  In  order  citations  for  to  last  determine  each  15 years  these  five-year  of data  proportions,  interval  was  were  the  cumulative  divided  cumulative frequency of citations to articles published in the Journal  from  by  the  in the same  five year interval. B y plotting this proportion for each time interval it is possible to determine  whether  an increasing citation level was due to 'favoritism' toward  citing articles published in Adult Education Table  Quarterly (see Table 14).  14  Citations in 1-35  the Adult  Volume 1-5 6-10 11-15 16-20 21-25 26-30 31-35  Education Quarterly  to  Its  Own Articles:  Volumes  0  Adult Education Quarterly Articles Cited :in Adult Education Quarterly 8 22 50 36 90 93 127  Total Citations 111 462 790 1030 1333 1727 1974  Proportion of Citing to Its Own Articles .07 .04 .06 .03 .06 .05 .06  53 As  can  be  seen  from  Table  14,  the  proportion of citations  to  Adult  Education  Quarterly has not increased over time, lending support to an  argument  that  rise in citing  simply  any  behavior  to  adult  education  research  is not  product of colleague familiarity on the part of the researchers  a  publishing in the  Journal.  A second matter  for discussion is related to the assumed values of this  study. It was the contention of this researcher  that the field of adult education  must increase the attention it pays to its own research if it is to build  theory  and accumulate knowledge unique to adult education. This building is assumed to be part  of the  maturation  of the  field.  Therefore,  the  increasing proportion of  citation to 'primary literature' was argued to be a sign of 'emergence' to be a positive course for adult education researchers of  course,  viewpoint, field  another looking  when the  side  to  this.  inward for  answers  may  Is  adult  answers  to  to be pursuing. There is,  education  questions  and felt  becoming  narrow  in  its  concerning problems  of  the  already be available in other disciplines? A t what  point does drawing from a field's research change from being a sign of strength to  a  sign of 'incestuousness'?  addressed  by the  research  The  answers  to  these  concerns  design of this thesis. These are  requiring a kind of evaluative answer  have  qualitative  not  been  questions  which the data collection and analysis in  this thesis do not confront. Further mention of this is made  in the  implication  section below.  When Boshier and Pickard (1979) concluded that their data supported the presence  of  a  growing unique  knowledge  base  in  adult  education,  they  also  54 cautioned: It is difficult to know whether the increasing citation of primary literature was due to editorial policies, the continued emergence of adult education as a discipline with a distinctive body of knowledge, or the interests and activities of authors who have made extensive use of Adult Education as a publication outlet over the most recent part of the ten-year period, (p. 39)  The period of analysis covered by this thesis was selected to limit the influence of overt changes in editorial policy at Adult Education Quarterly in order to more confidently adult  suggest  the results obtained were due to changes  education research.  Data  presented  in Table  14  in the  suggest  practice of  that  the  rising  proportions of citation to 'primary literature' was not an artifact of a particular group  of researchers  4,700  citations  citing  analyzed  each  other  represented  out  3,381  of 'familiarity'. different  authors  The  fact  that  was  evidence  the that  overall citation patterns were not unduly influenced by the fact that a few well known adult education authors are being cited frequently in the literature.  A l l of the hypotheses presented described study  in  the  conclusions  the  previous  inconsistent about  chapter.  nature  patterns  at  of  in Chapter 2 are supported by the  However, as some  times  data  was  illustrated throughout  were  a  24,  29,  (volumes  weakness 31  when  and  32  data the  forming  were  not  consistent with a pattern of increasing 'primary literature' citation).  Hypothesis  1 - that the percentage  of citations to "authors  of primary  literature" would rise, was supported in the data found in Table 9. In the 8-volume  period  41.6  percent  of  the  citations  were  to  "authors  first  of primary  literature". In the second 8-volume period 46.4 percent of the citations were to  55 these authors. This represents an increase of 4.8 percent from  the  first  to the  second period.  Hypothesis  2  -  increasingly identified as presented  that  the  "authors  most  frequently  cited  authors  of primary literature" was  would  be  supported by data  in Table 11 where of the twenty most frequently cited authors in the  initial 8-volume period 81.0 percent were "authors of primary literature" whereas, in  the  subsequent  literature",  a  8-volume  4.7  evidence supports  percent  period  85.7  percent  rise in "authors  were  "authors  of primary literature."  of  primary  Although  the  this hypothesis, as the increase was only one author  between  two 8-volume periods, this hypothesis is tentatively accepted until further  research  is  undertaken.  Hypothesis  3 - that  the  journals" would rise was supported  percentage  of citations  by data in Table  to  "primary  12, where the  literature percentage  in Volumes 21-28 was 31.7 percent and in Volumes 29-36 rose to 39.4 percent, a 7.7 percent rise.  Finally, journals  would  Hypothesis  4  increasingly be  which  predicted  "primary  that  literature  the  most  journals"  frequently  was  supported  data in Table 13 which identified three of nine most cited journals in the 8-volumes as "primary literature journals" rising to five of nine in the 8-volume period. Further analysis was necessary  cited by first  following  to determine if this increase in  "primary  literature journals" among the top cited was not a artifact of having  too  a  low  threshold  on  most  cited journals  included. Three  of the  top  nine  56 journals  most  frequently  cited  in  Volumes  21-28  were  "primary  literature  journals" and accounted for 70.8 percent of most frequently cited journal citations. Five  of  the  literature  top  nine  journals"  most  and  cited journals  accounted  for  in  84.9  Volumes  percent  of  29-36  were  most  "primary  frequently  cited  journal citations. This was an increase of two "primary literature journals"  In  Figure  proportions secondary  of  citations  literature"  intervals in the proportions predicted illustrates.  4 the to  are  Figure  This result  1  of this  "authors  study's  of  under  study  confirmed  also confirms  the  in measuring  literature"  representing  and the  "authors four  the of  4-volume  (1971-1986). The trend of increasing  "authors of primary was  procedures  primary  shown in a histogram  sixteen years  of citation to in  results  literature"  through  study  as  was hypothetically  findings  conclusions of earlier  as  Figure  4  studies  such  as  Boshier and Pickard (1979); and Allcorn (1985), that adult education is emerging as a field of study and practice.  57  Volumes Proportion of Citations to Primary Literature .70 .65 .60 .55 .50 .45 .40 .35 .30 .25 .20 .10 .05 .00 .05 .10 .15 .20 .25 .30 .35 .40 .45 .50 .55 .60 .65 .70 Proportion of Citations to Secondary Literature  Figure  21-24  25-28  29-32  33-36  .539 .475 .353  •••• •••• •••• •••• •••• •••• •••• •••• •••• •••• •••• •••• •••• •••• •••• •••• •••• .647  •••• •••• •••• •••• •••• •••• •••• •••• •••• •••• •••• •••• •••• •••• •••• •••• .525  .386  •••• •••• •••• •••• . •••• •••• •••• •••• •••• •••• •••• •••• •••• •••• •••• •••• •••• .614  •••• •••• •••• •••• •••• •••• •••• •••• •••• •••• •••• •••• •••• •••• •••• •••• •••• .461  4. Histogram of proportions of citations to "authors of primary literature" and "authors of secondary literature" in four intervals during the study period.  58 Limitations  There are  a number of limitations to take into account when reviewing  this study. The first is the matter  of the use of a citation's title to judge  the  material as 'primary literature'. The rationale used was one of face validity. If a title identifies it as being concerned with adult education, then the article is most likely about adult education phenomena.  A second and related limitation has to do with the matter of attributing all of an author's work to 'primary literature' if any work is designated so. The justification  for  this  education phenomena subsequent  procedure  was  are  than  more  writings cited in Adult  that  authors  likely  writing  affiliated  Education  with  Quarterly  directly the  even  about  field,  when  and  adult their  not entitled  manifestly adult education are still 'primary literature'.  Thirdly,  the  fact, that  only  journal  articles,  and  moreover,  only  one  journal's articles were examined, is a strong limiting factor. A s was noted in a Baath University Library (1979) publication that discussed the common limitations of citation analysis, "Usually only journals have been used as sources . . . there is  good reason  those  in other  interpreting the be  taken  confirming  no  for believing that sources"  (p.  5).  references This  in journals  limitation  is  a  may be different very  serious  one  from when  findings of this study. The generalizability of these results farther  research  than using  literature in adult education.  the  Journal  citation  from  analysis  which is  they  attempted  were on  obtained  other  forms  can until of  59 A  fourth  fundamental.  limitation  In short,  the  arises  from  rule was  the  fact  that  coding  rules  were  applied even when common sense dictated  otherwise. Thus, Havighurst was categorized as an "author of primary literature" because of terms used in the plethora  of  consistent  other  use  writings  of absolute  title of a single article, although the titles of a  by  him  rules  were  in  a  obviously  study  'secondary  of this  kind  is  literature'. acceptable  The when  results are derived by comparing consecutive periods internally. Thus, the impact of this type of error is minimized when it is a constant error.  Implications  Further  study  is  needed  on  the  broader  research  efforts  in  adult  education. As pointed out in the limitation section above, other forms of research reports  need to be analyzed in order to confirm the citation trends  in Adult  Education  Quarterly.  Future  studies  could  well  focus  literature which is accessible through 'Proceedings' of research  demonstrated  on the  'meeting'  conferences.  A second line of inquiry that is a natural extension of this study would be  investigation of the  trends  in  questions  the need  field. to  be  impact,  As  was  Further,  mentioned  addressed  intra-discipline citing and the  duration  on  the  'incestuous'  and in  the  matters  discussion of  risks inherent  the value of adult education research  be investigated.  significance of particular  the  section value  of  research qualitative increasing  in such citing behavior.  trends or lines of pursuit should  60 Finally, with depth  the present the  cited  the  implication  study's  Boshier  confirming results  materials  attempt to expand on the order to identify  of the  in adult  and Pickard  suggest  education  study  along  a need to analyze in more  research.  dichotomous categories  (1979)  Future  research  should  used in these past efforts in  more precisely the nature of the value researchers  are  finding  in 'primary literature'.  This  thesis  has  contributed empirical findings relevant when questioning  creation of knowledge in adult education. The knowledge base of adult education now  has  one  more  empirically  based  understanding of the field may be achieved.  characterization  by  which  greater  REFERENCES Aaronson, Steve. (1975). 'The Footnotes of Science.' Mosaic, 6(2), 22-7. Allcorn, Seth. (1985). 'The Knowledge Gap of Adult Education.' Lifelong Learning: A n Omnibus of Practice and Research, 8(5), 12-16. Andren, Gunnar. (1981). 'Reliability and Content Analysis.' In K a r l Erik Rosengren (Ed.), Advances in Content Analysis. Sage Annual Review of Communication Research, Volume 9. Beverly Hills, C A : Sage Publications, 58-65. Arlin, Marshall. (1978). 'Quantity and Impact of Scholarly Journal Publication in Canadian Faculties of Education.' Canadian Journal of Education, 3(1), 1-18. Baath  University Library. (1979). The Structure of Social Science Literature as Shown by Citations Design of Information Systems in the Social Sciences. (Research Reports Series A No.3). Bath, U . K . : Bath University.  Beals, Ralph A . & Brody, Leon. (1941). The Literature of Adult Education. New York: American Association for Adult Education. Boshier, Roger (Ed.). Learningpress.  (1980).  Towards  a  Learning  Society.  Vancouver:  Boshier, Roger & Pickard, Lynette. (1979). 'Citation Patterns of Articles Published in Adult Education 1968-1977.' Adult Education, 30(1), 34-51. Brookfield, Stephen. (1982). 'Adult Education Research: A Comparison of North American and British Theory and Practice.' International Journal of Lifelong Education, 1(2), 152-67. Brunner, Edmund deS. & Verner, Coolie. (1968). 'Adult Education.' in David L . Sills (Ed.), International Encyclopedia of the Social Sciences, Volume 1. N.P.: The Macmillan Company & The Free Press, 100-06. Brunner, Edmund deS.; Wilder, David S.; Kirchner, Corinne & Newberry, John S. Jr. (1959). A n Overview of Adult Education Research. Washington, D.C.: Adult Education Association Buss,  Allan R. (1976). 'Evaluation of Canadian Psychology Departments Based upon Citation and Publication Counts.' Canadian Psychological Review, 17(2), 143-50.  Darkenwald, Gordon G. & Merriam, Sharan B . (1982). Foundations of Practice. New York: Harper & Row.  Adult  Education  Dickinson, Gary. (1979). Contributions to a Discipline of Adult Education A Review and Analysis of the Publications of Coolie Verner. Occasional 61  62 Papers in Continuing Education Number 18. Vancouver: Continuing Education, University of British Columbia. Dickinson, Gary & Rusnell, Dale. (1971). ' A Content Education.' Adult Education Journal, 21(3), 177-85. Draper,  James A . (1985). 'Adult Education.' in The Volume 1. Edmonton: Hurtig Publishing, 10-11.  Centre  Analysis  Canadian  of  for Adult  Encyclopedia,  Garfield, Eugene. (1979). Citation Indexing Its Theory and Application in Science, Technology, and Humanities. Toronto: John Wiley & Sons. Garfield, Eugene. (1972). Science, 178(4060),  'Citation Analysis as 471-78.  a  Tool  in  Journal  Evaluation.'  Grabowski, Stanley M . (1974). 'The Literature of Adult Education: A Review Report.' Paper presented to the Commission of Professors of Adult Education, Adult Education Association of the U . S . A . (Miami, Florida, November 3rd, 1974). E D 102 413 Griffith, William S. (1966). 'Guidelines for Writers Preparing Adult Education.' Adult Education, 16(2), 85-91. Griffith,  Manuscripts  for  William S. & Roberts, Pearl J . (1981). 'Adult Education.' in H . M . Andrews & W. T. Rogers (Eds.), Canadian Research in Education: A State of the A r t Review. Report prepared for the Social Science and Humanities Research Council of Canada, Vancouver: University of British Columbia, 3-11.  Jams,  M . (1965). 'The Problem of Validating Content Analysis.' In Harold D. Laswell, Nathan Leites and Associates (Eds.), Language of Politics: Studies in Quantitative Semantics. Cambridge: MIT Press, p.5 7.  Jensen,  Gale; Liveright, A . A . & Hallenbeck, Wilbur. (1964). Adult Education Outlines of an Emerging Field of University Study. N . P . : Adult Education Association of the United States, forward.  Jones, Lyle V . (1980). 'The Assessment of Scholarship.' in Edward H . Loveland (Ed.), Measuring the Hard-to-Measure , (New Directions for Program Evaluation, 6). San Francisco: Jossey-Bass, 1-20. Knowles, Malcolm S. (1973). 'Sequential Research Needs in Evolving Disciplines of Social Practice.' Adult Education, 23(4), 298-303. Knowles, Malcolm S. (1960). Handbook of Adult Education in the Chicago: Adult Education Association of the U . S . A .  United States.  Krippendorf, Klaus. (1980). Content Analysis A n Introduction to Its Methodology. (Volume 5 The Sage CommText Series). Beverly Hills: S A G E .  63 Liveright, A . A . (1964). 'The Nature and Aims of Adult Education as a Field of Graduate Education.' in Gale Jensen; A . A . Liveright & Wilbur Hallenbeck (Eds.), Adult Education Outlines of an Emerging Field of University Study. N.P.: Adult Education Association in the United States, 85-102. Long,  Huey B . (1983a). Cambridge.  Adult  Learning  Research  and  Practice.  New  York:  Long, Huey B . (1983b). 'Characteristics of Adult Education Research Reported at the Adult Education Research Conference, 1971-1980.' Adult Education, 33(2), 79-96. Long,  Huey B . (1977). 'Publication Activity of Education.' Adult Education, 27(3), 173-86.  Selected  Professors  of  Adult  Long,  Huey B . & Agyekum, Stephen K . (1974). 'Adult Education 1964-1973: Reflections of a Changing Discipline.' Adult Education, 24(2), 99-120.  Merriam, Sharan B . & Simpson, Edwin L . (1984). A Guide to Research Educators and Trainers of Adults. Malabar, F l . : Kreiger Publishing.  for  Miller, Max D. (1982). 'Citation Patterns of Articles Published in the Journal of Medical Education, 1970-1980.' Journal of Medical Education, 57, 797-99. Myers,  C. Roger. Contemporary  (1970). 'Journal Citations and Scientific Eminence Psychology.' American Psychologist, 25(7), 1041-48.  Nelson,  Carnot E . (1972). 'Journal Publication of Material Presented Annual A E R A Meeting.' Educational Researcher, 1(8), 4-6.  at  in an  Peters, John M . (1980). ' A Systems Approach to Examining Adult Education.' in John M . Peters & Associates (Eds.) Building an Effective Adult Education Enterprise. San Francisco: Jossey-Bass, 115-57. Peters, John M . & Banks, Betty B . (1982). 'Adult Education.' in Harold Mitzel (Editor in Chief), Encyclopedia of Educational Research, Fifth Edition. New York: The Free Press, 83-87. Price,  Derek J . de Solla. (1970). 'Citation Measures of Hard Science, Soft Science, Technology, and Nonscience.' in Carnot E . Nelson & Donald K . Pollock (Eds.), Communication Among Scientists and Engineers. Lexington, Mass.: Heath Lexington Books, 3-22.  Roche, Thomas & Smith, David Lewis. (1978). 'Frequency of Citation as Criterion for Ranking of Departments, Journals, and Individuals.' Sociological Inquiry, 48(1), 49-57. Smith,  Nick L . & Caulley, Darrel N . (1981). 'The Evaluation of Educational Journals Through the Study of Citations.' Educational Researcher, 10(5), pp.11-12, 22-24.  64 Smith,  Robert M . ; Aker, George F . & Kidd, J.R. (Eds.). Adult Education. New York: Macmillan Publishing.  (1970). Handbook  Sork,  Thomas J . (1982). 'Meta-Research in Adult Education Through 1981: A n Historical Analysis and Critical Appraisal.' Unpublished Paper. Adult Education Research Centre, University of British Columbia. .  (1985).  A  of  Ulrich's  International Periodicals Directory, 24th edition Serials Bibliography. New York: Bowker, 532-534.  Bowker  Verner,  Coolie. (1960). 'The Literature of Adult Education.' in Malcolm S. Knowles (Ed), Handbook of adult education in the United States. Chicago: American Adult Education Association of the U . S . A . , 162-75.  Verner,  Coolie; Dickinson, Gary; Leirman, Walter & Niskala, Helen. (1970). The Preparation of Adult Educators. N . P . : ERIC Clearinghouse on Adult Education.  Xhignesse, Louis V . & Osgood, Charles E . (1967). 'Bibliographical Citation Characteristics of the Psychological Journal Network in 1950 and in I960.' American Psychologist, 22 (7), 778-91.  APPENDIX A  Journals Cited in Adult Education  Quarterly f  c ADULT EDUCATION IN FINLAND 2 o A D U L T E D U C A T I O N (UK) 13 © A D U L T E D U C A T I O N Q U A R T E R L Y 330 (J. A D . E D . , A D . E D . J . , AD.ED.) o A D U L T LITERACY A N D BASIC E D U C A T I O N 1 © A U S T R A L I A N J O U R N A L OF A D U L T EDUCATION 2 A M E R I C A N S O C I O L O G I C A L R E V I E W 32 A M E R I C A N E D U C A T I O N A L R E S E A R C H J O U R N A L 12 A M E R I C A N J O U R N A L O F S O C I O L O G Y 13 A M E R I C A N SOCIOLOGIST 6 A M E R I C A N J O U R N A L OF PSYCHOLOGY 4 A M E R I C A N PSYCHOLOGIST 9 AMERICAN EDUCATION 1 AMERICAN POLITICAL SCIENCE REVIEW 2 A M E R I C A N SCIENTIST 1 AMERICAN JOURNAL OF PSYCHIATRY 3 AMERICAN HERITAGE 1 A M E R I C A N ANTHROPOLOGIST 3 A M E R I C A N LIBRARY ASSOCIATION B U L L E T I N 1 AMERICAN JOURNAL OF P H A R M A C Y EDUCATION 4 A M E R I C A N BEHAVIOR SCIENCE 1 AMERICAN JOURNAL OF OCCUPATIONAL THERAPY 1 AMERICAN COUNCIL O N EDUCATION 1 A M E R I C A N A C A D E M Y OF POLITICAL & SOCIAL SCIENCE A N N A L S 2 AMERICAN QUARTERLY 1 AMERICAN J O U R N A L OF PHYSICS 1 A M E R I C A N POLITICAL SCIENCE REVIEW 1 A M E R I C A N REVIEW OF PSYCHOANALYSIS 1 AMERICAN JOURNAL OF ORTHOPSYCHIATRY 1 AMERICAN BAR FOUNDATION RESEARCH JOURNAL 1 A M E R I C A N J O U R N A L OF EDUCATION 1 ADMINISTRATIVE SCIENCE Q U A R T E R L Y 9 ADMINISRATIVE M A N A G E M E N T 1 ARCHIVES OF PSYCHOLOGY 2 AUDIO VISUAL COMMUNICATION REVIEW 3 A N N U A L REVIEW OF SOCIOLOGY 2 A N N U A L REVIEW OF PSYCHOLOGY 2 A L B E R T A J O U R N A L OF EDUCATIONAL RESEARCH 1 AGING A N D H U M A N DEVELOPMENT 2 ADMINISTRATOR'S NOTEBOOK 1 A U D I O V I S U A L INSTRUCTOR 5 AAUP BULLETIN 1 t- Of 381 Journals, 31 were designated ' • ' "primary literature journals", number following the journal title indicates frequency of citation 65  AUSTEN AMERICAN STATESMAN 1 ACROSS T H E BOARD 1 ACCOUNTING REVIEW 1 A N N A L S OF INTERNAL MEDICINE 1 ARCHIVES OF INTERNAL MEDICINE 1 AEDS JOURNAL 1 AZTLAN 1 BRITISH J O U R N A L O F E D U C A T I O N A L P S Y C H O L O G Y 3 BRITISH J O U R N A L O F S O C I O L O G Y 2 BRITISH J O U R N A L OF P S Y C H O L O G Y 3 BRITISH J O U R N A L O F S T A T I S T I C A L P S Y C H O L O G Y 2 BEHAVIOR SCIENCE 3 BUSINESS EDUCATION F O R U M 1 B U L L E T I N OF ATOMIC SCIENCE 1 • COMMUNITY EDUCATION JOURNAL 1 • C A N A D I A N J O U R N A L OF UNIVERSITY CONTINUING EDUCATION CANADIAN J O U R N A L OF HIGHER EDUCATION 1 CANADIAN JOURNAL OF PSYCHOLOGY 1 CENTURY 1 • CHAUTAQUAN 2 • CONTINUING EDUCATION 1 CHRONICAL OF HIGHER EDUCATION 5 • C O N V E R G E N C E 22 • COMMUNITY DEVELOPMENT JOURNAL 1 COLLEGE & UNIVERSITY 3 CANADIAN JOURNAL OF EDUCATION 1 CANADIAN PSYCHOLOGICAL REVIEW 1 COUNSELING EDUCATION & SUPERVISION 3 COUNSELING PSYCHOLOGIST 8 CALTFORNIAN M A N A G E M E N T REVIEW 3 CHANGE 3 C A N A D I A N J O U R N A L OF BEHAVIOR SCIENCE 1 CHILD STUDY J O U R N A L 1 COMMUNITY & JUNIOR COLLEGE JOURNAL 4 CHINA QUARTERLY 2 C H I N A R E C O N S T R U C T S lOt CAREER DEVELOPMENT ACTIVITY 1 COMMUNICATION EDUCATION 1 CANADIAN J O U R N A L OF PHILOSOPHY 1 • CONTINUUM 1 COLLEGE & UNIVERSITY BUSINESS 1 CONTEMPORARY EDUCATIONAL PSYCHOLOGY 3 COMMUNITY COLLEGE FRONT 3 CORRECTION T O D A Y 1 COMMUNITY M E N T A L H E A L T H JOURNAL 1 COMMUNITY COLLEGE REVIEW 1 CONTEMPORARY EDUCATION 1 C A N A D I A N REVIEW OF SOCIOLOGY & ANTHROPOLOGY 3 t F r o m a single article  COMMUNITY TEAM 1 CHILD DEVELOPMENT 4 C O L L E G E COMPOSITION & COMMUNICATION 1 COLLEGE BOARD REVIEW 2 CHILDREN TODAY 1 COMMUNICATION REVIEW 1 C A N A D I A N EDUCATION & R E S E A R C H DIGEST 1 COMPARATIVE EDUCATIONAL REVIEW 1 D E V E L O P M E N T A L P S Y C H O L O G Y 14 DAEDALUS 4 DIRECTOR & B O A R D 1 EDUCATIONAL LEADERSHIP 3 EDUCATIONAL CONSIDERATION 1 • E D U C A T I O N A L G E R O N T O L O G Y 14 EDUCATION & CULTURE 2 EDUCATION & PSYCHOLOGICAL MEASUREMENT 3 ERISTICS 1 EDUCATIONAL RESEARCH 6 ENGLISH EDUCATION 1 ENGLISH JOURNAL 1 EDUCATIONAL FORUM 1 EXCEPTIONAL CHILD 1 EDUCATION 1 EDUCATIONAL REVIEW 3 EDUCATIONAL TECHNOLOGY 4 EDUCATIONAL RESEARCH BULLETIN 1 EDUCATION BROADCASTING INTERNATIONAL 2 E D U C A T I O N A L STUDIES 1 E D U C A T I O N A L RECORD 1 EDUCATIONAL ADMINISTRATOR QUARTERLY 1 EDUCATIONAL THEORY 3 E D U C A T I O N FOR SOCIAL W O R K 1 EVALUATION 1 EVALUATION COMMENT 3 E V A L U A T I O N & H E A L T H PROFESSIONALS 2 • E X T E N S I O N INSIGHTS 1 FINDINGS 3 FUTURIST 4 FAR EAST S U R V E Y 1 F A M I L Y COORDINATOR 1 FREEMAN 1 FORUM 1 GERONTOLOGIST 9 GENETIC PSYCHOLOGICAL MONOGRAPHS 1 GERIATRICS 1 HIGHER EDUCATION 1 HIGH SCHOOL BEHAVIOR SCIENCE 1 H U M A N ORGANIZATION 3 H A R V A R D BUSINESS REVIEW 7 H A R V A R D E D U C A T I O N A L R E V I E W 21  H U M A N RELATIONS 5 H E A L T H SERV. & M E N T A L H E A L T H A D M I N . H E A L T H REPORTS 1 HIGH SCHOOL J O U R N A L 1 HUMAN DEVELOPMENT 5 HOSPITALS 1 H U M A N RESOURCE M A N A G E M E N T 1 HOSPITAL A N D COMMUNITY PSYCHIATRY 1 INDUSTRIAL RELATIONS 3 INDUSTRIAL GERONTOLOGY 1 INDUSTRIAL PIONEER 1 INTERNATIONAL SOCIAL SCIENCE J O U R N A L 1 IMPROVING COLLEGE & UNIVERSITY TEACHING 4 IMMIGRATION IN AMERICA REVIEW 2 INTERNATIONAL JOURNAL OF PSYCHOANALYSIS 1 INTERNATIONAL J O U R N A L OF AGING & H U M A N D E V E L O P M E N T 5 INTERNATIONAL JOURNAL OF PSYCHOLOGY 2 INTERNATIONAL MANAGEMENT 1 INTERNATIONAL REVIEW OF APPIED PSYCHOLOGY 1 I N T E L L E C T U A L DIGEST 1 INSTRUCTOR 1 INSTRUCTIONAL TECHNOLOGY 1 INTELLECT 1 ILLINOIS E D U C A T I O N A L REVIEW 1 INSTRUCTOR OF SCIENCE 1 INNOVATIONS IN HIGHER EDUCATION 1 INDIAN EDUCATOR 1 J O U R N A L O F E D U C A T I O N A L P S Y C H O L O G Y 26 JOURNAL OF MEDICAL EDUCATION 3 J O U R N A L O F A P P L I E D P S Y C H O L O G Y 12 J O U R N A L OF SOCIAL PSYCHOLOGY 6 JOURNAL OF PSYCHOLOGY 2 J O U R N A L OF HIGHER EDUCATION 4 • J O U R N A L OF CONTINUING EDUCATION & TRAINING 2 • J O U R N A L O F E X T E N S I O N 4 (J. C O O P E R A T I V E E X T E N S I O N ) J O U R N A L OF EDUCATION FOR SOCIAL WORK 1 • J O U R N A L O F A M E R I C A N SOCIETY F O R T R A I N I N G 1 JOURNAL OF READING BEHAVIOR 2 JOURNAL OF RESEARCH & DEVELOPMENT IN EDUCATION 6 • J O U R N A L OF T H E C O M M U N I T Y D E V E L O P M E N T SOCIETY 2 J O U R N A L O F G E R O N T O L O G Y 27 J O U R N A L O F R E A D I N G 17 JOURNAL OF HUMANIST PSYCHOLOGY 5 J O U R N A L O F SOCIAL ISSUES 2 J O U R N A L O F H I S T O R Y OF P H I L O S O P H Y 1 • J O U R N A L OF CORRECTIONAL EDUCATION 2 J O U R N A L O F A B N O R M A L & S O C I A L P S Y C H O L O G Y 15 J O U R N A L O F A B N O R M A L S O C I A L P S Y C H O L O G Y 5t JOURNAL OF CLINICAL PSYCHOLOGY 1 t F r o m a single article  JOURNAL OF PHYSICAL THERAPY 1 JOURNAL OF APPLIED BEHAVIOR ANALYSIS l i t J O U R N A L O F A P P L I E D B E H A V I O R S C I E N C E 10 J O U R N A L OF E X P E R I M E N T A L A N A L Y S I S OF BEHAVIOR 1 J O U R N A L OF N A T . ASSOC. OF W O M A N D E A N S & COUNSELORS 6 JOURNAL OF MARRIAGE & THE FAMILY 1 J O U R N A L OF HOME ECONOMICS 1 J O U R N A L OF RESEARCH IN SCIENCE TEACHING 5 JOURNAL OF M A R K E T RESEARCH 1 JOURNAL OF COMMUNICATION 6 JOURNAL OF EXPERIMENTAL EDUCATION 1 J O U R N A L OF APPLIED SOCIOLOGY 1 JOURNAL OF TEACHER EDUCATION 2 J O U R N A L OF B L A C K STUDIES 1 J O U R N A L OF INTERDISCIPLINARY HISTORY 1 J O U R N A L OF BRITISH SOCIETY OF P H E N O M E N O L O G Y 1 • J O U R N A L OF CONTINUING EDUCATION IN NURSING 3 JOURNAL OF RESEARCH IN TEACHING 1 JOURNAL OF AMERICAN INSTRUCTIONAL PLANNING 1 J O U R N A L OF GERONTOLOGICAL SOCIAL WORK 1 JOURNAL OF ALLIED H E A L T H 1 J O U R N A L OF M E D I C A L ASSOCIATION OF GEORGIA 1 J O U R N A L OF CRIMINAL L A W & CRIMONOLOGY 1 JOURNAL OF D E N T A L EDUCATION 1 J O U R N A L OF E D U C A T I O N A L SOCIOLOGY 1 J O U R N A L OF A M E R I C A N D E N T A L ASSOCIATION 1 JOURNAL OF PHILOSOPHY OF EDUCATION 1 J O U R N A L OF K E N T U C K Y MEDICAL ASSOCIATION 1 J O U R N A L OF H E A L T H & SOCIAL BEHAVIOR 1 JOURNAL OF EXPERIMENTAL LEARNING & SIMULATION 1 JOURNAL OF MEDICAL EDUCATION 4 J O U R N A L OF EDUCATION OF TECHNOLOGICAL SYSTEMS 1 J O U R N A L OF PERSONALITY & SOCIAL P S Y C H O L O G Y 2 J O U R N A L OF GENETIC PSYCHOLOGY 1 J O U R N A L OF INSTRUCTIONAL DEVELOPMENT 1 J O U R N A L OF PSYCHOSOMATIC RESEARCH 2 J O U R N A L O F SOCIOLOGY 1 J O U R N A L O F CROSS C U L T U R A L P S Y C H O L O G Y 1 JOURNAL OF CONSULTING & CLINICAL PSYCHOLOGY 1 JOURNAL OF COUNSELING PSYCHOLOGY 4 J O U R N A L OF Y O U T H A N D ADOLESCENCE 1 JOURNAL OF COLLEGE STUDENT PERSONNEL 8 J O U R N A L OF INDUSTRIAL TEACHER EDUCATION 1 JOURNAL OF AMERICAN P H A R M A C Y EDUCATION 2 J O U R N A L OF SOCIAL SCIENCE 1 JOURNAL OF EDUCATION 2 J O U R N A L OF NATIONAL INSTITUTE OF PERSONNEL R E S E A R C H 1 JOURNAL OF PROGRAMMED LEARNING 1 tTen from single article  J O U R N A L OF L E A R N I N G D I S A B I L I T Y 1 J O U R N A L OF C O N S U L T I N G P S Y C H O L O G Y 2 J O U R N A L OF H U M A N R E S O U R C E 3 J O U R N A L OF D E V E L O P M E N T IN READING 5 J O U R N A L OF EDUCATIONAL M E A S U R E M E N T 2 J O U R N A L O F E D U C A T I O N A L R E S E A R C H 13 J O U R N A L OF D E V E L O P M E N T A R E A S 2 J O U R N A L OF A M E R I C A N M E D I C A L ASSOCIATION 2 J O U R N A L OF M A R K E T I N G l i t J O U R N A L OF H E A L T H , P H Y S I C A L E D U C A T I O N & R E C R E A T I O N J O U R N A L OF E X P E R I M E N T A L P S Y C H O L O G Y 1 J O U R N A L OF R E G I O N A L S C I E N C E 1 J O U R N A L OF A P P L I E D S O C I A L P S Y C H O L O G Y 1 J O U R N A L OF P E R S O N N E L & S O C I A L P S Y C H O L O G Y 1 J O U R N A L OF P S Y C H O S O M A T I C R E S E A R C H 2 J O U R N A L OF ETHNIC STUDIES 1 JOURNALISM QUARTERLY 6 JUNIOR COLLEGE JOURNAL 1 K E Y REPORTER 1 LIBRARY TRENDS 1 • L I F E L O N G L E A R N I N G 90 ( A D U L T L E A D E R S H I P ) • L I T E R A C Y DISCUSSION 1 • L E A R N I N G 2 (CONTINUOUS L E A R N I N G ) MIDWEST SOCIOLOGY 1 M A N A G E M E N T REVIEW 1 McGILL J O U R N A L OF EDUCATION 1 MANAGEMENT 1 M B A 1 M O N T H L Y LABOR REVIEW 2 MARKETING FORUM 1 M A N A G E M E N T SCIENCE 1 MEDICAL & BIOLOGICAL ILLUSTRATION 1 M E A S U R E M E N T & E V A L U A T I O N IN GUIDANCE 1 MERRILL-PALMER QUARTERLY 2 MULTIVARIATE BEHAVIOR RESEARCH 4 M I C H I G A N S T A T E U N I V E R S T Y B U S I N E S S TOPICS 1 N E A U QUARTERLY 2 N A S S P BULLETIN 1 NOTES ON EDUCATION 1 NURSING RESEARCH 2 NURSING OUTLOOK 4 NURSING FORUM 1 • NOTES & ESSAYS ON A D U L T EDUCATION 1 NEW E N G L A N D JOURNAL OF MEDICINE 1 NATION'S SCHOOLS 2 N E W Y O R K U N . E D U C A T I O N Q. 2 N E W POLITICS 1 NEW OUTLOOK FOR THE BLIND 1 tTen from single article  71 NEW REPUBLIC 2 ORGANIZATIONAL DYNAMICS 2 QUARTERLY JOURNAL OF EXPERIMENTAL PSYCHOLOGY 1 Q U A R T E R L Y J O U R N A L OF ECONOMICS 1 PEDIATRICS 1 P E R C E P T I O N & M O T O R S K I L L S 6t P S Y C H O L O G I C A L REPORTS 7 PSYCHOANALYTIC QUARTERLY 1 P S Y C H O L O G Y B U L L E T I N 14 PSYCHOLOGY TODAY 8 PSYCHOMETRIKA 5 PSYCHOLOGICAL MONOGRAPHS 7 PSYCHOLOGICAL REVIEW 6 P S Y C H O L O G I C A L ISSUES 3 PSYCHOTHERAPY THEORY RESEARCH & PRACTICE 1 PHI D E L T A K A P P A N 6 PSYCHOMATIC SCIENCE 1 P S Y C H O L O G Y I N T H E SCHOOLS 2 PERSONNEL PSYCHOLOGY 2 PROGRESSIVE EDUCATION 1 PROGRESSIVES 2 PERSONNEL & GUIDANCE JOURNAL 3 PUBLIC OPINION QUARTERLY 3 PROGRAMMED LEARNING & EDUCATIONAL TECHNOLOGY 1 PERSPECTIVES ON BIOLOGY & MEDICINE 1 PUBLIC ADMINISTRATION REVIEW 2 PUBLIUS 1 PHYSICAL THERAPY REVIEW 2 PUBLIC H E A L T H REPORTS 1 POPULATION STUDIES 1 PEKING REVIEW 2 PROSPECTS 1 PEDAGOGICA EUROPEA 1 PUBLIC RELATIONS JOURNAL 1 PERSONNEL ADMINISTRATION 1 PHILOSOPHICAL STUDIES 1 PHYSICAL THERAPY 1 RESEARCH IN EDUCATION 1 RESEARCH I N HIGHER EDUCATION 1 RESEARCH O N AGING 1 RESEARCH ON MANAGEMENT 1 RESEARCH IN TEACHING OF ENGLISH 1 RELIGIOUS EDUCATION 3 R E W L E Y HOUSE PAPERS 1 READING RESEARCH QUARTERLY 6 R U R A L SOCIOLOGY 6 R E V I E W O F E D U C A T I O N A L R E S E A R C H 13 READING TEACHER 1 t F r o m single article  REVIEW OF RADICAL POLITICAL ECONOMICS 1 REVIEW OF EXISTENTIAL P S Y C H O L O G Y & PSYCHIATRY 2 SOCIOLOGIE E T S O C I E T E 1 SETTING THE PACE 1 SPEECH MONGRAPHS 1 SOCIOLOGY REVIEW 2 SEMINAR IN P S Y C H A T R Y 1 S C A N D I N A V I A N J O U R N A L OF P S Y C H O L O G Y 3 SCIENTIFIC M O N T H L Y 1 S C H O O L R E V I E W 10 SOCIOLOGY OF EDUCATION 3 SOCIAL BIOLOGY 1 SOCIAL POLICY 1 • S T U D I E S I N A D U L T E D U C A T I O N 14 SCIENCE EDUCATION 1 SOCIAL FORCES 5 SOCIAL R E S E A R C H 1 SOCIAL REVIEW 1 SOCIOMETRY 7 SCHOOL SCIENCE & M A T H 1 SOCIAL EDUCATION 1 SOCIAL P R O B L E M 4 SCIENCE 3 SCIENTIFIC A M E R I C A N 1 SATURDAY REVIEW 2 SOCIAL SCIENCE QUARTERLY 2 SOCIOLOGY TODAY 1 SOCIAL SCIENCE QUARTERLY 2 SOCIOLOGY Q U A R T E R L Y 1 SCIENCE & CHILDREN 1 SOCIOLOGY FOCUS 1 SCHOOL & SOCIETY 2 S C A N D I N A V I A N J O U R N A L OF E D U C A T I O N A L R E S E A R C H 1 SOCIAL WORK 2 SUPERVISORY M A N A G E M E N T 1 SECURITY M A N A G E M E N T 1 SLOAN M A N A G E M E N T REVIEW 1 • TRAINING 1 TRANS-ACTION 2 T E A C H E R C O L L E G E R E C O R D 12 • TRAINING & DEVELOPMENT J O U R N A L 4 • TUTOR'S B U L L E T I N OF A D U L T E D U C A T I O N 1 • UNIVERSITY EXTENSION WORLD 1 • U N I V E R S I T Y of N O R T H C A R O L I N A E X T E N S I O N B U L L E T I N 2 UNIVERSITY RECORD 1 URBAN LIFE 1 URBAN AFFAIRS QUARTERLY 1 VOCATIONAL GUIDANCE QUARTERLY 1 VIEWPOINTS 3 VISIBLE L A N G U A G E 1  WORKER'S EDUCATION QUARTERLY  JOURNAL  APPENDIX B Names of Experts Participating in Instrument Validation Procedures  Dr. Roger Hiemstra  - Syracuse U . , Syracuse, N . Y .  Dr. Sharon B . Merriam - Northern  Illinois U . , DeKalb  Dr. Huey B . Long - U . of Georgia,  Athens  Dr. Paul J . Ilsley - Syracuse U . , Syracuse, N . Y . Dr. Jerold W. Apps - U . of Wisconsin, Madison M r . David J . Littlet - University of Regina, Regina, Sask. Dr. Harold Beder - Rutgers U . , New Brunswick, N . J . Dr. Alan B. Knox - U . of Wisconsin, Madison Dr. Hayden Roberts - U . of Alberta,  Edmonton  Dr. Allen Tough - O.I.S.E., Toronto Dr. Stephen Brookfield - Teacher's College, Columbia University, N . Y .  t M r . Little is currently a doctoral candidate at the University Columbia, and a faculty member at the University of Regina. 74  of  British  APPENDIX C Letter  to  Experts  Requesting Their Participation Procedures  in Instrument  Validation  Adult Education Research Centre 5760 Toronto Road, Vancouver, V 6 T 1L2, B . C . September 6th, 1986. Dear Professor:  I of British and Tom to request  am an M . A . student in the Adult Education Program at the University Columbia. While meeting with my thesis committee: Drs. Roger Boshier Sork, it was suggested that I write to you. M y reason for writing is your assistance with the validation procedures of thesis.  The checklist on the reverse side of this letter contains names which have previously been categorized during my research effort. I am hoping to validate my research tool by having independent experts check off the professional adult educators from the names supplied. The process should take from five to ten minutes of your time.  I have enclosed a stamped, self-addressed envelope for your convenience.  I wish to thank you in advance complete research project to my committee.  for  helping  me  to  present  a  more  Yours Sincerely, Richard O. Kavanagh Graduate Student Administrative, Adult and Higher Education University of British Columbia  75  APPENDIX D  Validation Procedure and Results INSTRUCTIONS: Please indicate with a check (/) all of the following who are primarily known for their adult education activities. Name Knowles, Malcolm S. Houle, Cyril 0 . Verner, Coolie Tough, Allen Knox, Alan B . Freire, Paulo Lindeman, Eduard C. Boshier, Roger Havighurst, R . J . Maslow, A . H . London, Jack Cross, K . Patricia Darkenwald, Gordon G . Bergevin, Paul Johnstone, John W . C . Rivera, Ramon J . Dickinson, Gary Hiemstra, Roger Illich, Ivan Grabowski, Stanley M . Kerlinger, Fred N . Erikson, Erik H . Kidd, J.R. Dewey, John Ohliger, John Sticht, Thomas Miller, Harry H . Rogers, Carl Sheffield, Sherman Griffith, William S. Carlson, Robert A . Douglah, M . Litchfield, Anne Monette, Maurice Penland, Patrick R. Gagne, R . M . Blakeley, Robert J . Allport, Gordon W.  Rule-Based Classification primary primary primary primary primary primary primary primary primary secondary primary primary primary primary primary primary primary primary secondary primary secondary secondary primary secondary primary secondary primary secondary primary primary primary primary primary primary primary secondary primary secondary 76  Experts Agreeing 11/11 11/11 11/11 11/11 11/11 10/11 10/11 11/11 2/11 11/11 10/11 6/11 11/11 11/11 7/11 6/11 11/11 11/11 10/11 11/11 11/11 11/11 11/11 11/11 11/11 10/11 7/11 10/11 3/11 10/11 11/11 5/11 6/11 8/11 9/11 11/11 10/11 11/11  Levinson, Daniel J . Neugarten, B . L . Rubenson, Kjell Benne, Kenneth Brockett, R. Edwards, Allen L . Kotler, Philip Sheehy, Gail Skinner, B . F . Ausubel, David P. Cartwright, Morse A . Rokeach, Milton Fishbein, M . Vincent, John H . Corwin, Ronald G .  secondary secondary primary primary primary secondary secondary secondary secondary secondary primary secondary secondary primary secondary  11/11 11/11 10/11 5/11 10/11 11/11 11/11 11/11 11/11 11/11 7/11 11/11 11/11 1/11 11/11  

Cite

Citation Scheme:

        

Citations by CSL (citeproc-js)

Usage Statistics

Country Views Downloads
Japan 24 0
United States 9 1
Germany 5 34
Namibia 4 1
China 3 20
Norway 1 0
Poland 1 0
City Views Downloads
Tokyo 24 0
Unknown 12 40
Ashburn 8 0
Shenzhen 2 20
Beijing 1 0

{[{ mDataHeader[type] }]} {[{ month[type] }]} {[{ tData[type] }]}
Download Stats

Share

Embed

Customize your widget with the following options, then copy and paste the code below into the HTML of your page to embed this item in your website.
                        
                            <div id="ubcOpenCollectionsWidgetDisplay">
                            <script id="ubcOpenCollectionsWidget"
                            src="{[{embed.src}]}"
                            data-item="{[{embed.item}]}"
                            data-collection="{[{embed.collection}]}"
                            data-metadata="{[{embed.showMetadata}]}"
                            data-width="{[{embed.width}]}"
                            async >
                            </script>
                            </div>
                        
                    
IIIF logo Our image viewer uses the IIIF 2.0 standard. To load this item in other compatible viewers, use this url:
http://iiif.library.ubc.ca/presentation/dsp.831.1-0055771/manifest

Comment

Related Items