Open Collections

UBC Theses and Dissertations

UBC Theses Logo

UBC Theses and Dissertations

Flexible learning versus classroom lecture : a content analysis of undergraduate nursing students' learning… Musni, Sarah Marie 2017

Your browser doesn't seem to have a PDF viewer, please download the PDF to view this item.

Item Metadata

Download

Media
24-ubc_2017_may_musni_sarahmarie.pdf [ 9.4MB ]
Metadata
JSON: 24-1.0344005.json
JSON-LD: 24-1.0344005-ld.json
RDF/XML (Pretty): 24-1.0344005-rdf.xml
RDF/JSON: 24-1.0344005-rdf.json
Turtle: 24-1.0344005-turtle.txt
N-Triples: 24-1.0344005-rdf-ntriples.txt
Original Record: 24-1.0344005-source.json
Full Text
24-1.0344005-fulltext.txt
Citation
24-1.0344005.ris

Full Text

FLEXIBLE LEARNING VERSUS CLASSROOM LECTURE: A CONTENT ANALYSIS OF UNDERGRADUATE NURSING STUDENTS’ LEARNING USING CONCEPT MAPS  by SARAH MARIE MUSNI BSN, The University of British Columbia, 2009  A THESIS SUBMITTED IN PARTIAL FULFILLMENT OF  THE REQUIREMENTS FOR THE DEGREE OF  MASTER OF SCIENCE IN NURSING  in  THE FACULTY OF GRADUATE AND POSTDOCTORAL STUDIES   THE UNIVERSITY OF BRITISH COLUMBIA (Vancouver)  April 2017  © Sarah Marie Musni, 2017  	   ii	  Abstract Background: The ongoing difficulty that nursing educators face is managing finite class time and limited instructor-student interactions to achieve balance between discourse and active learning. To complicate this predicament, technology has infiltrated all aspects of daily life demanding that education must also meet the growing expectation of students to incorporate technology into curriculum. One approach that addresses this need is the flipped classroom (FC) format, which remains operationally ill-defined and the understanding of its effects on higher-level thinking are still nascent. Aim: The purpose of this study is to explore the differences in learning between the FC format compared to the traditional classroom lecture (TL) in the context of an undergraduate nursing course. Methods: A content analysis was performed on a previous study conducted in 2015. Concept maps were used to evaluate data from transcripts of undergraduate nursing students discussing a case scenario in either a TL or FC format.  Results: When comparing FC and TL groups, FC groups had a more complex concept map morphology and greater amount of identified subcategories and links. The FC groups exhibited more higher order thinking concepts compared to the TL cohort. An unexpected finding was the emergence of discussion tangents across both the FC groups and the TL cohort. Conclusion: Flipped classrooms have a place in the gamut of pedagogical approaches and this study demonstrates that the FC approach enhances student learning and aids in the development of higher-level thinking. Keywords: flipped classroom, concept maps, traditional lecture, flexible learning, content analysis 	   iii	  Preface This thesis is original, unpublished, independent work by the author, S.M. Musni.  I was responsible for identifying the topic of interest for this thesis and, through the help of my committee, was able to choose the research design and analysis approach for the research data. Dr. Bernie Garrett was my thesis supervisor and Dr. Maura Macphee and Cathryn Jackson were part of my thesis committee. All three members of my committee were responsible for helping me draft my thesis and contributed throughout the writing process. I conducted the analysis, using NVivo software 11.3.2, and wrote this manuscript on my own, with the inclusion of some comments and feedback from my committee.               	   iv	  Table of Contents Abstract……………………………………………………………………………………............ii Preface……………………………………………………………………………………………iii Table of Contents…………………………………………………………………………………iv List of Tables……………………………………………...……………………………………...vi List of Figures…………………………………………………...……………………………….vii Glossary of Terms……………..………………………………………………………………..viii Acknowledgments…………………………………………………………………..……………ix CHAPTER 1 INTRODUCTION………………………………………………………………...1  1.1 Background and Problem Statement…………………………………………………..1  1.2 Purpose………………………………………………………………………………...6  1.3 Significance……………………………………………………………………………6  1.4 Research Questions………………………………………...………………………….7  CHAPTER 2 LITERATURE REVIEW…………………………………………………………8  2.1 Search Question and Search Method………………………………………………….9  2.2 Sample………………………………………………………………………………..13  Findings   2.3 Knowledge…………………………………………………………………...15   2.4 Satisfaction…………………………………………………………………...16   2.5 Student Perceptions…………………………………………………………..17   2.6 Student Disposition…………………………………………………………..18   2.7 Gaps in Literature……………………………………………………………19  Limitations……………………………………………………………………………….21  Implications………………………………………………………………………………21  Conclusion……………………………………………………………………………….22  CHAPTER 3 METHODS………………………………………………………………………25  3.1 Approach……………………………………………………………………………..25  3.2 Study Design…………………………………………………………………………27  3.3 Sampling Plan………………………………………………………………………..28  3.4 Procedures, Data Collection and Ethical Consideration……………………………..29  3.5 Data Analysis………………………………………………………………………...30  3.6 Limitations of Study Design…………………………………..……………………..34   	   v	  CHAPTER 4 RESULTS………………………………………………………………………..36   4.1 Results………………………………………………………………………………..36   Categories and Subcategories……………………………………………………36   Links and Level of Thinking…………………………………………………......48  4.2 Summary……………………………………………………………………………..50  CHAPTER 5 DISCUSSION……………………………………………………………………51  5.1 Gross Similarities and Differences…………………………………………………..51  5.2 Discussion Tangents…………………………………………………………………53  5.3 Limitations…………………………………………………………………………...55  CHAPTER 6 CONCLUSION…………………………………………………………………..60  6.1 Summary……………………………………………………………………………..60  6.2 Implications for Future Practice……………………………………………………...61  6.3 Conclusion…………………………………………………………………………...65  REFERENCES…………………………………………………………………………………..67 APPENDIX A Literature Review Matrix………………………………………………...74 APPENDIX B  Composite Concept Maps………………………………………………..94 APPENDIX C  Table of Individual Group Subcategories and Links…………………...102 APPENDIX D Individual Group Concept Maps………………………………………..111                  	   vi	  List of Tables  Table 2.1 Scoping Review of Flipped Classroom and    Web-based Learning in Nursing Education………………………………….…..14 Table 3.1  Example of Content Unit, Condensed Content Unit   and Determination………………………………………………………………..31 Table 4.1 Number and Percentage of Group Discussion Devoted to Category…………….37 Table 4.2 Percent Average of Subcategories Identified by FC or TL Groups……………...38 Table 4.3 Percent Average of Links Identified by FC or TL Groups………………………38 Table 4.4 Number of Groups and Average Links    Identifying “Growth” Subcategory..................................................................…..39 Table 4.5 Number of Groups and Average Links    Identifying “Baby #2” Subcategory……………………………………………...41 Table 4.6 Average Number of Links Associated    with “Recommendation” Subcategory…………………………………………...42 Table 4.7 Average Number of Discussion Tangent Occasions &   Average Number of Links per Occasion………………………………………...46 Table 4.8 Number of Salient Concepts……………………………………………………..49  Table 4.9 Percentage of Higher Level Concepts per Major Category……………………...49               	   vii	  List of Figures  Figure 2.1 PRISMA Flow Diagram…………………………………………………………12 Figure 3.1 Example of Subcategory & Link Identification………………………………….31 Figure 4.1 Bottle Use in TL Composite Concept Map………………………………………40 Figure 4.2 Bottle Use in FC Composite Concept Map………………………………………40 Figure 4.3 TL Composite Concept Map Illustrating “Approach to Care”…………………..43 Figure 4.4 FC Composite Concept Map Illustrating “Approach to Care”…………………..44 Figure 4.5 TL Composite Concept Map for Main Category “Nurse”……………………….45 Figure 4.6  FC Composite Concept Map for Main Category “Nurse”……………………….45 Figure 4.7 Morphology of Discussion Tangents Composite Concept Maps………………..47 Figure 5.1 Example of Isolated Concept Map……………………………………………….57 Figure 5.2 Example of Integrated Concept Map…………………………………………….57                     	   viii	  Glossary of Terms  Flexible Learning: A broad term under which a variety of approaches are classified, two of which are online learning and flipped classroom. Flexible Learning speaks to two perspectives: 1. Logistics: flexibility in time or location pace of learning 2. Pedagogy: flexibility in activities/delivery of content/methods of instruction  Online Learning:  Content and course material are presented through web-based or computer-based methods allowing for students to participate in classes without physical attendance in a classroom setting on campus. When both in-person lectures and online activities are incorporated together, this is termed “Hybrid Learning” or “Blended Learning”. Other terms used interchangeably included: Web-based learning, Internet-based and Distance learning  Flipped Classroom (FC): The flipped classroom is a pedagogical method that reverses the traditional lecture and “homework” elements. Lectures and instruction are viewed online prior to class and class attendance is designated for activities, discussion or exercises.  Concept Mapping: A technique developed in the 1960s. It allows for focus on a specific subject matter and facilitates conceptualizing through a structured process. The result is a graphic representation of ideas and concepts and shows their interrelationship.       	   ix	  Acknowledgements  Bernie, Maura, and Cathryn: It has been a privilege to call all of you my committee. Thank you for your support, guidance and wisdom  Taline R.: Thank you for being the best second coder anyone could ask for! I truly appreciated all your hard work and enthusiasm. I wish you all the best in your nursing career.  To my parents: Thank you for the endless support, cheerleading, willingness to read all six of these chapters, and for believing in this work  To my brother, Mark: who in his own quiet way, affirmed what I do  All my friends and co-workers: Thank you for the interest in my thesis and all the pep talks to finish it.   Max: You are the best distraction a grad student auntie could ask for 	   1	  Chapter 1: Introduction    What is the ultimate purpose of higher education? While on the surface, higher education seeks to impart knowledge and discipline specific skills, the enduring goals of higher education are to foster and cultivate practical reasoning in the student, and more importantly, to nurture the capacity for continuing learning.   In its purest philosophical sense, higher education serves to help students understand that one’s purpose in life is not found solely in egocentricities, but in the pursuit of an intention greater than oneself (Sullivan & Rosin, 2008; Kreber, 2014). Indeed, the philosophical undertaking of higher education is lofty, but the objective of supporting individuals to question, to reason, to discern, to think more critically and finally to promote a disposition towards the continual pursuit of learning is not simply an ideal of higher education, but a requisite for our society to thrive (Abrami et al, 2008). For this objective to be achieved, higher education must understand more completely the learning process in the present conditions to ensure that it surpasses the basic conditions of imparting knowledge and skill towards cultivating a sense of meaningful contributions, not just in one’s chosen fields but also in the larger social context (Kreber, 2014). 1.1 Background and Problem Statement  The learning process is a complex undertaking that teachers and theorists alike have struggled to fully grasp and convey to students. How and when does learning occur? Under what conditions, context or circumstances can the learning process be maximized (Brown, 2014)? While these questions are considered enduring, the backdrop in which these queries present themselves is constantly changing. Transformations in technology, communication and accessibility of information have permeated all aspects of life. Consequently, our education must 	   2	  adapt. It affects what we learn, how we learn, how we teach and when and where we choose to do so. Most notably, areas in healthcare and healthcare education have seen significant change by the emergence and use of new technologies and communication tools. The effect is not only seen in the development of abstract concepts but also in the application of skills to real life situations.  The rapidly evolving information and communication technologies, innovations and increased demands in health care have led to the examination and restructuring of nursing curriculum and pedagogy (Petit dit Dariel et al., 2012). Nursing education does not seek to solely impart knowledge, but more importantly it demands to integrate theory with clinical practice. When the Internet emerged in the 1990s, the possibilities to address this gap between knowledge and practice grew dramatically. Over time, approaches to both learning and practice  have become more creative, diverse and sophisticated (McCutcheon et al., 2014). While innovation can act as a catalyst for change, improvement and excellence, the concern of nursing education should be that novelty is not confused with effectiveness. The quandary that nursing education grapples with is how to provide curriculum that is fitting and relevant to today’s learner in a healthcare context, a context that is progressive or inventive but does not diminish the quality or effectiveness of the learning process.  Pedagogical approaches no longer follow a strict model but can be tailored more practically to better suit course content, student demographics, teaching strengths or environmental constraints such as time or location. There is also evidence that students born after 1982, often termed “millennials” or the “net generation”, exhibit different learning styles and require different teaching strategies compared to those of previous generations (Simpson & Richards, 2015, p. 163). These students prefer environments that are supportive, interactive and 	   3	  engaging. They favour immediate feedback, group work and the opportunity to learn in their own way (Simpson & Richards, 2015). Knowledge construction in this setting becomes a social dialogue process (Pea 1993 as cited by Ertmer et al., 2011).  To address this change in the learning atmosphere and meet the learning needs of today’s students, strategies were developed and implemented to allow for more flexibility in learning. Physical and pedagogical constraints are addressed with flexible learning techniques, both of which are considered important to this generation of nursing students. In this manner, students are offered more freedom of time and place to learn and the course content can be presented in a form that maximizes interaction and scaffolding to promote higher order thinking (Simpson & Richards, 2015). Scaffolding is a term derived from Vygotsky’s (1978) “Zone of Proximal Development”. It suggests that students should be given supports or ‘scaffolding’ that allow them to build on towards more complex knowledge or skill, transitioning from lower levels of thinking towards the higher and more complex levels of thinking and problem-solving. Examples of such supports include coaching, prompting, questioning, modeling and feedback (van Merriënboer, Kirschner, & Kester, 2003). Higher order or levels of thinking is a concept based on learning taxonomies or cognitive patterns whereby lower order thinking pertains to recall of factual knowledge and higher order thinking is the incorporation of new and stored information and knowledge towards the creation, analysis or evaluation of real-life situations (Kantar, 2014; Lewis & Smith 1993).  With the advent of the Internet and its various technological presentations, web-based strategies were also integrated into flexible learning pedagogies. Web-based learning has many names including distance education, online education or E-learning. Its roots stem from the generation of correspondence courses, and, when technology became incorporated into higher 	   4	  education, developed into what we identify now as web-based education (Halawi et al., 2009).  Web-based learning, through the lens of constructivism, addresses the contemporary student’s preferred learning environment. Specifically, constructivism is considered the most suitable theory to address the educational needs of those in the nursing profession (Twomey 2004 as cited by Lahaie, 2008). Constructivism is a learning theory that defines formation of knowledge as developed by people through discovery and experiences (Brown, 2014). The responsibility of the learning process, therefore, no longer rests solely on the instructor but becomes more student-centered, fostering independence and self-discipline (Legg et al., 2009). Web-based learning, therefore, is not simply the presentation and delivery of course material but is also the learning process itself (Arving et al., 2014). Students are able to engage in higher level thinking skills and then are able to reconstruct this knowledge into new and meaningful ways (Lahaie, 2008). Interactions within web-based learning consist of “student to instructor”, “student to student” and “student to content”. The pedagogical goal of all these interactions is to increase student understanding of the course content. This is evident when there is a change in the learner’s understanding, perspective and cognitive structures of the learner’s mind (Moore 1989 as cited by Ertmer et al., 2011, p.158).  In an effort to address the growing demands of maintaining student engagement, increasing satisfaction and promoting student centered learning, higher education institutions recognize that the use of technology in addition to traditional scholarship is essential (O’Flaherty & Phillips, 2015). The ongoing difficulty that teachers face is managing finite class time and limited instructor-student interactions to achieve balance between discourse and active learning (Kim et al, 2014).  	   5	   Currently, a pedagogical approach that is categorized as a flexible learning strategy whilst incorporating web-based techniques is the Flipped Classroom. The basic description of this approach submits that what was once class content, the teacher-led instruction or didactic lecture, is replaced with what was previously considered “homework”, assigned activities for completion, to be done during class time (Pierce & Fox, 2012 as cited by O’Flaherty & Phillips, 2015). Students are expected to come to class prepared to participate, having completed the required preparatory learning on their own. By this definition, the flipped classroom model should provide some flexibility that students so desire while also imparting a greater responsibility for the learning process. One could consider, based on this interpretation, that flipped classrooms have been in existence within education for longer than reported through the requirement of students to complete preparatory work before attending class to discuss concepts in an in-depth manner (Strayer, 2012 as cited by O’Flaherty & Phillips, 2015).   Ideally, the flipped classroom would move passive traditional lecture or course content beyond the classroom. This asynchronous strategy would allow for more class time to be devoted to synchronous active learning activities including application of content to practical discussions and experiences (O’Flaherty & Phillips, 2015; See & Conry, 2014).  As Gilboy et al (2015) note, the flipped classroom model encourages the instructor to be less of a “sage on the stage”, where students are seated passively listening to a lecture, and more of a “guide on the side”, where students are exploring content independently or in groups but the instructor is available to provide assistance, correction and guidance (p.109). The use of technology and multimedia would provide opportunities for students to learn, that otherwise were not be possible with traditional approaches (Kim et al, 2014).  	   6	   The flipped classroom model in its current manifestation, however, remains operationally ill-defined and the web-based learning strategies incorporated vary significantly in structure, process and style (Bernard, 2015; Creedy et al., 2007).  As such, research determining effectiveness of both the flipped classroom model and web-based learning strategies remains in its early stages despite evidence of this flipped classroom model inception a decade ago and the use of technology based components in nursing education since the 1960s (Harrington et al., 2015; Gagnon et al., 2013). 1.2 Purpose  The purpose of this study was to explore the differences in learning between a flipped classroom-learning format compared to a traditional classroom lecture form in the context of an undergraduate nursing course. Research to date appears to be insufficient to confirm the effectiveness of the flipped classroom model though it is implemented widely throughout nursing education (Bernard, 2015, p.8). While current literature notes that flipped classroom courses encourage students and faculty to rethink how “learners learn and teachers teach”, further inquiry is needed to prove its value within higher education in healthcare (Simpson &Richards, 2015, p. 166; McCutcheon et al., 2014). Through the use of qualitative content analysis and concept maps, the goal is to determine if there are distinct differences between the two approaches and in what manner. 1.3 Significance  The findings from this study contribute to both educator and student perspectives. For educators, this study can contribute to knowledge of the value of the flipped classroom as an educational innovation, which can then inform future curriculum development. By understanding the flipped classroom approach more fully, educators will be able to more effectively translate 	   7	  the pedagogical strategy into practice. Results from this study will also contribute to the understanding of students’ dispositions and demonstration of higher order thinking within the context and dynamics of a flipped classroom format with web-based components or in the traditional lecture format as evidenced by the depth or breadth of meaning units identified through qualitative content analysis. 1.4 Research Questions 1. Does a flipped classroom strategy enhance small group learning?  2. What evidence supports the differences between flipped learning and a traditional lecture format with respect to higher order thinking? 	  	  	  	  	  	  	  	  	  	  	  	  	  	  	  	  	  	  	  	  	  	  	  	  	  	  	  	  	   8	  Chapter 2: Literature Review  A decade ago, the Flipped Classroom model was in its inception. Emerging from secondary school education, the approach was intended to decrease time spent re-teaching material to students who were absent (Tucker, 2012). Today, the flipped classroom framework is used across all levels of education as pedagogy for a more active, student-centered learning environment. Students are able to access course content outside of class time, which frees up “seat time”, the amount of time students spend in a classroom, and enables the time students do spend in class to be spent in dynamic learning activities.   Preparatory work and learning prior to class, however, is not a new notion and is certainly a practice older than 10 years. The flipped classroom model changes the sequence in which students build upon their knowledge: lower level knowledge acquisition on their own time and higher order thinking in class under the facilitation and support of instructors (See & Conry 2014). Additionally, as the Web and web-based technologies continue to be developed, education has incorporated software and multimedia into its pedagogical approaches, flipped classrooms included. While innovative, one must consider how studied the effectiveness of both the flipped classroom and web-based approaches are.  The purpose of this chapter is to explore current literature and its contribution to the understanding of flipped classrooms and web-based learning approaches to nursing curriculum and pedagogy. While these concepts are separate entities and compelling to be investigated individually, this review attempts to understand them together and in relationship to traditional face-to-face lecture format. The reasoning for this choice is that education, notably higher education, recognizes that technology and web-based approaches with or without traditional face-to-face pedagogy, is considered essential in order to promote learning and sustain student 	   9	  engagement and satisfaction (O’Flaherty & Phillips, 2015). Exploring what is known of flipped classrooms together with web-based strategies would highlight this aspect of an evolving learning environment. Using Arksey and O’Malley’s (2005) approach, a scoping review was conducted to determine the extent of current research, and to identify findings and potential research gaps within existing nursing literature. Unlike a systematic review which focuses on a well-defined question leading to a limited range of quality assessed studies, a scoping review, as defined by Arksey and O’Malley (2005), is inclined to address broader topics where different study designs might be relevant. In this sense, a scoping review is less likely to answer a narrow research question and subsequently does not assess in great detail the quality of studies chosen. Scoping reviews can be distinct projects on their own or can be cornerstones to larger research endeavors. In both cases, scoping reviews aim to map key concepts and identify main sources and evidence available. Four reasons have been identified for undertaking a scoping review:  1) To investigate the extent, range and nature of research activity;  2) To determine the value of pursuing a full systematic review;  3) To summarize and disseminate research findings or  4) To identify gaps in the existing literature findings (p.21). Methods 2.1 Search Question and Search Method  The question asked for this scoping review was: “What is known from the current literature about the use of flipped classrooms and non-traditional learning in nursing education?” It was anticipated that broad terms would aid in gathering all potentially relevant articles as wider definitions initially in a search would reduce the possibility of missing relevant studies and 	   10	  provide breadth of commentary (Arksey & O’Malley, 2005). It was planned that inclusion, exclusion and search parameters would then be applied once preliminary volume of study results could be identified. The search employed online databases CINAHL and MEDLINE. These two databases were used, as they are specific to nursing or health disciplinary research.   Initially the terms “Nursing Education”, “Non-Traditional”, “Flipped Classroom” and “Learning” were used. Contrary to assumption that these expansive terms would lead to a multitude of articles, the search only yielded two articles. The search question was amended to be: “What is known from the current literature about the use of flipped classrooms and web-based learning strategies in nursing education?” Since the definition of flipped classrooms can be ill-defined and web-based strategies can encompass many techniques, commonly associated synonyms were added to expand the number of results. The final search terms were entered as follows: Nursing Education AND Online OR Flipped Classroom OR Non-Traditional OR Blended OR Hybrid OR Web-based AND Learning    Inclusion and Exclusion Criteria. Studies considered in this scoping review were conducted and published between 2006 and 2016. The rationale for this range takes into 	   11	  consideration both the “novelty” of the flipped classroom model and the rapid refinement of web-based approaches and technologies. Other inclusion criteria were publications in English with full text accessible online. Included studies were peer reviewed and published in academic journals. As the large volume of potentially pertinent articles became evident, parameters were set to exclude studies involving simulation, post-RN populations pursuing a bachelor degree and health disciplines other than nursing.  With the determined terms, results totaled 157,763 articles. The inclusion and exclusion parameters were applied to narrow the results to a manageable number. Under these new parameters, results totaled 167 articles. Duplicates of studies and those that did not have full text online accessibility were removed, and the total was reduced to 159 articles. A title search was done and abstract suitability reduced the total count to 24 articles. This was achieved by reading the titles and abstracts of the 159 articles to establish if the study was addressing the area of interest, flipped classrooms with web-based strategies in nursing education, in some manner. These 24 articles were then read in their entirety. Decisions were made to include certain articles despite not addressing the area of interest completely as they offered insight, perspective or a different approach that appeared compelling to the understanding of the current literature. From this search, the article count for this scoping review was determined to be 15 articles.  As literature reviews are an iterative process, the database search was repeated a second time four weeks later to ensure that the most current and relevant articles published were found. No changes in the number of articles resulted in this second search. In addition to the online database search, a title search was conducted on the reference lists of the 15 articles chosen. This resulted in an additional seven articles. Using the inclusion and exclusion criteria and, by reading 	   12	  each article in its entirety, the seven articles were narrowed to four. In total, this scoping review consists of 19 articles.   	                     Figure 2.1: PRISMA flow diagram Studies	  identified	  through	  database	  searching	  (n	  =	  157,763)	  Studies	  after	  inclusion/exclusion	  &	  parameters	  applied	  (n	  =	  167)	  Studies	  after	  duplicates	  removed	  (n	  =	  159)	  Studies	  screened	  (n	  =	  159)	  Studies	  excluded	  (n	  =	  135)	  Full-­‐text	  articles	  assessed	  for	  eligibility	  (n	  =	  24)	  Full-­‐text	  articles	  excluded,	  with	  reasons	  (n	  =	  9)	  -­‐Not	  appropriate	  intervention	  (5)	  -­‐Not	  nursing	  profession	  (3)	  -­‐Not	  English	  (1)	  	  Studies	  included	  in	  review	  (n	  =	  15)	  Additional	  studies	  from	  reference	  list	  (n	  =	  4)	  Total	  studies	  in	  reivew	  (n	  =	  19)	  	   13	  2.2 Sample  The total sample for this scoping review is 19 articles. Of these articles, four were reviews and 15 articles were individual research studies. The motivation for including literature reviews to this scoping review was to determine the extent of what is understood of flipped classrooms and web-based learning within nursing education. Literature reviews offer a wider perspective as an adjunct to individual studies. The 19 articles were published between 2008 and 2015 and these studies were conducted mostly in the United States of America with studies also conducted in Canada, United Kingdom, Sweden, Netherlands, Norway, Spain, Slovenia, South Africa, Kuwait, Turkey, China, Japan, Korea, Thailand and Australia. This sample of 19 studies is heterogeneous in nature, as it is comprised of literature reviews, quantitative methods, mixed methods and qualitative approaches. These studies also explored different research areas and questions pertaining to flipped classrooms and web-based learning in nursing education. Therefore, they cannot truly be compared to each other with respect to quality of study, although as Arksey and O’Malley (2005) note, the aim of a scoping review is less concerned with the assessment of quality of the included studies.   The four reviews included in this search consisted of two systematic reviews that looked strictly at quantitative studies in the form of randomized controlled trials, one literature review and one systematic review that included both quantitative and qualitative approaches. No restrictions were placed on the individual studies related to study design, and the 15 studies included are comprised of eight quantitative, five mixed methods and two qualitative approaches. The sample populations of the studies and reviews included undergraduate students, post-graduate students and new graduate nurses in specialty course	   14	  Table 2.1: Scoping review of Flipped Classroom and web-based learning in nursing education Reference Type Sample Area of interest Conclusions Bernard, J.S. /2015 Lit Review 47 articles Flipped Classroom (FC): Student performance/ faculty+student perception/ faculty concerns Insufficient evidence to confirm effectiveness Du et al /2013 Systematic Review 9 RCTs Web-based learning & Knowledge acquisition/ skill performance/ self-efficacy/ satisfaction As effective  Lahti et al/2014 Systematic Review 11 articles (RCTs) E-learning & Knowledge acquisition/skill performance/ satisfaction No difference McCutcheon et al/2014 Systematic Review 19 articles Online vs Face-to-face (F2F) & Knowledge acquisition/ Skill performance/ self-efficacy/ satisfaction At least as effective Andrew et al/2015 Mixed methods 144 students Virtual Interactive Classrooms (VC) & Student satisfaction/participation Congruent w/ effective learning Arving et al/ 2014 Qualitative 53 students Blended Learning (BL) & Specialized nursing knowledge Effective dependent on content Cameron/2013 Quantitative 658 students Online vs. F2F in student characteristics/knowledge No difference Gagnon  et al/ 2013 Quantitative 102 students BL & Knowledge acquisition/ Satisfaction/ Self-direction Teaching model has no direct impact Geist et al/2015 Quantitative 86 students FC & Knowledge acquisition Differences during course, no difference in final Harrington et al/2015 Quantitative 82 students FC & Knowledge acquisition No difference Hsu&Hsieh/2014 Quantitative 99 students BL & student metacognition/attitudes/satisfaction + Correlation between freq of online chat & metacognition Lyons &Evans/2013 Mixed methods 46 students BL &Satisfaction/ learning experiences ↑perceived satisfaction Mgutshini/2013 Mixed methods 53 students Online vs F2F in Satisfaction/ learning experiences Comparable outcomes Missildine K et al/ 2013 Quantitative 589 students FC & Satisfaction/ academic success Difference r/t method, difference in satisfaction r/t method  O’Flaherty &Laws/2014 Mixed methods 101 students  VC & student motivation/ satisfaction/ knowledge/ perception VC=↑motivation/satisfaction/performance Post JL et al/2015 Qualitative 18 students FC & experiences/perceptions Can enhance but faces challenges Simpson & Richards/2015 Quantitative 157 students FC & satisfaction No difference but students in FC rated more + Sung, YH et al/ 2008 Quantitative 50 RNs BL & Knowledge/self-efficacy/satisfaction Can enhance knowledge, no difference in self-efficacy/satisfaction Trobec&Starcic/ 2015 Mixed methods 211 students Online & student knowledge/perceptions No difference  	   15	  Findings  The results of the four reviews and the 15 individual studies included in this investigation can be categorized into four areas of interest: knowledge, satisfaction, disposition and perceptions.  2.3 Knowledge   Nine studies measured knowledge acquisition. Five of the nine studies claimed there was no statistical difference in overall knowledge outcomes (Cameron, 2013; Gagnon et al, 2013; Geist et al, 2015; Harrington et al, 2015; Trobec & Starcic, 2015). Three of the five studies did indicate, however, a statistical difference noted in exams throughout a course period but overall did not show a difference between the traditional lecture strategy and a flipped or web-based approach (Gagnon et al, 2013; Geist et al, 2015; Trobec &Starcic, 2015). Du et al (2013) concurred in their systematic review that knowledge acquisition, retention and skill performance are at least equivalent in effects for both traditional face-to-face and web-based approaches. The remaining four of 9 studies indicated the students in a flipped classroom or web-based group had higher exam scores compared to their traditional in-class lecture counterparts (Mgutshini, 2013; Missildine et al, 2013; O’Flaherty & Laws, 2014). Missildine et al. (2013) indicated in their study there was significant difference according to teaching method. Students who participated in the flipped classroom approach with recorded lectures had higher exam scores than both the face-to-face lecture group and the face-to-face lecture with recording group. This result is notable as the study involved three varying teaching strategies for students compared to the other studies that employed the one strategy and the traditional lecture as a control.   Current State of Knowledge. It could be considered reassuring that the flipped classroom model and web-based learning are deemed similar in effectiveness to traditional 	   16	  lecture methods. Superiority, however, cannot be determined since factors pertaining to faculty, student, content and implementation have direct influence on effectiveness. Bernard (2015) observed that most findings in current articles do not provide details on the implementation of traditional classroom approaches or flipped classroom approaches. This, in turn, contributed to the varied interpretations and consequently the discrepancy in effectiveness.    2.4 Satisfaction  Of the six studies that explored student satisfaction, three noted no difference or similar satisfaction between traditional face-to-face lectures and flipped classroom or web-based approaches. Hsu & Hsieh (2014) observed a significant association between frequency of online discussions and level of satisfaction with blended learning and metacognitive qualities. Some discrepancies were observed in the reported results of certain studies. Simpson & Richard (2015) indicated no statistical difference in the course evaluations between face-to-face lectures and flipped classrooms, but also reported that students in the flipped classroom group rated the course more positively on the basis of its flexibility and practical application. Sung et al (2008) reported satisfaction was similar between the face-to-face group and the e-learning group; but found that the students in the e-learning group indicated that substantiality of the course was greater. Satisfaction scores in the Lyons & Evans (2013) study showed that in the experimental groups using online discussions, majority of the students disagreed with the value of these online discussions; but overall showed increased satisfaction with the course material and understanding of the course literature. This positive sense of satisfaction may be related to a component of the intervention, i.e., access to a librarian, but not the method by which students were able to reach the librarian. Similarly, the study by Mgutshini (2013) reported the online group expressed increased satisfaction but had varied responses over the material and clarity of 	   17	  the course. This contradicts the findings of the systematic review by Du et al (2013), which found that most studies reported high satisfaction with web-based modalities.  Current State of Knowledge. Findings have highlighted that satisfaction, as it pertains to the flipped classroom model and web-based strategies, is influenced by personal preference, experience and values. Suitability of content or implementation of strategy also exerted an impact on satisfaction. Results indicated that students’ sense of satisfaction was affected by inappropriate use of strategies or incongruence of course content to the implementation of the flipped classroom or web-based component. 2.5 Student Perceptions  Six studies explored the experiences and impressions of nursing students when using flipped classroom or web-based strategies. The majority of the results noted increased opportunity for interaction and collaboration with other students and more accessibility to instructors (Andrew et al, 2014; Arving et al, 2014; O’Flaherty & Laws, 2014; Trobec & Starcic, 2015). Mgutshini (2015) found that students in the campus group had higher perceived proficiency compared to the online group who felt less confident but nevertheless scored higher in exams than the campus group. In contrast, Post et al (2015) reported that students felt frustrated, disconnected from faculty and peers; and furthermore, depersonalized with a sense of having the flipped classroom model imposed on them. Students also perceived increased discipline was required of them in a flipped classroom than in exclusively web-based courses. Jump (2013) in the review by Bernard (2015) noted that students reporting negative satisfaction still performed better on exams than those in traditional lecture strategies, which would suggest that the “cognitive strain” that causes dissatisfaction could also promote superior performance (p.5). 	   18	   Current State of Knowledge. Findings for this area of interest inferred that experiences and impressions of flipped classrooms and web-based learning have a meaningful effect on its success as methodology. Student perceptions influence not only academic success but also shape student investment in the pedagogical approach. Results noted, interestingly, that web-based strategies had the dual ability to foster collaboration and accessibility or create disconnect from others.  2.6 Student Disposition  For this review, three studies were identified to explore “student disposition” which was seen to mean personal qualities inherent of mind or character, including motivation, self-direction, self-efficacy and metacognition. Gagnon et al (2013) indicated that motivation was the only variable to have a significantly positive effect with satisfaction. This study also measured self-direction using the “Self-Directed Learning Readiness Scale for Nursing Education” (SDLRSNE). The results stated the students in the control group (face-to-face learning) showed significant improvement if their baseline SDLRSNE scores were in the low to medium range. Comparatively, students with high SDLRSNE baseline scores in the intervention group, blended instruction with web-based tutorials and in-class sessions, reported greater improvement compared to the control group. These results would suggest self-directed learning is not suitable for all students and that teaching method was not directly related to student’s SDLR score.  The Sung et al (2008) study, on the one hand, investigated students’ self-efficacy related to traditional face-to-face instruction, blended learning with e-learning components and medication administration. They found no significant difference in self-efficacy between groups related to medication administration, measured pre-course and 6 months later. On the other hand, 	   19	  findings in a review by McCutcheon et al indicated an increase in self-efficacy and confidence related to blended learning strategies.  Hsu and Hsieh (2014) explored the metacognition of students and how blended learning might influence it. The authors hypothesized that blended learning would contribute to students’ metacognitive ability by enabling them to construct a framework for new knowledge through access, organization and evaluation of knowledge using technology and media. Results indicated those with a greater ability to analyze ethical dilemmas would also exhibit higher metacognitive abilities. This study also referenced the cultural disposition of Taiwanese students and how web-based approaches allow them to better express themselves online. Du et al (2013) indicated similar findings that web-based learning created an environment that is non-threatening allowing students to express themselves more effectively.  Current State of Knowledge. Findings indicated that the flipped classroom model and web-based strategies can influence the development of critical thinking, which is often regarded as the fundamental and overarching ideal of formal education (Abrami et al, 2008). That certain studies indicated no difference or that differences are only seen in specific conditions demonstrates that pedagogical strategy is not the sole predictor of academic success. 2.7 Gaps in Existing Literature  As Arskey and O’Malley (2005) indicated, one of the objectives of a scoping review is to identify gaps in the existing literature or areas in a given phenomenon that need further research. For this scoping review, gaps have been identified in all four themes.  All nine studies involving knowledge employed exam scores as the indicators of effectiveness of one approach compared to the other. This measure, while informative, does not provide the complete picture of knowledge acquisition and evidence of learning. Exam scores 	   20	  from one observed point in time are not sufficient to determine long-term knowledge retention and evidence of clinical reasoning. Bernard (2015) noted that some findings currently indicate the flipped classroom model has little influence on long-term learning. Lahti et al (2014) indicated more research is required to determine if e-learning is broadly applicable or only relevant to certain topics.   While satisfaction appears to be of great interest in the research of flipped classrooms and web-based learning in nursing education, it is inherently subjective and can be influenced by extraneous variables and inaccuracies in self-reporting. Satisfaction is an outcome, and more needs to be known of how flipped classroom approaches and web-based learning influence student satisfaction. While not pertinent to this specific review, faculty satisfaction with flipped classroom and web-based strategies has been minimally addressed in literature.  Like the satisfaction variable, current literature has measured student perceptions in addition to other variables but has not recognized the relationship between student experiences and implementation of successful flipped classroom or web-based interventions in nursing curriculum. What needs to be addressed in literature are these: a general understanding of what value this generation of students place on their education, and specifically what flipped classrooms and web-based strategies must do to meet those needs and provide meaningful learning experiences.  Current literature has suggested that flipped classrooms and web-based learning can further encourage certain dispositions in students such as critical thinking and self-awareness. It is still not fully established, however, what dispositions contribute to students succeeding in flipped classrooms and web-based learning, or what is required of these strategies to maximize learning and foster such dispositions as self-efficacy and motivation. 	   21	  Limitations of This Review  By its character and definition, a scoping review does not formally appraise the quality of findings or evidence in the studies chosen. This creates a limitation in understanding completely the topic in question. However, this limitation can be a catalyst for a more in-depth systematic review. Arksey and O’Malley (2005) stated likewise that a scoping review does not address the synthesis of findings, and therefore cannot contribute to the weight of evidence that favours the effectiveness of one approach over another. In like manner, decisions were made to narrow the scope (covering all available material) and depth (analyzing and evaluating in detail a smaller number of material) of this search in order to manage the quantity of data, which may have led to exclusion of relevant and informative articles. Although nursing education was the main focus of this scoping review, a fuller understanding of the concepts of flipped classroom and web-based approaches may have been achieved if other disciplines in the arts and sciences were included. Implications for Future Research  This scoping review has provided a sense of current literature regarding flipped classrooms or web-based approaches in nursing education. It is evident that our understanding of these pedagogical strategies remains incomplete and more areas for further scholarship remain untapped. As an example, future research designs must be more rigorous in order to confirm effectiveness of strategies (Bernard, 2015). Concurrently, instrumentation requires refinement for validity and reliability. Given the constant transformation of web-based approaches and applications of the flipped classroom model, instrumentation must also adapt to the changing education context.   Future research must also seek to understand the long-term retention of knowledge as a result of these pedagogical strategies as well as the influence or applicability of these approaches 	   22	  to other nursing courses and different populations (Geist et al, 2015). This paradigm shift in teaching methodology suggests that longitudinal assessments need to be established to measure outcomes other than knowledge recall, such as clinical reasoning; and to better understand the value and effectiveness of techniques (Harrington et al, 2015). Ideally, future research will involve greater investment and participation on the part of faculty, students and other stakeholders to have more robust and complete data that is representative of the population. Conclusion  Critical thinking poses a compelling paradox. One attitude values critical thinking as the most desirable purpose of education while the other recognizes critical thinking as a “mystical phenomenon” that is difficult to define (Dexter et al., 1997 as cited in Thompson & Rebeschi,1999). In an attempt to elucidate the definition of critical thinking, the American Philosophical Association, through a Delphi research project, developed a consensus characterization based on 46 published critical thinking theorists.  According to this iteration, critical thinking is:  “A purposeful process involving reflective judgment and reasoning of both inductive and deductive nature. The ideal critical thinker is habitually inquisitive, well-informed, truthful of reason, open-minded, flexible, fair-minded in evaluation, honest in facing personal biases, prudent in making judgments, willing to reconsider, clear about issues, orderly in complex matters, diligent in seeking relevant information, reasonable in selection of criteria, focused in inquiry and persistent in seeking results which are as precise as the subject and the circumstances of inquiry permit” (as cited in Thompson & Rebeschi, 1999, p.250).    Critical thinking has been characterized as containing the five elements of analysis, interpretation, inference, self-regulation and explanation for any given situation (Facione, 2013 as cited by Kaddoura et al., 2016). Concurrently, higher order learning processes include the assessment and classification of information, recognition of patterns, identification and prioritization of main ideas, comparison and contrast, connection of relationships and logical 	   23	  thinking (Jonassen, 1996 and Kinchin, 2000 as cited by Rosen & Tager, 2014). From these two definitions, the determination could be that critical thinking involves higher order thinking (Kaddoura et al., 2016). Even with these distinct features, critical thinking and higher order thinking remain complex and controversial, which further complicates its definition and its study (Abrami et al, 2008).  Critical thinking or higher order thinking is of the utmost importance in the formation of pedagogical approaches but beyond education, students who become critical thinkers tend to have better futures as functional and contributing members of their communities (Abrami et al., 2008).  This review endeavored to determine the scope of understanding of flipped classrooms and web-based learning approaches in nursing education. Of the 19 articles chosen, five of them indicated that flipped classrooms and web-based approaches are of equal effectiveness to traditional face-to-face pedagogies; while seven other articles stated that web-based strategies and flipped classrooms are important to nursing education and can enhance the learning experience. These findings are encouraging but are not enough evidence to determine superior effectiveness of learning compared to traditional pedagogies. Flipped classrooms and web-based methods cannot be considered individual or singular entities. Content, context and student characteristics must all be considered and accounted for when choosing media and its combinations of uses. Undoubtedly, this scoping review highlights the need for more in-depth investigations such as systematic reviews and rigorous studies, which are essential for the successful implementation of the flipped classroom model and web-based approaches and ultimately the progression of nursing education.  The subsequent chapters of this thesis will outline the secondary analysis of a study that sought to compare learning outcomes between the didactic classroom approach and the flipped 	   24	  classroom model with web-based modules. This scoping review is relevant because it serves as impetus and foundation for the secondary analysis. Findings illustrated the inadequate understanding of the flipped classroom model and little to no focus on its effects on students’ critical thinking. Through the use of content analysis and concept maps, the objective is to determine if the flipped classroom model using web-based modules enhances higher order thinking compared to the traditional face-to-face lecture format.  	  	  	  	  	  	  	  	  	  	  	  	  	  	  	  	  	  	  	  	  	  	  	  	  	  	  	  	  	  	  	  	  	  	  	   25	  Chapter 3: Methods 3.1 Approach  This study intended to provide some clarity on a pedagogical approach that might best enhance and support the development of higher order thinking and critical thinking. Subsequently, understanding of critical thinking and higher order thinking might be more precise and can guide in the development of more effective instructional interventions for critical thinking skills and dispositions (Abrami et al., 2008). In this chapter, the methods for this study will be outlined in detail. It is a secondary analysis of data from a previous study that was conducted in 2015. Through content analysis, this study endeavored to understand the differences between a flipped classroom model and traditional lecture format for a community health course in an undergraduate nursing program with respect to the critical thinking or higher order thinking demonstrated by students.  Qualitative content analysis has been used throughout nursing research and education allowing for the analysis of either manifest or latent content. In the case of this study, the content analysis explores more closely the manifest content of transcribed discussions between groups of students in either a flipped classroom (FC) model or a traditional lecture (TL) format. Statements made by students were coded and level of thinking was determined in order to establish if one approach led to greater higher-level thinking compared to the other approach. Data was analyzed using an inductive content analysis approach. Inductive content analysis is used when little is known of the phenomenon or knowledge regarding the phenomenon is disjointed (Polit & Beck, 2012).  Once the transcripts were analyzed, concept maps were generated to evaluate and compare the FC model to the TL format. Concept maps are a visual representation of a particular 	   26	  focus (Gurupur et al, 2015). For this study, concept maps were generated per group as a way to represent and understand individual discussions while broader concept maps were created to compare FC groups to TL groups. Novak and Gowin created concept maps in 1984 as a “schematic device to represent a set of concept meanings embedded in a framework of propositions” (as cited in Daley et al., 2016, p.631). In education, concept maps have been often related to critical thinking because concept maps provide a visual depiction of information, concepts and relationships identified and organized according to a student’s thinking (Senita, 2008; Mok et al, 2014). In evaluating a concept map, what becomes evident is the consideration given to evidence, context, conceptualization, methods and criteria and how it is depicted visually (Daley et al, 1999). Given that concept maps can serve as a tool for measurement of critical thinking abilities, the inference can be made that concept maps could also illustrate the critical thinking of students from codes derived from content analysis. Previous Study  A quasi-experimental study that compared student-learning outcomes between traditional didactic classroom approach and the flipped classroom strategy with web-based modules was conducted at the UBC School of Nursing in 2015. The sample population was a convenience sample of students in 4 different semesters that were enrolled in a community health undergraduate nursing course. Each class was comprised of 18-20 students. Inclusion criterion for this study comprised of students enrolled in the designated course. There were no exclusion criteria for this sample. The sample population consisted of students in the nursing program who were pursuing nursing as an additional degree or had at least 2 years of undergraduate education. They entered this course having already completed one full academic year of the nursing program. Students were recruited using email and printed cover letters and consent forms. 	   27	  Students submitted consent 24 hours prior to the discussion sessions. If a student did not consent, the group discussion was not recorded. To the knowledge of the research manager, this only occurred once in the final semester of data collection (K. Carino, personal communication, June 30 2016).  The study involved students receiving course content regarding nutritional and dental health in the traditional lecture or flipped classroom format. One semester received course content in the traditional face-to-face lecture format. These students had pre-class assigned readings and attended a lecture in class presenting content on nutrition and dental health. During the same class time, the students worked through the case study in small groups. The other three semesters experienced the course content through the flipped classroom format with web-based modules. This group of students had the same pre-class assigned readings as their traditional lecture counterparts but had access to online modules containing videos and embedded questions to guide learning. Students came to class to work through the same case study in their small groups. For the small group work, students in each class were divided into groups of 3 or 4 to discuss the case study involving the nutritional and dental health of a toddler and his parents in the community. The same professors taught all four semesters and the same case study was used for both the TL and the FC formats. Each group discussion was individually audio recorded and later transcribed by two research assistants. Current Study 3.2 Study Design   A qualitative content analysis approach was applied to the data from the previous 2015 study. Coding from the text data directly allows for observed specific occurrences to be unified into a larger whole or more general assertion (Elo & Kyngäs, 2007). This was also considered a 	   28	  descriptive single case study as this course and its pedagogical approaches could be considered a single entity, an intervention in a real-life context in which it occurred (Gerring, 2004). As a single case study, its analysis would inform understanding of the larger phenomenon of what pedagogical approach enhances higher order thinking (Yin, 2003 as cited by Baxter & Jack, 2008). Qualitative content analysis was best suited for this study because the data elements were words in students’ conversations during small group case discussions.  (Drisko & Maschi, 2015). Content from a transcript was analyzed and coded either within the context of a flipped classroom dynamic or a traditional didactic lecture format. Coding was of an emergent nature as the researcher and second coder progressed through each transcript. Due to the nature of the transcripts, content analysis was the most appropriate method of qualitative analysis because the data lacked identification of some nuances in context, such as non-verbal cues, which would contribute to meaning-making and latent content used in other qualitative methods (Graneheim & Lundman, 2004). Codes were then assembled into subcategories based on similar ideas or topics and then grouped into larger categories. These were later represented in individual group concept maps. The concept maps were assembled into a broader concept map to represent the FC model and the TL format overall. Morphology of maps, quantity of identified categories and links and the quality of those categories and links were assessed and evaluated to determine differences between the pedagogical approaches.  3.3 Sampling Plan  Sample Population. The sample for this content analysis contained 15 transcripts of the group discussions from the original study.  Inclusion and Exclusion Criteria. For this secondary analysis, all 15 transcripts were used for the sample, as it was determined that inclusion of all transcripts would provide a fuller 	   29	  perspective of the phenomenon in question. There were no exclusion criteria for this study’s sample.  Recruitment Method. No additional recruitment of data was needed for the content analysis. 3.4 Procedures, Data Collection and Ethical Consideration  No additional data collection or ethical approval was required for this work as it involved a secondary data analysis of the existing data from the 2015 study. Ethical approval for use of that data for subsequent analysis already existed, and it was made available to the researcher though the UBC School of Nursing.  Ethical approval was sought and given through the University’s Behavioural Research Ethics Board. The initial study of 2015 identified a minimal risk to students indicating potential discomfort with digital recordings. Students received both a paper copy and an invitation letter and consent form by email. Students were given a minimum of 48 hours to read consent form prior to class discussion. Participation in the initial study was not mandatory and did not affect students’ grades. No monetary compensation was offered for participation in the study. However, students were eligible for a token draw with nominal prizes.  The researcher and second coder for this analysis both received certification for Tri-Council Policy Statement: Ethical Conduct for Research Involving Humans Course on Research Ethics (TCPS 2: CORE). Confidentiality was maintained throughout the original study and the secondary analysis. Digital recordings and transcripts were kept locked with the Research Coordinator and were only granted access upon request. All identifying information was removed from transcripts.  	   30	  3.5 Data Analysis  The transcripts from the 2015 study were analyzed and coded using NVivo Software version 11.3.2. The 15 transcripts had been identified as either belonging to the flipped classroom group or the traditional lecture group. Four transcripts were from the traditional lecture format and the remaining 11 were groups in the flipped classroom format. Each transcript was coded separately. The researcher and second coder initially coded one transcript independently and then discussed in person choice of codes and interpretations. Periodically the researcher and second coder would meet to discuss the transcripts analyzed to ensure rigour and validity. Intercoder reliability refers to the extent to which coders agree with each other about the coding of the qualitative data. This is imperative, as it would indicate that the coders are measuring the same thing. Agreement would also indicate that the concept is not a fallacy of the researcher and enhances the validity of the code (Ryan & Bernard, 2003). Audit trails and reflexivity journals were also used to address and minimize any subjectivity or bias and to reflect on progress throughout the analysis process (Polit & Beck, 2012).  The analysis and coding approach was inductive. Transcripts were read in their entirety multiple times to establish a sense of the discussion. After multiple readings, key words or sentences were highlighted as content units and coded through NVivo software. Content units were defined as the words or sentences that stood out as noteworthy. These identified words or sentences were coded and condensed using the terms that emerged directly from the sentence. “Condensation” allows for shortening of a recognized concept without losing its core meaning (Graneheim & Lundman, 2004). As the analysis progressed, content units with shared commonalities were either grouped together to form subcategories or identified hierarchically as a subcategory and a link. Links were identified information associated with a subcategory. Once 	   31	  both the researcher and the second coder identified a subcategory, the text was searched for links associated with the subcategory. The progression of a group discussion made identification of links clear, as connections to a subcategory were often sequentially indicated after the subcategory in the dialogue. The example below illustrates the process of subcategory identification and link determination (Figure 3.1). Students identified the use of the bottle by the toddler in the scenario as noteworthy. Through the discussion one student identifies that the toddler uses the bottle at night and another student notes that this use will lead to dental caries. The use of the bottle was considered a content unit, which was condensed to “bottle use” as a subcategory code. The subsequent identification of when the toddler uses the bottle and consequences of the usage were determined as links to the subcategory. Content unit Condensed content unit Determination “Drinks from the bottle” Bottle Use Subcategory “Bottle at night” @night link “…Then he’s going to get dental caries” Dental caries link Table 3.1: Examples of content units, condensed content units, code determination   Figure 3.1: Example of subcategory and link identification  The coding process was also iterative. Often, as analysis progressed through a transcript, other codes and content units emerged and decisions were made to determine if such units were independent or needed to be combined with others to form a larger subcategory. As this was an 	   32	  inductive process, the final stage of the coding process was to determine if the subcategories could be united into an encompassing broader main category.  Once coding of a transcript was complete, the researcher and second coder each developed a concept map illustrating the codes and the associated linkages evident in that single group transcript. This was done for each individual group for subsequent comparison. When the individual concept maps from each group for each semester were completed, the researcher and second coder would meet to discuss the group of concept maps and individual findings. The researcher then analyzed and coded the transcript a second time. This acted, in part, like the process of double coding. Double coding is a process of coding a data set and then after a period of time, the researcher recodes the same data set and compares the results (Krefting, 1991 as cited in Baxter & Jack, 2008).  A table was created to chart individual group results of subcategories and links. This would later inform the findings of this study regarding commonly identified concepts and unique perspectives.  The researcher then integrated the concept maps from the individual groups into two composite concept maps (one for the FC groups, and one for the TL groups). Higher order thinking was determined through discussion of the researcher and the second coder. The agreed guideline for identifying lower level concepts was if information was rote from the case scenario, the lecture notes or textbook, it would be considered lower level thinking. Ideas or linkages that required combining information to form conclusions, inferences or questions would be considered higher-level thinking.    The final composite concept maps were then sent to the second coder for clarification or any additional perspectives. These concept maps were initially compared through their morphology or general visual structure. A concept map depicting higher order thinking would 	   33	  have a more complex appearance with more links and subcategories. Types of links such as singular/isolated, departmental or integrated cross-linkages would also indicate higher order thinking (Hung & Lin, 2015). Maps with departmental or isolated links depict lower level thinking as the linkages do not extend beyond the main category or subcategory originally identified. An integrated map would have crosslinks across multiple subcategories or main categories to depict understanding of how concepts are interconnected (Hung & Lin, 2015). The concept maps of this study were also compared in the level of thinking using the categories and links identified. Using the pre-determined guidelines agreed upon between the researcher and second coder, color-coding of concepts was done to illustrate if a concept was of a lower-level or higher-order thinking process. A concept map of a higher-level thinking group would have more concepts colored as higher-order thinking than lower level thinking. The figure 3.2 illustrates lower level concepts in blue and the higher order concepts in yellow  Figure 3.2: Example of color-coding for level of thinking 	   34	   Overcoming Difficulties. In qualitative content analysis, the researcher also serves as the instrument of coding and analytic decisions. Because of the dual role of researcher and instrument, reflexivity and self-reflection become important tenets to qualitative content analysis. Reflexivity refers to a researcher’s engagement in the explicit exercise of self-awareness. The purpose of self-reflection and reflexivity is to identity personal partialities or viewpoints and larger social issues that may affect decisions during analysis, be it conceptual or methodological (Drisko & Maschi, 2015). By identifying these biases, adjustments to methods can be made or justifications can be provided. Maintaining a journal and practicing reflexivity, the researcher and second coder were able to identify judgments or biases and these enabled intercoder discussions to determine if there were other perspectives or interpretations to consider. 3.6 Limitations of Study Design  One limitation of content analysis is that its design is less standardized and formulaic compared to other forms of analysis (Elo & Kyngäis, 2007). Its flexibility and lack of prescribed procedures suggest that results depend, to a certain extent, on the insights, analytic choices and style of the researcher (Elo & Kyngäis, 2007). Since the narrative material and analysis process are non-linear, this study design poses a challenge when elements that are coded can be relevant to subsequent categories as the coding process continues, and new meaning units emerge. In addition, the quantity of data can be vast, and can lead to compelling findings that are unrelated to the topic studied. This could lead to tangents or distractions from the research question (Elo & Kyngäis, 2007).   In the secondary analysis of written transcripts as data, a subsequent challenge to this study design is its dependence on the original study. There is a limited opportunity for clarification of data or transcription and no occasion to exercise the strategy of respondent 	   35	  validation with the original participants to clarify understanding of context and verify interpretation (Baxter & Jack, 2008). This study design was also potentially confronted by the limitations of the original study. Reactivity was a possible limitation in the previous study, in so far as students may have distorted their activity answers in their awareness of being observed and recorded. Depth, variety of answers and learning could have been credited to other factors such as experience in other courses, general disposition of students, maturation and may not be attributable to the methods of the course. Likewise, attrition is a common limitation to any behavioural study. In this instance, students may not have submitted consent, dropped out or not participated in the original study. Technology skills may have been a limitation of the original study. Students may have had varying skill levels with respect to technology, online or web-based applications and this may have affected their efficient use of course activity and knowledge acquisition. Similarly, online learning aptitude may be varied among students, while learning styles may have influenced knowledge acquisition. Transcripts of recordings could be incomplete or inaccurate due to audio and technology issues and auditory differences between transcribers.         	   36	  Chapter 4: Results  A retrospective qualitative analysis was conducted on transcript data from a previous study conducted in 2015. The purpose was to determine if the Flipped Classroom model (FC) enhanced student learning and development of higher level thinking skills compared to a Traditional Lecture (TL) format. The results through content analysis and evaluation of concept maps indicated some differences between the FC and the TL models. 4.1 Results Categories and Subcategories  To answer the first research question: “Does a flipped classroom (FC) strategy enhance small group learning?” the researcher and second coder looked to the categories and subcategories that emerged through the discussions from either pedagogical approach. It became quite clear that the researcher and second coder had a high degree of consensus in the choice of codes, interpretation and overall sense of the transcript.   Three major categories were identified from the analysis of transcripts: Child, Parents, and Nurse. When evaluating the individual concept maps (see Appendix D), the visual evidence showed what categories each group chose to focus on and to devote time during their discussion. Table 4.1 illustrates how one category often had more subcategories and linkages than the other two. This indicated more variety or in-depth discussion of a particular detail or area of the case scenario. However, no two groups were identical in their approach or discussion. An example would be FC group 4. Twenty five percent of their total identified subcategories were found in the “Nurse” category but 85% of their total identified links were in this category. These differences were particularly marked when each individual group’s categories and subcategories were assembled in a table format and links were identified (see Appendix C).   	   37	  Table 4.1: Number and percentage of group discussion devoted to a category  CHILD PARENT NURSE TANGENTS # % of total per group # % of total per group # % of total per group # % of total per group FC Group  1 Subcategories (n=13) 4 31%  2 15% 4 31% 3 23% Links (n=38) 20 53% 2 5% 8 21% 8 21% 2 Subcategories (n=16) 5 31% 3 19% 5 31% 3 19% Links (n= 64) 22 34% 10 16% 25 39% 7 11% 3 Subcategories (n=18) 5 28% 5 28% 4 22% 4 22% Links (n=50) 19 38% 5 10% 21 42% 5 10% 4 Subcategories (n= 16) 5 31% 2 13% 4 25% 5 31% Links (n=33) 2 6% 0 0% 28 85% 3 9% 5 Subcategories (n=23) 9 39% 5 22% 5 22% 4 17% Links (n=38) 12 32% 3 8% 19 50% 4 10% 6 Subcategories (n=18) 6 33% 8 44% 3 17% 1 6% Links (n=46) 13 28% 3 7% 29 63% 1 2% 7 Subcategories (n=16) 7 44% 3 19% 4 25% 2 12% Links (n=41) 10 24% 2 5% 26 64% 3 7% 8 Subcategories (n=23) 8 35% 7 31% 4 17% 4 17% Links (n=51) 14 28% 0 0% 23 45% 14 27% 9 Subcategories (n=26) 14 54% 4 15% 5 19% 3 12% Links (n=61) 17 28% 3 5% 34 56% 7 11% 10 Subcategories (n=22) 9 41% 6 27% 4 18% 3 14% Links (n=41) 13 32% 4 10% 18 44% 6 14% 11 Subcategories (n=12) 4 33% 4 33% 3 25% 1 9% Links (n=45) 8 18% 0 0% 36 80% 1 2%    CHILD PARENT NURSE TANGENTS     # % of total per group # % of total per group # % of total per group # % of total per group TL Group 1 Subcategories (n=20) 6 30% 8 40% 3 15% 3 15% Links (n=26) 5 19% 7 27% 13 50% 1 4% 2 Subcategories (n=20) 8 40% 6 30% 3 15% 3 15% Links (n=42) 17 41% 6 14% 14 33% 5 12% 3 Subcategories (n=22) 8 36% 8 36% 5 23% 1 5% Links (n=41) 14 34% 7 17% 18 44% 2 5% 4 Subcategories (n=23) 8 35% 7 30% 3 13% 5 22% Links (n=39) 7 18% 3 8% 19 49% 10 25% 	   38	   New concept maps were then created to compare all the Traditional Learning (TL) groups to the entire Flipped Classroom (FC) cohort in a more inclusive manner (see Appendix B). Using the same three categories of “Child”, “Parents” and “Nurse”, we were able to assess whether one pedagogical approach enhanced learning over the other approach. Tables 4.2 and 4.3 indicate percent average of total subcategories and links as identified by either the FC groups of the TL cohort. Table 4.2: Percent average of total subcategories as identified by FC or TL groups  Flipped Classroom (Flexible Learning)  (n=11 groups) Traditional Lecture  (n=4 groups) Percent average of subcategories (#)   CHILD  (total = 38 subcategories) 82% (31) 56% (22)    PARENT (total= 35 subcategories 66% (23) 49% (17)    NURSE (total = 6 subcategories) 100% (6) 100% (6)  Table 4.3: Percent average of total links as identified by FC or TL groups  Flipped Classroom (Flexible Learning) (n=11 groups) Traditional Lecture  (n=4 groups) Percent average of links (#)    CHILD (total= 193 links) 70% (135) 22%(43)    PARENT (total= 55 links) 58% (32) 42% (23)    NURSE (total= 331 links) 81% (267) 19% (64)   Within the category of “Child”, a wide range of subcategories was identified in the individual group concept maps that addressed physical, psychosocial, developmental and nutritional aspects of the case. In total, 38 subcategories were determined from the individual 	   39	  concept maps. Almost all the groups recognized “growth” as an important aspect of the scenario. The flipped classroom (FC) groups tended to make more associated links to this subcategory compared to their traditional lecture (TL) counterparts.  Table 4.4: Number of groups and average links identifying “Growth” subcategory  FC groups TL groups Number of groups identified “Growth subcategory 10/11 4/4 Average # of links associated with “Growth” subcategory 2.7 links 1.5 links   “Nutrients” and “Dental Health” subcategories were the next most commonly identified subcategories. The “Nutrients” subcategory was identified by most of the FC groups and one TL group. At least one link was made to this subcategory but of the FC groups that identified this subcategory, all but one made three or more links. Most of the FC groups and two of the TL groups also noted the “Dental Health” subcategory. The difference seen in the Dental Health subcategory was that some groups identified it but did not make additional links to it. These groups acknowledged dental health as a pertinent detail to the case scenario but did not delve more deeply into its importance or make connections associated with the dental health details despite it being distinctly part of the subject matter of this scenario or the objective for the activity. The remaining subcategories were perceived either by a single group or less than the majority of the groups in either pedagogical approach.  When comparing traditional and flipped overall, the morphology of the “Child” web of FC was more complex with more subcategories and links than that of the TL groups. The initial subcategories for “Child” in the TL web were more than the FC web since during the coding process, some subcategories were combined due to similarities. The FC web extended further with most subcategories having three or more links. The TL web had three subcategories, namely “bottle use”, “stomach pains” and “knows if no eat = candy”, with no links attached. The FC web 	   40	  had at least one link for every subcategory. This indicated that the TL groups suggested a particular concept was of importance but did not elaborate or delve deeply into its significance to the case study to create links. An example of this difference was the subcategory of “bottle use”. The TL groups indicated that bottle use was an issue for this child but the FC groups recognized that the child in the scenario drank a bottle at night. The groups also queried if this might be contributing to his lack of interest in eating or if his parents were weaning him off the bottle as it relates to his age.        Figure 4.1: “Bottle use” subcategory in the TL composite concept map  Figure 4.2: “Bottle use” subcategory in the FC composite concept map 	   41	   The major category of “Parents” had similar range in subcategories in the individual concept maps, with a total of 35 subcategories. Three subcategories arose as most commonly discussed or mentioned by a large portion of the overall study sample during the case study activity: “support”, “baby #2” and “pressure”. Unlike the “Child” category, associated links in the “Parents” category were generally fewer, with students in either pedagogical approach, identifying zero to two associated links related to a subcategory. Table 4.5: Number of groups and average links identifying “Baby #2” subcategory  FC groups TL groups Number of groups identifying “Baby #2” subcategory 9/11 3/4 Average number of links associated with “Baby #2” subcategory 0.3 links (only 2 groups had links associated with the subcategory, a total of 3 links) 0.7 links (total of 2 links for 3 groups)   When comparing traditional lecture (TL) to the flipped classroom (FC), their composite concept maps were visually similar. The TL groups identified more subcategories than the FL group but the TL groups have less links to these subcategories than the FC groups. The TL groups also had more subcategories without links (4 subcategories) than those of the FC groups (1 subcategory). This indicated that the TL groups, as seen in the “Child” category, mentioned these concepts but did not elaborate or associate them in detail to the case scenario. This observation, unlike the previous category, was not comparable to the FC groups, which did not identify the “link-less” subcategories that the TL did. The mutually identified subcategories between both groups show equal number of links or the FL web having one more link than the TL web.  The category of “Nurse” was the most succinct across flipped classroom (FC) and traditional lecture (TL) groups. There was little variance in subcategories of the individual maps, with a total of 6 subcategories identified from their discussions. “Recommendations” was a 	   42	  subcategory found in all the groups, as a required component of the case study activity. Associated links to this subcategory were most robust with three or more links identified by all groups. Of note, the TL groups averaged slightly more links to the “Recommendations” subcategory than the FC groups but only by 0.3 decimals. Table 4.6: Average number of links associated with the “Recommendations” subcategory  FC groups (n=11 groups) TL groups (n=4) Number of associated links to “Recommendation” subcategory 96 36 Average number of links per group 8.7 links 9 links   The “Assessment” subcategory emerged as groups in the FC and TL approaches identified what information they would want to know beyond what was given in the scenario. All the groups, except one in the FC cohort and one group from the TL format, indicated in some manner an assessment they would have performed as a public health nurse. Associated links to the “assessment” subcategory ranged from a single link to 17 links in the FC group and two links to 8 links in the TL groups. The “Education” subcategory encompasses what students determined as knowledge deficits in the case scenario parents and what they, as public health nurses, would teach them. Almost all groups in the FC approach and one group in the TL format indicated an education-related concept with links ranging from a single link to 13 associated connections seen in the FC groups and two links in the TL group. The less common subcategories of “Referrals” and “Resources” differed in that “Resources” were suggestions that pertained to educational material not provided by nursing directly. These included handouts or classes. “Referrals” comprised of people within the healthcare team that the students identified as needing to be part of care planning, such as dieticians, dentists or counselors. Two groups in the FC cohort and one group in the TL format identified “Resources” as a subcategory and links were only one to two 	   43	  connections. The “Referrals” subcategory was recognized by 7 of the 11 FC groups and half of the TL groups. Links seen in either group ranged from a single connection to three links. The final subcategory in the “Nurse” category was labeled “Approach to Care”. This was a unique subcategory as it addressed the demeanor or manner that nursing should take to provide care. These suggestions were more corresponding to values of nursing such as “compassion” and “non-judgment” than concrete physical actions. Figures 4.3 and 4.4 demonstrate differences seen between the FC and the TL groups with respect to the “Approach to Care” subcategory. The TL format initially has more subcategories but the FC composite illustrates more links per subcategory than the TL, which has 4 subcategories without links   Figure 4.3: TL composite map illustrating the subcategory “Approach to Care” 	   44	   Figure 4.4: FC composite map illustrating the subcategory “Approach to Care”  The assessment of TL and FC groups’ concept map morphology showed a stark contrast between the two groups. The category of “Nurse” is most telling of differences between TL and FC groups as they maintained the same subcategories of “Approach to care”, “Assessment”, “Education”, “Recommendations”, “Referrals” and “Resources”. While there were more groups in the FC cohort, the concept maps depicted identified concepts, not frequency. Similar concepts were combined and if multiple groups recognized the same concept, the map only represented it once. The intricacy of the FL web and the number of links per subcategory were clearly greater than that of the TL web. 	   45	   Figure 4.5: TL format’s composite concept map for the main category of “Nurse”  Figure 4.6: FC approach’s composite concept map illustrating the main category of “Nurse”  	   46	   A fourth category emerged and named as “discussion tangents”. When two or more students work together, it is anticipated that there will be some tangents or off-topic discussions. This was seen in all groups of this study. Like the three major categories noted above, the number of tangents and subcategories related to each tangent was a reflection of the amount of discussion devoted to social talk. These tangents, for the most part, were not grossly irrelevant to the case study activity and were often elicited by an aspect of the case study or the discussion activity. As with the other categories, the FC web was more intricate than the TL web. The two approaches had initially the same number of subcategories but off-topic connections were greater in the FC group than the TL group which may indicate some correlation between format and generation of information. Table 4.7: Average number of discussion tangent occasions & average number of links per occasion  Flipped Classroom (Flexible Learning) (n=11 groups) Traditional Lecture (n=4 groups) Average # of discussion tangent occasions 3  (total of 33 tangent occasions) 3 (total of 12 tangent occasions) Average # of links associated to a discussion tangent occasion 5.4 (total of 59 links in tangent occasions) 4.5 (total of 18 links associated with tangent occasions)     	   47	     Figure 4.7: Morphology of discussion tangent composite concept maps    	   48	  Links and Level of Thinking  Support to answer the second question, “What evidence supports the differences between flipped learning and a traditional lecture format with respect to higher order thinking?” was assessed through the links and appraisal of level of thinking of the concepts ascertained in the discussions.   Due to the nature of most groups’ discussions, links were often easy to identify as they were details or information mentioned by a student after the identification of the subcategory in the conversations. Similar to subcategories, links that were alike were combined and multiple identical links were only noted once in the concept map formation. Flipped classroom (FC) groups tended to have more links per subcategory than the Traditional Lecture (TL) groups. More links revealed that groups were able to determine how details, concepts and information would relate to a specific scenario and how this knowledge and relationships would inform care and address family needs.   Level of thinking was determined as either “rote information or lower level thinking” or “higher level thinking”. The researcher and second coder determined that information directly repeated from the case study or information that was taken from lecture slides or a textbook was determined as rote information or lower level thinking. Conclusions or statements that resulted from extrapolating or connecting information beyond the case scenario were considered higher order thinking. If a group queried missing information or made inferences based on understanding of the scenario, this was also considered higher-level thinking. Considering concept saliency or concepts that were particularly striking, the FC groups only slightly surpassed the TL groups in number when presenting details or information that was noteworthy. 	   49	  It should be noted that overall aside from discussion tangents, the concepts identified by the FC and TL groups were all valid and pertinent to the case study activity. Table 4.8: Number of salient concepts identified per group per category  FC groups TL groups Number of salient concepts   CHILD 14 10 PARENT 7 8 NURSE 17 6   Subcategories such as “allergies”, “birth complications” or “discipline issues” were identified by students in FC groups and the researcher and second coder noted that these concepts were unexpected yet spoke to an increase level of thinking. TL groups also noted surprising details or subcategories such as “cultural practices, “speech development” or “RN interactions impacting health”. When assessing the number of higher-level concepts between FC and TL groups, FC exhibited a slightly higher percentage of higher order concepts compared to the TL. Table 4.6 depicts the percentage of higher-level thinking concepts per major category for the FC and the TL groups. Table 4.9: Percentage of higher-level thinking concepts per major category per FC or TL group  FC groups TL groups Percentage of higher-level thinking concepts per category   CHILD  63%  (total of 106 concepts) 52% (total of 54 concepts) PARENT 62% (total of 47 concepts) 47% (total of 36 concepts) NURSE 56% (total of 192 concepts) 46% (total of 61 concepts)     	   50	  4.2 Summary  This chapter presented the findings from a content analysis evaluating differences between a flipped classroom approach and a traditional lecture format. Using concept maps, it was determined that FC groups had a more complex morphology of concepts and greater number of identified subcategories and links. By analyzing identified concepts, FC groups also exhibited a greater number of higher order level answers to the case study activity than their TL counterparts. Both FC and TL groups engaged in discussion tangents but FC groups participated in more occasions of social talk than the TL cohort.                	   51	  Chapter 5: Discussion  The findings from this study showed some significant differences between students exposed to the flipped classroom method and those exposed to a traditional lecture format. Differences were seen in the morphology of the concept maps as well as in the level of thinking. Morphology, or the visual appearance of a concept map, addresses the quantity of major categories, subcategories and links. If one concept map is more intricate and complex in appearance than another, one would suppose that the complex concept map experienced an enhanced learning opportunity compared to the other. Quality of identified subcategories and links is related to level of thinking. This characteristic considers the detail and the comprehensive information collected to form conclusions, statements or questions. Considering these attributes, the results from our study indicated that the flipped classroom format had more intricate concept maps and more complex categories and links that suggest a higher level of thinking.  5.1 Gross Similarities and Differences  When examining the concepts and categories identified in both the flipped classroom (FC) and traditional lecture (TL) groups, the terms identified were not grossly different between them and most groups identified subcategories that ran in similar veins. Both groups identified concepts and links that could be allocated into the three major categories of “Child”, “Parents” and “Nurse”. Given that both groups developed similar initial answers and categories to the case study, the learning setting, in this instance the FC format or the TL approach, did not dramatically influence outcomes (Trobec & Starcic, 2015). This is in agreement with other study findings that established comparability or equal effectiveness between flipped classrooms and traditional lectures with respect to course success (Cameron, 2013; Gagnon et al, 2013; 	   52	  Harrington et al, 2015; Mgutshini, 2013; Simpson &Richards, 2015; Trobec & Starcic, 2015). It should be noted that most of the early research on this phenomenon is quantitative using exam scores or grades and therefore cannot be directly likened to the results of this study. Appraisal of the implementation of a new pedagogical approach should include the evaluation of conceptual knowledge using diverse designs, tools and instrumentation such as clinical reasoning and critical thinking measures (Benner et al, 2010 as cited by Harrington et al, 2015).   The findings in this study appear in line with the results from previous studies which indicated that flipped classroom with web-based components are of value to nursing education and have the potential to enhance learning (Andrew et al, 2015; Arving et al, 2014; Geist et al, 2015; Lyons & Evans, 2013; Missildine et al, 2013; Post et al, 2015; Sung et al, 2008). Ostensibly, the two pedagogical formats had similar categories, subcategories or links. This suggests is that both pedagogical formats would have similar results on lower levels of thinking. However, the flipped classroom groups experienced enrichment to their learning that is unique to the approach as evidenced by the higher quality and sophisticated discussions.  A distinction emerged that the FC groups had a larger quantity of identified subcategories and thought more deeply of the case scenario to find more links and meaningful connections. The FC groups exhibited a greater percentage of higher-level thinking concepts within their total identified concepts compared to their TL counterparts. FC groups not only identified more details or concepts (quantity), more of the concepts they identified were considered of a higher order thinking level (quality). The pitfall of lecturing is that it compels students to memorize and regurgitate information when required, such as during exams, but are unable to retain it afterwards (Rotellar & Cain, 2016). The flipped classroom, on the other hand, allows time for 	   53	  reflection on information learned before needing to apply the content, increasing likelihood of retention of knowledge (Vaughan, 2014).  The number of salient concepts was not wholly different between FC and TL groups but FC groups identified slightly more striking details compared to the TL groups. All concepts identified by either group were valid compared to total links. This is a difficult characteristic to compare, as the distinctly noteworthy concepts identified by the FC group were not identified by the TL group and vice-versa. Prominent concepts are a signal of critical thinking and the result that there are similarities between the two groups denotes that pedagogical approach might not be influential in this aspect.  5.2 Discussion Tangents  Unanticipated findings were the discussion tangents that developed in both the FC and TL approaches. The existence of off-topic discussions during the small group activity was unexpected. Upon analysis, it was also noteworthy that the flipped classroom groups maintained similar density in their tangent discussions as their on-task scenario discussions. If we consider the constructivist perspective, creation of knowledge is a social process and learning is entrenched in the socializing experience (Etmer et al, 2011; Brown, 2014). Flipped classrooms allowed students more opportunities to share their own learning experiences and be exposed to the perspective of their peers in active discussion. These in turn helped them to build up their own frameworks and understanding of the concepts of the scenario (Hung & Lin, 2015). It was through collaboration and the non-substantive or off-topic elements of these discussions that students were able to compare ideas and arguments as a way to negotiate and form meaning (Paulus, 2009; Gan et al, 2008). Furthermore, as Hou et al. (2015) noted, off-topic discussions and social interactions are critical to building a better learning environment of knowledge 	   54	  sharing and positive group work. Subsequently, this leads to an increased learning performance and we could infer that by having more discussion tangents, the FC groups are promoting a more positive learning environment for knowledge sharing. This is evidenced by the increased complexity of their discussion of the case scenario. It was observed in the transcripts that there was on-topic and serious discussion following an off-topic tangent. This is in line with what Chen and Wang (2009) perceived in their research that on-task messages were followed immediately after brief social talk. The incorporation of social talk into a “serious” task, seen in small group activities, acted as a bridge between diverse group members to facilitate engagement in a joint purpose of learning by establishing common experiences which in turn lead to more comfortable membership (Chen & Wang, 2009).   If social talk can provide these benefits, we recognize through this finding that we cannot discount the value of these types of social interactions or deem them irrelevant and ineffective to learning. The use of small group activities in class as part of the flipped classroom model allows for peer-to-peer instruction or cooperative learning. While social talk and discussion tangents might be considered an unwanted effect, the activity benefits students by maximizing their own learning and each other’s learning through interaction and interdependence (Johnson et al, 2014 as cited by Brame & Biel 2015). Students are able to express their ideas, develop understanding and can hear the perspective of their peers more, versus a traditional lecture where students mainly listen to the instructor. Facilitation by the instructor can help foster a social environment while helping students stay on task with the activity but it was unclear through our transcription data if this occurred in our study.   	   55	  5.3 Limitations  Limitations of this study can be seen in two areas: study design and participants (or sample). Study Design   As a secondary analysis, this study is cognizant of the limitations that result from the data and design of a previous study. Given this inherent limitation, our analysis was dependent on the data of the previous study as presented. Due to the time difference between initial data collection and this study, we were unable to clarify and verify observations and understanding with participants, professors or transcribers. This further added to a limited perspective of context. Because we did not witness first-hand the group discussions, this content analysis was restricted to the words of the transcripts. Context and non-verbal cues were not noted in the previous design and cannot be determined for this study. Context and non-verbal cues would have enhanced the analysis that would have provided an additional perspective to understand the dynamics between students’ higher order thinking development and pedagogical approach.  Transcripts of recordings also posed a limitation because their accuracy and completeness could not be verified. Additionally, we were not able to access the audio recordings for all groups. Variances between transcribers in terms of style and format caused some confusion and difficulties identifying number students were participating in dialogue. Issues pertaining to technology and audio discernibility also limited transcription and subsequently our analysis, due to recording malfunctions and multiple voices speaking over each other which led to potentially significant data being lost in indecipherable noise.  By nature, content analysis and secondary analysis of a previous study look at data from a particular moment. In this case, the findings from this study are limited to the point in time of 	   56	  these group discussions and do not account for or observe longitudinal changes related to the development of critical thinking. The identification of critical thinking or higher-level thinking was done solely through verbal expression of thoughts. A student may have had additional higher-level concepts in mind but did not to verbally share with the group.   Additionally, the aim of a qualitative study is not generalizability. The findings of this study may only be generalizable to this specific course of community health nursing and not an overall program. Ideally, the findings here would provide some foundation or inference that can be applied and transferable to other courses, programs or educational situations.  In content analysis using concept maps there is, to a certain extent, an inherent degree of subjectivity. Coding and interpretation of transcripts and context were colored by our own experiences and biases. We attempted to reduce this by having a second coder and ensuring consensus was reached through audit trails, debate and discussions. Difficulties between the researcher and second coder were infrequent during the analysis. Any disagreements were addressed in person and involved discussion and justification of choices. Consensus was always reached. Reflexivity through journal keeping was also used to address personal difficulties such as preconceptions, judgments, bias, reflections and progress throughout the coding and analysis process (Polit & Beck, 2012).  The concept maps generated also highlight the limitations of analyzing group perceptions. Unlike evaluating students’ individual concept map that reflects their understanding of a given subject matter, we generated maps based on a group’s dialogue. The end results are maps that have a particular form the shape of “Isolated Mapping” (Hung & Lin, 2015).   Isolated maps typically have a main concept with subordinate concepts linked to it but lack cross linkages to each other. Hung and Lin (2015) indicated that the ideal morphology of 	   57	  concept maps is one of “Integrated Mapping” (p. 5). In an integrated shape map, subcategories and links are not only connected to the main concept but cross linkages to each other depict understanding that one link to a particular subcategory might also be related to an associated link or subcategory elsewhere on the map, indicating how aspects of a particular subject matter are interrelated and influential to each other.   Figure 5.1: Example of an isolated map  Figure 5.2: Example of an integrated map  Making those integrated connections for the concept maps of this study, however, would be a product of our own extrapolations and not a reflection of students’ expressed capacity to make links, as the researchers created the mappings. It should be noted that while the concept maps do not depict it, it is possible that students were able to recognize the interrelation of links and concepts concerning this scenario and their knowledge of nutrition and dental health. However our data and study were unable to be conclusive on this assumption. Participants and Sample  Reactivity such as the Hawthorne effect is a common limitation in human behavioural research. Students may distort their activity answers when they are aware of being observed and 	   58	  recorded (Polit & Beck, 2012). This was especially evident when group members expressed concern related to not knowing that they would be recorded or that they could be identified based on their voice.   Attrition is a common imperfection in all studies. Students may not participate, may not submit consent or may drop out of the course altogether, which consequently affect sample characteristics. It is unclear from the previous study the extent of attrition of participants, if any. According to the research manager of the previous study, there was one group discussion that was not recorded because one member did not consent to the audio recording (K. Carino, personal communication, June 30 2016). A student choosing to be absent from class or not participate in a group discussion, though present in class, would also be considered attrition and we were not able to ascertain if that occurred through our transcription data.  Confounding variables also pose a limitation to this study as they can diminish the strength of results as attributable to the methods of this study, specifically higher order thinking and the pedagogical approach used. Depth and variety of answers and learning may be attributable to other factors seen in precursor and concomitant variables. Precursor variables such as previous experience in other courses, general disposition of students, individual experiences or maturation may adulterate response to the pedagogical approach. Similarly, concomitant variables that arise during the experience of the discussion can influence outcomes. Variables such as course or activity specifics, group dynamics or situational events occurring during the discussion can all impact the results of the activity (Schwartz, 2013). Technology skills and online learning aptitude bear some influence on the results of this study. Students may have varying skill level with respect to technology, online or web-based applications, which can then 	   59	  affect use of course activity and knowledge acquisition. Students may also have varying styles of learning and pedagogical needs, which also influence learning or motivation to learn.                       	   60	  Chapter 6: Conclusion 6.1 Summary  Does a flipped classroom strategy enhance small group learning? With the use of concept maps as a tool to evaluate the level of thinking, the findings from this study suggested that the flipped classroom format did enhance small group learning overall compared to the traditional lecture format. This is supported through the results of content analysis and use of concept maps as a means to evaluate levels of learning. This analysis showed that the FC groups had more subcategories and links than the TL groups. The amount of subcategories indicates the FC groups touched on more aspects of the case scenario. The number of links indicated the FC groups made more connections and delved more deeply into the significance of the concepts.  What evidence supports the differences between the flipped classroom and a traditional lecture format with respect to higher-level thinking? Differences between the FC approach and the TL format can be seen in the quality of links and level of thinking found in these connections. The FC group not only had more links per category but the majority of those links were of a higher-level thinking. Level of thinking was determined by the information and knowledge stated by students. Rote or lower level thinking was identified as information stated verbatim from lecture notes or textbook or with little connection to the case scenario. Higher order thinking was when students were able to take information from the case scenario and knowledge from their preparatory work to make connections, conclusions and inferences about the care required for this case scenario family. The FC groups were able to articulate more clearly and more frequently higher order thinking as evidenced by their connections, queries and conclusions between the case scenario family and knowledge acquired from their flipped classroom models and preparatory work. 	   61	   The presence and amount of discussion tangents that occurred during both pedagogical approaches was unexpected. While both the TL and FC groups engaged in social talk, the FC group engaged in discussion tangents more often. This finding was considered significant and may be an indication of an additional relationship or effect between the pedagogical format and the sharing knowledge in small group work. 6.2 Implications for Future Practice Nursing Education  Flipped classrooms (FC) with web-based components have a real place in the gamut of nursing educational approaches. This study supports that flipped classrooms are just as effective if not slightly more effective at enhancing learning for students to develop higher order thinking. What this means for the future of nursing education extends across the entire spectrum of learning circumstances.  On a course and classroom level, we have seen the effects of flipped classrooms with web-based components on the development of higher-level thinking. The FC model offers students time to grasp knowledge at their own pace, reflect upon it and then be able to relate it in a meaningful way in class with their peers (Vaughan, 2014). As the findings of this study indicated, the increased number and depth of concepts demonstrates higher order thinking and an enhancement to learning that occurs when students are able to acquire knowledge and reflect upon it prior to class. The FC approach can be used for a single unit within a course or for an entire course in a semester. The use of FC as an adjunct pedagogical format or as an overall structure of a course can diminish student boredom, increase engagement and promote a collaborative learning environment (La Marca & Longo, 2017). Further research is required to determine elements of FC that would further enhance the benefits that increase higher order 	   62	  thinking such as web-based applications, faculty facilitation during in-class portions, activities or specific courses that are better suited for the FC approach (Etmer et al, 2011).   For professional programs (such as nursing) and higher education institutions, the FC format addresses two emerging trends: technology innovation in education and economic feasibility. The incorporation of web-based applications or technology components into the flipped classroom model and nursing courses not only elevate nursing programs to meet the contemporary climate of education and health care but also, through innovative use, can be a more attractive choice for prospective students (O’Flaherty and Phillips, 2015). From the perspective of education institutions, the FC model speaks to some outcomes associated with budgetary and fiscal constraints. Strained resources lead to large class sizes and decreased instructor to student interactions. By employing the use of flipped classrooms, students are able to seek immediate support, guidance and feedback from instructors that cannot be achieved in large lecture settings and can still stay within the boundaries of program funds (Kim et al, 2014). Class size should still be factored into course design when considering facilitation during in-class portions. Advocates and some research indicate that flipped classrooms can be used and provide benefit to any class size. However, more research is still required (Rotellar & Cain, 2016). Institutions and nursing programs must be cognizant of the initial increase in time and resources needed to implement a flipped classroom format. Over time the investment of time and resources to maintain the format, course or program diminishes (Bernard, 2015). Furthermore nursing programs and institutions must consider potential financial implications of mandated technology use to students (Critz et al, 2013). The timeless quandary that educational institutions and nursing programs face is whether such pedagogical choices are feasible and if they ultimately serve their purpose. Undoubtedly technology and its videos, simulations and online tools will 	   63	  continue to infiltrate all pedagogical approaches but it remains uncertain if and how their potential will be realized (Critz et al, 2013).  Ongoing professional development can be addressed through the use of flipped classroom approaches (Vaughan, 2014). The flipped classroom format offers an appropriate avenue for nursing continued education and for specialized skill certifications (Sung et al, 2008). If flipped classrooms offer enhanced benefits for students, similar advantages would be seen in registered nurses who are interested in developing new skills or who are maintaining the professional standard of continued education. This model could also be used in orientation of new nurses and newly hired nurses to specific areas. The flipped classroom model is not constrained to the formal education of nursing programs but can also be developed and utilized in occupational education (Sung et al, 2008). In addition, the construction of a flipped classroom activity or course would act as a professional development opportunity for nursing educators. It remains imperative to learn the skills and technology needed to develop the modules and activities to maximize learning (Vaughan, 2014). Nursing Research  Findings from this study not only align with previous research, but also emphasize a nascent understanding of the potential for flipped classrooms in nursing education. Further research is required to determine long-term effects of flipped classroom approaches on the development and continuation of critical thinking skills. Early research suggests that FC model could have a carryover effect, which would contribute to the success in future course work (Bernard, 2015). To date, however, there are no studies found that explore students’ higher order thinking over a period of time longer than a specific course. Nursing pedagogy would benefit 	   64	  from the evaluation of flipped classrooms on knowledge retention and progression of critical thinking skills over the period of an entire nursing program and into nursing practice.  Future research should also explore different methods, more rigorous designs and instrumentation for determining the effectiveness of the flipped classroom model (Bernard, 2015). The majority of studies found employed exam grades as a means of establishing value of flipped classrooms (O’Flaherty & Phillips, 2015). While exam scores are of importance, they are not the sole determinant of worth of a pedagogical approach. Other qualities such as critical thinking, engagement, satisfaction or experiences of students and faculty are of merit in assessing quality of the flipped classroom format for a particular course, student demographic or nursing program (Gilboy et al, 2015).  Additionally, research into characteristics of students and nursing courses that best suit the flipped classroom with web-based components would provide another perspective that would inform implementation and increase the odds of effectiveness. Previous research often points to this generation of students, the Millennials, as learners who, from a very young age, have been engaged in technology and are digital natives (Vaughan, 2014). The age range of Millennials is quite wide with individuals born between the years of 1982 and 2002. Those born in the earlier years of this generation identify more as the technological generation who learned in class (Post et al, 2015). This assessment could indicate why findings to date are more subdued as to the effectiveness of the flipped classroom model. Further research into learner characteristics and FC models involving technology and web-based applications would help educators and programs to adjust curriculum in an effort to meet the needs of current students while anticipating the needs of future generations (Vaughan, 2014). 	   65	   An area of interest that was not addressed in this study was the perspective of faculty and the implementation of flipped classrooms. Little is addressed in current literature regarding perception and experiences of faculty and the use of flipped classrooms in nursing education. Given that the format of FC with web-based components requires increased investment of time and resources, potential need for education, skills or qualifications of faculty and the adjustment of curriculum and content, it would be of significance to understand the perspective of faculty who must implement this pedagogical approach for the benefit of their students (Bernard, 2015). 6.3 Conclusion  The immediate intention of nursing education is to develop proficient and skilled nurses. But like all professions, disciplines and formal education, the loftier objective is to help nurture the evolution of an individual into a more critically thinking, self-directed, motivated and contributing member of society. The exceptional competencies, which serve nursing, such as the desire for continuing education, clinical reasoning, collaboration, adaptation to unfamiliar situations and participation in change are not achieved through passive learning in successive lectures (David &Patel, 1995). The flipped classroom model fosters in students an increased investment and responsibility in their own learning. FC formats, through flexibility, allow for students to pace learning to meet their own needs and allows time for reflection which would lead to more self-aware students (Post et al, 2015). Nursing education, therefore, must recognize the changing climate that technology has brought about through its interconnectedness with all aspects of daily life and incorporate pedagogical approaches that will challenge nursing students. The presence of technology in education is not sufficient but requires purposeful implementation related to course content and adapted so to provide the appropriate conditions for higher order thinking (Szabo & Swartz, 2011). The responsibility of nursing education goes beyond ensuring 	   66	  the mastery of profession-specific skills but more significantly it must provide pedagogies and an environment that promotes critical reflection, provokes questioning and scrutiny, encourages active participation and personally investment in learning and engages students in relating abstract academic content to concrete real-life situations of social significance (Kreber, 2014). It is in this atmosphere that students are transformed into nurses who are life-long learners with a deeper understanding of self, an identity rooted in a sense of responsible agency, commitment to others, and who ultimately are able to contribute to their communities for change and progression for the common good.                   	   67	  References  Abrami, P.C., Bernard, R.M., Borokhovski, E., Wade, A., Surkes, M.A., Tamim, R., Zhang D.  (2008). Instructional interventions affecting critical thinking skills and dispositions: A  stage 1 meta-analysis. Review of Educational Research, 78(4), 1102-1134.  doi:10.3102/0034654308326084   Andrew, L., Ewens, B., Maslin-Prothero, S. (2015). Enhancing the online learning experience  using virtual interactive classrooms. Australian Journal of Advanced Nursing, the, 32(4),  22-31.   Arksey, H. & O’Malley, L. (2005). Scoping Studies: Towards a Methodological Framework.  International Journal of Social Research Methodology, 8(1), 19-32.  doi:10.1080/1364557032000119616  Arving, C., Wadensten, B., Johansson, B. (2014). Registered Nurses’ Thoughts on Blended  Learning in a Postgraduate Course in Cancer care—Content Analyses of Web Surveys  and a Focus Group Interview. Journal of Cancer Education, 29(2), 278-283.  doi:10.1007/s13187-013-0594-6  Baxter, P., & Jack, S. (2008). Qualitative case study methodology: Study design and  implementation for novice researchers. The qualitative report, 13(4), 544-559.  Bernard, J.S. (2015). The Flipped Classroom: Fertile Ground for Nursing Education Research.  International Journal of Nursing Education Scholarship, 12(1), 99-109.   doi:10.1515/ijnes-2015-0005  Brame, C.J. &Biel, R. (2015). Setting up and facilitating group work: Using cooperative  learning groups effectively. Retrieved [March 22 2017] from  http://cft.vanderbilt.edu/guides-sub-pages/setting-up-and-facilitating-group-work-using- cooperative-learning-groups-effectively/.  Brown, L. (2014). Constructivst Learning Environments and Defining The Online Learning  Community. Journal on Scholl Educational Technology, 9 (4), 1-6  Cameron, N.G. (2013). Comparative descriptors of applicants and graduates of online and face- to-face master of science in nursing programs. Nursing Education Perspectives, 34(6),  372-376.   doi:10.5480/11-507  Chen, F., & Wang, T. C. (2009). Social conversation and effective discussion in online group  learning. Educational Technology Research and Development, 57(5), 587-612.  doi:10.1007/s11423-009-9121-1  	   68	  Creedy, D.K., Mitchell, M., Seaton-Sykes, P., Cooke, M., Patterson, E., Purcell, C., Weeks, P.  (2007). Evaluating a Web-Enhanced Bachelor of Nursing Curriculum: Perspectives of  Third Year Students. Journal of Nursing Education, 46 (10), 460-467.  Critz, C.M. & Knight, D. (2013). Using the flipped classroom in graduate nursing education.  Nurse Educator, 38(5), 210-213.  doi:10.1097/NNE.0b013e3182a0e56a  Daley, B., Morgan, S., Black, S.B. (2016). Concept maps in nursing education: A historical  literature review and research directions. Journal of Nursing Education, 55(11), 631-639.  doi:10.3928/01484834-20161011-05  Daley, B.J., Shaw, C.R., Balistrieri, T>, Glasenapp, K., Piacentine, L. (1999). Concept maps: A  strategy to teach and evaluate critical thinking. The Journal of Nursing Education, 38(1),  42-47.  David, T.J. & Patel, L. (1995). Adult learning theory, problem based learning, and paediatrics.  Archives of Disease in Childhood, 73(4), 357-363.  doi:10.1136/adc.73.4.357  Drisko, J. W. & Maschi, T.(2015). Qualitative Content Analysis in Drisko &Maschi (Eds)  Content Analysis. Oxford, NY: Oxford University Press & UPSO eCollections  (University Press Scholarship Online).   doi:10.1093/acprof:oso/9780190215491.001.0001  Du, S., Liu, Z., Liu,S., Yin, H., Xu, G., Zhang, H., Wang, A. (2013). Web-based distance  learning for nurse education: a systematic review. International Nursing Review, 60, 167- 177.  Elo, S., & Kyngäs, H. (2008). The qualitative content analysis process. Journal of Advanced  Nursing, 62(1), 107-115.   doi:10.1111/j.1365-2648.2007.04569  Ertmer, P.A., Sadaf, A., Ertmer, D.J. (2011). Student-content interactions in online courses: the  role of question prompts in facilitating higher-level engagement with course content.  Journal of Computing in Higher Education, 23(2), 157-186.       doi:10.1007/s12528-011-9047-6  Gagnon, M., Gagnon, J., Desmartis, M., & Njoya, M. (2013). The impact of blended teaching on  knowledge, satisfaction, and self-directed learning in nursing undergraduates: A  randomized, controlled trial. Nursing Education Perspectives, 34(6), 377-382.   doi:10.5480/10-459   	   69	  Gan, Z., Davison, C., & Hamp-Lyons, L. (2009). Topic negotiation in peer group oral  assessment situations: A conversation analytic approach. Applied Linguistics, 30(3), 315- 334.   doi:10.1093/applin/amn035  Geist, M.J., Larimore, D., Rawiszer, H., Al Sager, A.W. (2015). Flipped versus traditional  instruction and achievement in a baccalaureate nursing pharmacology course [research  brief]. Nursing Education Perspectives, 36(2), 114-115.  doi:10.5480/13-1292  Gerring, J. (2004). What is a case study and what is it good for?. American political science  review, 98(02), 341-354.  Gilboy, M.B., Heinerichs, S., Pazzaglia, G.(2015). Enhancing Student Engagement Using the  Flipped Classroom. Journal of Nurtirtion Education and Behaviour, 47(1), 109-114.  doi:10.1016/j.jneb.2014.08.008  Graneheim, U.H. & Lundman, B. (2004). Qualitative content analysis in nursing research:  concepts, procedures and measures to achieve trustworthiness. Nurse Education Today,  24, 105-112.  doi: 10.1016/j.nedt.2003.10.001  Gurupur, V.P., Jain, G.P., Rudraraju, R. (2015) Evaluating student learning using concept maps  and Markov chains.  Expert Systems with Applications, 42, 3306-3314.  doi:10.1016/j.eswa.2014.12.016  Halawi, L.A., McCarthy, R.V., Pires, S. (2009). An Evaluation of E-learning on the Basis of  Bloom’s Taxonomy: An Exploratory Study. Journal of Education for Business, 84(6),  374-380.  doi:10.3200/JOEB.84.6.374-380  Harrington, S. A., Bosch, M. V., Schoofs, N., Beel-Bates, C., & Anderson, K. (2015).  Quantitative outcomes for nursing students in a flipped classroom. Nursing Education  Perspectives, 36(3), 179-181.   doi:10.5480/13-1255  Hou, H., Wang, S., Lin, P., & Chang, K. (2015). Exploring the learner's knowledge construction  and cognitive patterns of different asynchronous platforms: Comparison of an online  discussion forum and facebook. Innovations in Education and Teaching International,  52(6), 610-620.   doi:10.1080/14703297.2013.847381  Hsu, L-L.& Hsieh, S-I.	  (2014). Factors affecting metacognition of undergraduate nursing  students in a blended learning environment. International Journal of Nursing Practice,  20(3), 233-241.   doi:10.1111/ijn.12131 	   70	   Hung, C-H. & Lin, C-Y. (2015). Using concept mapping to evaluate knowledge structure in  problem-based learning. BMC Medical Education, 15(1), 212, 1-9.  doi:10.1186/s12909-015-0496-x  Kaddoura, M., Van-Dyke, O., Yang, Q. (2016). Impact of a concept map teaching approach on  nursing students’ critical thinking skills: Concept maps for critical thinking. Nursing &  Health Sciences, 18(3), 350-354.   doi:10.1111/nhs.12277  Kantar, L. D. (2014). Assessment and instruction to promote higher order thinking in nursing  students. Nurse Education Today, 34(5), 789.   doi:10.1016/j.nedt.2013.08.013  Kim, M.K., Kim, S.M., Khera, O., Getman, J.(2014). The experience of three flipped classrooms  in an urban university: an exploration of design principles. Internet and Higher  Education, 22, 37-50.   doi:10.1016/j.iheduc.2014.04.003  Kreber, C. (2014). Rationalising the nature of ‘graduateness’ through philosophical accounts of  authenticity. Teaching in Higher Education, 19(1), 90-100.  doi:10.1080/13562517.2013.860114  Lahaie, U.D. (2008). Is Nursing Ready for WebQuests? Journal of Nursing Education, 47(12),  567-570.   doi:10.3928/01484834-20081201-05  Lahti, M., Hätönen, H., Välimäki, M. (2014). Impact of e-learning on nurses’ and student nurses  knowledge, skills, and satisfaction: A systematic review and meta-analysis. International  Journal of Nursing Studies, 51 (1), 136-149.   doi:10.1016/j.ijnurstu.2012.12.017  La Marca, A. & Longo, L. (2017). Addressing student motivation, self-regulation, and  engagement in flipped classroom to decrease boredom. International Journal of  Information and Education Technology, 7(3), 230-235.  doi:10.18178/ijiet.2017.7.3.871  Legg, T.J., Adelman, D., Mueller, D., Levitt, C. (2009). Constructivist Strategies in Online  Distance Education in Nursing. Journal of Nursing Education, 48 (2), 64-69  doi:10.3928/01484834-20090201-08  Lewis, A., & Smith, D. (1993). Defining higher order thinking. Theory into Practice, 32(3), 131- 137.  doi:10.1080/00405849309543588  	   71	  Lyons, T.& Evans, M.M. (2013). Blended learning to increase student satisfaction: An  exploratory study. Internet Reference Services Quarterly, 18(1), 43-53.  doi:10.1080/10875301.2013.800626  McCutcheon, K., Lohan, M., Traynor, M., Martin, D. (2014). A systematic review evaluating the  impact  of online or blended learnig vs. face-to-face learning of clinical skills in  undergraduate nurse education. Journal of Advanced Nursing, 71(2), 255-270.  doi:10.1111/jan.12509  Mgutshini, T. (2013). Online or not? A comparison of students’ experiences of an online and an  on-campus class. Curationis, 36(1), 1-7.   doi:10.4102/curationis.v36i1.73  Missildine, K., Fountain, R., Summers, L., Gosselin, K. (2013). Flipping the classroom to  improve student performance and satisfaction. Journal of Nursing Education, 52(10),  597-599.   doi:10.3928/01484834-20130919-03  Mok, C. K. F., Whitehill, T. L., & Dodd, B. J. (2014). Concept map analysis in the assessment of  speech-language pathology students' learning in a problem-based learning curriculum: A  longitudinal study. Clinical Linguistics & Phonetics, 28(1-2), 83-101.  doi:10.3109/02699206.2013.807880  O’Flaherty, J.A. & Laws, T.A. (2014). Nursing student's evaluation of a virtual classroom  experience in support of their learning bioscience. Nurse Education in Practice, 14(6),  654-659.  doi:10.1016/j.nepr.2014.07.004  O’Flaherty, J. & Phillips, C. (2015). The use of flipped classrooms in higher education: A  scoping review. Internet and Higher Education, 25, 85-95  doi:10.1016/j.iheduc.2015.02.002  Paulus, T. M. (2009). Online but off-topic: Negotiating common ground in small learning  groups. Instructional Science, 37(3), 227-245.   doi:10.1007/s11251-007-9042-5  Petit dit Dariel, O., Wharrad, H., Windle, R. (2012). Exploring the underlying factors influencing  e-learning adoption in nursing education. Journal of Advanced Nursing, 69(6), 1289- 1300.   edoi:10.1111/j.1365-2648.2012.06120.x  Polit, D.F. & Beck, C.T. (2012) Nursing Research: Generating and Assessing Evidence for  Nursing Practice (9th Edition). Philadelphia, PA: Wolters Kluwer Health/Lippincott  Williams & Wilkins  	   72	  Post, J.L., Deal, B., Hermanns, M. (2015). Implementation of a flipped classroom: Nursing  students’ perspectives. Journal of Nursing Education and Practice, 5(6), 25-30.  doi:10.5430/jnep.v5n6p25  Rosen, Y. & Tager, M. (2014). Making student thinking visible through a concept map in  computer-based assessment of critical thinking. Journal of Educational Computing  Research, 50(2), 249-270.   doi:10.2190/EC.50.2.f  Rotellar, C. & Cain, J. (2016). Research, perspectives, and recommendations on implementing  the flipped classroom. American Journal of Pharmaceutical Education, 80(2), 34, 1-9.  Ryan, G. W. & Bernard, H. R. (2003). Techniques to identify themes. Field methods, 15(1), 85- 109.  doi: 10.1177/1525822x02239569  See, S. & Conry, J.M. (2014). Flip My Class! A faculty development demonstration of a flipped- classroom. Currents in Pharmacy Teaching and Learning, 6, 585-588.  doi: 10.1016/j.eptl.2014.03.003  Senita, J. (2008). The use of concept maps to evaluate critical thinking in the clinical setting.  Teaching and Learning in Nursing, 3(1), 6-10.   doi:10.1016/j.teln.2007.08.002   Simpson, V., & Richards, E. (2015). Flipping the classroom to teach population health:  Increasing the  relevance. Nursing Education in Practice, 15(3), 162-167.  doi:10.1016/j.nepr.2014.12.001  Schwartz, M. (2013, March). Best Practices in Experiential Learning [Blog Post]. Retrieved from  http://www.ryerson.ca/lt/resources/newsletters/best_practices/pasttopics/mar2013/  Sullivan, W. M., Rosin, M. S., & Carnegie Foundation for the Advancement of Teaching.  (2008). A new agenda for higher education: Shaping a life of the mind for practice (1st  ed.). San Francisco, CA: Jossey-Bass.  Sung, Y.H., Kwon, I.G., Ryu, E. 2008). Blended learning on medication administration for new  nurses: Integration of e-learning and face-to-face instruction in the classroom. Nurse  Education Today, 28(8), 943-952.   doi:10.1016/j.nedt.2008.05.007  Szabo, Z & Schwartz, J. (2011). Learning methods for teacher education: the use of online  discussions to improve critical thinking. Technology, Pedagogy, and Education, 20(1),  79-94.  doi: 10.1080/1475939X.2010.534866  	   73	  Thompson, C.& Rebeschi, L.M. (1999). Critical thinking skills of baccalaureate nursing students   at program entry and exit. Nursing and Health Care Perspectives, 20(5), 248-252.  Trobec, I., Starcic, A.I. (2015). Developing nursing ethical competences online versus in the  traditional classroom. Nursing Ethics, 22(3), 352-366.  doi:10.1177/0969733014533241  Tucker, B. (2012). The flipped classroom: Online instruction at home frees class time for  learning. Education Next, 12(1), 82.  van Merriënboer, J. J. G., Kirschner, P.A., & Kester, L. (2003). Taking the load off a Learner’s  mind: Instructional design for complex learning. Educational Psychologist, 38(1), 5-13.  doi:10.1207/S15326985EP3801_2  Vaughan, M. (2014). Flipping the learning: An investigation into the use of the flipped  classroom model in an introductory teaching course. Education Research and  Perspectives, 41, 25-41.  Vygotsky, L. S. & Cole, M. (1978). Mind in society: The development of higher psychological  processes. Cambridge, MA: Harvard University Press.  West, D.C., Pomeroy, J.R., Park, J.K., Gerstenberger, E.A., Sandoval, J. (2000). Critical thinking  in graduate medical education: A role for concept mapping assessment? Jama, 284(9),  1105-1110.  doi:10.1001/jama.284.9.1105    	   74	  Appendix A: Literature Review Matrix   Literature Reviews   Author/Date/Title Purpose Sample Design/Methods   Findings Comments Bernard, J.S. /2015  “The Flipped Classroom: Fertile Ground for Nursing Education Research”	  Critique and synthesis of the current state of the art and science of the Flipped Classroom Model (FCM) in order to explicate a clearer path for implementation within the context of nursing education (p.1)	  47 articles	   LIT REVIEW  -Articles written from 2000-2014 (FC not used in higher education until 2000)  -39 research articles, 2 dissertations, 2 lit reviews, 4 theoretical papers  Data eval: Mostly USA (UK, Canada, South Africa, Kuwait), mostly in science/tech/econ/math. Only 8 studies used pre/post test in quasi or nonexp design. Small sample sizes  Data analysis: would need richer/more accurate ax of FCM to capture genuine learning 	  Definition: remains operationally ill-defined “There is no single model for implementing the flipped classroom approach and our review of the literature indicates that the approach is still in a stage of innovation (Love et al 2014 p.319) Themes: (a) Student performance w/ FCM: 24 studies measured FCM effectiveness (b) Student perception of FCM: linked to student performance (c) Faculty perception of FCM: Time investment, “techness”,  (d) Faculty concerns w/FCM: limit time to address essential content; replace qualified instructors Implications: lack rigorous design ∴ insufficient to confirm effectiveness  Comprehensive lit review  “can enhance” 	   75	      Author/Date/Title Purpose Sample Design/Methods Findings Comments Du et al /2013  “Web-based distance learning for nurse education: a systematic review” Systematic literature review of web based distance learning for nursing 9 RCTs  UK Taiwan Japan Thailand Spain Netherlands No time limiter English MeSH terms: Internet, distance learning, e-learning, online learning, computer, www, web, case-based learning, education, nursing, randomized controlled trial, random Peer-reviewed Snowball search 1 reviewer screened title/abstract 2nd reviewer separately read random sample of titles and abstracts. Then worked independently evaluating selected articles for quality critical appraisal then snowball search 3rd reviewer to resolve disagreements Themes: - knowledge:  ↑ knowledge post test (3) ∅ changes (2) ↑ at different times for control vs exp. (1) -skills performance: ↑skills (3) ↓ skills (1) -participant satisfaction: ↑ satisfaction (4) -self-efficacy ↑ SE (1) ∅ change (1) - others tech issues time tech literacy   Heterogeneity N  Small N  Unconventional article selection  Foundation for future research/highlights gaps in current understanding and quality/scope of present body of knowledge  “equivalence” 	   76	  Author/Date/Title Purpose Sample Design/Methods Findings Comments Lahti et al/2014  “Impact of e-learning on nurses’ and student nurses knowledge, skills, and satisfaction: a systematic review and meta-analysis” Review the impact of e-learning on nurses’ and nursing students’ knowledge skills and satisfaction related to e-learning 11 publications (European) between 2004-2010 Quantitative analysis  Criteria:  -RCT/CCT/pre&post control -Excluded simulation/mobile phones /mannequins/ non-English  Cochrane extraction tool  Independent extraction of data by one reviewer   Issues extracted: - purpose of trial - sample - details of the intervention - outcomes - measurements used  methodological quality assessed using Cochrane Handbook - only 11/177 studies reporting effectiveness of e-learning on participant’s knowledge level -review indicates that e-learning is not a superior learning method to traditional learning methods - need to explore more carefully content related to limitations of the use of e-learning…more relevant method to provide education on specific topics - no evidence for assumption and no stat diff between e-learning and traditional to improve skills--- need to integrate content into clinical -e-learning methods are still fairly new in field and may not be considered in existing educational interventions---existing education interventions cannot adequately respond to participants’ needs/experiences in clinical practice -not able to answer question due to missing stat info (r/t impact on learning satisfaction)  implications for practice: educational policymakers  different educational methods are available as means of educating nurses/nursing students  educators/researchers cannot treat web-based learning methods as a single entity—consider the needs of the students/context when deciding medium, media or other combinations  Strengths: systematic approach, large search area, broad inclusion  Limitations: Selective reporting, judgments made by review authors (bias), lack of numerical data = issues w/ validity of inclusion procedures, ?randomization (appropriately done), heterogeneous interventions, small N in studies, lack power calculations, overestimation of effects of experimental interventions  Need further studies to assess impact of e-learning in nursing education  “comparable”     	   77	  Author/Date/Title Purpose Sample Design/Methods Findings Comments McCutcheon et al/2014  “A systematic review evaluating the impact of online or blended learning vs. face-to-face learning of clinical skills in undergraduate nurse education” To determine whether the use of an online or blended learning paradigm has the potential to enhance the teaching of clinical skills in undergraduate nursing    19 papers (1995-2013) = 14 quant (13 had comparison group and 1 had observational with no comparison), 1 qual, 3 mixed methods, 1 integrative review  UK Turkey Thailand Norway Korea Australia USA Systematic review protocol of McColgan & Blackwood (2009)  Searched existing systematic reviews in DARE   MEDLINE, CINAHL, BREI, ERIC, AUEI  English publications and checked bibliographies from relevant studies  Inclusion: RCTs and QCTs, qual also included if detailed nursing students’ perceptions on impact or effect of online learning , systematic reviews of above topic also included. Pilot studies with powered N and effect   Exclusion: Pilot studies based on intervention refinement and small N   Participants: students in online learning modality at any stage in undergraduate training (excluded = postgrad and postRN), studies with only instructor/teacher experience were excluded, excluded if not RN students primarily  quality appraisal (blinded) Knowledge: 13 papers 7 reported sig results indicating ↑ level knowledge post online modality 2 reported ↑ level knowledge post f2f 5 reported no sig diff in knowledge between online/f2f Skill: 13 papers 6 reported sig results r/t students’ skill performance post online 1 reported sig diff w/ only one of the pre-surgical care skills and found all other skills no sig diff 1 reported no sig diff between groups @ 2 wk but sig diff at 8wk F/U 1 study reported that 1 group lack of adherence to performance standards than groups taught by lecture 6 studies = no sig diff Self-efficacy: 3 papers 1= no sig diff pre/post 1 study noted sig increase with SE post online video 1 qual noted that students found online stressful/anxiety majority found it increased confidence Satisfaction: 11 papers 1 study found students wanted didactic then wanted online  5 students indicated students had higher satisfaction with online 1 study reported that students found online learning interactive/exciting but felt disadvantaged 4= no sig diff in satisfaction 1 study reported no sig diff in time spent engaging in learning between f2f and online  limitations: 4 studies excluded due to language restriction, pilot/feasibility studies excluded, included studies regardless of quality  lack of available international evidence r/t blended learning  evidence available is lacking in both quantity and quality (? r/t lack of research or publication)  need for robust methodologically strong studies in implementation process to help close the gap in evidence base  “equivalence” 	   78	     2 JBI data extraction forms used  quant: weak quality, non-validated instruments and lacked detail of research tools/reliability/validity, limited generalizability  qual: met criteria for quality appraisal…integrated into overarching synthesis of results  mixed methods: lacked detail and depth of response (reliability issue)…weighted more toward quant with qual ad hoc  review: not resynthesized into review’s results  -1 study noted steep learning curve and time required to produce online product -1 study noted younger students perform better = need for educators to be aware of range of students’ learning capacity -online learning proved to have similar if not improved benefit to students’ clinical skill knowledge (based on 10/13 studies) -4 studies were poorly designed with imbalance of educational instruction between control/intervention groups therefore sig results can be d/t supplementary training (BIAS)---though can infer that supplementary training can help close theory/practice gap  -can infer from 13 studies that students obtained higher/similar level of clinical skill post online (may be d/t ability to repeat activities/review content at own pace)—can create lack of consistency between study arms (bias) -age, computer experience, learning style and attitude toward tech have been ID’ed in other studies as affecting the students’ level of engagement/satisfaction -can conclude based on this review that online appears to be at least as effective as traditional -blended learning vs traditional = limited evidence available to enable conclusive opinion -need to research future development of blended learning for clinical skills 	   79	  Journal Publications   Author/Date/Title Purpose Sample Design/Methods Findings Comments Andrew et al/2015  “Enhancing the online learning experience using virtual interactive classrooms” Development, design, implementation and evaluation of pilot project involving virtual interactive classrooms  “To enhance online learning in two nurse theory units through a pilot project to design, implement and evaluate the introduction of interactive classroom technology”             perceptions 144 nursing students = 130 undergraduates, 14 postgraduates  Australia Interactive tutorial classrooms  Surveys, descriptive cross sectional surveys (MC/open questions/free text)  Descriptive stats and thematic analysis through consensus 53% of UG/79% PG and 9% accessing recordings  demographics: high rep of “non-traditional” students…inequality as underrep of aboriginal and torres strait islanders  reasons for non-participation: family/employment commitments  majority actively participated in real time  apparent enjoyment of social interaction before and after weekly session = relevance in promoting learning and development of social engagement “of value”  	   80	         Author/Date/Title Purpose Sample Design/Methods Findings Comments Arving et al/ 2014  “Registered Nurses’ Thoughts on Blended Learning in a Postgraduate Course in Cancer care—Content Analyses of Web Surveys and a Focus Group Interview” To describe registered nurses thought on the blended learning format in a specialist nursing programme in cancer care                  perceptions 53 RNs in Uppsala Sweden  4 RNs (1 M 3 F) for group interview Descriptive study   Open ended questions in a web-based questionnaire and focus group  2 oncology courses  convenience sample for group interview  content analysis by Malterud % of RNs answered web questionnaire  themes of focus group: 1. facilitates learning: thoughts on blended learning/thoughts on interactions with teachers and other RNS/Thoughts on interactive tech used in course 2. Knowledge we need as nurses: thoughts on focusing on nursing/Thoughts on evidence based nursing education Positive thoughts with web based lectures/  interaction with other RNs and teachers needed improvement   content in seminars needed to focus more on nursing knowledge/practice and skills but should be evidence based not taught as stable facts/procedures  developed new perspectives through thinking and repeated encounters with concepts  Small N and only one focus group interview  “can be successful if…” 	   81	     Author/Date/Title Purpose Sample Design/Methods Findings Comments Cameron/2013  “Comparative Descriptors of Applicants and Graduates of Online and Face-to-Face Master of Science in Nursing Programs” To compare the online and F2F formats within a single nursing college’s MSN programs in terms of applicant differentiating characteristics graduate admission and graduating GPAs and family nursing practitioner certification first time examination pass rates.            knowledge F2F and Online MSN Applicant Analysis N=507  Admission/Graduation F2F and Online MSN Students  N=151  Tennessee SECONDARY ANALYSIS F2F reporting higher BSN GPAs, higher admission score  No diff between average BSN GPA of online and F2F. No sig diff in mean graduating MSN GPA between online and F2F  No sig association between program format and passing FNP cert exam  Expectations of student performance are equivalent  No diff in 1st time FNP cert exam pass rate = graduates from both formats were well prepared with the knowledge needed to pass the cert exam  Experience between applicants in the two programs is declining = advisor would feel confident recommending either format to prospective students   Limited data re: applicants/admitted students/graduates  Explore applicant/admitted/graduate preferences would enhance knowledge base   “of equivalence” 	   82	      Author/Date/Title Purpose Sample Design/Methods Findings Comments Gagnon  et al/ 2013  “The Impact of Blended Teaching on Knowledge, Satisfaction, and Self-Directed Learning in Nursing Undergraduates: A Randomized Controlled Trial”  To assess the effectiveness of a blended teaching intervention using internet-based tutorials coupled with traditional lectures in an introduction to research undergraduate nursing course               Knowledge Satisfaction Self-direction N 102 No sig diff in demographics between groups  Canada Two-group randomized controlled design with participants recruited from a population of first year nursing undergrads (Quebec)  Intervention group: blended instruction with internet-based tutorials and inclass sessions  Control group: f2f classroom teaching  Satisfaction and SDLRSNE (self-directed learning readiness scale for nursing education) scales used   Knowledge: no diff in global results on both exams…interaction effect was seen between the students’ level of motivation and the teaching method. Unmotivated students in IG performed sig better than unmotivated students in CG -A sig diff found between groups in first exam at mid-sem (IG performed sig better than CG)---no diff in second exam  Satisfaction: no sig diff between groups re: course satisfaction (CG slightly more satisfied but not sig) Motivation was only variable with sig + effect  SDLR: no diff in SDLR between groups Adjusting for motivation = sig diff in SDLR score found post intervention (depended on SDLR score at baseline) -Low SDLR baseline= CG improved sig more than IG -Med SDLR baseline = CG sig improvement than IG -High SDLR baseline= IG improved sig vs CG  Teaching model does  not have direct impact on knowledge acquisition, satisfaction and SDLR -Motivation and teaching method had interaction effect on knowledge acquisition -Motivation is positively associated with satisfaction -Less motivated students might benefit from e-learning although weak methods to determine this (also contrary to literature) -Teaching method also has no direct association with students’ SDLR score -Higher SDLR score sig improved score in IG whereas lower SDLR score at baseline sig improved in CG Students given option to switch groups???   “comparable” 	   83	  Author/Date/Title Purpose Sample Design/Methods Findings Comments Geist et al/2015  “Flipped Versus Traditional Instruction and Achievement in a Baccalaureate Nursing Pharmacology Course” To gather scientific data on the effects of the flipped classroom on knowledge acquisition based on one instructor implementing the flipped classroom model in a nursing pharmacology course               knowledge Tennessee  Two cohorts with equivalent admission standards  Control= 40 students w/ ave GPA 3.8 with midcurricular Health Education Services Inc (HESI) exam score of 965  Exp group= 46 students w/ ave GPA 3.7, HESI 911 Pretest-posttest non-equivalent control group quasi experimental quantitative design  “Is there a significant difference in content knowledge gained when comparing methodology of traditional lecture to the flipped classroom?”  100% participation, course content and instructor were the same for both cohorts, identical unit tests and final exam  control taught traditional methods, exp had flipped classroom plus HESI pre & online questionnaire post for students’ perceptions of pedagogical approach no missing data, miscodes or outliers  covariate was ↑ sig for each of 4 D variables   f  value for method (traditional/flipped) for unit tests = ↑ sig  1st test—diff in method accounted for 52% of variance in performance F[1.86]= 90.5, p=0.000 η= 0.52  2nd-- ↑ sig diff in method accounted for 15% of variance F[1.86]= 14.38, p=0.000 η=0.15  3rd-- ↑ sig showed that diff in method accounted for 34% of variance in student performance f[1.86]= 43.59, p=0.000 η=0.34  Variance in performance on 3 unit test was highly associated with teaching method used  Final exam= not sig diff F[1.86]= 1.95, p=0.167 η=0.02 FC treatment performed better in 3 unit tests during course even no diff in final exam  Future research: other nursing courses, LT gains in content knowledge  Repeat study larger scale with different populations (associate degree, private uni, larger state uni, community college etc)  Quantitative data for FC  Foundation for future study  “+ for cont. use”   	   84	      Author/Date/Title Purpose Sample Design/Methods Findings Comments Harrington et al/2015  “Quantitative Outcomes for Nursing students in a Flipped Classroom”  To objectively compare learning outcomes of two pedagogies as measured quantitatively through exam questions, quiz scores, and course grades.                 knowledge Convenience sample of 82 students (undergrad)  2nd sem of BSN program   Midwest USA (Michigan) Students met at same time/day of week for same amount of time in different classrooms. All took identical 3 exams/24 quizzes/written paper  Data analyzed by SAS software  Descriptive/inferential stats to compare scores between groups  Method of equivalence intervals used to demo similarities  MANCOVA 3% diff on ave between students in traditional classroom and those in flipped environment (meaningful as increase of 3 percent raises a student to next letter grade in nursing course)  no stat diff between groups in overall scores (MANCOVA) no stat diff in exam scores specific to knowledge and application (Wilkes lambda)  no sig diff between mean course grades after adjusting for prereq (ANCOVA)  did not demonstrate stat sig differences between pedagogies…equivalence interval results imply equal effectiveness for student learning  conceptual knowledge should be evaluated using novel tools  needed: improvements and accuracies in critical thinking assessments and problem-solving teaching methodologies Small N therefore lack generalizability  Academically homogenous  Faculty inexperience?  Outcomes assessed using traditional measurements (test scores)…?paradigm shift for longitudinal assessment looking beyond test scores (clinical reasoning?) 	   85	    Author/Date/Title Purpose Sample Design/Methods Findings Comments Hsu LL & Hsieh, SI/2014  “Factors affecting metacognition of undergraduate nursing students in a blended learning environment” To examine the influence of demographic, learning involvement and learning performance variables on metacognition of undergraduate nursing students in a blended learning environment                Analytical ability(?) Attitudes Disposition Purposive sampling  99 female students from two ethics classes in 2009  a priori G power done (minimum N=70)  Taiwan Quantitative study using SPSS  3 ethical scenarios used and students exchanged views/ideas via online chat  Instruments:  CAAS for attitudes towards ethical dilemma analysis  CASES assessed ability to analyze ethical dilemma  BLSS to determine perception towards online learning (designed thru pilot project by authors)  MS for metacognitive abilities Simple Linear regression = sig association between frequency of online chats (p=0.001), attitude towards case analysis (p<0.001), self-evaluation of case analysis (p=0.006 or p<0.001), and level of satisfaction w/ blended learning and metacognitive qualities among sample  ↑CASES = ↑ MS   Multi linear regression= stat sig r/t predictors of age, freq of online chat, attitude towards case, self-eval, satisfaction  Adjusting for age/attitude= sig independent predictors of total metacognition (1 unit ↑ in freq online chat or BLSS= 0.31-0.35 ↑ in MS) (48% of variance in metacognition r2= 0.48, adjusted r2= 0.44) Positive correlation between sample freq of online chat and metacognitive qualities (p<0.02)  CASES might be indicator for metacognitive power as those who are good at case analysis registered ↑ scores in CASES  ↑ satisfaction w/ online resources used in blended learning experience  Taiwanese students less outspoken and less inclined to actively participate in class discussions and would be more comfortable expressing online  Limitations: same college (characteristics not representative of population) lack of objective assessment of performance to compare to self-assessment 	   86	      Author/Date/Title Purpose Sample Design/Methods Findings Comments Lyons &Evans/2013  “Blended Learning to Increase Student Satisfaction: An Exploratory Study”  To determine if online discussions improve the learning experience  EXPLORATORY PILOT STUDY                    Satisfaction Two semesters consented:  20 in sem 1  26 in sem 2   Northeastern USA (Pennsylvania) Blended teaching approach: incorporated online discussions as a supplement to classroom experience  4-hour weekly class for 6 weeks and participate in online discussions outside class time.  Discussions were compulsory and part of grade for 1 sem  Mixed methods: quantitative research with qualitative content analysis  Students indicated positive outcome of being connected to librarian  Second semester at 63% disagreed with value of discussion boards  Disparity in number of postings between 2 semesters  Increased demand for more online interactions: students felt distant from activity  Increased perceived satisfaction with course material and deeper understanding of course literature Self-selection bias?  Study lacks info on design/methods  Small N  Different requirements/expectations per sem  Lack control for covariates/confounding variables   “potential value” 	   87	    Author/Date/Title Purpose Sample Design/Methods Findings Comments Mgutshini/2013  “Online or not? A comparison of students’ experiences of an online and an on-campus class” To describe the comparative experience of online and campus-based students who took the same undergraduate nursing course   To present insider insights with respect to how online and campus based modalities compared on a range of criteria including content mastery, attrition and student satisfaction          Knowledge Satisfaction Perceptions N=53 (23 campus based, 30 online)   Those that did not participate did not appear to differ academically from those who completed the course  South Africa Mixed Methods One instructor taught both modalities  Krippendorf’s content analysis used  Comparative study using 2 data collection questionnaires of 9 items related to performance  Satisfaction questionnaire (SIR-II) Comparable academic outcome Campus based were younger versus online (traditional students from highschool tend towards campus based), gender similar representation -3 exams computer based timed/proctored. Online had higher scores (87 to 81) -Online had superior running GPA compared to campus based upon entering course -Online group reported spending 24 % more time on course-related study per week -Campus based students reported more consistently perceived proficiency compared to online group -Online students felt less confident about content mastery despite scoring better in exams  SIR-II:  - appropriateness of taught material: varied report of appropriateness of course material -clarity: varied response for clarity -both appropriate and clarity focused on psychiatric terminology -level of tutorial support: campus based students reported more + in that they felt the option of accessing tutor before/during/after each session -level of engagement: campus discussion seen as voluntary online felt compelled to contribute as it was assessed and part of grade -overall learning experience: campus was individual eval of instructor and relationship with class. Online more multifactorial assessment of instructor and mode of teaching -online had greater satisfaction with learning experience -within online class, each student ahs equal opportunity to express viewpoint and be heard versus campus -online out performed campus in formative and summative exams even if reported less confident Lacks transferrable relevance   “comparable” 	   88	   Author/Date/Title Purpose Sample Design/Methods Findings Comments Missildine K et al/ 2013  “Flipping the Classroom to Improve Student Performance and Satisfaction” to determine the effects of a flipped classroom and innovative learning activities on academic success and the satisfaction of nursing students          Knowledge Satisfaction Convenience sample  589 participants (majority were White, female, mean age 24.32 SD 6.71) over 3 semester  USA (Texas) Quant: Quasi-experimental design (non-equivalent groups?)  Adult health courses with 3 different options:  Lecture only (LO)- by faculty and via interactive TV Lecture plus lecture capture back up (LLC)- live lecture and prior captured Lecture capture plus innovation (LCI)- flipped classroom  16 point questionnaire on 4 point Likert scale – ↑ scores = greater satisfaction Cronbach’s alpha = 0.98 SPSS, level of sig p=0.05 (two tailed)   ANOVA: sig diff according to method of teaching. F(2.586)= 10.69, p<0.001,ω2= 0.032  Exam scores were higher in LCI group than LLC group and LO group  Satisfaction survey completed by 445 students was faculty developed (lacked validity)—Kruskal Wallis test for LCI compared to LLC and LO x2(n=445_= 66.10 p<0.001 which indicates difference in satisfaction across the 3 delivery types  LCI students were significantly less satisfied than LO group  LCI students had significantly less satisfied compared to LLC group  No sig results between LO and LLC  LCI required more work and did not perceive value of interactive learning Limitations:  Infrastructure: classroom availability, limited access to high speed internet  Student perception of loss of supportive social system—needed structured opportunity for questions and periodic assessment for earlier problem identification and more effective intervention   “can improve learning” 	   89	   Author/Date/Title Purpose Sample Design/Methods Findings Comments O’Flaherty &Laws/2014  “Nursing student’s evaluation of virtual classroom experience in support of their learning Bioscience”   to report on the integration of virtual classrooms for external (off campus/distance) students in anatomy & physiology course   to explore the external students’ utilization and perceived benefits from participating in a web-based virtual classroom and provided a comparison of scholastic outcomes using course grades           Knowledge Perceptions Satisfaction 1st year BSN students   N=101 completed web-based questionnaire  N=89/101 responded to open-ended comments  Australia Mixed Methods Students attended pre-sem trial runs to familiarize with virtual classroom  Surveys of open and closed end questions (anonymous) –2 points of collection (before/after)  Engagement was measured by comparison of their final practical exam to previous year’s course offerings of cohorts who did not use web-based intervention Reported increase motivation  Share meaningful knowledge/experiences in real time without being isolated from situations  Mutual collaboration  Good access to instructors  Increased satisfaction  External students performed on ave 33% higher than internal students (for exams). Also fewer non-attendants (“disposition”)  Can miss out on real-time collaborative learning opportunities availed in f2f counterpart  	   90	  Author/Date/Title Purpose Sample Design/Methods Findings Comments Post JL et al/2015  “Implementation of a flipped classroom: Nursing students’ perspectives” “to understand the lived experience of students participating in the flipped classroom”                      Experiences/ Perceptions 18 participants (15 female, 3 male, mean age 28.7 years)  Convenience sample  Southern USA (Texas) Qualitative: Descriptive phenomenological inquiry  3 focus group interviews  Data analysis using Giorgi’s approach  Triangulation (data/investigator) 4 themes: 1) Frustration: unable to ask questions, new system 2) Disconnect: ↓ faculty interaction, time delays re: questions answered 3) Depersonalization: did not allow for different learning styles, unable to benefit from others’ questions 4) Time: time management, slow internet/computer limitations, prof talks too fast, monotone, recording issues, ↑ workload “technological generation who learned in class” benefits: repetition, flexibility, re-listen to lecture, go at own pace, reinforcement of ideas  requires more discipline than other online classes challenges: lack of option (had to take this format of class), financial challenges (r/t upgrade software etc), felt like participating in two different classes (online and in class), impersonal video lecture, increased work outside of class -Students did not embrace this strategy (↑ anxiety/discomfort to changed format), lack of prep/lack of choice  Sample N  May not be representative of whole population  Inaccuracies in self-reporting (subjective and recall bias)   “can enhance learning”  	   91	       Author/Date/Title Purpose Sample Design/Methods Findings Comments Simpson & Richards/2015  “Flipping the classroom to teach population health: Increasing the relevance”     To discuss the rationale for course revision, describe the flipped classroom design and to provide preliminary evaluation of this revised course              Satisfaction  N= 157 2nd/ 3rd year nursing students  Midwest USA (Indiana) Quantitative: Descriptive and exploratory  Voluntary online university sponsored course eval and inclass paper survey specific to flipped classroom design  NOT MANDATORY therefore more completed inclass survey than online  Descriptive stats for summary  Paired t-test to compare courses using university sponsored evals No stat sig diff in university course evals between traditional versus flipped  Students in FC rated course more + than traditional class  FC felt there was increased practical application of subject matter and greater use of multiple methods to involve their learning  Majority of students indicated the new course design allowed them greater flexibility and ability to control the pace of their learning  Increased feelings of being more responsible for their own learning  Students who prepared ahead of time were more engaged Small N  N were typical traditional students from one geographic area therefore may not represent all undergrads  Rethink how learners learn and teachers teach  Active learning strategies can help to foster the development of intentional learners who are able to pull together info from multiple sources to support decisions  Eval of level of content retention and critical application would determine if such design increased the level of student understanding and ability to critically think through issues 	   92	  Author/Date/Title Purpose Sample Design/Methods Findings Comments  Sung, YH et al/ 2008  “Blended learning on medication administration for new nurses; Integration of e-learning and face-to-face instruction in the classroom” To determine the effects of a medication education program utilizing blended learning with e-learning on new nurses’ knowledge of medication, self-efficacy of medication administration, and satisfaction with the learning program          Knowledge Self-efficacy Satisfaction 50 RNs (26 blended learning ,24 traditional)  fairly homogenous sample  Korea Quantitative Quasi-experimental study with non-equivalent groups Traditional classroom= control Blended with e-learning= experimental  Knowledge/self-efficacy measured pre- course. Face-to-face instruction given (both groups) for 2 wks and worked in field for 4-8 wks. Experimental group had access to web-based e-learning program  Post-course, a post-test done for knowledge, self-efficacy and satisfaction plus preceptor evals  Self-efficacy measured again 6 months later  Knowledge: 40 questions r/t basic knowledge Self-efficacy: 100 point scale consisting of 23 questions (likert scale) Med-admin: 21 questions 5 point scale Satisfaction: survey questions (8 questions)  SPSS data analysis Knowledge: sig ↑ in experimental group (t= 6.284, p= 0.000)  Time effect was significant in self-efficacy of med admin (F= 124.944, p=0.000) but a group x time interaction (F=2.507, p= 0.111) and group main effect (F=0.161 p=0.693) were not significant  Knowledge increased 14.3 points higher in exp group  No sig diff between groups r/t self-efficacy of med admin, no sig diff r/t med admin ability  Satisfaction was similar in both groups but substantiality of educational content was higher in exp group  Reduced lecture time d/t e-learning can be helpful to strengthen practical education on med admin (↓ human resources, ↑ cost effectiveness) Controlled for computer usage abilities  Cannot control for other variables affecting self-efficacy at the 6 month mark…and other extraneous variables such as sampling biases, length of education program, difference in instruction    “can enhance learning” 	   93	    Author/Date/Title Purpose Sample Design/Methods Findings Comments Trobec&Starcic/ 2015  “Developing nursing ethical competences online versus in the traditional classroom” To investigate whether students have good achievement in online learning and whether there are differences between students’ competences learning online and in traditional classroom           Knowledge Perception N=115 1st year students (Case C)  N= 40 (Case A)  N= 56 (Case B)   Slovenia    Mixed methods  Quasi experimental  Multiple case study for investigating 3 separate groups of students   Theoretical part done as lecture for EG and CG  Random assignment to tutorials where role play/discussion of scenarios was conducted in small groups (half of students in each section –groups of 5-6 completed tutorials in trad and other half in online classroom)  Asynchronous and synchronous methods used but synchronous used less frequently  Group C had no experience with online and group A/B were engaged in online learning prior to experiment Case A:  Mann-Whitney shows no stat sig diff in test scores between EG and CG Case b: Mann-Whitney shows no stat sig diff in marks between EG and CG Case C: Stat sig diff in marks between EG and CG. Half the ave values of marks are higher in the EG and half in control but the diff are not stat sig in this case …therefore no absolute advantage for any group  Content analysis themes: - collaboration/team work -communication/ interpersonal relations -autonomous decision making -motivation and approach to learning - teacher support for learning -the learning setting does not essentially influence the learning outcomes and there is no diff in learning outcomes between learning ethics online or in trad. Setting -students can achieve approximately the same results learning online and traditionally -no diff in learning outcomes between online setting and trad -cannot disregard the fact that working with people is crucial to nursing which requires nurses to demonstrate communicative and interpersonal competences that are best acquired in an environment that enables f2f contact in small group SO COMPLICATED!!!  Subjective nature of qualitative analysis/interpretations        “equivalence” 	   94	  Appendix B: Composite Concept Maps  TRADITIONAL LECTURESCHILDanxietyexertingcontrolgrowth20th %ilefrom 34thnutrientsFedeficiencydentalcaries/decaylots ofdairydeterringhungertension= acceptable?2x ave.intakeresearchr/t soy &childrenfortified =↑ supplements& minerals pickyeaterwhypressuresolidspoorexperiencenot excitedmouthteethdev.jaw muscledev.?unableto chewability toeatplay w/handsfingerfoodfeedinghimselfbottleuseknows if∅ eat =candymouthcoordinationdrinkingfrom cupdevelopmentfine motorskillssick/tired soreteethaffectingappetiteaversionto texturesoy milkwt for agestill onformulamust beoff by 1 yrnot an issuealternativeto milktoo full c liquidsunfortified =badspeechconversationsexpress feelingsto parentssleep thrunightstomachpainsincorrect infor/t %ile?leadingtothereforesensingsuchasr/tabletohave&perhapstherfore??but?determinedsharingr/tr/tsuchas?suchasleadstoleadstothereforethereforeleadstor/twhichiswhichisbutif thensuchassuchasthereforecan?considerif??rote informationor lower levelthinkinghigher levelthinking	   95	   	   96	   	   97	   	   98	   	   99	   	   100	   	   101	  	   102	  Appendix C: Individual Groups’ Subcategories and Links 	  	    Flexible Learning Traditional Lecture April 2015 October 2015 December 2015 May 2015 Group 1 2 3 4 5 1 2 3 1 2 3 1 2 3 4  Topics pertaining to: CHILD ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓  Subcategory: growth ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓  ✓ ✓ ✓ ✓ ✓ ✓ Associated links 1 3 5 1 2 4 3 2  3 3 1 2 1 2  Subcategory: weight       ✓  ✓       Associated links       1          Subcategory: bottle use ✓   ✓  ✓ ✓  ✓ ✓ ✓  ✓ ✓ ✓ Associated links 1      1  1  2      Subcategory: eating behaviours ✓            ✓   Associated links 8            3    Subcategory: try new things        ✓        Associated links                 Subcategory: nutrients ✓  ✓ ✓ ✓    ✓ ✓ ✓ ✓    Associated links 10  8 1 3    3 3 3 1     Subcategory: milestones  ✓       ✓ ✓      Associated links  5        4       Subcategory: fine motor dev         ✓       Associated links         2        Subcategory: speech dev               ✓ Associated links               2 	   103	      Flexible Learning Traditional Lecture April 2015 October 2015 December 2015 May 2015 Group 1 2 3 4 5 1 2 3 1 2 3 1 2 3 4 Topics pertaining to: CHILD (con’t)  Subcategory: soy          ✓    ✓ ✓ Associated links          1    1 2  Subcategory: unfortified               ✓ Associated links               1  Subcategory: liquids         ✓      ✓ Associated links                                Subcategory: formula         ✓ ✓    ✓  Associated links         4 2    1                  Subcategory: dental health  ✓  ✓ ✓ ✓  ✓ ✓ ✓  ✓  ✓  Associated links  2    3  4 3   1  3   Subcategory: digestion health  ✓   ✓ ✓ ✓  ✓      ✓ Associated links  4     2  2                       Subcategory: food  ✓ ✓   ✓   ✓       Associated links  8 1   6   1                       Subcategory: milk   ✓  ✓  ✓    ✓  ✓   Associated links   2  6  2      3    Subcategory: night routine   ✓    ✓  ✓      ✓ Associated links   3    1          Subcategory: picky eater    ✓ ✓   ✓     ✓ ✓  Associated links     1        2 1  	   104	  	       Flexible Learning Traditional Lecture April 2015 October 2015 December 2015 May 2015 Group 1 2 3 4 5 1 2 3 1 2 3 1 2 3 4 Topics pertaining to: CHILD (con’t) Subcategory: turned off to eating        ✓        Associated links                 Subcategory: self-regulating        ✓        Associated links                 Subcategory: eat when hungry     ✓           Associated links                 Subcategory: allergies     ✓           Associated links                 Subcategory: attention-seeking     ✓           Associated links                 Subcategory: anxiety            ✓    Associated links            1     Subcategory: dairy        ✓    ✓    Associated links        8    1     Subcategory: tension            ✓    Associated links                 Subcategory: candy             ✓   Associated links                 Subcategory: mouth             ✓   Associated links             4    Subcategory: solids          ✓   ✓   Associated links             3    Subcategory: sick/tired        ✓      ✓  Associated links              2  	   105	            Flexible Learning Traditional Lecture April 2015 October 2015 December 2015 May 2015 Group 1 2 3 4 5 1 2 3 1 2 3 1 2 3 4 Topics pertaining to: CHILD (con’t) Subcategory: ability              ✓  Associated links              5   Subcategory: stomach size      ✓          Associated links                 Subcategory: fullness       ✓         Associated links                 Subcategory: comfort measures         ✓       Associated links                 Subcategory: negative psychology         ✓       Associated links                 Subcategory: not getting enough to eat         ✓       Associated links         1        Subcategory: will take on new baby’s habits          ✓      Associated links                 	   106	       Flexible Learning Traditional Lecture April 2015 October 2015 December 2015 May 2015 Group 1 2 3 4 5 1 2 3 1 2 3 1 2 3 4  Topics pertaining to: PARENTS ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓  Subcategory: support ✓ ✓ ✓  ✓ ✓   ✓ ✓  ✓ ✓ ✓  Associated links 1 2 1  1    1   2 2 2   Subcategory: child ✓ ✓              Associated links 1 4               Subcategory: family unit dynamics  ✓          ✓  ✓ ✓ Associated links  4            1   Subcategory: concern       ✓ ✓        Associated links       1          Subcategory: genetics       ✓ ✓        Associated links       1          Subcategory: baby #2   ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓  ✓ ✓ ✓ Associated links         2 1   1 1   Subcategory: bottle vs BF   ✓             Associated links                 Subcategory: approach   ✓             Associated links   3              Subcategory: birth      ✓          Associated links      2           Subcategory: coping   ✓             Associated links   1              Subcategory: stress              ✓  Associated links              1   Subcategory: anxiety    ✓            Associated links                	   107	       Flexible Learning Traditional Lecture April 2015 October 2015 December 2015 May 2015 Group 1 2 3 4 5 1 2 3 1 2 3 1 2 3 4  Topics pertaining to: PARENTS (con’t) Subcategory: exhaustion      ✓  ✓        Associated links                 Subcategory: frustration     ✓     ✓      Associated links     1     1       Subcategory: bonding     ✓           Associated links     1            Subcategory: pressure     ✓ ✓  ✓  ✓ ✓ ✓ ✓ ✓ ✓ Associated links      1      1   1  Subcategory: cereal             ✓   Associated links                 Subcategory: cultural            ✓    Associated links                 Subcategory: husband      ✓   ✓  ✓ ✓   ✓ Associated links            1     Subcategory: practical reasons            ✓    Associated links            1     Subcategory: prenatal nutrition            ✓ ✓  ✓ Associated links            2     Subcategory: discouraging solids            ✓    Associated links                 Subcategory: team effort             ✓   Associated links             3    Subcategory: breastfeeding (BF)              ✓  Associated links              1  	   108	  	       Flexible Learning Traditional Lecture April 2015 October 2015 December 2015 May 2015 Group 1 2 3 4 5 1 2 3 1 2 3 1 2 3 4 Topics pertaining to: PARENTS (con’t) Subcategory: formula              ✓  Associated links              1   Subcategory: Socio-Economic (SES)              ✓  Associated links                 Subcategory: distraction                ✓ Associated links               1  Subcategory: temperament               ✓ Associated links               1  Subcategory: time management      ✓          Associated links                 Subcategory: sit down meals      ✓          Associated links                 Subcategory: when giving things        ✓        Associated links                 Subcategory: homemade soy drink         ✓       Associated links                 Subcategory: “keyed up” parents          ✓      Associated links                 Subcategory: lack of knowledge          ✓      Associated links          2       Subcategory: overwhelmed           ✓     Associated links                 	   109	   	          Flexible Learning Traditional Lecture April 2015 October 2015 December 2015 May 2015 Group 1 2 3 4 5 1 2 3 1 2 3 1 2 3 4  Topics pertaining to: NURSE ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ Subcategory: recommendations ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ Associated links 3 9 7 8 11 11 12 6 8 10 11 7 11 5 13  Subcategory: resources ✓    ✓         ✓  Associated links 2    1         1   Subcategory: referrals ✓ ✓ ✓ ✓ ✓    ✓ ✓   ✓ ✓  Associated links 2 2 3 2 2    2 2   1 3   Subcategory: education ✓ ✓ ✓  ✓ ✓ ✓ ✓ ✓ ✓ ✓  ✓   Associated links 1 4 6  4 11 3 7 2 4 13  2    Subcategory: assessments  ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓  ✓ ✓ Associated links  8 5 17 1 7 10 6 16 2 12 2  8 2  Subcategory: approach to care  ✓  ✓   ✓ ✓ ✓   ✓  ✓ ✓ Associated links  2  1   1 4 6   4  1 4 	   110	                      Flexible Learning Traditional Lecture April 2015 October 2015 December 2015 May 2015 Group 1 2 3 4 5 1 2 3 1 2 3 1 2 3 4  DISCUSSION TANGENTS ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ Subcategories 3 3 4 5 4 1 2 4 3 3 1 3 3 1 5 Associated links 8 7 5 3 4 1 3 14 7 6 1 1 5 2 10 	   111	  Appendix D: Individual Group Concept Maps  	   112	   	   113	   	   114	   	   115	   	   116	   	   117	   	   118	   	   119	   	   120	   	   121	   	   122	   	   123	   	   124	   	   125	   

Cite

Citation Scheme:

        

Citations by CSL (citeproc-js)

Usage Statistics

Share

Embed

Customize your widget with the following options, then copy and paste the code below into the HTML of your page to embed this item in your website.
                        
                            <div id="ubcOpenCollectionsWidgetDisplay">
                            <script id="ubcOpenCollectionsWidget"
                            src="{[{embed.src}]}"
                            data-item="{[{embed.item}]}"
                            data-collection="{[{embed.collection}]}"
                            data-metadata="{[{embed.showMetadata}]}"
                            data-width="{[{embed.width}]}"
                            async >
                            </script>
                            </div>
                        
                    
IIIF logo Our image viewer uses the IIIF 2.0 standard. To load this item in other compatible viewers, use this url:
http://iiif.library.ubc.ca/presentation/dsp.24.1-0344005/manifest

Comment

Related Items