You may notice some images loading slow across the Open Collections website. Thank you for your patience as we rebuild the cache to make images load faster.

Open Collections

UBC Theses and Dissertations

UBC Theses Logo

UBC Theses and Dissertations

Evaluating the effects of an online community of practice : the case of Openpediatrics Sarmast, Hamideh 2018

Your browser doesn't seem to have a PDF viewer, please download the PDF to view this item.

Notice for Google Chrome users:
If you are having trouble viewing or searching the PDF with Google Chrome, please download it here instead.

Item Metadata


24-ubc_2019_february_sarmast_hamideh.pdf [ 1.86MB ]
JSON: 24-1.0375827.json
JSON-LD: 24-1.0375827-ld.json
RDF/XML (Pretty): 24-1.0375827-rdf.xml
RDF/JSON: 24-1.0375827-rdf.json
Turtle: 24-1.0375827-turtle.txt
N-Triples: 24-1.0375827-rdf-ntriples.txt
Original Record: 24-1.0375827-source.json
Full Text

Full Text

   i  EVALUATING THE EFFECTS OF AN ONLINE COMMUNITY OF PRACTICE:  THE CASE OF OPENPEDIATRICS  by HAMIDEH SARMAST   A THESIS SUBMITTED IN PARTIAL FULFILLMENT  OF THE REQUIREMENTS FOR THE DEGREE OF  MASTER OF ARTS  in  THE FACULTY OF GRADUATE AND POSTDOCTORAL STUDIES (Educational Studies)  THE UNIVERSITY OF BRITISH COLUMBIA (Vancouver)    December 2018 © Hamideh Sarmast, 2018         ii The following individuals certify that they have read, and recommend to the Faculty of Graduate and Postdoctoral Studies for acceptance, a thesis/dissertation entitled:  Evaluating the effects of an online community of practice: the case of Openpediatrics  submitted by Hamideh Sarmast  in partial fulfilment of the requirements for the degree of Master of Arts in Educational Studies   Examining Committee: Thomas J. Sork Supervisor  Niranjan Kissoon Supervisory Committee Member  Chris Lovato Supervisory Committee Member Judith Walker Additional Examiner   Additional Supervisory Committee Members: Traci Wolbrink Supervisory Committee Member Jeffrey Burns Supervisory Committee Member    iii Abstract Many factors contribute to preventable child and infant mortality globally including a shortage of well-educated health care professionals, heavy workloads in pediatric departments, and lack of time for professional development. These factors are particularly acute in developing countries. Online Communities of Practice (OCoPs) create opportunities for those with limited access to high quality learning experiences, to join virtual communities that encourage the creation and exchange of knowledge and experiences. OPENPediatrics (OP), an OCoP, was created to connect clinicians worldwide involved in the care of children to enable them to share their knowledge and experiences, to improve their professional competencies, to help ill children and save lives.  The purpose of this study was to evaluate OP as an online community of practice, to determine its effects from the perspective of its users and to recommend ways that OP and other OCoPs can be made more effective as platforms for professional development.  An outcomes-logic-model was created using the theory of Communities of Practice as a conceptual framework. Drawing on elements of Utilization-focused Evaluation (UFE), six research questions were developed to examine the significance and outcomes of OP.  Findings showed that OP, as an OCoP, helps pediatric clinicians learn, increase their competencies and deliver better care. Results of this study also identified some limitations of OP such as lack of awareness of its many features among users, contextual problems in using OP in developing countries, and low levels of interaction among members.  OPENPediatrics plays a significant positive role in users’ learning, professional development and quality of care although there is room to improve how it functions as an OCoP. Recommendations are offered to the leadership of OP and to those who may wish to conduct research on the effectiveness of online communities of practice.     iv Lay summary High rate of disease and death among children is a major health care problem in many parts of the world. A global shortage of well-educated health care workers, heavy workloads in pediatrics departments, and lack of access to professional development opportunities are major contributing factors. Online Communities of Practice (OCoPs) can be an effective way to help health care professionals continue learning at times and places that are convenient to them. OPENPediatrics (OP) as an OCoP was designed to connect these busy professionals worldwide, facilitate their learning and development, and contribute to better care of children worldwide. This study investigated the effects of OP from the perspective of users and offers recommendations for improving its role as an online community of practice.       v Preface This thesis is an original, unpublished and independent intellectual product of the author,  Hamideh Sarmast. As of the date of this thesis, no part of this research has been published (partially or wholly).  The online survey conducted as part of this research was developed collaboratively with Dr. Dennis Daniel at OPENPediatrics who also managed its distribution to OPENPediatrics’ users and forwarded the data collected to Hamideh Sarmast for analysis and inclusion in this thesis.  This research was approved by the University of British Columbia’s Behavioral Research Ethics Board under certificate (H16-02284) as well as by Institutional Review Board (IRB) of Boston Children’s Hospital (IRB-P00026019).     vi Table of contents  Abstract ......................................................................................................................................... iii Lay summary ................................................................................................................................ iv Preface ............................................................................................................................................ v Table of contents .......................................................................................................................... vi List of Tables .............................................................................................................................. viii List of Figures ................................................................................................................................ x List of Abbreviations ................................................................................................................... xi Acknowledgements...................................................................................................................... xii Dedication ................................................................................................................................... xiv Chapter 1: Introduction ............................................................................................................... 1 1.1. Introduction ............................................................................................................................................1 1.2. Background ............................................................................................................................................3 1.3. Evaluation approach, research paradigm, and conceptual framework ...................................................7 1.4. Problem statement ................................................................................................................................ 16 1.5. Purpose of the study ............................................................................................................................. 17 1.6. Research questions ............................................................................................................................... 17 Chapter 2: Review of the literature ........................................................................................... 18 2.1. Communities of Practice (CoPs) in health care ................................................................................... 18 2.2. Themes ................................................................................................................................................. 20 Chapter 3: Methodology ............................................................................................................. 37 3.1. Evaluating Online Communities of Practice (OCoPs) ......................................................................... 37 3.2. Evaluation approach: Utilization Focused Evaluation (UFE) .............................................................. 38 3.3. Process of conducting the OP evaluation ............................................................................................. 39 Chapter 4: Findings .................................................................................................................... 53 4.1. Part one: Quantitative findings ............................................................................................................ 53    vii 4.2. Part two: Qualitative findings .............................................................................................................. 84 Chapter 5: Discussion ............................................................................................................... 107 5.1. Research Question 1. How well does OP cover topics of interest to the community? ....................... 107 5.2. Research Question 2. To what degree are knowledge, experiences and stories being shared across the community?  ................................................................................................................................................................... 110 5.3. Research Question 3.  To what extent are members engaging in productive and sustained interactions via the OP platform? .................................................................................................................................................... 114 5.4. Research Question 4. How effective is OP in supporting professional development of members? ... 117 5.5. Research Question 5. How does OP impact health care practices across the community? ................ 119 5.6. Research Question 6. Overall, how helpful is OP in delivering better care and in what way (s) has it fallen short?  ........................................................................................................................................................... 120 5.7. Unexpected findings .......................................................................................................................... 123 Chapter 6: Summary, Conclusions and Recommendations.................................................. 126 6.1. Methodology ...................................................................................................................................... 126 6.2. Key findings ....................................................................................................................................... 127 6.3. Limitations ......................................................................................................................................... 128 6.4. Conclusions ........................................................................................................................................ 129 6.5. Recommendations .............................................................................................................................. 131 6.6. Closing comments/ final reflections .................................................................................................. 138 References .................................................................................................................................. 142 Appendices ................................................................................................................................. 151 Appendix A: Online questionnaire ................................................................................................................... 151 Appendix B: Focus group/ interview questions ............................................................................................... 159    viii List of Tables Table 3.1: Logic Model for OPENPediatrics program as a global OCoP .................................................................... 41 Table 3.2 Short-term Outcome Evaluation Framework ............................................................................................... 43 Table 4.1: Distribution of survey participants by age (year) ....................................................................................... 54 Table 4.2 Distribution of participants by gender ......................................................................................................... 54 Table 4.3 Distribution of participants by profession .................................................................................................... 55 Table 4.4 Distribution of participants by specialty ...................................................................................................... 56 Table 4.5 Distribution of participants by country of registration ................................................................................. 57 Table 4.6 How long have you been using OPENPediatrics? ....................................................................................... 57 Table 4.7 Which type(s) of internet connection do you most often use to access OPENPediatrics? ........................... 58 Table 4.8 Where do you most commonly access OPENPediatrics? ............................................................................ 58 Table 4.9  What device do you most commonly use to access OPENPediatrics? ....................................................... 59 Table 4.10 Do you use Internet Explorer as your main browser to access OPENPediatrics? ...................................... 59 Table 4.11 Distribution of respondents by familiarity with different features of OP .................................................. 60 Table 4.12 I have used a Group on OPENPediatrics. .................................................................................................. 60 Table 4.13 Regarding OPENPediatrics Groups, which of the statements below apply to you? ..................................60  Table 4.14 What are common reasons for you to use OPENPediatrics? ..................................................................... 62 Table 4.15 Approximately how many times in total have you visited OPENPediatrics? ............................................ 63 Table 4.16 How frequently do you visit OPENPediatrics? .......................................................................................... 63 Table 4.17 User-friendliness of OPENPediatrics ........................................................................................................ 64 Table 4.18 Distribution of the participants by their perception about appearance of the OP website ......................... 65 Table 4.19 Perceptions of the participants about the OP website ................................................................................ 66 Table 4.20 Positive impact of OP on clinical practice ................................................................................................. 67 Table 4.21 Perceptions of participants about videos on the OP website ...................................................................... 68 Table 4.22 Do you ever watch OPENPediatrics World Shared Practice Forum videos as part of a group? ................ 69 Table 4.23 Distribution of the participants by their perception about the World Shared Practice Forum videos. ....... 70 Table 4.24 Distribution of participants by their perceptions about OP Guided Learning Pathways ........................... 72 Table 4.25 Why did you not complete an OPENPediatrics Guided Learning Pathway? Select all that apply: ........... 73 Table 4.26 Which of the OPENPediatrics simulators listed below have you used in the last three months? .............. 73    ix Table 4.27 Distribution of the participants by their perception about OP simulations ................................................ 75 Table 4.28 What technical barriers, if any, do you face in accessing the platform? .................................................... 76 Table 4. 30 Themes surfacing from the open-ended question .....................................................................................78            x List of Figures Figure 1.1 Legitimate Peripheral Participation (based on Lave and Wenger's description)  ....................................... 10      xi List of Abbreviations BC: British Columbia CoP: Communities of Practice OCoP: Online community of practice OP: OPENPediatrics PICU: Pediatric Intensive Care Unit UFE: Utilization Focused Evaluation DEU: Dedicated Education Units  UNICEF: United Nations Children's Fund USA: United States of America       xii Acknowledgements Special acknowledgement and thanks is due to my committee members, Dr. Thomas Sork, Dr. Chris Lovato, Dr. Niranjan Kissoon, Dr. Traci Wolbrink and Dr. Jeffery Burns for their time and assistance. I truly appreciate all of their guidance and support as I navigated this journey! This research would not have been possible without their impressive support and assistance. A special note of thanks is due to Dr. Thomas Sork, my supervisor, for encouraging my research and for allowing me to grow as a research scientist. His advice on research has been priceless. “Dear Tom, my journey through graduate school was amazing, and I owe a huge part of it to you.” A special note of thanks is due to Dr. Chris Lovato, for giving her time, talent and great experiences to my project.  “Dear Chris, having a professor with your knowledge and experience to confer with has been a valuable gift over my study. “ A very special thanks is due to Dr. Niranjan Kissoon, who taught me the true meaning of being human and who introduced me to OPENPediatrics and supported me whenever I needed it throughout my study. “Dear Tex, it was a great feeling knowing that I always could come to you for your wonderful advice and incredible support. Thanks for being the greatest mentor, advisor, and liaison I could ever asked for…”  I would like to extend my special appreciation and thanks to Dr. Traci Wolbrink from bottom of my heart for her immeasurable supports right from the beginning of my study, especially extra work she did for obtaining ethics approval from Boston Children’s Hospital. “Dear Traci, I feel fortunate and lucky for being one of the many students that you have influenced with your great mentorship.”     xiii I would like to extend my special appreciation and thanks also to Dr. Jeffery Burns for being an honorable member of my committee and for priceless advice and his precise and instructive comments.  “Dear Jeffery, there is no doubt I will always remember your kind support.”  I also wish to thank the physicians, nurses, allied health and all other staff in the field of Pediatric Medicine who participated in this research from many different countries.      xiv Dedication  Kindly dedicated to:  My parents who always loved me and taught me to work hard for things I desire to achieve.   My siblings and their families for their kind support especially taking care of my aging parents in my absence.  My sister Fariba and her lovely husband Kevin who supported me in many ways over this long journey & Mirkaber My husband, for being a constant source of love, support, guidance and encouragement during the challenges of my graduate study. “Dear Mirkaber you are not only my husband but also are my best friend and soul mate. Thanks for being in my life.”  Aryan My son, for all the patience he showed over years of my study. “Dear Aryan, it was my job to make a successful man out of you, but you made a successful mother out of me. I am truly thankful to God for having you in my life.”     1 Chapter 1: Introduction 1.1. Introduction One hundred years ago, the elimination of infectious diseases resided in the realm of fantasy (Gotsch, 2001). In the past 50 years, however, advances in public health have made it possible to combat infectious diseases and advances in acute care, have made elimination of infectious diseases a reality. These and other innovations have resulted in doubling of life spans among many people (Frenk et al., 2010). Despite all the advancements that are frequently celebrated, sharing medical knowledge equitably was sadly failed both within and between countries (Frenk et al., 2010; Lee & Sadana, 2011). Based on the Julio Frenk et al. 2010 report, about one million new doctors, nurses, midwives, and health care professionals are trained every year worldwide through 2,420 medical schools, 467 public health departments, and other nursing educational institutions. However, there is a huge institutional and professional shortage of skilled personnel due to the poor distribution of resources, both between and within countries. For example, four countries (China, India, Brazil, and USA) each have more than 150 medical schools, whereas 36 countries have no medical schools. In sub-Saharan Africa, 26 countries have one or no medical school (Frenk et al., 2010b). Moreover, the burden of disease is disproportionately concentrated in countries with fewer health care workers. Thus, health systems in disadvantaged societies are confronted with shortages of skilled health care professionals to effectively address emerging health issues such as new infectious diseases, environmental, and behavioral risks, which threaten the health security of all. According to Scheffler et al., around 57 countries have a shortage of 2-3 million physicians, nurses and health care professionals, which leads to busy clinics and hospitals and suboptimal quality of care and medical education (Scheffler, Liu, & Dal, 2008). Faced with this reality, the care of children is especially compromised due to their physiological fragility and the lack of skilled clinicians to provide care. As a consequence of these factors, mortality and morbidity    2 among children are high especially in low and middle-income countries (Lee & Sadana, 2011). This situation worsen because of the inadequate number of pediatricians, coupled with busy departments, and high turnover in pediatric units that limits the time available for professional learning and development for pediatric clinicians (Spedding, Jenner, Potier, Mackway-Jones, & Carley, 2013). World Health Organization (WHO) states that the worldwide shortage to achieve the Sustainable Development Goals needs about 43 million health care workers requiring enormous training and maintenance of skills (Scheffler et al., 2016).  Emerging technological innovations and Internet-based learning in the 21st Century has brought opportunities for professional development and seems to be an effective tool to reverse this need. Advances in technology have transformed medical education nowadays to the extent that almost all medical students, residents, and fellows use computer-based resources at some time during their training.  They have found Web-based learning very helpful toward their educational advancement because they can learn at their convenience, out of their duty hours and with no need for geographic proximity to a classroom (Wolbrink & Burns, 2012). Medical e-textbooks are now outnumbering printed medical textbooks, and the quantity and quality of Internet-based medical education, e-learning platforms and websites have significantly increased. OPENPediatrics (OP) as an online learning platform has been developed by Boston Children’s Hospital (in collaboration with the World Federation of Pediatric Intensive and Critical Care Societies) in a response to the potential of the Internet to dramatically transform health care education, including postgraduate medical education. It is an open access, online educational platform that allows pediatricians and other pediatric health care professionals worldwide to advance their professional development and improve their competencies.  OP is based on Lave and Wenger’s theory of learning in Communities of Practice (CoPs). According to them, CoPs are groups of people who have a common interest or the same concern and gather together in a community or create a network to communicate    3 one another to create or exchange knowledge in order to learn more about their interests, solve problems and improve their practice. (Global Solution Networks, 2014; Lave & Wenger, 1991; Wenger, 2000, 2002; Wenger, McDermott, & Snyder, 2002a). OP as a CoP aims to improve capacity for better care for children globally. It provides related knowledge for health care  professionals and connects them globally helping them to share their latest research findings, best practices, patient care examples, simulations, and other experiences in order to improve their competencies and save lives (Beasley, 2013; Tucker, 2014; UNICEF, 2013). The purpose of this study was to evaluate the role of (OP) and contributing factors in learning, professional development and quality of care delivered by pediatric clinicians worldwide. This chapter provides background information about OP and presents the evaluation approach, research paradigm and the conceptual framework, which frames my research. It includes a statement of the problem and purpose of the research and my research questions. 1.2. Background Millions of children under the age of five die from preventable causes worldwide each year and many health problems stay unresolved despite the availability of life-saving medical solutions. This is largely because of medical staff shortages, poorly educated workforce, unevenly distributed resources, outdated and static curricula in medical schools, weak stewardship, and separation between education and health system within and between countries. In addition to modern equipment, adequate supplies and medicine, what is clearly needed is a reform in health care  professional education toward a more dynamic system; a competency-driven approach to promote inter-professional and trans-professional education in order to eliminate professional silos, increase collaborative and non-hierarchical relationships and use the power of new technology for learning (Frenk et al., 2010b). These needs were addressed in The Commission on Education of Health Care  Professionals for the 21st Century, which was launched in the Harvard School of Public Health in January 2010 (China    4 Medical Board, 2010). The Commission aimed to reform health professional education by identifying gaps and opportunities and offering helpful recommendations. The Commission was led by co-chairs Julio Frenk, Dean of the Harvard School of Public Health, and Lincoln Chen, President of the China Medical Board. It involved a group of 20 Commissioners and academic leaders from different countries, who came together to create strategies for education in medicine, nursing, and public health worldwide. The Commission adopted a global vision and a multi-professional perspective with a systems approach (Frenk et al., 2010b). Throughout 2010, the Commission conducted researches, collected information and analyzed data to create a vision with practical recommendations to transform the education of health care professionals in all countries. The results revealed from the researches presented problems that hinder health care  professionals from getting their needed education and from becoming equipped to work in today’s complex health systems such as insufficient funding, inadequate medical schools and static curricula (China Medical Board, 2010).      The key recommendations offered by the Commission were first, instructional reform in medical schools about what ought to be taught and how.  A suggestion was made for a competency-based, inter-professional and team-based approach, which harnessed the Information Technology (IT) revolution and new technologies in the workplace rather than traditional learning methods (Mathur, 2011). Another recommendation was institutional reform toward enhancing collaborative activities between the education and health sectors in order to prepare health care professionals to match health needs in every country. For example, they suggested adopting joint planning mechanisms to engage key stakeholders such as ministries of education and health, professional associations, and the academic communities to overcome health care education issues (Mathur, 2011).      To achieve these goals, the Commission called for global alliances of educators, health care and education professionals, universities, non-governmental organizations, international agencies, donors, and foundations, for a global movement to promote a new century of transformative    5 professional education.  They hoped to create a joint education and health planning system to help health care  professionals to catch up with new knowledge in their field, as well as a global networks among hospitals and health care  units for the result of more equitable and better health services world-wide (Tanja, Lindgren, Wohlin, Lagervall, & Andersson, 2015). 1.2.1. Guatemala case and origin of the idea When Dr. Traci Wolbrink, a pediatrician from Boston Children’s Hospital, was working in a remote hospital in Malawi to save a little girl’s life, she realized that a lack of facilities and medical supplies prevented her from treating the child as effectively as she knew she could do at Boston Children’s Hospital. She used available hospital equipment and created her own tool to save the child’s life. When she returned to Boston, she shared her experiences with colleagues but she wondered if there would be a way to share this experience or even other creative experiences and new knowledge in the field with colleagues all over the world (Tony & Conner, 2015).  Later Dr. Wolbrink and Dr. Jeffrey Burns, Chief of Critical Care Medicine at Boston Children’s Hospital, started to work on an innovative way to share medical knowledge and experiences globally to enhance medical practice. Meanwhile, Dr. Burns received a phone call from a colleague in Guatemala asking his advice to save a little girl’s life as he had an experience of treating the same case. Dr. Burns was not in Guatemala. He had to work on the child’s treatment remotely via video link with his Guatemalan colleagues (Tony & Conner, 2015). Working online together, Dr. Burns and his colleagues saved the child’s life, but technical issues caused many challenges for the medical team because of the poor quality of video and sub-optimal data transmission. Later, when Dr. Burns was watching his son playing an Xbox game he realized that his son had better communication with other people through the Xbox game than they had in Guatemala during a medical emergency. Consequently, he proposed the creation of a high-quality connection via an online program to link clinicians together all around the world. This would provide opportunities to help one another to learn and solve medical problems,    6 beyond the confines of medical schools and without any concern about geographic distance. The power of digital tools also impressed Dr. Burns while he was watching the Masters Golf Tournament. The players were coached through a digital connection to find their way around the course and to be more competitive. Impressed, Dr. Burns aimed to apply the same idea to the coaching and mentoring of health care  professionals around the globe to improve their practice (Tony & Conner, 2015).  The idea was innovative and required work at the intersection of medicine, education and technology. Dr. Burns’ position and his educational background helped him to connect different disciplines. He developed collaborations with related companies such as IBM for technical support, the World Federation of Pediatric Intensive and Critical Care Societies for their medical expertise and education experts for educational technologies. They worked together and created a cloud-based platform to share pediatric knowledge and related experience worldwide. The key advantage of using a cloud-based platform is that it is accessible on any web-connected device and it reduces the problem of poor phone and video connections. The cost of building a global cloud-based platform for this purpose was more than $4 billion (Malamut, 2013). was launched in 2012, integrating IBM’s technology infrastructure (including social networking, cloud, data analysis, video and simulation technologies), educational experts from Harvard University (faculty of education) and the medical expertise from Boston Children’s Hospital. The OPENPediatrics (OP) platform offers virtual training, tools for sharing knowledge and communicating with other health care professionals, and a library of resources including videos, lectures, device simulators, and protocols. OP is currently used by clinicians in more than 145 countries on six continents (OPENpediatrics, 2013). Clinicians connect to the platform to learn, improve and share their knowledge.    7 1.2.2. The OPENPediatrics platform developers Dr. Burns is a Professor of Anesthesia, Harvard Medical School and Co-Director of OPENPediatrics. His academic work over the past two decades has focused on innovations in postgraduate medical education.  Dr. Wolbrink is an Assistant Professor of Anesthesia in Harvard Medical School and Co-Director of OPENPediatrics. She has led the development of the platform’s content, coordinated the development team, and worked with technical collaborators IBM and Genuine Interactive. Dr. Wolbrink’s academic interests include the application of innovative medical educational technologies globally.  Dr. Niranjan "Tex" Kissoon the Past President of the World Federation of Pediatric Intensive and Critical Care Societies, Vice-President, Medical Affairs at BC Children’s Hospital and professor in Acute and Critical Care – Global Child Health, also serves in OPENPediatrics as a member of external advisory team and content reviewer. Dr. Kissoon’s international work has included time in China, India, Bangladesh, Brazil and Africa, often in areas of vulnerability and limited resources for critically ill children. 1.3. Evaluation approach, research paradigm, and conceptual framework  In this section I describe the use of utilization-focused evaluation study design for this study, pragmatic research paradigm, and of Communities of Practice (CoP) as the conceptual framework.  1.3.1. Evaluation approach: Utilization-Focused Evaluation I have applied the principles underlying Utilization-Focused Evaluation (UFE) which is complementary to a pragmatist approach (Patton, 2008a). I studied UFE, its steps and details and tailored its principles into my study.  I used UFE approach because of the following reasons:    8 • UFE provides stakeholders with information they need to determine if the program is achieving its objectives and to make improvements to the OP platform, especially in relation to the CoP components,  • involving stakeholders from the beginning increases feelings of stakeholder ownership that in return will increase the likelihood that findings will be used for program improvement. UFE is different from traditional program evaluation, which was often a collection of information about the program and its activities for making judgments. UFE’s practical nature promises applying evaluation results and suggestions to solve problems or improve programs. Several problems can be addressed in a single UFE design; however, it is better to decide where limited evaluation resources will be focused. In this case the UFE will focus on improvement of the program (Patton, 2000). UFE is conducted with contributions from various stakeholders, and timelines, resources, stakeholders’ involvement, as well as descriptions of implementation, context and outcomes are all important elements (Patton, 2008b). UFE is based on a pragmatist view because it aims to address practice-based evaluation questions that are useful to the primary users of the evaluation who, most often, are decision-makers. The criticism made for this approach is that, focusing only on questions that are important to the primary users may result in a bias in the kinds of questions that are asked.   1.3.2. Research paradigm My research paradigm was pragmatism, as I used a mixed methods approach and assumed it would support the use of both qualitative and quantitative methods to generate evidence to answer my research questions (Shaw, Connelly, & Zecevic, 2010). “[P]ragmatism evaluates theories or beliefs in terms of the success of their practical application”(Stevenson, 2010). In other words, truth can be measured by its practical consequences. Pragmatism is an outcome-oriented approach and focuses on practical solutions to social problems (Shannon-Baker, 2016).      9 When the paradigm is pragmatism, practical consequences and the effects of concepts and behaviors are crucial components of meaning and truth. Pragmatism assumes there are singular and multiple realities and focuses on solving practical problems in the real world (Feilzer, 2010). Using the paradigm of pragmatism, the researcher is free of mental and practical challenges between post-positivism and constructivism and does not have to choose between or use any particular research methods or techniques (Feilzer, 2010). The epistemology of pragmatism helps the researcher study phenomena in the way he or she considers the best. Pragmatism makes program evaluation easier by allowing the researcher to decides what she/he wants to research and study, what she/he thinks is important, and what methods are appropriate to answer research questions (Shaw et al, 2010). The practical nature of pragmatism helps researchers answer questions and leads them to practical solutions. The results of pragmatism approach satisfy quantitative researchers with quantitative data driven from the research and qualitative researchers with naturalistic data collected in a qualitative and personal manner (Tashakkori & Teddlie, 1998). 1.3.3. Conceptual framework Communities of Practice (Learning Theory) Working collaboratively is essential for individuals, teams and organizations in order to navigate the rapid changes and increasing advances of the new millennium. Communities of Practice (CoPs) are ways of learning that connect people together to enhance learning toward best practice and to increase individual, group and organizational development through shared knowledge and experiences (Cambridge & Kaplan, 2005). CoPs are powerful informal learning settings. They usually develop naturally by a group of people for sharing interests, learning together and solving problems. Being informal, they tend to reside outside of formal education. Participants in CoPs are not typically pursuing formal qualifications, but rather share their knowledge and experiences to improve work performance. Participants in CoPs can    10 use multiple ways to communicate. They may meet face-to-face socially or at work, or they can participate in online or virtual communities of practice through internet connection (Bates, 2015). The concept of “Communities of Practice” does not represent a new phenomenon; in fact, this type of learning has been around since human beings started to share their knowledge and experiences through storytelling (Lave & Wenger, 1991). Jean Lave and Etienne Wenger, however, were the scientists who first introduced the term into the academic mainstream. Initially, they called it “situated learning” (1991).  In 1998 Wenger extended the term to “Communities of Practice” (Lave & Wenger, 1991; Wenger et al., 2002a). Lave and Wenger, knowing that learning is not limited to classrooms, were interested in exploring how learning happens outside of classrooms. Studying many learning groups, they found that in each group, usually, there is a core group of individuals who lead the process of learning among group members. Other individuals join the group as newcomers and gradually move ahead and become an “old timer” members of the group (Lave, 1991b). When newcomers join a group or a community, it takes a while to get grounded by observing others and doing some simple tasks. As they gradually learn more, they advance and do more complicated tasks (Lave & Wenger, 1991; Wenger et al., 2002a). This way they move from a novice position to an established one and become an expert. “Legitimate peripheral participation” is the phrase Lave and Wenger use for this centripetal process (Lave, 1991b) (Figure 1.1).  Occasional participation Peripheral participation Active participation     Core group Figure 1.1 Legitimate Peripheral Participation (developed by the researcher based on Lave and Wenger's description) (Lave, 1991b)    11 Wenger defined Community of Practice as a “group of people who interact, learn together, build relationships, and in the process develop a sense of belonging and mutual commitment” ” (Wenger, McDermott, & Snyder, 2002b, p. 34). After they shared their study results on CoPs in the early 1990s, it became a popular topic for scholars as a learning theory and a method of knowledge management.   According to Lave and Wenger, every CoP consists of three components as follows: • Domain:  Domain is a common ground for members that differentiate them from non-members. It has defined boundaries based on which members decide what is worth sharing or how to share their ideas.  • Community: Community creates the social atmosphere, which facilitates knowledge exchange through interactions and relationships with others.  • Practice: Practice includes a set of shared resources such as documents, ideas, experiences, information, and ways of solving problems. In this sense, Practice is the knowledge and knowledge tools the community develops, shares, and maintains. Creation of knowledge and sharing experiences can be optimized if these three elements work well together in a mature CoP (Gunawardena, Beth, & Carol, 2002). Knowledge Management Knowledge Management (KM) is a term that has recently become popular in organizational settings as a crucial tool to help them to keep up with rapid changes. The perception behind KM is that human beings are not able to use the full potential of their brains and therefore organizations are not able to fully utilize the knowledge that professionals possess (King, Chung, & Haney, 2009). The promise of  KM is that it helps an organization provide its best potential knowledge and support its maximum effective usage. This in turn positively influences organizational performance.  It  is generally believed that if organizations can increase their effective knowledge utilization even just by only a small percentage, they will gain great benefits (King et al., 2009). Knowledge has the power to    12 keep companies competitive in the era of globalization by helping organizations adapt to trends and adopt new technologies (Jelenic, 2011). KM is vital also in managing tacit knowledge which is a type of knowledge that not all people in the organization possess, and it cannot be transferred in usual ways such as reading, writing or conversation. Tacit knowledge is transferred in practice through social networking and effective interaction between the people who possess it and other employees (LeMay, 2009a; Parboosingh, 2002). Studies have shown that those organizations lacking in KM, experience a significant reduction in their productivity and competencies, especially when they lose members who possess a wide variety of knowledge and experience (King et al., 2009). Rapid changes in workplace environments challenge organizations in developing, acquiring, and modifying knowledge for developing new products, services or innovations more quickly in response to global competitiveness (Zboralski, Salomo, & Gemuenden, 2006). I order to overcome this challenge, organizations implement knowledge management systems. Knowledge management system use knowledge resources more effectively and efficiently to stay active and maintain their competencies through their power to control access to opportunity and advancement (Jelenic, 2011). CoPs have recently gained increased attention from practitioners and scholars and appear to be the best strategy to foster knowledge management, and to enhance knowledge sharing and learning in organizations (Wenger, 2000, 2002; Wenger et al., 2002a; Zboralski et al., 2006). From a management point of view, supporting existing CoPs in organizations, by providing resources and establishing the necessary conditions, is an efficient and effective way of implementing KM (Zboralski et al., 2006). Due to the efficiency of CoPs in KM, organizations support and even create CoPs. They help their members capture and integrate existing knowledge embodied in the expertise of other members, in order to improve their competencies, productivity, reputation, and innovation (LeMay, 2009a; Ranmuthugala et al., 2010; Ranmuthugala, Plumb, et al., 2011). A report on CoP-based KM programs reviewing over 200 manuscripts, confirmed that CoPs have positive role in KM in organizations. The    13 analysis was based on a qualitative case study of CoPs, which were intentionally created inside a large division of a multinational oil company. Considering that the largest amount of knowledge possessed by firms is embedded in the minds of employees and considering geographical, cultural, and organizational barriers which make it difficult to manage knowledge capital possessed by organizations, creating and supporting CoPs is the best way to promote effective retrieval, sharing and reusing of existing knowledge (Wenger et al., 2002a). Separate or geographically distance units, differences in background, values, language, culture, and behavior are barriers to knowledge sharing in organizations, which lead in creation of knowledge islands. CoPs are thus an effective way to bridge such islands to promote knowledge sharing, organizational learning, and rapid innovation (Scarso, Bolisani, & Salvador, 2009). Online Communities of Practice (OCoP) Since 1972, availability of accessible technology, specifically the Internet, has allowed people to share information and communicate at a faster pace and on a larger scale (Hall, 2017). Today people have more access to the Internet, mobile phones and other technologies than ever before. They connect to each other through social networks, share information and create change. They gather around ideas and work they are interested in, without regard for geographic boundaries. Millions of people in thousands of organizations are strengthening and expanding a global culture of learning. Online learning connects people together to engage in creative and collaborative work. People who want to make the world better are bringing their skills, talents and questions into online conversations (Tony & Conner, 2015). Online communication started in the 1960s when messages were sent through a network created by the US government. People began to look for new ways to work together by e-mail using this network for exchanging messages. As users continued to share their ideas via online communication, they formed virtual communities. This was the beginning of an Online Community of Practice (OCoP) (Wenger et al. 2009).     14 Online networking grew as people realized the potential of learning in online communities. Wenger et al. (2009) explained that how technology has affected the way Communities of Practice (CoPs) work and how they could work in a virtual world by using wikis or blogs to bring people together to exchange their knowledge and experiences. An OCoP includes active members who are practitioners, or “experts,” in the specific domain of interest. Members participate in a process of collective learning within their domain. Social structures are created within the community to assist in knowledge creation and sharing. Community members learn through both instruction-based learning and group discussion. There are also facilitators who provide long-term management and support interactions (Tsai, 2012). OCoPs are different from traditional CoPs in some aspects. Traditional communities are land-bounded, norms play significant role in their membership and group dynamics is crucial in these communities. Additionally, there is a clear line between members and non-members in traditional CoPs; people are either members or they are not. However, the main factor for the existence of an OCoP is identification with an idea or task (not a place); activities (tasks) have strong roles in their creation, they are mainly need-based, and they have fluid boundaries rather than formal boundaries. Norms play fewer roles in membership in these communities. Instead of land, in fact, the Internet turns into a place for OCoPs and plays a significant positive role in their development and maintenance as well as in interactions among members.  OCoPs allow people to share new tools or ideas instantly with others via the Internet. OCoPs allow a larger number of group members to become connected than traditional CoPs. Although CoPs can extend beyond a geographic location, OCoPs have capacity to grow much bigger and easier than traditional CoPs (Baylor, 2014). Capabilities of applications such as Facebook, LinkedIn and Twitter for social networking enable local casual socializations similar to what happens in normal CoPs and developing learning networks if they are used for sharing knowledge (Gaál, Szabó, Obermayer-Kovács, & Csepregi, 2014). Creating, using or facilitating an OCoP, however, is not a static and one-time event.    15 The platform is just the technical part of the architecture, which assists members in communication and collaboration. The more important part is the social architecture of the community. While the technical part supports the community’s function, the social architecture helps the community members interact toward the goal of achieving best practice. Technical and social architectures together create the container for the community (Cambridge & Kaplan, 2005). Similarity and diversity among members’ knowledge and skills also lead to creating different types of communities. For example, similarities in a group of specialties keep them together and help them to get along well and work together toward best practice. Diversity among members’ knowledge and skills on the other hand complement one another skills and competencies and creates a multi-disciplinary team effort for a common goal.  A large part of lifelong learning is now happening in OCoP through sharing of knowledge and experience and developing new ideas and forms of practice. These communities of learners are mostly open and admission is free; thus they are an alternative to the high-priced lifelong learning programs being offered by many universities (Bates, 2015). Taking advantage of OCoP for knowledge management is a wise move for organizations especially in many low-resource countries where management of existing knowledge is a huge challenge. A platform for knowledge management in health care financing in sub-Saharan Africa, showed better implementation of policies in low-income countries through OCoP (Meessen et al., 2011). The problem was a disconnection between scientists, policy makers and practitioners, which would make the implementation of policies more challenging because each party simply would ignore the knowledge possessed by the others. Most of the issues were related to knowledge management such as insufficient coordination, lack of international evidence, incomplete policy, inappropriate design, inadequate involvement of different stakeholders, poor communication and lack of monitoring and evaluation which would make the policies fail to produce expected results (Meessen et al., 2011). The solution for this conundrum, as the researchers have pointed out, was to create a platform to let    16 different knowledge holders communicate with each other and exchange knowledge. The results showed that working together within a virtual community of practice, exchanging ideas with peers on technical issues and supporting each other was a powerful way to understand each other and to transfer tacit knowledge for better policy implementation (Meessen et al., 2011).  Barriers such as members’ withdrawal, cultural diversity, lack of in-depth discussion, lack of relevant resources, delay in replies and interactions (lack of right in-time or synchronous communication and interaction) can preclude development of an OCoP that need to be considered (Hew & Hara, 2006; Thrysoe, Hounsgaard, Dohn, & Wagner, 2010, 2012).  1.4. Problem statement Millions of children under the age of five die from preventable causes every year all around the world, mainly in developing countries (UNICEF, 2013). Many factors account for those deaths; however old medical education system and shortage of well-trained health care professionals play a major role. Traditional educational models restrict medical education to the confines of classrooms and a health worker shortage results in suboptimal care. These factors have significant negative effect on health conditions in all countries but more so in developing countries. The consequences are worsened health conditions and millions of preventable deaths. Unfortunately, children, who are more vulnerable than adults, are disproportionately affected. Their health problems remain poorly addressed, worsen and cause many potentially preventable deaths mostly in developing countries.  OP as an OCoP connects clinician involved in the care of children including pediatricians worldwide, to help ill children and save lives (OPENPediatrics, 2015). Gathering data on the effectiveness of OP as an example of an OCoP and how it might be improved will not only contribute to a more effective OP, but also will provide advice to others who wish to develop online Communities of Practice intended to enhance professional competencies.    17 1.5. Purpose of the study The purpose of this study was to evaluate the role of (OP) and contributing factors in learning, professional development and quality of care delivered by pediatric clinicians worldwide. 1.6. Research questions This research study is a program evaluation of OP as an Online Community of Practice. The following questions guided this mixed-methods research to ascertain the significance and outcomes of OP. The results will be used to improve the program by building a more robust and effective virtual learning platform. 1. How well does OP cover topics of interest to the community? 2. To what extent are knowledge, experiences and stories being shared across the community? 3. To what extent are members engaging in productive and sustained interactions via the OP platform? 4. How effective is OP in supporting the professional development of members? 5. How does OP impact health care practices across the community?  6. Overall, how helpful is OP in delivering better care and in what way(s) has it fallen short?      18 Chapter 2: Review of the literature Overview Communities of Practice (CoPs) are ideal learning approaches to help health care  professionals exchange knowledge and skills and discuss best practices in order to improve the quality of care and meet the educational needs of their members (OPENPediatrics, 2015).  The purpose of this literature review is to identify, summarize and evaluate the literature on the effectiveness of CoPs (face to face & online) aimed at improving the health care outcomes across the world.  Using PsycINFO, ERIC, PsycARTCLES, EBL (E-book Library), EBSCO, and Online Educational Databases I conducted a research using key words such as Communities of Practice, health care, health, practice and Online Communities of Practice. I included peer-reviewed articles published in English since 2000, studies that used Communities of Practice as their theoretical framework; studies that focused exclusively on describing interventions with no data on evaluation, were excluded. Once the publications had met the inclusion criteria, the full texts were considered. The search generated more than 500 titles and abstracts, of which 163 remained after application of initial exclusion criteria. Results reported in findings and discussion sections of the studies were reviewed to find major themes. Results of review were organized by dominant themes.  Learning, legitimate peripheral participation (LPP), factors encouraging knowledge sharing in CoPs, benefits of CoPs (individuals, organizations, communities) and barriers to CoPs were five dominant themes that were extracted and reported in this chapter. 2.1. Communities of Practice (CoPs) in health care  The concept of CoP was introduced by Lave and Wenger and has become of interest in the field of health care and education as a learning theory. Through mutual engagement in CoPs, health care    19 professionals exchange knowledge and skills, create tools, routines, stories, shortcuts, and other resources in order to do their job better. CoPs are not limited to face-to-face interaction. In fact, advances in technology and social media have developed and promoted learning through online interactivity in virtual CoPs and allowed individuals to collaborate even if they are geographically apart. Health care, professionals usually spend time together, share information and ideas, give advice and discuss problems. They create tools and standards together and design manuals or other documents. Over time, they acquire a unique perspective and a body of common knowledge on their field, practices and approaches. They develop friendships and increase their participation in the field, they create a common identity and finally they become a community of practice (LeMay, 2009a; Wenger et al., 2002a). CoPs play a crucial role in the outcomes of the health care system, in knowledge sharing, professional development, and improving health care professionals’ confidence to apply the shared knowledge in the practice. They provide opportunities for health care  professionals to enhance their effectiveness and efficiency and create social, human, organizational and professional capital to increase quality of health care services (LeMay, 2009a). Furthermore, CoPs help to deliver high-quality health care services within economically constrained environments across the world that meet the needs of the people especially in developing countries.  A literature review on the effects of CoPs in health care in Australia reported significant improvement in outcomes, such as developing guidelines and policies, creating assessment tools, as well as using of screening tools. It also reported more positive impacts such as patient involvement in decision making, reducing frequency of insurance liability claims received by hospitals and improving rates of adherence to evidence-based process indicators (Ranmuthugala et al., 2010).      20 2.2. Themes 2.2.1. Learning Considering the fact that learning affects every aspect of a human being’s life, it is not surprising that it has become a main concern in contemporary life and at various managerial levels. For example, governments are concerned about their citizens' learning, companies think of their employees learning and schools try to find the best way for teaching their students (LeMay, 2009a). However, despite the importance of learning, learning styles have always been controversial due to lack of innovative methods to foster learning in different situations. In 1991, Lave and Wenger introduced an innovative model of learning that helps people learn through socialization, and they called it “situated learning” (Lave & Wenger, 1991). Learning through CoP is mostly based on problem solving which allows individuals to explore real life situation to find the answer through interaction with others in social contexts (Lave & Wenger, 1991; Roeckelein, 2006). In other words, socialization is crucial in this way of learning. It has been shown that learners who gravitate to communities receive more benefit and attain more knowledge from knowledgeable members (Lave & Wenger, 1991; LeMay, 2009a). According to Sayer (2014), dominant educational theories usually tend to ignore the social context of learning and consider learning as transferring knowledge and skill from one individual to another.  Situated learning theory, however, regards learning as a social phenomenon that happens through participation in a social practice and believes it must be situated and learned in a real context (Lave & Wenger, 1991).   Sayer explored practice teachers’ understanding of learning through coaching community nursing students. She used situated learning theory as a framework to study nursing student learning processes among CoPs. The findings indicated that students’ learning takes place by participation in a social context (Sayer, 2014). Ranse and Grealish also found that students learn better if they have a chance to engage in real clinical settings and care for real patients (Ranse & Grealish, 2007). In CoPs    21 students have an opportunity to learn from their peers and staff, exchange knowledge and enhance their confidence. Their study did not fully address how the students evaluated their experience and the way they responded to these pedagogies and the strategies used by staff and the university to include students in the community also was not clearly articulated.   According to Wenger, the dominant learning type in the workplace  is informal learning via socialization in CoPs (Henning, 2004; Wenger et al., 2002a). Informal learning has been embedded in practice and workplace relationships, however, it is often considered as part of the daily job interactions not an effective way of learning. Boyd and Lawley showed that informal learning is the best way for expert nurses to teach and support the nursing students and adapting in their new position. They learned about their new job, its rules, regulations, and academic language and so on. Authors believe that it could be even more helpful if they would get more support in adopting a critical stance toward their professional identity (Boyd & Lawley, 2009). Results of Walsh study about nursing students illustrated that they could learn more in hospital settings by interactions with their peers while teaching them or learning from them informally (Walsh, 2015).   Considering the nature of tacit knowledge as “a form of knowledge that is highly personal and context specific and deeply rooted in individual experiences, ideas, values and emotions” (Metaxiosis, Karrillo, & Yigitcanlar, 2010, p. 216)), it is clear that CoPs are the best place for practicing tacit knowledge. It is a crucial way to increase competencies in the workplaces, increase their productivity and overcome challenges. One paper in this literature review referred to a study, which investigated the effect of participating nurses who were working with homeless people, in an OCoP. The study showed that tacit knowledge transferred in the CoP among nurses, had a significant influence on reducing the stigma of homelessness among nurses and encouraged them to work with and help homeless people (Valaitis, Akhtar-Danesh, Brooks, Binks, & Semogas, 2011). The work was not without challenges though since creating synergic across the community was difficult due to groups’ diversity. Time    22 restriction also was an important obstacle in participating in the community since they had to devote their personal time (Valaitis et al., 2011).    According to the literature reviewed here, knowledge created in COPs is either shared orally through storytelling and narratives in places such as lunchrooms at work, in the field, on the factory floor, or online via discussion boards and newsletters. Knowledge also shared through written materials such as books, published works, protocols and standards. The results of review showed that health care professionals believe that CoPs could be a good way to share stories, narratives and information across the country, however, in order to get more out of OCoPs, there is a need to provide computing knowledge for the members and a good facilitation system for the community. This would avoid technical issues and inefficient interactions. Community developers also need to consider inter-organizational corporations and facilitate it in some case. For instance, in the case of homeless people study, dealing with multiple complex issues, community leaders may want to include social workers, housing providers or other groups in the research (Valaitis et al., 2011).   The International Peer Review (IPR) project, as an OCoP, was created after the Universities 21 Conference (global network of research universities for the 21st century) in Hong Kong in 2007. It brought dentistry students together via an online platform to work in a learning community. The platform enabled students to learn from each other by exchanging their assignments and their professional and practical dentistry knowledge (Gardner, Bridges, & Walmsley, 2012).  Themes emerging from the qualitative evaluation illustrated a significant increase in the dentistry knowledge, the ability to communicate in a professional manner with colleagues and tech-communications. Difficulties running this OCoP included a disinclination among some students to participate in the study and difficulties in recruiting enough students from other dental disciplines. This affected developing parallel spaces within the platform. Authors believe that an additional survey is needed    23 among graduated students to determine the effectiveness of participating in the OCoP on their continuing and professional learning.  A study done by Lisa Baylor from Walden University investigated the effects of OCoP on the technological self-efficacy among student teachers. They developed lessons by using iPads for classroom instruction and integration of technology in teaching. Findings showed a significant difference in pre/post-test survey and a large increase in self-efficacy among student teachers who participate in the online community and learned from expert teachers (Baylor, 2014). The study had some limitations such as different number of participants in the pre/post-test (35 and 20 respectively), and a short evaluation period of only one semester. Also, all participants were female, which may limit the generalizability of this study.  A USAID-funded Capacity Project, which aimed for sufficient and appropriate distribution of educational resources, established an ongoing web-based CoP. The CoP focused on health-related training institutions in developing countries and provided online teaching and acquisition of competence in family planning. The results of the study showed significant success in teaching and learning processes and a global alliance for nurses and midwifes. The study results provided a strong example of the successful use of an OCoP to reach many participants in a range of settings. Findings of the study suggest that to get optimum benefits it would be better if OCoP members had at least one and when possible more than more than one opportunity to meet face-to-face with other members (Thomas, Fried, Johnson, & Stilwell, 2010).  A research conducted in the University of Bristol, UK - Faculty of Health and Social Care on an OCoP for health care students revealed positive effects on developing essential characteristics of higher education. Results confirm that students were able to engage in the online community and develop essential elements of CoPs such as mutual engagement, joint initiatives and shared inventory even    24 though there were issues with connectivity especially among those students lacking computer skills (Moule, 2006)..  Children with chronic disease usually stay in hospitals for a longer time and parents of these children are in communication with health care professionals during the period of treatment. A hospital used this situation as an opportunity to create an online community called child-health care practice consisted of parents and health care professionals from hospital to work together to manage the children’s chronic conditions and to increase their care quality. Childhood chronic kidney disease (CKD) was the children’s problem, which is one of the complex disorders and need an optimal care by skilled parents at home under remote supervision and support of health care professionals. Parents of these children usually search online for care-giving information, however, accessing appropriate and medically reliable information among a great amount of unreliable, misleading and inaccurate websites is difficult for most families. Creating child-health care online community seems to be the best way to help parents and health care professionals to share responsibility for management of the situation. Carolan and colleagues (Carolan, Smith, Hall, & Swallow, 2014) investigated the effectiveness of this community. Results of the study demonstrated that, by getting involved in this OCoP and helping with clinical tasks, parents received many advantages, such as reliable information, mentorship from health care professionals and achieved expertise in care. Parents also mentioned participating in the community helped them with service transition, their psycho-social life, their children’s chronic illness, and even language and cultural barriers. Health care professionals also benefited by the communities of child-health care practice and support parents of children with chronic kidney disease (Carolan et al., 2014). Limitations for this study included resistance of some families to collaborate in clinical care via CoP due to lack of their clinical skills, English language deficiencies or having other family obligations.    25 2.2.2. Legitimate Peripheral Participation Legitimate Peripheral Participation (LPP) is the second theme surfaced in this review. It explains how a newcomer enters into a CoP and grows and advances to the center of the community. LPP is considered a centripetal movement that participants begin in the periphery and gravitate towards the center of the community as they gain knowledge and confidence (Floding & Swier, 2011; Fuller, Hodkinson, Hodkinson, & Unwin, 2005; Lave & Wenger, 1991, 2002; Sayer, 2014; Wenger et al., 2002a). In the peripheral position, the level of engagement is low, and participants are merely observers or the tasks they are involved are easy to complete and manage. At this stage, participants have minimum meaningful contribution in the outcome of the CoP. Over time, by observing the more experienced members, they absorb knowledge, learn tasks, language, norms, rules and culture of the community and gradually increase participation in complex tasks. The process of transition from periphery to the center of community is transformational in that the new comers go through an identity change; they start with a novice identity at the periphery and transform into experts through interactions in the community that helps them observe and absorb how experts are interacting and doing tasks. Katja Zboralski and colleagues supported this situation (LPP) by the obvious improvement in the positions of the active CoP members from peripheral to central positions. Through delivering valuable information from the CoP to their none member colleagues in their units, a novice members gained a better network position and increased their levels of engagement in activities (Zboralski et al., 2006).  This journey (periphery to expert) has it own difficulties for newcomers though. They may experience stress, embarrassment, frustration and even fear. Thrysoe and his colleagues found that there are problems with the transition of Newly Qualified Nurses (NQNs) into the center of their community in the clinical setting. They reported that acceptance in the CoP is a very important factor in newcomers’ smooth transition to the center of community in clinical settings (Thrysoe et al., 2012). Experienced nurses have a major influence on students’ level of participation. The authors consider experts as    26 “door-openers” (Thrysoe et al., 2010). Sometimes challenges are so intense that lead new nurses toward job dissatisfaction and even leaving the job. However, a good communication between novices and experts help novices to learn, grow, thrive in the community, enjoy their work and gave them a sense of belonging to the community (Thrysoe et al., 2012). A study done by Grealish et al. showed that a good relationship between nursing students and staff helped new nurses change their negative attitude towards taking care of older patients in an aged care unit after participation in the community and receiving mentorship in the workplace (Grealish, Bail, & Ranse, 2010).  In the International Peer Review (IPR) project, Boyd and Lawley explored how clinical experts felt as newcomers when they were appointed as lecturers to teach and support nursing students (Boyd & Lawley, 2009). Legitimate Peripheral Participation (LPP) was the central idea of this project whereby a learner acquires new knowledge through engagement with expert members of the group (Gardner et al., 2012). An ethnographic study at the pediatric oncology unit in Sweden with 16 months fieldwork also revealed interesting results.  They engaged young patients at the children’s hospital as members of the ongoing CoP in the hospital (Rindstedt & Aronsson, 2012). This research indicated that by including older children as members of clinical community they could help greatly by participating in their own treatment. Children learn by observing in hospital, especially those who are dealing with a long-term illness, gradually learn a lot about their disease, the medications, the side effects, the diet and the schedule and dosages of the treatment protocol just by observing the process of treatment. In this study doctors and nurses let the older children participate in the treatment process by creating a local CoP. The children learned, not only related medical procedures by participating in actions, but also, they talked about their test results, blood counts, medications, and other medical terms as naturally as if all were always part of their daily conversations (LPP). These children were assisting the medical staff with various tasks at the unit, working side-by-side with nurses and doctors and as a result of this    27 participation they were a great assistance for children’s hospital in the treatment process and in increasing quality of care (Rindstedt & Aronsson, 2012). A limitation for this kind of participation is that some children may overwhelmed by the responsibilities involved because they are young patients above all else.   2.2.3. Factors encouraging knowledge sharing in CoPs All CoPs need to grow and improve over the time; otherwise, they will no longer exist or no longer be vibrant. CoPs leaders should facilitate members’ ongoing interactions and vibrant communications to keep the CoPs active and helpful. Creating and posting work related questions to the community website, sharing new information, creating new knowledge and problem solving could be a few ways to name. Adding a large number of newcomers into a CoP without a robust plan for creating new knowledge, without managing process of knowledge sharing, or without taking care of vital aspects of a successful CoP such as communications, or supporting the novice and knowledge providers is not effective (LeMay, 2009a; Wenger et al., 2002a). The success of these communities in sharing and building common operational knowledge is dependent on how participants are recognized and supported by the members and organizations in the domain of interest. Meessen et al suggested that to support the CoPs in health care, ministries of health, academic organizations, agencies of aid and NGOs, along with others, allow and encourage their personnel to contribute as ‘experts’ to these CoPs (Meessen et al., 2011).  CoPs in health care settings, as reported in almost all studies reviewed here, play an important role in creating a successful and helpful community. They provide opportunities to empower both students and experts and improve their organization’s status (Boud & Middleton, 2003; LeMay, 2009a; Ranmuthugala et al., 2010; Ranmuthugala, Plumb, et al., 2011; Wenger et al., 2002a). University of Canberra in the Australia has created an educational unit in hospitals called Dedicated Education Units    28 (DEU) that creates optimal knowledge sharing opportunities for nursing students with support from experts (Ranse & Grealish, 2007). In these units, students discuss their work and professional topics with their peers and staff twice a week. Ranse and Grealish report a significant beneficial effect on both students’ learning and increasing staff-student relationships. They found that acceptance was an important factor for integrating students into the community. Participating in CoP in the DEU helped students to establish a positive relationship with staff and consequently felt welcomed and invited to engage in activities and work related conversations (Ranse & Grealish, 2007). This relationship was not limited to professional relationships rather; it included social relationships with the staff. The staff invited students to join them during their break times (for meal or tea) and to their social events outside of the clinic (Boyd & Lawley, 2009; Ranse & Grealish, 2007; Thrysoe et al., 2010). Accountability which is another important factor also affects CoPs (Ranse & Grealish, 2007). According to these researchers, nursing students who are given responsibilities for patient care and are trusted by the staff for completing allocated jobs, show more accountability in their work (Ranse & Grealish, 2007).   Another study done by Thrysoe and colleagues focused on exploring the reasons of job dissatisfaction among newly qualified nurses (NQN) which lead them to quit their job (Thrysoe et al., 2012). Results showed that negative interaction between the NQN and expert nurses was the reason for job dissatisfaction. Thrysoe and his colleagues found that quality of professional and social interaction between NQN as newcomers and the longer-term staff members can support or discourage newcomers in the process of moving from periphery towards the center of the CoP and determine their level of participation. Mutual relationship gives the NQN a sense of belonging so that they do not feel isolated or marginalized. This study indicated that receiving support from experts and becoming more valued members of CoP, increased new comers’ job satisfaction and they decided to stay in their jobs (Thrysoe et al., 2010, 2012). Grealish and colleagues also explored the effect of dedicated education units (DEU) in hospitals as a model for building social relationships, which provide learning opportunities for both    29 students and the staff. The results revealed that the expert nurses see their help to students, as an investment for future workforce (Grealish et al., 2010; Sayer, 2014). Other studies focused on factors such as self-selection for participation (no pressure for sharing knowledge), willingness to improve (nursing profession or improve patient care), reciprocity of knowledge sharing (they give knowledge because they also receive knowledge via community), non-competitive atmosphere, effective moderator (monitoring communications in the community to limit conversation to the area of expertise and avoid disruptive behavior) that all have significant positive effects on the success of CoPs especially online formats (Hew & Hara, 2006). Studies have also indicated significant positive effects of technology and social media to develop and promote social learning through interactivity in online or virtual CoP. OCoPs allow members with specialized interests who are geographically dispersed to connect and share ideas with their colleagues around the world. OCoPs create a dynamic continuous education environment, which helps members to access to the source of information to discuss in a common platform and to exchange their ideas. An OCoP for the creation and sharing of new knowledge in evidence-based physiotherapy guideline found several advantages to a socially constructed CoP. For instance, participants were able to model and build on the postings of others, which improved the quality of the discussions and supported the collaborative sharing of knowledge (Evance, Yeung, Markoulakis, & Guilcher, 2014). Chun Tsai showed the same effects of technology used in online communities, such as increasing members’ satisfaction and social interactions (Tsai, 2012).   By assessing user satisfaction in virtual CoPs Jiménez-zarco demonstrated that users’ satisfaction is also one of the key elements to foster knowledge sharing in OCoPs. The popularity of CoPs in the health care sector and the availability of technology have allowed health care professionals to use virtual CoPs for learning and exchanging ideas of best practice. This study investigated the satisfaction of 130 Spanish health care professionals who are participating in an online community. The    30 results of the study showed a higher satisfaction level among OCoP users because they were able to exchange knowledge about patient diagnosis and treatment, cost reductions, optimal  management, and also increase their social networks (Jiménez-zarco, González-gonzález, Saigí-rubió, & Torrent-sellens, 2015). 2.2.4. Benefits of CoPs (individuals, organizations, communities) Sharing knowledge in CoPs has numerous potential benefits that can be categorized into three levels: individuals, organizations and communities (LeMay, 2009a). According to Millen et al (Detlor, 2004), individual benefits include improved reputation, better understanding of other co-workers, increased trust, more interest for learning new knowledge, access to experts and higher knowledge, and increased confidence and competency.  At the organizational level, collective knowledge acquired in the CoPs can help organizations benefit from CoPs. Some of the benefits that have been reported in the organizational level are tangible outcomes such as, successful projects, increased new products and efficiency.  At the community level, CoPs can provide added benefits by creating new ideas, improving quality of knowledge, expanding problem solving and creating a common context (Detlor, 2004; Evance et al., 2014) .  The Cooperative Extension Service is a unique network that links extension professionals in more than 3,000 counties/parishes in the 50 states of the U.S. and delivers programs through an OCoP known as eXtension. An evaluation of the eXtension OCoP indicated benefits such as working across disciplines and in multi-state programs, learning from peers, teaching peers, reducing redundancy, and engaging the discipline in a more innovative and in-depth manner than by participating in the non-online community of practice (Kelsey & Stafne, 2012).   A study done at the University of Akron, USA also investigated the outcomes of an OCoP in supporting pre-service (student teacher) and in-service teachers learning how to teach, found that using    31 online connection influences members’ social ability, sense of community, and learning satisfaction (Tsai, 2012). Tsai et al studied how teachers benefit and learn to teach by participating in Sakai (“A free, community source, educational software platform designed to support teaching, research and collaboration”). Taking advantage of the accessibility of technology and affordability of the Internet, universities and researchers have been establishing and testing online learning through OCoPs and considering the potential of OCoPs in filling the gaps and disconnection between the stages of teachers’ professional development. Results included significant positive changes in members’ perceptions regarding the social network, usefulness of the tools used for communications and satisfaction with network experiences. Members also found the social network very effective in supporting their current/future teaching and bringing them a sense of community (Tsai, Laffey, & Hanuscin, 2010). The assessment of professional learning communities where teachers learn skills to improve their ability to teach resulted in improvement in their practice. This study assessed students’ achievement, compared teachers’ practice before and after participating in the community and interviewed teachers about the changes in their teaching practice. Teachers who were interviewed expressed the effectiveness of OCoP in their current or future teaching, changes in their sense of community via interacting with peers in online discussions or chats, changes in their perception of other members’ social presence via Sakai. This study showed the ability of the community to develop a sense of community, engage teachers through computer-mediated learning and work, and provide them with knowledge, confidence, and perceptions of themselves as effective teachers (Tsai et al., 2010).  Ranse and Grealish’s study demonstrated that participation in CoPs in the clinical setting increased confidence and accountability among the members and gave them more chances to become an expert after being accepted into the CoPs (Ranse & Grealish, 2007; Sayer, 2014). Participating in a CoP also increased nursing students’ job satisfaction and their willingness to continue working as a nurse (Thrysoe et al., 2012) while expert nurses regarded helping students in a CoP as a future    32 workforce investment (Grealish et al., 2010).  Thrysoe and Uys reported personal growth and improved confidence, experience of teaching and learning and improved motivation for work due to participation in CoPs (Thrysoe et al., 2010; Uys & Middleton, 2015). Participation in CoPs with peers also helped students to reduce the gap between theory and practice (Walsh, 2015). Boyd and Lawley noticed that nurses’ participation in a collaborative and supportive CoP encouraged them to learn better and create a better network with other colleagues (Boyd & Lawley, 2009).   Millen et al pointed out in her book “Towards Knowledge Portals”
organizations (hospitals, clinical and health care settings) also benefit from CoPs. By creating a CoP unit called Dedicated Education Unit (DEU)1, the University of Canberra was able to facilitate better relationships between students (the novice) and staff (the experts) that led to better care of patients (Grealish et al., 2010; Ranse & Grealish, 2007; Thrysoe et al., 2010; Walsh, 2015). CoPs also increased nursing students’ accountability, which was beneficial for both hospital and university (Ranse & Grealish, 2007). In Thrysoe and colleagues’ study, hospitals were faced with nurse shortage because of a high rate of turnover among the nurses due to job dissatisfaction. Nursing students are often not introduced effectively into the clinical setting because of time limitations and overloaded experienced nurses that limit mentoring possibilities. This situation leads to student dissatisfaction with their job and to quitting shortly after. However, after creating a DEU in the hospital and asking nurses to communicate and support students, communication and job satisfaction among the students improved. Consequently fewer nurses quit their jobs (Thrysoe et al., 2012). Tsai et al showed positive effects in members’ learning and professional improvement as well as cognitive changes and changes in their teaching                                                  1 Dedicated Education Units (DEUs) are health care units developed in hospitals in order to create an optimal clinical learning environment for nursing students (Edgecombe, Wotton, Gonda, & Mason, 1999).     33 practices through online social interaction with their peers (Tsai et al., 2010). The CoP was also beneficial in addressing a shortage of nurses and in improving beliefs of expert nurses regarding communication with nursing students. This led to better support for novice students and resolution of the nursing shortage in aged care units (Grealish et al., 2010). Research on Collaboration for Higher Education of Nurses and Midwives in Africa (CHENMA) showed significant benefits for a school of nursing in terms of improving the international status of the school (internationalization) (Uys & Middleton, 2015). A CoP also helped in reducing the stigma associated with working with homeless people, improving quality of care provided and impacting existing policies for more support in the organizational level (Boyd & Lawley, 2009; Valaitis et al., 2011).  Studying 114 business students in Australia, Moule and colleagues reported the benefits of online group communication in achieving higher performance (Moule, 2006). Katja Zboralski reported the potential of the CoPs in supporting the development, exchange, and managing of knowledge in an organization in order to improve the organization’s performance. They investigated 36 communities in a multinational corporation. The results demonstrated a positive relationship between CoP activities and business performance. They learned that members became involved and learned through a knowledge sharing process through participation in the CoP. The knowledge gained improved their network influence as they passed information on to non-community colleagues in their primary organization, as well as application of learnings positively influenced their organizational performance (Zboralski et al., 2006). 2.2.5. Barriers CoPs may encounter various barriers, which will impact their effectiveness at both the individual and organization level. For example, in Thrysoe and colleagues’ study, uncertainty among NQNs was a significant barrier, which hindered their full participation in the community. As Thrysoe et al discussed, NQNs were unsure about their professional capability and therefore hesitated in    34 participating in professional discussions (Ranse & Grealish, 2007). Lack of dialogue and new knowledge were also reported as barriers that could make CoPs inactive and difficult to join (Hew & Hara, 2006; Ranse & Grealish, 2007). Group norms have a significant influence in participation and knowledge sharing (Ranse & Grealish, 2007). Most of participants, especially nurses in the clinical settings, considered time and workload as a barriers to their participation in the CoP (Hew & Hara, 2006). Lack of knowledge and comfort in the use of technology for sharing knowledge sometimes hindered nurses’ participation in OCoPs. In Hew’s study, for example, one nurse explained that sometimes he or she finds it difficult to communicate some things clearly in words for fear of running the risk of being misunderstood by someone else (Hew & Hara, 2006). Sometimes barriers also occurred due to nurses’ unfamiliarity with the topic shared in the OCoP simply because some topics are outside of nurses’ areas of expertise or far from their daily practice (Hew & Hara, 2006). Inappropriate interaction between newcomers and old-timers (experienced members) can also be an obstacle. The experienced members play an important role in establishing productive interaction with the novice. Thrysoe refers to an expert nurse as a “door opener” since a good relationship with a newcomer can open the doors for a student’s full participation (Boyd & Lawley, 2009; Thrysoe et al., 2010; Walsh, 2015). In the case of OCoPs lack of face-to-face interaction can be a limitation for CoPs’ productivity, however, as Thomas et al. have pointed out, the effects of this limitation can be reduced by preparing opportunities for members to gather together occasionally to interact, share best practices and learn strategies from one another during international conferences or forums. By participating in conferences they can share the results of their works, experiences, strategies to strengthen the community and its productivity (Thomas et al., 2010). Other barriers for OCoP effectiveness are access issues, such as problems accessing a computer with an Internet connection, difficulties for members lacking computer skills, technical issues and problems with accessing to the learning environment. Lack of necessary skills and trust can also limit relationships within the groups (Moule, 2006). Gender and culture    35 differences can appear to limit communication within the community or in discussion boards. There is evidence that members fail to engage in community endeavors, learning and sharing ideas due to gender and cultural differences (Moule, 2006).  To summaries, the literature review revealed that CoPs are powerful informal learning, which evolve naturally in organizations to address common interests and problems and can be considered as an opportunity for delivering better health care services. CoP is an ideal learning mechanism in the health care system that can also be created purposefully to help health care  professionals share their tacit knowledge and experiences and discuss best practice in order to improve quality of care (LeMay, 2009a; Parboosingh, 2002). CoPs play an important role in knowledge management, generating innovation, smoothing integration of new staff, and creating social capital and adding organizational value. These features then make CoP popular in health care system as a tool for managing the existence knowledge and improving the quality of care (Ranmuthugala, Cunningham, et al., 2011).  CoPs support developing individual professional identities beyond geographical boundaries and are increasingly promoted in the health care sector to foster knowledge exchange and to improve organizational performance (Moule, 2006). This is especially true for online forums, which have grown significantly in recent years. OCoPs have provided a unique opportunity for dynamic continuous education environments, with access to repositories of information and a common platform for the discussion and exchange of ideas. Studies have shown that OCoPs support knowledge management system (KMS), and enable health care  professionals to create, share, and utilize knowledge (Alali & Salim, 2013). OCoPs improve health care  professionals’ learning and professional development due to their ability to allow professionals to learn at their convenience regardless of geographic location (Patel, 2007; Spedding et al., 2013).  Despite all the proposed benefits of OCoPs, there are reports as well on barriers and problems that hinder them from best functioning. Problems such as uncertainty and hesitation in communication    36 especially among novices, lack of trust, lack of dialogue and new knowledge, time restriction and workload, lack of knowledge and comfort in the use of technology, lack of face-to-face interaction in OCoPs, lack of accessing a computer, Internet connection, and even gender and culture differences that need to be managed in order to get all potential benefits out of them.  .    37 Chapter 3: Methodology Overview  In this chapter, I will describe my research methodology. In so doing, first, I will discuss briefly evaluation among Online Communities of Practice (OCoPs). That will be followed by a description of the evaluation approach used for evaluating OP in this research. Then, I will describe the process of evaluation and finally study design in my research.  3.1. Evaluating Online Communities of Practice (OCoPs) Despite all the potential benefits accounting for OCoPs, little is known about their performance outcomes (Zboralski et al., 2006) or the processes that lead to the successful creation of knowledge-based structures because the wide variety of OCoP characteristics such as membership, purpose, and structure, makes it difficult to draw conclusions about overall effectiveness. Evaluating of OCoPs is also difficult because outcomes are highly dependent on the way they are structured and operated (Ranmuthugala, Plumb, et al., 2011). The effects cannot always be directly linked to activities of the OCoP, as they could be the result of other contextual factors. Furthermore, there is usually a time lag for outcomes to become obvious. Due to the nature of OCoP activities, most outcomes are intangible and, therefore, difficult to measure, (Adler & Kwon 2002; Bontis & Choo 2002; Carmeli, 2004). Assessing the cost-benefit of a community is also challenging because the expenses include human resources for maintaining rather than just launching the platform and supporting active engagement with the community.  Despite all difficulties in assessing the value of OCoP (Zboralski et al., 2006) there is a need to examine how they have been applied and how they affect improving health care (Ranmuthugala, Plumb, et al., 2011; Zboralski et al., 2006).     38 3.2. Evaluation approach: Utilization Focused Evaluation (UFE) According to Hallie and Nathalie, the role of an evaluation should be to acquire information that informs decision-making and action taking (Preskill & Jones, 2009a).  Considering the role of evaluation, I decided to use utilization-focused evaluation (UFE) as the primary framework since UFE’s focus  is on the utility of evaluation findings and helps decision makers to improve the quality of services (Adams, Nnawulezi, & Vandenberg, 2015). UFE considers how evaluations will affect a program and how program owners and authorities - potential users of evaluation results- will apply findings (Patton, 2000; Ramírez & Brodhead, 2013).  Based on this approach, for a successful evaluation, individuals who are interested in using evaluation findings for improving a program (normally program owners or developers), are included in the research process from the beginning in order to get benefit from their guidance during the evaluation process (Preskill & Jones, 2009a). Engaging intended users is crucial for creating relevant evaluation questions in this approach. Moreover, it is important to include individuals who have expertise and who bring various perspectives and experiences to all aspects of the program being evaluated. These individuals are responsible for the use of the results to improve the program. They are also key advocates who leverage buy-in and support throughout the evaluation design and implementation. In this research, I included stakeholders to ensure the questions are useful and relevant. This is one of the main tenets of UFE.  In order to involve all potential stakeholders, potential stakeholders were categorized in three groups. The first group consisted of stakeholders whose engagement was vital for the success of the evaluation and who would use the findings. The second group included those whose engagement was important due to their practical and influential positions. And finally a third group, those stakeholders who were somehow helpful to be included (Preskill & Jones, 2009b).   To evaluate OP, I contacted the first group of OP primary stakeholders-- Dr. Wolbrink, Dr. Burns, and Dr. Kissoon--because of their key roles in applying evaluation findings and implementing    39 recommendations (Adams et al., 2015; Patton, 2000; Wiesenberg, 2000). After studying OP and creating the first version of the evaluation questions, I sent them the questions and asked for their thoughts and comments. Then I modified the questions several times based on comments I received until we all agreed they would capture the data we need for the OP evaluation (Michael & Patton, 2005; Wiesenberg, 2000).  3.3. Process of conducting the OP evaluation 3.3.1. Method This is a case study research evaluation. Willing to learn about OCoPs, I decided to study OP as a case in order to have a better understanding about all OCoPs. Case study allows researcher to explore and understand complex issues while helping her/him for a holistic and in-depth investigation. Case study is an appropriate method for issues with regard to education, sociology and community-based problems (Fletcher et al., 1997).  To study OP closely, I used an outcomes-logic-model approach to learn about the effects of participating in OP on improving pediatricians’ professional development and practice. A logic model provides a structure for making explicit the logic of the program, the relationship between the design and operations of the program and outcomes observed.  I applied the logic model approach from the Program Evaluation Toolkit created by Ontario Centre of Excellence for Child and Youth, Mental Health (Ontario Centre of Excellence for Child and Youth Mental Health, 2013). Using CoP learning theory as the program’s conceptual framework, I constructed a logic model for all three components of the learning theory - domain, community and practice (Ontario Centre of Excellence for Child and Youth Mental Health, 2013) (Table 1: Logic Model for OPENPediatrics program as a global OCoP). Activities for the OP domain (members common interests) included identifying desired and needed competencies, technology capabilities and related learning and educational topics, as well as    40 identifying web-designers and preparing learning and educational materials (e.g.: library, videos, lectures, simulations). To achieve those activities a well-designed web and well-developed set of learning materials based on identified competencies and needed topics are required.  For the community, activities included identifying and reaching out the primary target population, which are pediatricians and health care professionals. Identifying knowledge producers (core group) and deciding on how they would communicate and connect as professionals and how to encourage others to engage in knowledge sharing. Facilitating and supporting member’s engagement also is an important activity in this regard. After conducting these activities, it was estimated that the program would have a clear target population for the community as well as a clear core group of experts. It was also predicted that various groups such as specialists in ICU, sepsis, burns, cardiology etc. would be created and collaborative activities would be utilized. Pediatricians and health care professionals would engage actively in the community and support the community and the number of centers and individuals joining the community would be increased.  Practice activities consisted of creating and sharing training tools, producing new knowledge, educational materials, and sharing explicit and tacit knowledge such as educational tools, reports, stories, experiences, ideas, and work done through communication. Through these activities, it was assumed that participants’ attitudes, skills, competencies and professionalism would be enhanced. Long-term outcomes include the development of a global pediatrician CoP, improved professional development among pediatricians, increased capacity with regard to pediatricians and health care professionals worldwide, as well as improved competency and job satisfaction. The program would reach its ultimate goal, which is reducing mortality and morbidity rates among critically ill children.   41 Table 3.1: Logic Model for OPENPediatrics program as a global OCoP (Ontario Centre of Excellence for Child and Youth Mental Health, 2013)        Producing knowledge and knowledge tools  Sharing explicit and tacit knowledge (tools, reports, stories, experiences, ideas, works done…)  -Identify competencies (subjects, number of subjects, etc.) -Identify learning and educational topics -Develop learning and educational materials  -Well -designed platform     Activities  Short-term outcomes  Medium term outcomes PROGRAM COMPONENTS  Problem:  1. worldwide shortage of pediatricians and health care professionals taking care of critically ill children 2. Inadequate educational opportunities in pediatric critical care   3. Lack of enough time for professional development among pediatricians and health care professionals                                                                  Rationale:  The creation of OCoPs among health care professionals will improve the health care system by: 1. Improving knowledge sharing, professional development and promoting best practice and delivering high quality care  2. Making available more professionals to care for critically ill children  Doman:  A common interest of members  (Pediatrics)  Identify desired and needed competencies  Identify the technology capabilities globally  Identify web-designers Identify related learning and educational topics Preparing learning and educational materials in relation to the competencies/ topics (eg: library, videos, lectures, simulations)  Long-term outcomes -Organized set of educational packages for each needed competency -Bank of educational packages for all needed competencies -Ongoing material updates  -Clear target population for the community and for each competency -Clear core group containing of experts and speakers -Various groups developed (e.g.: ICU, sepsis, burns, cardiology, sepsis …). -Clear support groups, communication group  Facilitating group for members’ interactions -Active engagement of the members in the COPs -Active supporting the COPs  -Clear communication among the members of communities -Increasing the number of centers and individual joining the community -Vibrant and active community of practice  Decreasing mortality and morbidity among critically ill children Improvement in: -Number of knowledge tools produced (guidelines, articles, videos, books etc.) -Number of experiences and stories shared  -Collaborative activities utilized by members -Improvement in attitudes, skills and competencies, professionalism (quantity and quality) -Improvement in evidence-based practice (a) Pediatrician CoPs (one big CoP and various subject and centre-specific CoPs)  (b) Improved professional development among target groups  (c) Increased capacity with regard to pediatricians and health care professionals worldwide, Improved competence, job satisfaction  Identify target population (Pediatrics & Health care professionals) Identify speakers and experts Communicating with target population  Connect professionals Encourage for engagement and knowledge sharing Facilitate/support engagement  Required resources:   Learning and educational material,  Identify potential members:   Identify health care specialties (all and interested people for engagement), content experts, facilitators, and support staff  Identifying required resources:   Communication tools and channels  Knowledge tools (such as: protocols, guidelines, stories, reports) Ultimate goal Community:  A Social atmosphere for knowledge exchange (Pediatricians, nurses, health care professionals)  Practice:  A set of shared resources (Documents, ideas, experiences, information, ways of solving problems)  Program Goals:  1. To create cops among pediatricians and health care professionals worldwide  2. To improve Professional development through sharing knowledge, empowering and promoting best practice 3. To improve the capacity for caring for critically ill children  Assumptions:  1. Program developers and leaders are committed to the program and are dedicating time and recourses needed to the program. 2. Pediatricians and health care professionals engage in the community and share their knowledge and experiences, create new knowledge and apply the new knowledge and skills in their daily work     42 3.3.2. Evaluation questions, data collection methods and tools Considering the research situation and conditions, mixed method design, as well as research questions, I created evaluation questions for short-term outcomes, which was the aim of this study. Measurement indicators help evaluators to provide evidence of the outcomes the program achieved. To create a measurement indicator, I needed to define the specific measurable indicators or change that represented achievement of the outcome by OP (see Table 3.2).  For the quantitative part, statistics measures such as number and percentage were used to describe and summarize OP’s level of goal achievement. In so doing an online survey questionnaire were distributed via the OP platform.   For qualitative portion, I used methods such as interviews and focus group discussion to get a deeper understanding of the program and its users’ perceptions. I will explain data collection tools and methods later in this chapter.          43   Table 3.2 Short-term Outcome Evaluation Framework Evaluation Questions (What do we want to know about this program?)  Link to outcomes in logic model (What outcome from the logic model does the evaluation question relate to?) E.g., Increased self-esteem Indicator(s) (What is one possible measurable approximation of the outcome?) E.g., Increased score on the Rosenberg Self-Esteem Scale Data Collection Method(s) (What data collection method will be used to measure the indicator? e.g., Survey, focus group, interview, document review, etc.) Data Collection Tool(s) (What specific tool will be used? Specify the name and whether it is a standardized tool or internally-developed) Respondent(s) (Who will provide the information needed? For example, parent, child, clinician, teacher, program staff, etc.) Person(s) Responsible for Data Collection (Who is responsible for ensuring the data are collected?) Timing of Data Collection (When will the data be collected?)      How well does OP cover topics of interest to the community?  Providing needed educational material   Usefulness Qualitative Questions (survey, interview and focus group) Program users  Evaluators Program developers Web staff      2017    Relevancy Peer reviewed Up dated  To what degree are knowledge, experiences and stories being shared across the community?  Knowledge sharing among groups Increased shared knowledge Qualitative Questions (survey, interview and focus group) Program users  Program developers  Web staff  Evaluators  Program developers  Web staff     2017     Quantitative Web analytic To what extent are members engaging in productive and sustained interactions via the OP platform?  Increased productive communications Asking and answering questions Qualitative Questions (survey, interview and focus group) Program users  Program developers  Web staff   Evaluators  Program developers  Web staff  2017  Referring to references Quantitative Web analytic Problem solving How effective is OP in supporting the professional development of members?  Increased professional development  Individual change (knowledge, expertise and identity) Qualitative Questions (survey, interview and focus group) Program users   Evaluators Program developers  2017  How does OP impact health care practices across the community Increased best practice Improving work Quantitative Questions (survey, interview and focus group) Program users  Evaluators Program developers  2017  Qualitative Overall, how helpful is OP in delivering better care and in what way(s) has it fallen short? Increased quality of care Improving organizations’ output/cost  Quantitative Questions (survey, interview and focus group) Program users  Evaluators Program developers Evaluators Program developers  2017  Qualitative   44  3.3.3. Study design In order to answer my research questions, I employed a descriptive mixed-methods research design. In so doing, I used a combination of both qualitative and quantitative data collection. Quantitative data was collected using an online questionnaire and qualitative data was collected through interview and focus group to better inform issues that were not explored in the survey of OP users.  My standard for comparison was derived from the literature review and was based on what I learned from related studies. In other words, my comparison standard was an ideal community of practice that has a clear domain of interest which all members care about it, a vibrant community with adequate interactions among members and lots of collaborative activities and products. The best possible features of a CoP that intended to bring about change and improve practice. Quantitative phase Results produced from the quantitative data collection methods are easy to summarize, compare, and generalize. Data collection  In the quantitative phase, I collected statistical and quantifiable data to describe the OP reality and its quantitative effectiveness toward best practice. Then, I summarized the data to make generalizations about effects of OP in more broadly. The quantitative data was collected using a data collection instrument, which was a questionnaire. This questionnaire was made available to all OP users via the OP website and was answered by some portion of the OP users.    45 Online Surveys  Many community platforms administer online surveys. Online surveys are a simple and an inexpensive way to access the views of large numbers of community members. Online surveys have advantages that make them suitable for deeper and more reliable inquiry (Evans & Mathur, 2005). Some of these advantages are as follow: • Allow researchers to ask participants more directly about issues and opportunities and to produce more direct evaluations of the community, • Help researchers learn what is important in the community, what is working and what is not working for different types of participants in the community, • Can be used very broadly to not only learn about members’ need, concerns, and problems, but also to find out about later potential issues, • Allow participants to stay anonymous and tell leaders things they would not say more directly in other ways. I used an online survey to collect quantitative data. For this purpose, Dr. Daniel from OP and I worked together to develop the annual OP survey questionnaire about the members’ feelings, opinions, experiences, and technical needs in regard to OP. This questionnaire was posted to OP website and users of OP were encouraged to participate in the annual survey and answer the questions. The survey data was stored in the Monkey platform and was made available to me for my analysis and interpretation. I independently analyzed, interpreted and presented the data in my thesis.  Online questions asked in the OP annual survey solicited three types of information: demographic; users’ perceptions about the website (educational materials and its different features); and open-ended responses intended to elaborate on the quantitative responses.   46  In total, the online survey included 32 questions; 29 closed-ended and three open-ended questions (Appendix A). The closed-ended questions encompassed: (a) demographic questions (including age, gender, occupation, level of education, language used, location and years using the OP platform); (b) questions asking about how they connect to OP and what kind of tools or devices they use in order to connect to the platform; (c) questions focusing on connectivity and that itself consisted of three subsections: (1) questions asking participants’ view on how well OP reflects adult learning and education principles; (2) questions inquiring about OP as an online community of practice and its ability to provide members the opportunity to create and share the knowledge they need for best practice. (3) questions that investigated about OP as an online learning website, its usefulness, the relevancy and currency of learning materials, and its connectivity. The open-ended questions were included at the end of the survey that asked participants’ opinions on the major strengths, weaknesses, and features most liked about OP as well as comments to improve the program. Face and content validity of the questionnaire has been checked by content experts (Dr. Sork, Dr. Lovato, Dr. Wolbrink, Dr. Kissoon, and Dr. Burns). Data analysis Quantitative data were analyzed using simple descriptive statistical techniques in Excel. Frequency tables were used to summarize and illustrate findings. Qualitative phase Qualitative research has much to offer in the health care field. It was the best fit for the portion of my research focusing on the thoughts, ideas and experiences of OP users. Qualitative methods with their special features and their systematic way of gathering, classifying, and   47  interpreting qualitative data were the best way to answer my qualitative questions about OP (Hancock, Windridge, & Ockleford, 2007; Holloway & Wheeler, 2010a; Kvale, 1996; Lincoln & Guba, 1985; Malterud, 2001; Yin, 2011). In the qualitative phase I investigated how the health care professionals and pediatricians make sense of their experience in connecting to OP.  In this phase I conducted two online focus groups, and four face-to-face interviews about OP to understand, describe and interpret members’ ideas about using OP for learning and obtaining information as a member of the community and in helping others to learn and achieve best practice. Data collection Focus group discussion Focus group discussions (FGD) are a well-established, valuable, mainstream qualitative research tool which is being used in multiple fields of study (Woodyatt, Finneran, & Stephenson, 2016). Focus groups are widely used in health service research presently, either as an independent data collection method or in combination with other methods such as surveys, and often in combination with individual in-depth interviews.  A focus group, by definition, is a group of people who are selected to discuss a specific topic and share their perceptions in a safe environment (George & George, 2012; Liamputtong, 2011). Focus groups usually focus discussions on specific topics such as social issues, health topics and wellness, or common concerns, opinions and experiences. (Liamputtong, 2011; Pope & Mays, 1995).  People participating in the focus group are normally from similar social and cultural backgrounds or have similar experiences or concerns. A moderator helps participants engage in a   48  dynamic discussion for one or two hours with no distractions or fear of judgments by others in the group. Focus groups can reveal participants’ attitudes, opinions or perspectives through a natural social interaction in an attempt to understand more about the topic of interest. Reaching a consensus is not the aim of focus group discussion (Kitzinger, 1994; Liamputtong, 2011). Group dynamics created in the focus group helps researchers capture shared and collective perceptions and the lived experiences of the participants. Focus groups can uncover hidden aspects of a topic that may not be accessible by other methods. They are also an opportunity for participants to have a voice and indicate what is important in understanding their thoughts, feeling and experiences (Liamputtong, 2011; Morgan, 1996). I used virtual focus groups for data collection in order to uncover the collective and shared perceptions and experiences of the OP participants. I believe the group dynamic in focus groups made this data collection method an ideal approach for exploring experiences, opinions, beliefs, needs and concerns of individuals and groups in OP (Kitzinger, 1994; Liamputtong, 2011). Online focus group discussion Having explained the suitability of FGD for data collection in qualitative phase, I conducted two online focus group discussions (OFGD), each about one hour. OFGD can be delivered through a range of online platforms such as message/forum-based or real-time/“chat room” groups (Woodyatt et al., 2016).   It can provide a unique and an inventive opportunity for collecting qualitative data and allow researcher to cross time and space barriers (Turney & Pocknee, 2005). Conducting focus group online has become more popular and important recently as a way of collecting insights and information, since the companies understand the advantages of online research especially with   49  the rapidly increasing number of consumers around the world. (Sweet, 2001; Turney & Pocknee, 2005). OFGD has ability to reach distance members and record discursive data. It has enormous potential for collecting qualitative data with no concern about time difference or geography distance (Turney & Pocknee, 2005). OFGD can be initiated and conducted in real-time or asynchronous depending of the situation (Sweet, 2001). I conducted real-time OFGDs in studying OP in order to permit participation of OP members from all over the globe and to let them to discuss their ideas and share their opinions and concerns regarding OP.   Conducting OFGD enabled me to record discursive data while running the focus group. It also provided safe, secure, and anonymous environments for participants to discuss their views freely (Turney & Pocknee, 2005).   For conducting the OFGDs I decided to use existing university infrastructure, UBC IT services platform called Blue Jeans to connect participants. Blue Jeans allows multipoint conferences that can include participants using different protocols and devices such as video conferencing systems and software, web browser, telephone and mobile app (for Apple and Android devices). Using Blue Jeans, I was able to host an OFGD from anywhere in the world with any device, as long as I had an Internet connection. It allowed me to communicate across different locations and time zones and record the focus group sessions.      50 Recruitment for the focus groups Participants were recruited electronically through online user lists. While distributing the survey questionnaire, OP users were informed about the focus groups and interviews, which would be following the survey. In addition to that, invitation letter and information card also were posted on the OP platform. For some audiences or audiences who are less likely to respond online, telephone recruiting was used to assure their attention and responses. I invited members who had connected to OP and were willing to share their experience with other members around prepared questions (Appendix 3-1 and Appendix 3-2 focus group invitation letter and script, respectively).  I provided a consent form and asked them to sign and return it prior to the focus group session. Out of 12 people who expressed interest in participating in OFGD, 4 of them participated in two OFGD, two people each. These participants were from different countries including Guatemala (n=1), Latvia (n=1), and The United States (2). Participants were from different practice fields that included various pediatrics units and with different positions. Their work experiences ranged from six months to more than 10 years. They used OP for different purposes, mainly for their learning, or teaching others. Participants were mostly English speakers except for one Latvian and one Spanish speaker.  I conducted two OFGDs. I started the sessions with explaining the purpose of my research, the voluntary nature of participation as well as the rules of a safe and sound focus group session. I asked questions and made an audio recording of each session with some help from an assistant. Number of focus group questions Based on the literature, while ten questions is optimal, and eight is ideal, twelve questions is the maximum number for one group (Eliot & Associates, 2005; Holloway & Wheeler, 2010b;   51  Kroll & Neri, 2009). I decided to have no more than six questions in order to get deeper understanding of participants’ perceptions of and experiences with OP. Questions were all open-ended since they encourage participants to expand on their opinions (Nagle & Williams, 2011). I used different types of questions; engagement questions, exploration questions and exit questions (Eliot & Associates, 2005; Holloway & Wheeler, 2010b; Krueger & Casey, 2001; Spradley, 1979; Yin, 2011). I asked them to discuss the benefits, barriers and drawbacks of OP, and the extent to which OP met their expectations or fell short. Topics of interest and the website itself also were subjects that I asked their opinion about since these are of concern to OP stakeholders. I guided the focus group in a way to elicit in-depth information (Appendix 5). Interview Interviews are another way to collect data from individuals. They are a good way for collecting in-depth information about people’s opinions, perceptions, attitudes, experiences, feelings and so forth. Interviews are useful when the topics of inquiry are complex or related to issues that need more in-depth information (Dicicco-bloom & Crabtree, 2006; Fitzpatrick & Boulton, 1996). Interviewees are encouraged to speak more in detail freely and openly. From four different types of interviews including structured, semi-structured, unstructured and non-directive, I conducted semi-structured interviews and used the same questions as with the focus groups (Green & Thorogood, 2004). I conducted four face-to-face interviews with four pediatricians who are members of the OP CoP. Recruitment for the interviews happened using the same process for the focus group discussions. After participants expressed their interest in focus groups, if they were not able to participate in the focus group discussion for any reason such as time difference between countries or time restrictions, they were encouraged to   52  participate in the interviews. Participants signed consent form for participating in the interviews. I recorded all interviews, transcribed and analyzed them along with focus group results. Data analysis Qualitative data were entered into ATLAS ti a qualitative data management and analysis software (Thomas, 2017) and analyzed using thematic analysis techniques (Trottier, 2014). I started data analysis with open coding. In so doing, I formed the categories of information about the participants’ experience of OP.  Then I moved on to axial coding. I reassembled the data in various ways to find a central category and looked for relationships between various categories and subcategories (Holloway & Wheeler, 2010a). Next, I used selective coding for writing the story line that connects the categories. The results of data analysis and coding were the results I was looking for in relation to the OP and its effects on health care.    53  Chapter 4: Findings Overview The overall purpose of this study was to evaluate the effectiveness of online communities of practice (OCoPs). The OPENPediatrics (OP) platform was chosen as a case to be studied in this regard. OP is an OCoP designed to help with the professional development of pediatricians, nurses and other health care professionals and to enhance quality of care delivered by them worldwide. The program’s previous evaluations have generated some useful data, primarily quantitative. Present study used a mix of qualitative and quantitative methods in order to gain a deeper understanding of the OP users’ experience, its commitment to a global education, and its impact on health care practice. This study helped me to do a deeper investigation of users’ perceptions, their experiences in connecting to the platform, the benefits they gain, and difficulties they face. In this chapter I present the findings of both quantitative and qualitative phases of the evaluation. Findings have been structured in two parts: Part one summarizes quantitative findings and part two presents qualitative findings of this study. Quantitative findings are based on the data that was received from an online survey distributed via the OP website. Two hundred eighty users out of a total of approximately 20,193 registered users completed the survey in October 2017 (1.4%). Not all participants answered all questions. Qualitative findings are based on the data coming from eight people, as I elaborated earlier. 4.1. Part one: Quantitative findings In this part, I will elaborate on the demographic characteristics of the participants and then will move on to main questions. At the end, I will explain my findings from open-ended questions that gave insights about the qualitative part of this research.      54  4.1.1. Demographic characteristics of survey participants The largest proportion of respondents (42%; 116/279) was in 35-49 age group and the second largest age group (31%; 87/279) was in the 50-64. (Table 4.1). Approximately half of the respondents were females (53%; 148/277) (Table 4.2).   Table 4.1: Distribution of survey participants by age (year) Age group Frequency   % Count Less than 20 1 3 20-34 19 53 35-49 42 116 50-64 31 87 65 and above 7 20  Total answered 279  Skipped 1     Table 4.2 Distribution of participants by gender Answer Choices Responses  % Count Female 53 148 Male 46 126 Other 1 3  Answered 277  Skipped 3     55  In terms of profession, respondents were mainly physicians (60%; 167/280) including attending physicians, fellows, residents and assistants. Nurses were the second largest group of participants in the survey (20%; 55/280) that included general nurses, clinical nurse educators and nurse practitioners (Table 4.3).   Table 4.3 Distribution of participants by profession (Please select the professional background representing your current practice.) Answer choices Responses  % Count Physicians  50.6% (167) Physician – Attending/Consultant 43.2 121 Physician – Resident/Registrar 5.7 16 Physician – Fellow 5.4 15 Physician Assistant 2.9 8 Surgeon – Attending/Consultant 1.4 4 Surgeon – Resident/Registrar 0.4 1 Surgeon – Fellow 0.7 2 Nurses  19.6% (55) Nurse 12.1 34 Nurse Educator 4.3 12 Nurse Practitioner 3.2 9 Others 20.8% (58) Other Health care Professional 4.6 13 Child Life Specialist 4.3 12 Respiratory Therapist 4.2 12 Non-Health Professional 3.2 9 Health care Administrator 1.1 3 Medical Student 1.1 3 Emergency Medical Technician 0.7 2 Clinical Officer 0.4 1 Laboratory Technician 0.4 1 Pharmacist 0.4 1 Physiotherapist 0.4 1 Biomedical Engineer 0.0 0 Answered 280 Skipped 0   56  Not surprisingly, as can be seen in Table 4.4, most participants selected pediatrics as their specialty area (82%; 179/218).  Table 4.4 Distribution of participants by specialty (Please select the specialty area best representing your current practice.) Answer Choices Responses  % Count Pediatrics 82.1 179 Anesthesiology 5.5 12 Emergency Medicine 5.5 12 General Surgery 1.8 4 Neurological Surgery 0.9 2 Physical Medicine and Rehabilitation 0.9 2 Allergy and Immunology 0.5 1 Family Medicine 0.5 1 Internal Medicine 0.5 1 Neurology 0.5 1 Orthopedic Surgery 0.5 1 Thoracic Surgery 0.5 1 Urology 0.5 1  Answered 218  Skipped 62  Distribution of the participants by their country of registration (the country from which they registered for OP) is shown in Table 4.5. As can be seen, participants were mostly from North America (44%; 122/278).      57  Table 4.5 Distribution of participants by country of registration (Please indicate your country of registration) Answer Choices Responses  % Count North America 44 122 Europe 20 55 Asia 17 48 South America 8 21 Africa 6 17 Central America 3 9 Australia 2 6  Answered 278  Skipped 2  The largest proportion of participants (43%; 103/236) have been using OP for two or more years with 25% using it between one and two years (Table 4.6).  Table 4.6 How long have you been using OPENPediatrics? Answer Choices Responses  % Count Two or more years 43.6 103 Between one and two years 24.6 58 Between 6 months and one year 13.6 32 Between 3 and 6 months 3.8 9 Less than 3 months 9.8 23 Not sure 4.7 11  Answered 236  Skipped 44  As can be seen in Table 4.7, more than half of the participants used the Internet at work to access OP (59%;136/231) and almost the same number (57%;131/231) used the Internet at home.    58  Table 4.7 Which type(s) of internet connection do you most often use to access OPENPediatrics?  Answer Choices Responses  % Count Work internet 58.9 136 Home internet 56.7 131 Cellular data network 19.1 44 Other (please specify): 1.7 4 Satellite 0.9 2  Answered 231  Skipped 49  In terms of place of accessing and using OP, 41% (97/236) of respondents indicated that they accessed OP at home, and the second most frequent (35%; 84/236) access to OP happened from workplace (Table 4.8).   Table 4.8 Where do you most commonly access OPENPediatrics? Answer Choices Responses  % Count Home 41.1 97 Work (clinical setting) 35.6 84 Work (non-clinical setting) 18.2 43 During my commute 5.1 12  Answered 236  Skipped 44  Respondents used a variety of devices to access OP, however, the most commonly used device was a computer (75%; 177/236). Smartphones with 19% (44/236) were the second most frequently used devices (Table 4.9). In terms of type of browser, 39% (91/236) of the respondents mentioned that they used Internet Explorer as their main browser for accessing OP (Table 4.10).   59  Table 4.9  What device do you most commonly use to access OPENPediatrics? Answer Choices Responses  % Count Computer 75.0 177 Smartphone 18.6 44 Tablet 5.5 13 Other (please specify) 0.9 2  Answered 236  Skipped 44  Table 4.10 Do you use Internet Explorer as your main browser to access OPENPediatrics? Responses   % Count No 61.4 145 Yes 38.6 91  Answered 236  Skipped 44  4.1.2. Main questions Participants were asked about their familiarity with different features of OP. As can be seen in Table 4.11, the highest familiarity among the respondents was related to OP videos (92%; 191/207) and simulators (76%; 157/ 207). Familiarity with the “World Shared Practice Forum” feature of OP among the respondents was 73% (151/207), the third highest familiarity. Of note, only 28% (57/207) of the respondents expressed familiarity with Group features of OP, and only 11% (7/62) of respondents used OP groups (Table: 4.12).     60   Table 4.12 I have used a Group on OPENPediatrics. Answer Choices Responses  % Count No 88.7 55 Yes 11.3 7  Answered 62  Skipped 218  In fact, only six (out of nine) people reported joining an OP group and just one respondent (out of nine respondents) reported creating a group (Table: 4.13).   Table 4.13 Regarding OPENPediatrics Groups, which of the statements below apply to you? (Please select all that apply) Answer Choices Responses  % Count I have joined an OPENPediatrics Group. 66.7 6 I have browsed an OPENPediatrics Group. 22.2 2 I have created or been an administrator for an OPENPediatrics Group. 11.1 1  Answered 9  Skipped 273 Table 4.11 Distribution of respondents by familiarity with different features of OP (I am aware that OPENPediatrics offers the following features. Please select all that apply.) Answer Choices Responses  % Count Videos 92.3 191 Simulators 75.9 157 World Shared Practice Forum 73.0 151 Guided Learning Pathways 71.0 147 Geggel's Congenital Heart Disease Library 34.3 71 Groups 27.5 57  Answered 207  Skipped 73   61  Findings showed that three of the most common reasons for using different features of the OP platform are as follows (Table 4.14): • Learning more in-depth information about a topic using features that included World Share Practice forum (WSP) (73%; 69/95 respondents), videos (62%; 98/159 respondents), Guided Learning Pathway (GLP) (56%; 33/59 respondents), and simulators (52%; 32/62 respondents) • Learning the latest advances or development in the field using features that included World Share Practice forum (WSP) (62%; 59/95 respondents) and videos (56%; 89/159 respondents)  • Learning how to deliver safer or more effective patient care using features that included Guided Learning Pathway (GLP) (49%; 29/59 respondents), simulators (37%; 23/62 respondents), World Share Practice forum (36%; 34/95 respondents) and videos (33%; 52/159 respondents) (Table 4.14).     62  Table 4.15 What are common reasons for you to use OPENPediatrics? Please select up to THREE of the options below: Answer Choices Videos Simulators Guided Learning Pathway (GLP) World Share Practice (WSP)  % Count % Count % Count % Count Learn basic information about a topic 23.3 37 38.7 24 45.8 27 27.4 26 Learn more in-depth information about a topic 61.6 98 51.6 32 55.9 33 72.6 69 Learn how to solve a medical problem 14.5 23 24.2 15 25.4 15 11.6 11 Learn how to deliver safer or more effective patient care 32.7 52 37.1 23 49.2 29 35.8 34 Review things I already know 31.5 50 37.1 23 30.5 18 29.5 28 Learn the latest advances or developments in an area 56.0 89 11.3 7 23.7 14 62.1 59 I was assigned to watch the video by a supervisor 22.6 36 3.2 2 3.4 2 2.1 2 World Shared Practice Forum videos are part of the educational curriculum at my institution 1.9 3 29.0 18 18.6 11 6.3 6 Other (please specify) 22.0 35 1.6 1 3.4 2 2.1 2  Answered 159 Answered 62 Answered 59 Answered 95  Skipped 121 Skipped 218 Skipped 221 Skipped 185      63  About 31% (73/236) of respondents in total had visited OP more than 10 times (Table 4.15) and most of them (33%; 79/236) had visited OP on a monthly or weekly basis (Table 4.16).  Table 4.16 Approximately how many times in total have you visited OPENPediatrics? Answer Choices Responses  % Count Once 3.4 8 2-5 times 14.0 33 6-10 times 18.2 43 More than 10 times 30.9 73 More than 50 times 21.2 50 More than 100 times 8.9 21 Not sure 3.4 8  Answered 236  Skipped 44   Table 4.17 How frequently do you visit OPENPediatrics? Answer Choices Responses  % Count Daily 5.1 12 Weekly 31.4 74 Monthly 33.5 79 Occasionally (less than once a month) 23.7 56 Not sure 3.0 7 Other (please specify) 3.4 8  Answered 236  Skipped 44  As can be seen in Table 4.17, most of the respondents (85%; 182/214) agreed that the OP website is a user-friendly website.   64      Table 4.18 User-friendliness of OPENPediatrics (The OP website is easy to use.) Disagree (5.6%; 12/214) Neither agree nor disagree (9.4%; 20/214) Agree (85%; 182)   Strongly  disagree 1 2 3 Neither agree nor disagree 5 6 Strongly agree 7 Total Weighted  Average % Count  % Count  % Count  % Count  % Count  % Count  % Count    1.9 4 1.4 3 2.3 5 9.4 20 25.2 54 25.7 55 34.1 73 214 5.7              Answered 214              Skipped 66   65  Most of the respondents (80%) agreed that the OP website was visually appealing, except 3% (7/214) of the respondents which didn’t find OP a visually appealing (Table 4.18). Table 4.19 Distribution of the participants by their perception about appearance of the OP website (The OPENPediatrics website is visually appealing.)  Disagree (3.3; 7/214) Neither agree nor disagree 16.8 (36/214) Agree (79.9; 171/214)    Strongly disagree 1 2 3 Neither agree nor disagree 5 6 Strongly agree 7 Total Weighted Average  % Count  % Count  % Count  % Count  % Count  % Count  % Count     1.4 3 0.9 2 0.9 2 16.8 36 20.6 44 29.9 64 29.4 63 214 5.6               Answered 214               Skipped 66   In general, a large majority of the respondents (87%; 227/262) agreed that the OP website is a useful resource (Table 19). Of those, 68% (179/ 262) strongly agreed with the usefulness of the OP website as a resource. In terms of the relevance of the OP website, 83% (217/ 262) of the respondents found the website and contents of the website relevant to their learning needs. Of these, more than half of the respondents (58%; 152/ 262) strongly agreed with the relevancy of the OP website resources. There was high agreement among the respondents (86%; 224/262) that the website reflected current best knowledge and evidence. As a result, most of the respondents (88%; 230/ 262) indicated they were planning to use the OP website in the future. A large majority of the respondents (86%; 225/262) indicated that they would recommend OP to their peers (Table 4.19).     66  Table 4.20 Perceptions of the participants about the OP website  Disagree Neither agree nor disagree Agree      Strongly disagree 1 2 3 Neither agree nor disagree 5 6 Strongly agree 7 Not applicable/ I don't know Total Weighted Average  % Count % Count % Count % Count % Count % Count % Count     Overall, I consider OPENPediatrics to be a useful resource. 3.8 10 1.2 3 1.5 4 3.1 8 5.0 13 13.4 35 68.3 179 3.8 10 262 6.3 The resources on OPENPediatrics are relevant to my learning needs. 3.4 9 1.5 4 2.3 6 4.6 12 8.0 21 16.8 44 58.0 152 5.3 14 262 6.11 The resources on OPENPediatrics are up-to-date and reflect current best knowledge and evidence. 3.4% 9 0.8 2 1.9 5 3.8 10 6.1 16 19.1 50 60.3 158 4.6 12 262 6.22 I plan to use OPENPediatrics in the future. 3.1 8 1.9 5 1.9 5 2.7 7 5.0 13 12.2 32 70.6 185 2.7 7 262 6.33 I would recommend OPENPediatrics to my peers. 2.7 7 1.9 5 2.3 6 4.6 12 2.7 7 12.6 33 70.6 185 2.7 7 262 6.32                 Answered 262                 Skipped 18    67  As can be seen in Table 4.20, most of the respondents (76.6%; 199/260) agreed that OP had a positive impact on their clinical practice. Of those, (46%; 119/260) strongly agreed with the positive impact of the OP platform on their clinical practice.  Table 4.21 Positive impact of OP on clinical practice Disagree (5.4%; 14/260) Neither agree nor disagree (12.7%; 33/260) Agree (76.6; 199/260)     Strongly disagree 1 2 3 Neither agree nor disagree 5 6 Strongly agree 1 Not applicable/ I don't know Total Weighted Average % Count  % Count  % Count  % Count  % Count  % Count  % Count      2.7 7 1.9 5 0.8 2 12.7 33 8.1 21 22.7 59 45.8 119 5.4 14 260 5.57                Answered 260                Skipped 20  In terms of the different educational features of the OP website and their criteria respondents shared their perceptions as follows: Videos:  A large majority of respondents (91%; 145/159) agreed that the videos are relevant to their clinical practice and 95% of the respondents (151/159) agreed that the videos reflected current best knowledge and evidence. A vast majority of the respondents (91%; 145/ 159) also found the videos engaging and most of them (96%; 152/159) intended to watch more OP videos in the future (Table 4.21).      68  Table 4.22 Perceptions of participants about videos on the OP website (The following questions relate to how you feel about OPENPediatrics videos.)  Disagree Neither agree nor disagree Agree    Strongly disagree 1 2 3 Neither agree nor disagree 5 6 Strongly agree 7 Total Weighted Average  % Count  % Count  % Count  % Count  % Count  % Count  % Count    OPENPediatrics videos are relevant to my clinical practice. 1.9 3 0.0 0 1.9 3 5.0 8 13.8 22 30.8 49 46.5 74 159 6.08 OPENPediatrics videos are up to date and reflect current best knowledge and evidence. 1.3 2 0.0 0 1.3 2 2.5 4 12.6 20 32.1 51 50.3 80 159 6.23 I find OPENPediatrics videos to be engaging. 1.9% 3 0.0 0 0.0 0 6.9 11 17.6 28 31.5 50 42.1 67 159 6.01 I find the length of OPENPediatrics videos to be just right. 1.9 3 0.6 1 2.5 4 6.3 10 18.2 29 36.5 58 34.0 54 159 5.84 I will watch OPENPediatrics videos in the future. 1.3 2 1.3 2 0.0 0 1.9 3 9.4 15 27.0 43 59.1 94 159 6.35               Answered 159               Skipped 121   69  In terms of World Shared Practice Forum(WSP) videos, 35% of the respondents (33/95) reported watching these videos as a group (Table 4.22).  Table 4.23 Do you ever watch OPENPediatrics World Shared Practice Forum videos as part of a group? Answer Choices Responses  % Count Yes 34.7 33 No 65.3 62  Answered 95  Skipped 185  As demonstrated (Table 4.23), a very strong majority of respondents (93.6%; 89/ 95) agreed that OPENPediatrics World Shared Practice Forum videos are relevant to their clinical practice. Also, most of the respondents (95.8%; 91/95) agreed that these videos are up to date and reflect current best knowledge and evidence and most of the respondents (92.6%; 88/95) indicated that they will watch OPENPediatrics World Shared Practice Forum videos in the future.    70   Table 4.24 Distribution of the participants by their perception about the World Shared Practice Forum videos.  Disagree Neither agree nor disagree Agree    Strongly disagree 1 2 3 Neither agree nor disagree 5 6 Strongly agree 7 Total Weighted Average  % Count  % Count  % Count  % Count  % Count  % Count  % Count     OPENPediatrics World Shared Practice Forum videos are relevant to my clinical practice. 0.0 0 1.1 1 0.0 0 5. 5 16.8 16 32.6 31 44.2 42 95 6.13 OPENPediatrics World Shared Practice Forum videos are up to date and reflect current best knowledge and evidence. 0.0 0 0.0 0 0.0 0 4.2 4 13.7 13 31.6 30 50.5 48 95 6.28 I will watch OPENPediatrics World Shared Practice Forum videos in the future. 1.1 1 0.0 0 0.0 0 6.3 6 6.3 6 30.5 29 55.8 53 95 6.32                Answered 95                Skipped 185     71  More than 97% of the respondents (57/59) agreed that Guided Learning Pathways (GLPs) of the OP website are relevant to their clinical practice (Table 4.24). Additionally, most of the respondents (93%; 55/59) agreed that GLPs were up to date and reflected current best knowledge and evidence. Ninety-five percent of the respondents, therefore, plan to use more guided learning pathways in the future.    72   Table 4.25 Distribution of participants by their perceptions about OP Guided Learning Pathways (The following questions relate to how you feel about OPENPediatrics Guided Learning Pathways.)  Strongly disagree 1 2 3 Neither agree nor disagree 5 6 Strongly agree 7 Total Weighted Average  % Count  % Count  % Count  % Count  % Count  % Count  % Count    OPENPediatrics Guided Learning Pathways are relevant to my clinical practice. 1.7 1 0.0 0 0.0 0 1.7 1 6.8 4 25.4 15 64.4 38 59 6.46 OPENPediatrics Guided Learning Pathways are up to date and reflect current best knowledge and evidence. 1.7 1 0.0 0 0.0 0 5.1 3 8.5 5 18.6 11 66.1 39 59 6.39 I will use OPENPediatrics Guided Learning Pathways in the future. 1.7 1 0.0 0 0.0 0 3.4 2 3.4 2 20.3 12 71.2 42 59 6.53                Answered 59                Skipped 221    73  Table 4.25 reveals the most frequent reason for not completing an OP Guided Learning Pathway among respondents to this question, which is lack of enough time (79%; 33/42). Table 4.26 Why did you not complete an OPENPediatrics Guided Learning Pathway? Select all that apply: Answer Choices Responses  % Count  Not enough time 78.60 33 I am still working on the Guided Learning Pathway and intend to complete it 19.10 8 I ran into technical barriers when using the Guided Learning Pathway 16.70 7 Other (please specify) 7.10 3 I did not find the content engaging 4.80 2 I was not required to complete the Guided Learning Pathway by my supervisor 4.80 2 I did not find the content relevant to my practice 2.40 1  Answered 42  Skipped 238  Of the three simulators available on the OP website, the Mechanical Ventilation Simulator (76%; 47/62) was the feature that most of the respondents had used in the last three months (Table 4.26).  Table 4.27 Which of the OPENPediatrics simulators listed below have you used in the last three months? Answer Choices Responses  % Count  Mechanical Ventilation Simulator 75.80% 47 Peritoneal Dialysis Simulator 21.00% 13 None 12.90% 8 Hemodialysis Simulator 12.90% 8  Answered 62  Skipped 218    74  As it has been pointed out in Table 4.27, a vast majority of the respondents agreed with the relevance of the OP simulators to their clinical practice (92%; 57/62). Less than 2% (1/62) of the respondents disagreed that OP simulators were relevant to practice and up-to-date while 94% (58/62) of the respondents agreed that the simulators are up to date and reflect current best knowledge and evidence. Interestingly, 85% of the respondents (53/62) agreed that simulators enable them to better perform key tasks (improve quality of the care they provide).  Ninety percent of the respondents (56/62) indicated that they intended to use OP simulators in the future.      75    Table 4.28 Distribution of the participants by their perception about OP simulations (The following questions relate to how you feel about OPENPediatrics simulators.)  Strongly disagree 1 2 3 Neither agree nor disagree 5 6 Strongly agree 7 Total Weighted Average  % Count  % Count  % Count  % Count  % Count  % Count  % Count    The OPENPediatrics simulators are relevant to my clinical practice. 0.0 0 0.0 0 1.6 1 6.5 4 22.6 14 22.6 14 46.8 29 62 6.06 The OPENPediatrics simulators are up to date and reflect current best knowledge and evidence. 0.0 0 1.6 1 0.0 0 4.8 3 25.8 16 24.2 15 43.6 27 62 6.02 The OPENPediatrics simulators enable me to better perform key tasks. 0.0 0 3.2 2 1.6 1 9.7 6 21.0 13 22.6 14 41.9 26 62 5.84 I will use the OPENPediatrics simulators in the future. 0.0 0 3.2 2 0.0 0 6.5 4 19.4 12 21.0 13 50.0 31 62 6.05                Answered 62                Skipped 218      76  With regard to barriers, around 29% (18/ 62) of the respondents indicated that the simulator activities take too long to complete. In terms of technical barriers to access OP, while most of the respondents (56%; 123/218) reported not having technical issues in accessing OP, the rest reported various technical issues. The most frequent technical issues addressed in accessing OP among the respondents were problems with an Internet connection (20%; 43/218) and limited access to network (16%; 34/218) (Table 4.28). Table 4.29 What technical barriers, if any, do you face in accessing the platform? Answer Choices Responses  % Count  I have not encountered any technical problems using OPENPediatrics 56.4 123 Problems with Internet connection 19.7 43 Limited internet access 15.6 34 I do not want to use data on my cellular phone plan 10.6 23 Lack of adequate computers or other devices 6.4 14 OPENPediatrics is not compatible with my device(s) 4.6 10 Other 7.8 17  Answered 218  Participants were also asked about those features of OP that improve their performance. In response to this question (Table 4.30), downloadable materials were considered to be the most important feature for the respondents to this question (85%; 196/232). Offline browsing capability was considered to be the second most important feature among the respondents (68%; 155/232). Sixty-five percent of respondents (147/ 228) considered a standalone mobile app to be an important feature.   77     Table 4.29 Please rate the importance of the following items in terms of improving your OPENPediatrics experience.  Not at all important 1 2 3 Neither important nor unimportant 5 6 Very important 7 Total Weighted Average  % Count  % Count  % Count  % Count  % Count  % Count  % Count    Offline browsing capability 5.2 12 3.1 7 0.9 2 23.1 53 15.3 35 18.3 42 34.1 78 229 4.97 Downloadable materials 0.9 2 1.3 3 0.9 2 11.7 27 14.4 33 20.9 48 50.0 115 230 5.5 Standalone mobile app 4.4 10 1.8 4 2.2 5 27.2 62 14.5 33 18.4 42 31.6 72 228 4.96              Answered 232              Skipped 48     78  4.1.3. Open-ended questions    Open-ended questions revealed interesting findings which could be categorized under six major themes: (1) Community of Practice among pediatricians, (2) the most important/popular/or helpful features, (3) meeting users’ expectations of op/adult education, (4) advantages of op, (5) barriers to use and problems with op (6) suggestions and solutions to problems. These themes have been illustrated in Table 4.30 and will be described briefly in the following sections.  Table 4. 30 Themes surfacing from the open-ended question Theme Examples of answers to open-ended question 1. OP, Community of Practice among pediatricians • Appears to be highest level of knowledge being shared  • The opportunity to find answers to my questions • The open-minded approach to teach and ask every day practice issues • Sharing a knowledge and to stay up to date • The use of experts in the relevant fields in presentation of topical issues. • That information is shared from multiple paediatric centres, different countries and in such a professional way  • The expert practitioners sharing their knowledge. • The speakers are renowned 2. The most important/popular/or helpful features of OP • World share practice • Simulators videos • World expert discussions- WSP • Videos by world experts • Up to date top rated lectures • Videos, simulations, global activities      79  Table 4. 31 Themes surfacing from the open-ended question (Cont.) Theme Examples of answers to open-ended question 3. Meeting users’ expectations by OP • Wide coverage of topics  • Variety of topics • Expert speakers • Loading resources specific to paediatrics • [Including] cardiac defect presentations as it pertains to my field relevant • To get access to teaching resources 4. Advantages of OP • High quality updated open access educational material of practical use  • Great free and open-access medical resource for all levels of training  • Ease of use • Open to everybody everywhere • Interactive tools • The speakers are renowned • The varied approaches to content 5. Barriers/ problems of OP mentioned by users • Not all topics I would like to see are included • Internet connection problems  • Not convenient for slow internet connection  • A bit 'Americanised' at times • Language barriers: three quotes from three different users included: When I don't understand because my English its not so good; Many people in my country use Korean, not English; Not available in French • I wish I knew there was so much more to explore • It takes longer than I like to get to the lessons I’m looking for • It needs high speed Internet and for many lectures only audio is available… It is good experience if we have videos. 6. Suggestions/ solutions mentioned for problems by users • Needs to update posted guidelines • Lack of education for neonatal practices • More neonatology topics • Collaboration to reduce Newborn mortality • Case reviews for CME would be great • Cardio-thoracic surgery in real time • It would be nice if you have some written down statements/guidelines with the video • Would love to see modules that would be suitable for just-in- time learning for both procedures and for post-op cardiac patients.  • Would be willing to help with them if that is needed   80 CoP among pediatricians  On reviewing answers received for the open-ended questions, it became apparent that the OP platform could play a significant positive role in creating a CoP among health care professionals. Access to pediatrics experts and speakers who are well known in the field of pediatrics can be regarded as a core group for the OP community. The core group produces and shares explicit and/or tacit knowledge, especially with novice members. There are also members who connect to the community to use the knowledge provided and apply it in their daily practice. Respondents mentioned that they found it interesting sharing knowledge across different countries in a professional way.  One of the major themes that appeared among the answers to open-ended questions is the respondents’ perception about the most important, popular and helpful features of OP. They mentioned that they enjoy joining the World Share Practice Forum. They appreciate the Simulators for their ability in facilitating participants’ learning and improving their practice. Videos and experts’ lectures were considered very helpful, and in general they enjoyed global aspect of the OP platform. Meeting users’ expectations by OP: adult education As educated adult learners, users of OP have high expectations. A review of the results of the open-ended questions showed that respondents value the updated educational recourses posted to OP website. They like the educational content, which is free, open access, up-to-date and expert reviewed. The platform meets users’ expectations as adult learners in terms of access to interactive and user-friendly tools from any location at any time. Simulations and videos meet users’ needs with regard to hands-on learning, which is an important principle of adult education and fundamental for an online CoP. Users also found the material relevant to their previous and   81  current knowledge and helpful for their daily practice. Clear and easy to understand information and the variety of topics met their educational expectations. Advantages of OP In answering the open-ended questions, respondents pointed out some interesting advantages in regard to the OP platform. They found OP to be a very well developed educational platform with high quality materials and a variety of educational approaches to its contents. They like the capability of the platform for downloading the videos, illustrative graphic materials and the guides for action with critically ill children. They enjoy learning via watching experts’ interviews and like being able to select what they want to view. Accessibility of the platform and possibility of access to experts were also considered of important advantages. Respondents found the content very useful, comprehensive and easy to understand. Open access and being free of charge are also features appreciated by the users. Barriers and challenges of using OP platform (users’ perspective) Having said about many advantages accounted for OP, it became apparent that there are some issues as well. For example, Internet access is “a requirement” for use of this platform. This would be a problem in areas with less reliable Internet access or lower Internet speeds. High-speed Internet is needed to load the videos. With lower Internet speeds, users were able to access only to the audio part of the audiovisual materials. Reliable Internet connections are problematic in some areas. The platform logs out after a few hours, which is a big concern especially during the guided learning pathway courses. It cannot progress without answering questions in Q& A's session whereas some users would like to skip that section. Respondents also found it challenging and even annoying that they have to go through several chapters before they could have access the specific topic they are interested in. They think there is unnecessary   82  repetition in some of the sessions. Forgetting passwords is also an issue as there are no "remember me" or "keep me logged in" options. Members therefore have to type their id & password every time they are logging in, even from their personal computer.  For some members, downloading lessons or videos to watch them offline is difficult and waiting time for videos to be loaded, especially in remote countries, is a big challenge. Lack of clear guidance for quick access to the educational materials was also perceived frustrating. Users were not able to save search results and each time they have to start all over again. Some participants in the survey mentioned that OP is limited and does not cover all needed and foundational topics. They have found some topics too general while the expectation is more in-depth presentation of these topics. Time is another issue for busy pediatricians as they claim that the educational materials take a lot of time to be completed. Non-English speakers understandably found language to be another barrier that limits the proper use of OP.  Participants in the survey have mentioned that the Guided Learning Pathway lack the functionality of skipping a section, and it makes them start all over again each time. They asserted that OP lacks in inviting variety of speakers and that usually have the same people as speakers. They believe that it is very easy to get overwhelmed by the volume of information.  They believe an iPad app for the program. Could be very helpful.  They would like to see their name on their computer screen after obtaining the GOLD star for completing the ventilator simulation.  Results of open-ended questions also indicated that users do not have enough information about different aspects of OP. Some of participants believed that “the platform is a bit 'Americanized' at times” meaning that content is more relevant to North America situation and does not reflect or cover poorer countries educational needs.   83 Suggestions and solutions suggested by users When asked what they preferred to see on OP, respondents listed following suggestions: • Adding more simulation on other topics such as ventricular assistive devices, pulmonary hypertension, pulmonary vein stenosis, heart transplant, Extracorporeal Membrane Oxygenation (ECMO) monitoring simulator, neuro-critical monitoring, education for neonatal practices, radiology database, toxicology and more neonatology and pediatric cardiology topics.  • Updating clinical practice guidelines on an ongoing basis, • Offering free Continuing Medical Education (CME), • Providing an iPad app, • Adding case reviews for CME, • Providing trauma hours (counting trauma educations as training hours), • Adding educational series to address the medically fragile population, • Clearly indicating how to communicate with providers of OP platform,   • Helping clinicians to better understand the choices families make with regard to palliative care, • Helping clinicians with the Medical Orders for Scope of Treatment (MOST), • Collaborating to reduce Newborn mortality, • Adding Cardio-thoracic surgery in real time, • Providing easy access to quality information,  • Shortening the World Shared Practice videos,  • Adding some written statements/guidelines to the videos,   84  • Adding clear guidance to access different features,  • Adding modules suitable for just-in-time learning,  • Adding modules for post-op cardiac patients, • Adding educational materials from other countries such as Latin-America,  4.2. Part two: Qualitative findings Six qualitative questions were discussed in focus groups and interviews, which were recorded and transcribed. All transcripts were read once very carefully to get a general idea about the participants’ opinions regarding OP. Then the data was analyzed using ATLAS Ti software (Thomas, 2017). Data were coded to find the main themes.  In the following sections, I will present findings of qualitative part of the study. These findings are based on thematic analysis of focus group and individual interviews questions that were asked from 8 participants. These findings have been grouped in 10 major categories as follow:  4.2.1. Using OP (time, place and device)  Not surprisingly, users connect to the OP platform in various ways and at different times of the day. They use different types of devices such as PCs, laptops, iPads, and smart phones to connect the OP platform. They usually use OP at home in their free time, and some participants also use it at work when the workload is not high (for example during night shifts). Some participants use the platform as a group learning activity at work. For this purpose, they play the videos and mix it with group discussion.   “… we have used the Open Pediatrics platform for groups of our trainees, and some of the stuff that we do, from time to time is to use, the Open Pediatrics material, and what we’ll have is an attending or consultant, [as] part of the group, and what we do is, we run the material, stop to discuss it, and go through the process of well, so that’s what they’re talking about, in North America…”  P 7: interview 6.rtf - 7:6 (33:39)   85   In some hospitals, OP is being used as a teaching tool for residents. In this case the professor asks residents to watch, listen or read a topic and discuss it in the group. 4.2.2. Recent version of OP vs first version Participants were very satisfied with the recently revised version of OP compared to the previous version. They indicated various reasons for this satisfaction. For example, they believed that the previous version mainly focused on critical care with less videos, unorganized and scattered, had low quality material (specially animations) and less user friendly. However, the recent version offers greater variety of content (such as neurology) with more videos rather than focusing on critical care, covers more areas, is very organized and easy to follow, has high quality materials (including animations), is very user friendly and in fact has much more educational potential. In so doing, the recent version, participants believed that, addresses most of the weaknesses found in the first version: “They changed the interface, from what it used to be. It’s been, I think it’s a lot more   user-friendly now and it put up a lot more videos. So, the potential is definitely there.”  P 3: interview 2.rtf - 3:12 (33:33)   Although almost all participants showed their interest and acknowledged improvements found in the recent version of OP platform, some believed that still it does not cover all subjects and it cannot be considered as primary resource, rather it could be a complementary resource to textbooks and traditional sources of information. They found textbooks more organized and validated as primary resource compared to OP.  4.2.3. CoP among pediatricians and health care professionals As a result of the focus groups and interviews it became apparent that OPENPediatrics has the potential to be regarded as a typical community of practice as Lave and Winger have   86  described (Wenger et al., 2002a).  In other words, participants believed that OP provides an opportunity for groups of pediatricians, nurses and other health care professionals who connect to the platform from all around the world to benefit from the knowledge and experience and resources (such as clinical practice, guidelines and tools such as sepsis screening tools) provided in the community by leaders, core group and knowledge producers. They believed that the OP platform is useful for all disciplines (such as physicians, nurses and allied health) and has something to offer to all health care professionals:   “P: I think it’s very useful, relevant and it’s up to date, yeah.  And it’s not just, catering to physicians. It’s catering to nursing. You can have pediatric ICU resident was rotating, plus physiotherapy. So, I think it’s cater to everybody, like. There’s something in it for everyone, who’s working in pediatric ICU environment.” P 3: Interview 2.rtf - 3:20 (115:121)  Participants showed their great interest in being part of the community and the way that the community connects them to a wide range of audiences. Participants like the opportunity that the platform provides for learning about others’ experiences in the same field. They looked at this as an opportunity for learning what is happening in other parts of the world in their field and as an opportunity for increasing their knowledge by interacting with others via forums:   “Um, I think there’s, it’s also great that you have the opportunity to comment, on what you’re seeing, what you’re doing and to kind of get the experience from your own center. Because that gives you a way of, you know beyond the person that’s speaking what else people are doing around the world…” P 4: interview 3.rtf - 4:18 (115:117)   Knowledge gained through OP is also being shared with colleagues in their units and participants believe that the OP platform considers the principles of adult learning and education and meets their educational needs as adult learners: P: “…Um, so you know, for myself, I need to involve, at least two of my senses um, in order to retain information that I’m being given and, and so I like for instance the feature that, you have the video and the text is next to it. And, they highlight as they’re, as they’re talking   87  where they are, so that you can both read and listen at the same time. Um so I find that use of, the ability to use multiple senses very, works for me. It helps me be able to retain the information...” P 6: interview 5.rtf - 6: (177:179)  They watch the videos, simulators and lectures, think about them, and plan to practice them in their workplace s (Kolb’s learning cycle) (Wolbrink & Burns, 2012). Connecting to OP perceived to be a great factor in users’ professional development. CME, for instance, is one of those opportunities, which helps members increase their competencies:     “[They are from] North America, it’s really, useful to have, to see all these, you know celebrities, on these um forums, and uh hear their thoughts. And, and not only to theoretical background, but also what they, what they use in their own practice, and in their hospitals where they work. So, it’s, it’s a really useful tool for me. I’m really, I’m really happy it’s out there.” P 6: interview 5.rtf - 6:46 (195:195) Opportunities to access leaders and experts in the field was also noted as a very important feature of OP and as a typical CoP, that could help them to stay on top of the most recent advancements in knowledge of their own field:  “So, these are all people with immense experience and they actually tell you how things have changed over time. And, what’s happening right now. So, I think it’s, it’s a very, very uh good, summative uh you know summative knowledge that they give you.” P 3: interview 2.rtf - 3:6 (21:21)     4.2.4. Meeting expectations To participants, OP meets their expectations as an online educational platform. It is a learning and teaching resource that covers most of the educational topics they need. Topics offered by OP are viewed as relevant, current and useful. Users like the quality of the videos, especially in the recent version. They can connect to OP and use it easily. The platform therefore meets their expectation as both learners and educators so that they are impressed about this feature of the platform:   88  “I’m very impressed about how, how they try to talk about all the, the points of view of, of a critical patient. You know, family things, [inaudible] things, pain, I, I think we, we as a group, here in my hospital, personally are very impressed” P 5: h interview 4.rtf - 5:40 (355:357)  They believe that all educational materials are relevant, useful and timely:  “I, yeah, um, so um, I, find them fairly timely and I tend to go into something that I’m involved with at the time” P 6: interview 5.rtf - 6:11 (65:67)     4.2.5. Advantages of OP Participants indicated various advantages of the OP platform that included resource validity, using other languages, meeting educational needs, meeting adult education principles and connectivity whenever/ wherever. In the following sections, I will briefly describe each of these advantages. Resource validity It was interesting to hear how important learning resources are for the members. They considered resource validity as an important advantage for the OP platform. They perceived OP and its materials credible and trustworthy. This way, they respect Boston Children’s Hospital very much and enjoy learning from this resource.  “I think you know, it’s coming from Boston Children’s, at least when you are in pediatrics in the US, and everybody knows that it’s one, one of the, among the top five, children’s centres in the country. And so, I would look to them for authoritative, work that, I would be willing to use it, to use as, to incorporate into my own practice and, also a lot of the names, of the people that are doing it are well known in the, you know, pediatric critical care field. So, so, they’re people that you recognize and you wanna hear them speak.  Um, and I think it’s great that they got the, the true people who are, who are major names in the field to be able to, share their thoughts.” P 6: interview 5.rtf - 6:7 (37:37)    89  They like to listen to the well-known and credible people in their field as instructors and lecturers. They called these people as “celebrities of their [academic] world”, people whose publications have been studied by the participants. In fact, participants preferred validity [reputations] of the celebrities over validity of the content:      “I think uh, I prefer the names over topics. If it’s a good speaker then uh, almost any topic is good. Some, and as I… said, some of them, you know these are, celebrities of our world, that have specialty and, it’s always very interesting to watch them, you know them from book chapters. You know them from, from the papers. You know them from the conferences and it’s very, interesting to, to hear their thoughts. I almost prefer, the people over the topic”. P 6: interview 5.rtf - 6:12 (73:75)  Participants also expressed that “hearing about the history of pediatrics from people [who] were in this field for a longer time and with more knowledge and experiences is very nice and helpful”: “they bring in experts from their field. And they’re giving you know, an up to date summary of what, what’s been going on in that field. So I think that’s very, very, helpful like, so. For example, for a person like me who has only been in this field for the last three or four years, it’s nice to hear about the history of how things started out, and how it came to what’s being done today.” P 3: interview 2.rtf - 3:22 (145:153).” Using other languages Participants appreciate that OP is offered in other languages. There was one Spanish, one French and one Latvian speaker among focus group and interview participants. They pointed out that having more language options makes OP even more useful for a broader range of users worldwide although they acknowledged that nowadays more people know English:  “We have some co-workers that have kind of problems with, the, English and very happy to have a Spanish version.” P 5: Interview 4.rtf - 5:27 (259:265)       90 Meeting educational needs I was curious to hear participants’ opinions on the educational materials offered by OP. Based on our focus group discussion and interviews, members are happy with the educational coverage by OP. They all agree that the materials are simple, concise and clear. They are more focused and more specialized in pediatrics and as one of them mentioned:   “…What they’re gearing this towards, because they’re nice and, simple and direct. And, um, helpful.” P 6: Interview 5.rtf - 6:48 (197:197)  Participants in the focus groups and interviews believe that the educational materials are regularly updated. They like simulation-based videos, as well as the World Share Practice Forum and the curricula-based content. In fact, all participants in the focus groups and interviews agree that OP platform is a very well-developed website and could be regarded as a good educational asset for post graduate studies such as medical residency and fellowship programs specially in developing countries:    “So, the fact is the site would be a big asset for any residency and fellowship program, and for the people in developing countries. Where they want to kind of see how the things work.” P 5: interview 4.rtf - 5:11 (119:123). Meeting adult education principles The blended methods of teaching as a suitable form of teaching adult learners were considered as a good feature of OP. Participants in the focus groups and interviews found the blended and multi method educational approaches used by OP very helpful in their learning and transferring this learning into practice in their workplaces (Kolb’s learning cycle). Users of the OP believed that variety of educational methods used in the OP platform, such as videos, simulations, lectures and so forth, plays strong role in scaffolding and facilitating their learning and practicing their learning.    91 Connectivity whenever/wherever Participants were very happy with OP’s connectivity and its accessibility. They found the platform and its contents very user-friendly and easy to reach. They found the possibility of reviewing the World Share Practice Forum in their leisure time useful and interesting. As busy people, they enjoyed the convenience of being able to use the platform at their home and at work during their free time or less busy times (such as during night shifts).  4.2.6. Barriers/ problems of OP mentioned by users Despite all the benefits that OP has provided to the field of pediatrics, like any other educational program, participants pointed out some barriers to the use of the platform. These barriers included lack of awareness about the OP platform, lack of community networking, lack of English language skills (among the users), lack of proper use of equipment, difference between countries, lack of organization in website materials, older generation versus new technology, and finally existence of many other competing learning resources. In the following sections, I will briefly describe each of these perceived barriers. Lack of awareness about the OP platform The first and most important barrier surfaced was lack of awareness about the platform and its various features. As participants mentioned, many people in the field of pediatrics don’t even know that the OP platform exists. Most of them had no idea about the platform until their colleagues introduced them to it or they found it through a search engine. Additionally, among those who are aware of the program, many do not know about the different features that OP offers. In other words, many of the pediatricians, nurses and health care professionals either are not aware of the existence of the OP platform or they do not have enough information about variety of the features that the platform offers. For example, OP is not very well known among   92  nephrologists and participants barely knew about the OP newsletter. Participants believed that lack of enough knowledge about the platform might lead to under-utilization of such a great resource:  “Yeah, so, I feel as a tool probably it’s being underutilized…” P 3: interview 2.rtf - 3:10 (33:33) Lack of community networking One of the important features of any CoP is social interaction among members and between members and core group. Creating or joining groups that members are interested in, also is another important aspect of CoP (Jiménez-zarco et al., 2015).  Although OP is doing an excellent job in supporting the learning by a group of educational leaders (core group), it is not as effective in connecting members as it could be.  OP members rarely join groups so there are fewer group activities than what is expected in an active CoP:   “I haven’t joined a group. No.” 4: Interview 3.rtf - 4:25 (163:169)    While most of the members would like to join some groups, get information or share their experiences, they are not aware of the process of navigating the platform, joining to different groups offered by OP, process of engagement, sharing their knowledge and participation in content development:  P2: Yes. I want to join mainly with the pediatric nephrologists. But, [pause]  Yeah, I didn’t have [to] know how to access and navigate your website, but if you can help me in sending the information how I can, get involved in that, and then I can add a lot of simulation, or can give some idea. I can give some ideas, even like how to do the, um, access evaluation and all those modules. So that can help a lot.” P 5: interview 4.rtf - 5:20 (181:187     93  Group interactions are, therefore, very weak as participants either do not know about the groups or have joined in some groups when creating their profile, however, they have not used it for interacting with others via OP afterwards: “I’ve subscribed to some groups, but I haven’t used it so much, like in terms of networking”. P 3: interview 2.rtf - 3:7 (29:29)     Since there is no proper link between members or between members and leaders, they sometimes find it easier to ask their questions or share their information with their colleagues or mentors at their local hospitals, rather than communicating with others via OP.  Considering the definition of the CoP by Lave and Wenger, “…a group of people who share a concern or a passion for something they do and learn how to do it better as they interact regularly” (Ranmuthugala, Plumb, et al., 2011), it appears that OP is weak in this regard. One of the participants, for instance, mentioned that there is no real connection between nephrologists via OP even though they are members of a big nephrologists’ community within their country. Lack of English language skills Lack of English language skills among non-English speakers was found to be another important barrier for using the platform. For people with English as their second language or for those with less English skill, OP is not as beneficial as it is for English speakers. Non-English- speaking users apply different strategies to get as much as they can out of OP. For example, in Latvia, they are trying to translate materials into Latvian.            “Um, as I said the, we are working on translating them in Latvian…”  P 6: interview 5.rtf - 6:18 (83:83)     One of the participants in the focus group mentioned that her colleagues usually use Google Translate in order to learn from the OP platform which may is not an ideal tool for   94  translation. Another participant pointed out that in her country her colleagues listen to the videos multiple times in order to understand the topic.  “Uh, no I don’t [have language barriers], but I have some co-workers, that they have kind of problems with, with this. That’s why, they used a translator maybe, but they tell me that they had to listen, several times the videos to understand, to try to understand. Because maybe they speak just a little of English, or, or they have some problems with the language.” P 5: interview 4.rtf - 5:28 (271:273)     Some participants found lip reading very helpful in understanding the material and overcoming the language barriers. For example, one participant said that she needs to see faces clearly in the videos in order to recognize words and understand the topics they are discussing:  “English is not my first language. Sometimes if I can’t see the faces really well it’s hard for me to follow the English. So that would be one of my suggestions.  Um, I have to kind of go back and re-listen sometimes, where they speak too fast, or I couldn’t see their faces and it was kind of harder to understand. So I would have to rewind a lot.” P 4: interview 3.rtf - 4:13 (65:65) Lack of proper use of equipment The issue associated with educational materials such as simulations and videos. In poorer settings such as developing countries, poor maintenance of high tech equipment (for example ventilators) precludes optimal use of these equipment as advised in the simulations and videos in OP:   “For example, in one country “a hospital got four ventilators donated from outside but they didn’t have water heating humidifiers on those ventilators. And because it is quite expensive to run humidifiers, and, the components of it, so they’re running them dry, which has consequences in terms of tubes blocking. But and they don’t have heat moisture exchanges, which might be an alternative. And then, you actually start scratching, and you say, well, so when this patient comes off the ventilator, what happens to this ventilator? And they say, well, you know, we, we use it on the next patient. And you say, but, who changes the filters? Who calibrates the machine? Who does the maintenance? Do you know when the maintenance was done? Is there budget for that? And suddenly people are saying, well, no we’re not sure. We don’t, really know how that, works.” P 7: Interview 6.rtf - 7:1 (11:15)    95  Lack of proper use of equipment was also associated with lack of enough knowledge and skills of trouble shooting the equipment. In other words, users of OP either use the equipment inappropriately or can’t manage if a problem develops. There is normally no special technician for the equipment and those who use the equipment do not know how to troubleshoot: “And so, for, often, for people in those, in poorer settings, they have to do it themselves. And they have to be able to troubleshoot, what’s going wrong on the circuit.” P 7: Interview 6.rtf - 7:1 (11:15)   Another issue with regard to medical equipment is that equipment recommended or used in the OP platform are not normally available in all hospitals especially in remote areas or small cities. One of participants shared this during group discussion that in her country private hospitals own equipment but not all hospitals. She said: “we usually refer our patients to those hospitals that own the equipment when the patient need those.” Difference between countries Another barrier indicated in interviews was about differences between the countries (contextual differences). Interviewees believe that there are huge differences between countries around the world especially between the first world and poor countries in terms of work styles and cultural issues that makes using OP either difficult or less valuable. They believed that work styles that suite to first world countries may not be the best option(s) for other parts of the world that may have restricted financial resources, different priorities and different culture compared to developed countries:  “Because often the educational material that they’re getting, is coming from the big textbooks that people are writing from it. But there needs to offset it and say, well, in different parts of the world, this is different. Your priorities are different. The investigations you’re gonna do are different. And, and people need to get that confidence of saying, okay, so I don’t have to believe the text book. I need to collect my own data,   96  and then decide what, what actually the priority is and, and maybe what the experts do, isn’t right for where I am. And some of that would be really exciting to explore” P 7: interview 6.rtf - 7:24 (84:84)   Participants believed that culture and religion play important roles in poorer countries, which lead people to act differently and adapt different styles. Additionally, poorer countries are only able to deal with day-to-day issues. They have many issues to consider in their daily work in order to be able to work in those situations. Participants, believed that consideration of the realities and acknowledging the difference and variation between countries that produce OP material and those countries that are users of the OP platform would improve application of the recommendations of the materials included in the OP website: “The reality is that if you are in poorer areas, you’re not trying to run modern state of the art intensive care. You’re trying to deal with things, at a very different level. And often, you know it’s not about the high tech, it’s about getting, an appropriate dose of an appropriate antibiotic, into the patient early, finding a pragmatic way of controlling reasonable fluid balance when you don’t have all the technology of infusion pumps and syringe drivers. And, you know, it’s a bit like the [inaudible] study. That the things that we have assumed that we know or we think we know, work in, the first world, have the, exactly the opposite effect there. Now I’m not sure that it always works as well as we think it works in the high technology countries. But the reality was that when they did a pragmatic study, that used the available resources in those places they came out the confusion that standard therapy was wrong.” P 7: Interview 6.rtf - 7:20 (66:68) Lack of organization in website materials Lack of organization in the OP website was another barrier addressed by some participants. They believed that there are no clear guidelines for the formatting of publications made available on the OP platform and some publications haven’t provided references.  “You know I don’t think, and maybe it’s in the website and I didn’t see it, but, you know, for most magazines or books or etcetera they have really precise guidelines that they have to adhere to, before they can publish something. And those are available to the general public. Um, and I haven’t seen on the website, which guidelines people publishing or people posting on this website have to adhere to” P 4: interview 3.rtf - 4:17 (111:113)     97  Some believe that the educational materials in the earlier version leaned toward critical care and now are more toward heart and lungs. Some of the educational materials are quite simple while others are complicated and more in depth. Although some of the interviewees find this helpful, others believe that it could be seen as a problem. For most participants videos are too long whereas they prefer short two-minutes videos. How about videos? Lectures? Other?  P: Um, [sigh] H: Have you had chance to take a look? P: Well I listened to a few um, I think a lot of them are, long, too long. I really like, sometimes I listen to podcasts, from, other websites. And I really like those kind of two minutes, talk on a subject. P 4: interview 3.rtf - 4:21 (131:137)      Quality of videos was another issue discussed by participants. A participant for whom English is a second language can understand the words in videos if she can see the faces clearly that helps her providing possibility for lip reading. However, she finds that the faces are not clear enough in videos that have been captured from faraway.  Some participants believed that Boston Children’s Hospital has high validity and credibility. Consequently, this creates an expectation among the users that materials located on the OP website will be high quality materials and trustable. However, they believed that sometimes materials located in the OP are missing expected quality. For example, some participants reported that citation of the materials sometimes were found to be problematic or missing: "You know? It’s not because Boston is a good hospital and a good Centre that everything they publish is necessarily to be trusted. No. "    98 Poor Internet connection Although OP has tried to resolve the connectivity issues by using a cloud-based platform, people in remote areas and poorer countries still face problems such as interruptions in power supplies.  “And I think, one of the realities is that, despite the changes, across the world, things like the Internet in many of those countries, is not the reliable platform that you’re used to. And that frequent things like, interruptions to the Internet, the interruptions to power supplies…” P 7: interview 6.rtf - 7:15 (55:55)     Internet connections cost money for those who want to connect to the platform, and access in remote areas like South Africa is poor and difficult.  “But even when you look at things like cell phones, the data fees in places like South Africa are massively higher than they would be in the US and North America and Europe” P 7: interview 6.rtf - 7:16 (55:55) Lack of time and time difference The issue of time had two aspects that included lack of enough time and also time differences between countries. Lack of enough time was another barrier addressed by the users. Pediatricians, nurses, allied health and other clinicians are reporting that they do not have enough time to fully explore OP. In other words, even though one of the main purpose of developing the OP platform was to overcome the issue of time restrictions among clinicians and providing a resource that is accessible at any time and any place for learning, professional development and improving competencies of these clinicians, users still are encountering time restrictions for fully exploring the OP platform. One important issue reported in this regard was numerous activities that in some cases the OP users were expected to do or different steps that they were expected to   99  go through in the process of utilizing the platform or using desired materials. This was an additional factor to the issue of time the precluded further exploration of OP:   “P: Uh because a lot of the activities require you to, well, all of the activities, or all of the things on this website require you to sit down with a computer, or tablet. H: Ah, okay. P: And, I think we’re quite busy. And so, because that’s what was expected of me, was to know that, you know, to go through those courses and everything. It didn’t leave me a lot of time to go to other resources.” P 4: interview 3.rtf - 4:5 (25:29)     In terms of time differences, despite participants’ huge desire to participate in the World Share Practice Forum in real time, this participation is not practical for many users due to the time difference between countries. People cannot connect to the platform at the same time, therefore, interaction is not ideal. One of the participants from Latvia pointed out that normally there are few participants to create good real-time interactions and communication. This issue, limited number of participants participating in real-time interaction, restricted the capability of the platform for real-time participation and real-time knowledge exchange in different parts of the world. Technically, this means, “The only thing you [as OP user] miss is you can’t participate”. Sometimes, it is only one hospital participating in these forums that makes the forums to turn into local discussion groups creating opportunities for local Communities of Practice: “P: I can tell you that, we watched two videos in Country X on Tuesdays. And those, whether it’s, I think it’s the last Tuesday of the month, or the fourth Tuesday. Uh, but then because of the time zone difference, we were the first, to watch them. So no one in the US would have seen them there. And we were the only ones, commenting. We didn’t see the other comments.” P 6: interview 5.rtf - 6:9 (51:51)       100  Some participants in the World Share Practice Forum mentioned that Tuesday is not the best option for running it.  They suggested that real-time participation would be improved if it could be switched to Wednesdays:  “P: I haven’t, I end up usually doing them after, because, they come out on Tuesdays and, that’s a brutal day for me. I have, you know, if you’re, asking me about whether I’ve done it. But I, it’s one of my goals is if I can free up that time during the forum, but after the fact. The only thing you miss is you can’t participate. But, you know being able to see it, at your leisure is always, is always a great option. “  “So [chuckle] uh, so it’s probably for best, for us it’s best to watch on Wednesdays, rather than Tuesdays, because, because of the time zones.” P 6: interview 5.rtf - 6:8 (45:45) Older generation versus new technology Difficulty with technology while using OP was the last issue discussed by participants. They believe that older generations are not as comfortable as younger generations with technology even though they can connect and use the platform.  “Um, but, I think often things like, apps, those sort of things, may also be into the younger generation, are using much more, than some of the older ones [inaudible] might be doing” P 7: interview 6.rtf - 7:17 (55:55) Existence of many other competing learning resources  One of the participants believed that there are so many other resources rather than OP that are more organized and user-friendly that can be used for professional development and updating their knowledge. These resources have more organization and come with high level of scaffolding, meaning that they are concise and organized from easiest to hardest helping the users easily go through the entire learning package:  “H: What’s the difference? are you comfortable with that one, or you find, better information there?  P: I just feel it’s a bit more concise. And it’s a bit more, organized. There’s maybe 20 lectures to go through and, they’re organized in order of easiest to hardest. And so the   101  way it’s organized for my learning, it really helped me to go through them all. And so I did them kind of sequentially” P 4: interview 3.rtf - 4:7 (35:41)     4.2.7. The most important, popular and helpful features Participants in the focus groups and interviews noted three different features of the OP platform that they found useful and helpful for learning and teaching purposes that included simulations, videos and more specifically, short videos. In the following sections, I will briefly describe each of these features. Simulations Participants in the focus groups and interviews believed that simulations create opportunities for learning and practicing with the same equipment used in a real-life experience. This method creates an opportunity for hands-on experience and learning by doing and makes learners comfortable to manage critical situations. This way, they believed, users learn how to do the same job in their daily practice. It reinforces other methods of learning and teaching. Simulation is considered a better method for learning than videos and reading. It is considered a great way to teach residents specially regarding Accreditation Council for Graduate Medical Education (ACGME). Learning via simulation is considered especially useful for users from developing countries where mostly they want to know how things work. Additionally, participants in the focus groups and interviews believed that simulation and World Share Practice Forum are the most behavior change resources. Videos Videos were perceived as another important feature of the OP platform. Participants in the focus groups and interviews believed that videos are important because they are able to show human interactions, voice and body language while are transferring updated and new information   102  to the learners. In fact, participants believed that videos were important features of the OP platform since they contain different levels of learning, easy to hard which suits very well with medical residency education. They elaborated that “residents are in different levels of learning and [attending physicians have] found videos very suitable for [this] situations” (Focus group). Short videos Of particular interest were short videos on the OP platform. Participants considered short videos as important features of the platform since they believed that these vides contain a summary of latest knowledge and this way they cover a lot more topics. Additional reason for short videos being perceived as important feature of OP was related to the reduction of attention span for human being over time making short videos efficient in facilitating learning among the users of OP: “Short videos are very good as you move along in your life, your attention span tends to, get a little less focused and so I find those little short talks really good, they’re a minute or two.”   4.2.8. Suggestions After discussing barriers to the use of the OP platform, participants’ suggestions about the OP platform were sought. Participant in the focus groups and interviews addressed a few suggestions as follow: Proper promotion and advocacy about the platform The first suggestion was properly promoting about and advocating for the OP platform. Different strategies were suggested for this purpose. For example, one suggestion was to clearly advertise about OP, as one of the participants stated, “if the main idea is to connecting members in the community it’s better for them to make it obvious”. Another participant mentioned that OP   103  “…need[s] a way to capture people….” Another interesting suggestion was to introduce pediatricians, nurses and health care professionals to the platform as part of their orientation package in hospitals, especially in pediatrics unit. One of the participants who had learned about OP in an international conference also believed that conferences and seminars could be another opportunity to promote about and advocate for the platform. Participants believed that lack of awareness about the existence of the platform or its various features is one of the main barriers to its use that would be overcome by proper promotion and advocacy. Improvement in content and educational materials The second group of suggestions was mostly about improvements that participants believed need to be made in the content and educational materials on the OP platform in order to improve its utilization and impact. In so doing they suggested educational materials should be linked to guidelines and literature. Another suggestion was improvements in the educational videos. They believed that videos should be short (two minutes long), have subtitles in English as well as other languages. This, they believed, would involve more senses of the audiences. Additionally, they suggested creating clear videos by including close-ups of people speaking in order to enable lip-reading for non-English users. They believed these considerations would make educational contents easy to follow for people whose first language is not English. They also suggested materials and topics, in general, to be offered in different languages. Participants also suggested OP to categorize the educational contents by dividing them into beginner level (simple materials) and advanced level (more in-depth and complex materials). This, they believe would make materials easy to follow. Furthermore, participants believed it is hard to navigate through the website. For this purpose, they suggested materials in the website to be originated in a systematic way, so it is easy and straightforward to find materials in different levels (beginner   104  to advanced level). They suggested contents to be logically organized as system based or disease based. In order to improve optimize navigation, they also suggested OP to add some guidance in the website about how to navigate the website.  Participants suggested OP to provide opportunities for users to give feedback or comments about the topics and contents of the materials included in the website. They believed that users would like to be engaged more and interact with the platform and content managers by asking questions and sharing their ideas. Participants suggested materials and contents to consider context and environmental factors. For this purpose, they suggested OP as a global platform to adapt styles that match with other contexts since different contexts have different working styles. They also suggested educational contents to include information and data from developing countries.  Participants suggested that OP should balance the learning and teaching methods used in the platform for facilitating learning of the included content. They suggested more hands-on experiences, rather than more lecture methods. Also, they suggested adding podcasts. As one of the participants indicated, “I like to listen to podcast, you can use it anywhere at any time while cycling, while driving ...” In order to implement all these suggestions about the educational materials and content, participants believed that OP needs to have a strong committee for content and provide strong support for content committee and core committees for their commitments and reward committee members and “give credit to the committee.” Proper use of equipment introduced in simulations Participants suggested OP leaders should teach the users about proper use of the equipment such as ventilators. They believed that taking some courses before using the equipment would be   105  helpful. For example, teaching the staff and doctors how to troubleshoot and deal with problems related to the use of equipment. Some even suggested using checklists to audit this learning about proper use of the equipment. Participants suggested OP should teach people to develop skills that could help them to work in different contexts and adapt to and work with different work styles style. Some participants believe that for users from developing countries it is better to learn from OP but adjust their learning to the contexts of their workplace as needed. Supporting the OP Community of Practice Participant had interesting recommendations in order to improve interactive aspect of the OP platform.  They suggested OP should send email notifications to the members, informing them about the news and new features or changes. They encouraged OP to connect people to each other by including interactive question and answer (Q&A) functionality under the group tab to allow members to enter their questions and receive answers by other members.  They also suggested peer education as another way for improving interaction via OP. For this purpose, they suggested members who are good at English could help others in using the platform and navigating the OP materials. Participants suggested that OP create opportunities for users to feel confident to speak up about their real needs. They believed that OP should provide opportunities for users to share their opinions about the topics they would like to learn. They believed that OP needs to consider opinions from people who are expert in the field, as one participant mentioned “to be honest I think you’ve actually gotta ask the people on the ground”. Some even suggested to go beyond and create opportunities for people who are interested to be part of the OP content committee. In fact, they suggested OP to provide opportunities for users even to evaluate those OP materials that related to their field.   106  In total, participants in the focus groups and interviews suggested that OP create opportunities for members to connect with others, join the groups and share their work and experiences with fellow members. They believed that OP is not doing well enough in this regard and “…want[ed] this to be fixed” for example, they asked OP to create opportunities for creating an interactive group for nephrologists by connecting them via OP. They indicated that they “… Want[ed] to help…”. They also suggested OP to invite well-known people in the field of pediatrics to contribute to OP.  4.2.9. Suggesting OP to others  I was curious to know if participants would recommend OP to others such as their colleagues and friends. Interestingly all the participants in the focus groups and interviews had a very positive attitude with regard to suggesting OP to others. In fact, almost all of them mentioned that they had already started doing this. They mostly suggested the platform to colleagues, friends and their residents. 4.2.10. Impact of study on the participants Participating in this study had a positive impact on the participants in the focus groups and interviews and improved their attitudes and intention to use the platform. It was surprising that participating in this study motivated the participants to go back to the platform and look at it carefully. They were happy for this and regretted not having done it earlier. Now they have more ideas about the program and are happy to use it and suggest it to the others. One of the participants mentioned that he was going to create a group for nephrologists in his country and will ask others to join the group. He was so excited about his plan.    107  Chapter 5: Discussion Overview This study attempted to answer six questions about OP as an OCoP. Questions were created based on initial conversations, meetings and teleconferences with the primary intended users of the evaluation results including program developers, other stakeholders and committee members. This chapter discusses each research question individually, followed by a discussion of unexpected findings. Organization of this chapter will be based on the research questions that my thesis has addressed.  5.1. Research Question 1. How well does OP cover topics of interest to the community? Considering pediatrics education reform and in an attempt to resolve related health care educational issues, participants perceived that OP has provided topics of interest in the field of pediatrics, topics that should be relevant, useful, and updated in order to facilitate pediatrician’s learning in a new way and with new technologies.  This research question tried to find out if OP successfully identified a shared domain among users and provided topics and educational materials that the community needed in order to meet users’ learning needs.  Aside from educational materials, this question also tried to find out if users perceived that OP meets structural or technical criteria of a well-designed online learning website. Findings about this research question are discussed in two sections:   108  5.1.1. Identifying and providing related topics Domain is one of the core elements of every CoP including an OCoP. A high-quality domain plays a crucial role in attracting a variety of the members to the community. In an OCoP members with different levels of expertise, from different locations, with different interests join together to share their knowledge via an online platform. This way, the domain sets the stage for the community to actively participate in the process of collective knowledge sharing and learning within their domain (Tsai, 2012).Some of criteria considered for this research question (Q1) included educational contents, their scope of coverage, as well as meeting users’ educational expectations in terms of usefulness, relevancy, accuracy and being up-to-date.   Both quantitative and qualitative data showed that participants in this study are generally satisfied with the educational contents.  A vast majority of survey respondents agreed that the OP platform is a useful educational resource and most of them found it relevant to their field of practice and confirmed that the content is updated regularly. Themes surfaced from open-ended questions also confirmed users’ satisfaction with the platform, educational materials, tools and expert speakers.   Findings from qualitative data (focus groups and interviews) also confirmed that the website is meeting users’ educational expectations, especially as adult learners, and majority of them have found scope of the educational coverage very impressive. This indicates that participants in this study perceived that OP has successfully addressed the domain of interest in developing its community of practice.  5.1.2. Developing a well-designed online platform Internet-based learning is well suited to the needs of adult learners for many reasons. It allows learners to have control over their desired educational topics and their time. They can also   109  use different modalities based on their style of learning as Kolb classified them in four categories: concrete experimenters, abstract conceptualizers, active experimenters, and reflective observers (Wolbrink & Burns, 2012).   OP offers virtual training tools for sharing knowledge and communicating, and a library of resources including videos, lectures, device simulators, and protocols (OPENPediatrics, 2013). Findings of this study indicated that users with different learning styles have found these features of the platform very helpful. They like the possibility of downloading materials, illustrative and graphical materials and most importantly guidance for action. The utilization of multimedia educational methods in preparing and presenting educational materials also appeared to be very effective and helpful feature of OP, a finding that was supported by other researchers as well (Ruth & Mayer, 2011). By using multimedia features, participants believed, OP meets almost all types of users’ needs and different learning styles as addressed by Kolb (Ruth & Mayer, 2011; Wolbrink & Burns, 2012). Multimedia features can significantly enhance learning if it is well designed and well implemented (SEG Research Team, 2008). Studies show that visual channels transfer less information compared to auditory channels (Miler, 2005). However, using both visual and auditory channels while presenting information, working memory can handle more information in general and in this way can increase the amount of information that brain can process (Fred, Renkl, & Sweller, 2003). In other words, presenting information via two or more channels increases the likelihood that learning will occur (Junaidu, 2008).  A vast majority of respondents agreed that the OP website is a user-friendly and visually appealing platform and connecting to the platform is easy. Non-English speakers also appreciated that the content is available in other languages such as Spanish and Turkish. OP is   110  regarded by users as a well-designed online platform. Study participants indicated they will be using OP in the future and will recommend it to others.   5.2. Research Question 2. To what degree are knowledge, experiences and stories being shared across the community?  This research question investigated how knowledge was created and shared especially in regard to tacit knowledge. Pediatricians, nurses and other health care professionals connect to the OP community worldwide and use the knowledge being shared to improve their knowledge and skills. Surveys, interview and focus group questions inquired whether or not respondents believe that knowledge is being well managed in the community. This research question was answered in two parts: 5.2.1. Whether users learn from each other or share their knowledge, experiences and stories with others in the community When reviewing the literature, it became apparent that Knowledge Management (KM) is a crucial tool for organizations to keep up with rapid changes in the 21st century, especially in health care sector due to the overwhelming body of knowledge produced in this area. Health care professionals, including pediatricians and pediatrics nurses, are dealing with a huge need to keep up with new knowledge produced in their field of specialty (Alali & Salim, 2013). However, health care workforce shortage requires them to work harder and longer hours. This interferes with their professional development (Carolina & Carolina, 2010). They don’t have enough time to learn new information and to update their skills, jeopardizing quality of care and patient safety (Burnette, Ramundo, Stevenson, & Beeson, 2009). In addition to this is shift work and lack of motivation that interfere with in-service education during clinicians’ off hours. OCoPs have great   111  potential to address this gap and to meet clinicians’ learning needs by creating a knowledge management system which distributes knowledge and resources more effectively and efficiently among members of a community (Jelenic, 2011; King et al., 2009). While reviewing literature, one study among dentistry students showed that their connection to an online dentistry CoP helped them to learn better and more effectively. They exchanged their knowledge, reviewed assignments, and shared their professional and practical experiences (Gardner et al., 2012). Findings of my research on the OP community showed that members and interested people connect to the platform around the world (almost 145 countries) in order to learn, enhance or exchange knowledge.  Results of the survey, focus groups and interviews, however, showed that knowledge is not being exchanged or shared within the community and between the users to the extent that is expected. My research showed that there is a core group of knowledgeable experts in the OP community who take the responsibility for creating and delivering knowledge. Most users of OP play a passive role as knowledge consumers rather than collaborating in knowledge production or even knowledge sharing. An important aspect of KM, participation in collaborative activities, which create new knowledge and share work experiences, is missing in this platform. Users are so happy with the amount and quality of the educational resources they receive from the platform that they don’t think of the possibility of learning from each other, or even generating new knowledge in collaboration with other members. Findings showed that most of the users were not aware of facilities provided by OP for letting them connect to others and share experiences, knowledge, tools, etc. Focus groups and interviews were very helpful in digging deeper into this issue. I found that most of the users see the platform as an online learning website, rather than an OCoP. As Zboralski et al. suggested, providing necessary resources is important, however   112  establishing appropriate conditions for encouraging the sharing of existing knowledge among members is an efficient and effective way of implementing KM that needs active participation of all members of the community (Zboralski et al., 2006).  The fact that users consider OP as an online learning system rather than an OCoP is very important as it shows that OP is lacking active participation of the users in knowledge production and knowledge sharing. Lack of participation in producing and sharing tools indicates that practice, the third element of a CoP, is missing in the OP platform as well, or is not addressed enough. When practice is missing, it is hard or impossible to share tools, resources and tacit knowledge owned by different members of a CoP (LeMay, 2009b; Parboosingh, 2002; Wenger et al., 2002a). Exchanging knowledge in CoPs saves resources such as time, budget and human resources which is an important feature to help countries with health care professional shortage. Thus, lack of interactions among members and lack of collaborative activities reduces the effectiveness and efficiency of the OP community. This may also induce the feeling of lack of belonging to the community and users may feel isolated. This as a result will prevent users from legitimate peripheral participation and the ability to move from the periphery to the center of the community. It will prevent change in identity that is a main feature of learning in the CoP (LeMay, 2009b; Wenger et al., 2002a). Considering those missing features, OP really needs to increase participation of the members in producing content, tools and materials and improve the sharing of tacit knowledge and the tools they have, and they produce in order to optimize the platform as an OCoP.     113  5.2.2. If tacit knowledge is being shared Tacit knowledge is a type of knowledge that only some people possess it in each organization because of their special talent, knowledge, experience, ideas and values. Tacit knowledge cannot be passed to others in conventional ways and needs to be transferred through social interaction (LeMay, 2009a; Parboosingh, 2002). CoPs are ideal venues for passing on tacit knowledge as they include social interaction, which is a requirement for transferring tact knowledge. Members of CoP can share their special and unique experiences, inspiring stories and or their extraordinary knowledge and talent with others in the community. Transferring tacit knowledge via CoPs is one of the best ways to increase health care professionals’ competency and productivity and to help them overcome workplace challenges (Valaitis et al., 2011).  My research showed that users of OP are not connecting to other users via OP and consequently do not have any chance to pass or receive tacit knowledge. One of the primary purposes of OP as an OCoP is to help members capture and integrate the tacit knowledge embodied in the experts and other members in order to improve their competencies, productivity and innovation through connections and interactions in OP (LeMay, 2009a; Ranmuthugala et al., 2010; Ranmuthugala, Plumb, et al., 2011). However, in the current version of OP, the platform is only able to “deliver” knowledge by a core group to other members through lectures, videos, simulators, forums etc. with little or no social interaction.  Lack of opportunity to share tacit knowledge in OP limits access to knowledge distributed among all members of the community which will consequently reduce the effectiveness of the community (McKellar, Pitzul, Yi, & Cole, 2014). In clinical professions (such as medicine, nursing and other health care professions), staff need to have access to tacit knowledge in   114  addition to explicit knowledge coming from literature. In fact, tacit knowledge is even more important in these professions in order to know how to apply recommendations of the literature (Valaitis et al., 2011). Sharing experiential or tacit knowledge in CoPs can prevent mistakes and medical errors and help the staff deliver better care (LeMay, 2009b; Parboosingh, 2002; Wenger et al., 2002a). Users’ lack of awareness about the full potential and capabilities of OP is also considerable. Informing OP users about the possibility of sharing their explicit knowledge, experiences and tacit knowledge as well as tools via this platform may increase participation of the clinicians in sharing their domain-specific knowledge and also may improve their feeling of belonging to the OP community which will set the stage for more learning and moving to the center of CoP as professionals (Hsu, Ju, Yen, & Chang, 2007).   5.3. Research Question 3.  To what extent are members engaging in productive and sustained interactions via the OP platform?  In health care systems and hospitals, professionals spend time together, share information and ideas, give advice and discuss to solve problems in an interactive way. This kind of interactions and communications among colleagues and staff is routine in every health care setting, and it is part of health care culture. They usually work together to create tools and standards. They develop guidelines, design manuals or other documents in a cooperative way. This type of relationship helps them over time to acquire a unique perspective on topics, practices and approaches and leads to friendships, and as a result, increases their participation in the field. They create a common identity and finally they become a CoP (LeMay, 2009a; Wenger et al., 2002a).    115  Considering the way that a CoP is being created among health care professionals, it is obvious that a website by itself cannot be regarded as a community of practice. As it discussed earlier website is just a technical part of the community and it needs the other part to be completed, which is social architecture. To explain more, having the same job, position and education does not form a CoP until they interact with each other in an active and productive way, learn from each other, and exchange knowledge and stories. Problem solving, for instance, is fundamental for learning in any CoP which allows members to explore real life situations to find answers through interaction with others in social contexts (Lave & Wenger, 1991; Roeckelein, 2006).  In my research I sought the OP users’ opinions about interactions via the OP platform and inquired their experiences in this regard. I found that, there is not enough regular interaction among users of the OP. They mostly join the platform to learn from resources such as experts, videos, simulators, lectures, forums and so forth. Although they gain significant benefit in this way, they miss learning from each other. Literature has shown that learners who gravitate to communities and interact with others receive more benefit and attain more knowledge from knowledgeable members (Lave & Wenger, 1991; LeMay, 2009a). Despite the possibility provided in OP for creating groups or joining to existing groups, users either do not create or join groups or do not use this option to the degree that is expected. Lack of connection and consequently lack of interaction between OP users can lead to the lack of engagement of the users in creating shared educational tools, problem solving and collaborative activities. Thus, they may miss a huge potential benefit of online communication, learning and sharing their tacit knowledge with other members (Baylor, 2014). Survey results illustrated that OP has been used by a majority of participants for more than two years in almost 74 countries   116  and they mostly have access to different electronic devices such as computers, tablets and smartphones to connect the platform that is a huge opportunity for sharing tacit knowledge. However, they still have no or minimal interactions with other members. To learn more about this problem, focus group and interview questions were developed and directly discussed with participants. Results of the focus group and interviews showed that users actually do like to interact with other professionals via the platform. However, they are either not aware of the possibility of joining with others and groups or do not have enough time to do so. This lack of knowledge implies a need for raising awareness of the possibility to join colleagues on the website. Even though social interaction is vital in facilitating learning, professional development and achieving the goal of “best practice” (Cambridge & Kaplan, 2005), OP members only learn from a core group (who develops the educational contents) and miss any social interaction. The OP core group is reportedly doing an excellent job in creating and sharing new knowledge, helpful methods and useful experiences; however, they do not seem to be very active in communicating with members or trying to help members connect to each other. The OP members are very happy to receive useful and practical information from the core group. However, they are missing the benefits they can gain through interactions with others in the community, such as better understanding other co-workers, increasing trust and becoming more interested in new knowledge, having access to other’s knowledge, experiences, stories, solutions and finally increasing confidence and competency (Detlor, 2004). Instead, they use the knowledge provided by the core group passively without becoming involved in the process of producing or sharing knowledge.    117  5.4. Research Question 4. How effective is OP in supporting professional development of members? Lack of professional development is one of the issues that pediatric professionals are dealing with due to health workforce shortage, busy pediatrics units, workload and time restrictions (Spedding et al., 2013). Literature reviewed in this study showed that OCoPs can improve health care professionals’ learning and professional development due to their ability to allow them to learn at their convenience regardless of geographic location (Patel, 2007; Spedding et al., 2013). Tsai et. well showed positive effects in members’ learning and professional improvement, cognitive changes and changes in their teaching practices through online social interaction with their peers (Tsai et al., 2010).  I found that members’ connections in the OP community are underdeveloped. However, the good news is that connection happens outside of the OP community within local communities, in the hospitals, either within members of OP or between OP members and non-OP members. To explain more, findings of this study revealed that local and small (mini) communities have been created around the OP community in different countries. Members of these mini communities are mostly also members of OP. Members in these mini communities (I call them mini OPs) discuss whatever they have learned via OP with their friends, colleagues and medical students in their workplaces. Interestingly, findings showed that group connectivity and collaborative activities are very good among these mini communities. They even use OP together as a group. They join the World Share Practice Forum together and share their opinions and ideas. In some hospitals they run a learning session once in a week, which is called “academic day”. In academy day, community members – pediatricians and other pediatric staff – spend one day in a week to review and update their knowledge and chat with each other. For this purpose,   118  they first watch a video about a specific topic from the OP website and then discuss about the topic. They have a scheduled plan for topics review on academic days.   Members of locals also share the knowledge gained from OP in their workplace with people who are not members of OP. They use this knowledge to teach nurses, medical students and residents. This activity can be regarded as the same process as legitimate peripheral participation (LPP) that Lave and Wenger have mentioned when they explained about how novices enter into a community of practice, and grow and advance to the center of community (Floding & Swier, 2011; Fuller et al., 2005; Lave & Wenger, 1991, 2002; Sayer, 2014; Wenger et al., 2002a). To explain further, members of mini OPs, can be regarded as novices when they connect to the OP platform and learn from it and when, they share their knowledge in their workplace with non-member colleagues or use their learning to teach residents, they are regarded as experts and serve as educational leaders in their local communities. This can also be explained by some researchers’ findings regarding a typology of Online Communities of Practice (OCoPs). OCoPs can be classified into different groups based on a variety of characteristics. Wenger classified them according to their size, life span, geographic dispersion, boundary span, as well as their creation process, and the extent of their formality (Wenger et al., 2002a). When homogeneous professionals (clinicians from the same specialty, for example) are involved in an OCoP, boundary crossing will be low. If they have more specialty groups from the same health care system (for example, the same hospital) involved, they may have medium boundary crossing. However, when members of different health care systems (such as specialists from various hospitals, universities or countries) are covered by an OCoP, they will have a high level of boundary crossing. This is the case for OP. A high level of boundary crossing may lead to psychological distance and a lower level of trust among members   119  of different groups that consequently may result in reduced knowledge sharing. This addresses the possibility that users of OP and similar OCoPs might “fracture” into discrete specializations and related cultures and sub cultures (Dubé, Bourhis, & Jacob, 2006; Wenger et al., 2002a).  5.5. Research Question 5. How does OP impact health care practices across the community?  This question investigated how being part of the OP community affects members’ practice, how many knowledge tools are being produced, to what extend members use whatever they learned via OP in their workplace, and if learning through OP enabled them to do their job better. Participants in my study reported that being part of the OP community and learning from it had positive effects on clinical work and helped them to do their job better and with more confidence. They it helped them learn to solve medical problems and to deliver much safer and more effective patient care. They learned about the latest advances and developments in the pediatrics field through connecting to the World Share Practice Forum. They stated that they also learned to do procedures better by watching simulators. Hands-on teaching methods offered by OP helped members to learn better and gain more confidence in doing the same task at work. Participants in the focus group discussions mentioned that after participating in OP they felt more comfortable at the bedside at clinics. The body of literature reviewed in this study confirmed that CoP could be an ideal strategy to improve health care systems as it helps health care  professionals to share tacit knowledge, create and share tools, discuss problems and find solutions (LeMay, 2009a; Parboosingh, 2002; Wenger et al., 2002a; Zboralski et al., 2006). LeMay’s study also demonstrated that CoP can improve the quality of health care services (LeMay, 2009a). OP offers many educational resources created with different features such as videos, lectures, device simulators and protocols. Each feature has the potential to help users   120  with different learning styles to learn better and practice better. These resources are all created by the core group of OP. All educational content and methods for teaching and learning are provided by the platform. Members are not becoming involved in collaborative activities as they are not connecting to each other via OP. This is a missing link coming from disconnections between members that could be an important obstacle toward best practice. A literature review about CoPs in Australia revealed that the connections of members have significant effects on health care practice and outcomes through creating and applying new guidelines, new policies, the assessment and screen tools and improves patient safety and quality of care (Ranmuthugala et al., 2010). Despite lack of collaborative works in OP, mini OPs are helpful in developing many collaborative activities and improve the quality of the practice. Tsai, Laffey and Hanuscin, in their study with regard to teachers CoP teachers, demonstrated that connecting to the community helped teachers learn new teaching skills and improve their practice (Tsai et al., 2010). 5.6. Research Question 6. Overall, how helpful is OP in delivering better care and in what way (s) has it fallen short? Two main obstacles to deliver better care in health care system worldwide are as follow: First obstacle is the worldwide shortage of pediatricians, nurses and health care professionals, which limits their engagement in professional development. Another obstacle is the traditional medical education system that limits learning out of classrooms and geographical boundaries. OP was developed to address these problems by trying to set up an OCoP to provide opportunities for learning, knowledge share and professional development among pediatric health care professionals. It aimed to provide the latest educational materials in the field of pediatrics and to make them available for pediatric health care professionals anywhere in the world at any time. It tries to use the best and latest available online technology to make these   121  materials available. Recognizing that all users would be adult learners, OP attempted to follow adult learning and education principles in establishing OP. Making all educational materials appropriate and available for everyone across the world, OP tried to help pediatricians, nurses and other health care professionals to improve their educational competency and deliver better care.  Evidence from this study showed that OP has had a very positive impact on pediatricians’ clinical practice and helped them deliver better care, as discussed earlier. Participants in this study perceived OP successful in providing the latest knowledge in the field of pediatrics and delivering it via a very well-developed platform. However, OP still needs to add variety of topics and specialties to its educational recourses to attract a significant portion of its potential audiences who are not yet attracted to OP due to its focus on limited number of specialties.  As it was mentioned earlier, OP aimed to connect pediatricians, nurses, allied health and other health care professionals via an OCoP in order to facilitate sharing their knowledge and experiences worldwide with no concern about geographical boundaries. For this purpose, OP created a cloud-based online platform with global access. The idea was to make experts accessible for members of the community and to provide a professional, yet friendly, community to let the members connect one another, exchange knowledge, and share their work experiences to deliver better care. However, OP has not met this aim completely for the following reasons: • OP developers have not had an effective marketing plan to introduce the platform. OP may be losing potential members because people simply do not know about it. Results of this study showed that many pediatricians, nurses and other health care professionals do not become members because they are not aware of OP and its potential benefits.  Many current users also do not know about different available features of the platform   122  such as the possibility of creating or joining to groups. Therefore, they miss the opportunity to connect to other professionals in their own field. • OP does not seem to have a specific strategy to encourage members to connect to each other. As discussed earlier, creating an OCoP is not a static and one- time event. Creating the platform is just the technical part of the job. The social part of the architecture is as important as the technical part. While the technical part supports the community’s functioning, the social part helps the members interact actively in order to achieve each other’s tacit knowledge and implement best practice (Cambridge & Kaplan, 2005). • There is no clear guidance for members to create or join groups. CoPs cannot achieve their potential goals if they add a large number of new members to the community without planning for social interaction, sharing knowledge and experiences, and integrating the novice for supporting knowledge providers (LeMay, 2009a; Wenger et al., 2002a). In a successful CoP, participants are recognized and supported by other members and the core group.  Some participants in this study pointed out that they neither know how to create or join a group nor they are aware what they are expected to do in group activities. • There is limited or no communication between members and the core group. Good relationships between members and the core group give members a sense of belonging to the community rather than feeling like outsiders (Thrysoe et al., 2012). Studies indicate that connecting to and receiving support from experts increases job satisfaction and better care (Thrysoe et al., 2010). Findings of my research showed that despite members’ desire to communicate with OP core group or knowledge providers, they see no chance to do so.   123  5.7. Unexpected findings This study set out to look at the role of OP in contributing to learning, professional development and quality of care delivered by pediatric clinicians. Findings of this study addressed the problem and the research questions, as elaborated earlier. However, there were some unexpected findings that also emerged from the data collected via survey, focus group discussions and interviews. One of the most interesting finding was about differences between countries in getting benefit out of OP especially between developed or Western countries and under-developed or poor countries. In the open-ended part of the survey, there was a quote saying “[OP] is a bit Americanize[d] at a time…”; another one suggested that the program developers should “add something from Latin-America”. This became more apparent in the qualitative phase when people shared their opinions freely, either in focus group discussions or interviews. Members from those developing or under-developed countries, where the highest children’s mortality happens and who are the main audiences of the OP community, cannot benefit from the program as was hoped for several reasons. First, their educational and clinical needs are different from Western countries. That is to say, whatever comes from textbooks or is called “standard of practice” or whatever works in Western countries, does not necessarily work in other countries. For instance, things with high priority in the first world may be regarded as a low priority in poorer countries. One of the participants mentioned that when Western countries talk about pathogens, they probably worry about a small number of bacteria or some viruses. However, in some (under-developed) countries they are worried about a whole set of infections such as malaria, tuberculosis, meningitis that are uncommon in the first world. The situation in poor countries is very complex and different so, most of the time solutions and methods of action that come from textbooks do not work in those countries and they have to act in a very different   124  way. One participant pointed out that they use OP, but they do not follow the exact directions. Instead, they adjust them with their local conditions:  “I need to collect my own data, and then decide what, what actually the priority is and, and maybe what the experts do, isn’t right for where I am. And some of that would be really exciting to explore.” P 7: Interview 6.rtf - 7:1 (11:15)  The second problem is regarding medical equipment. In poorer countries, they either do not have the same equipment as they see in OP (simulators for example) or do not know how to properly maintain and troubleshoot the equipment they have. Participants believe that Western countries have technicians to run the equipment, whereas in developing countries everything is the responsibility of the pediatricians. OP has apparently not considered this. For instance, in one of the hospitals they have some ventilators, but they do not have water-heating humidifiers for them. It is very expensive to run humidifiers, so they run them dry. This way, after a couple of times technical problems develop and they have no idea what to do. They do not change filters properly, they do not calibrate the machine and they do not even have budget allocated for this purpose. In some hospitals the doctors have tried to solve this problem by developing a ventilation course and a checklist to go through, before starting to use the ventilator. Third, in developing countries, users still have to deal with Internet connection problems. OP in those countries is therefore not a reliable source because of poor Internet accessibility, Internet interruptions, power supply interruptions and expensive fees.  And, finally, there is a great amount of difference between Western countries and developing countries in terms of culture and religion. For example, sometimes in poorer countries health care professionals have to accept children with special conditions in the hospital while they know it is too late for any help in anyways. However, they accept them in the hospital and use their limited resources and time to work on those cases because in their culture they   125  cannot reject any patient. This takes their time and budget and reduces access for other children that actually can benefit from these services.    126  Chapter 6: Summary, Conclusions and Recommendations Millions of children around the world die each year from preventable diseases before they reach the age of five, mostly in developing countries. Among the many problems accounting for this situation, lack of enough medical schools, traditional medical education systems, a poorly educated workforce, and a shortage of well-trained health care professionals are the most important factors. Pediatricians and other health professionals don’t have adequate time to keep up with the huge body of knowledge and evidence produced, adapt to ongoing changes in their workplace, and improve their work competencies. Online communities of practice (OCoPs), have the potential to help improve this situation. OPENPediatrics (OP), an online community of practice, was developed to help health professionals and pediatricians worldwide to keep their knowledge updated, to access modern technologies, to learn the latest practices, and to connect with their professional colleagues to share knowledge and experiences at any time and in any place where they can connect to the virtual community. The OP platform gives them a way to communicate with one another, to ask questions and solve problems at times that are convenient to them. The purpose of this study was to evaluate the role of (OP) as an online community of practice in the learning and professional development of pediatric clinicians, and in the quality of care they deliver. 6.1. Methodology In order to examine the effects of participating in OP on pediatricians’ knowledge and practice, an outcomes-logic-model approach was developed considering three primary concepts   127  to discuss its main components: domain (common ground and interests), community (social interactions) and practice (shared knowledge and knowledge tools).   Using elements of Utilization-focused Evaluation (UFE), six research questions were developed to examine the significance and outcomes of OP through a mixed-methods research design.  A survey questionnaire was distributed to registered users via the OP website to collect quantitative data. For collecting qualitative data, two online focus groups and four face-to-face interviews were conducted in order to understand, describe and interpret members’ views about using OP and its effects on their knowledge and practice.  Survey respondents were recruited through an announcement posted to users of OP. Focus group and interview participants were recruited through an invitation posted to the OP website and follow-up reminders.  6.2. Key findings Findings showed that the OP platform, as an online community of practice, plays a significant positive role in learning, professional development and quality of care delivered by those pediatric clinicians who participated in the evaluation. Participants believed the user-friendly platform follows adult education principles and meets many of the users’ learning needs. Users found the educational content valid, relevant, up to date and useful for their learning and practice. Some also expressed appreciation for the availability of content in two other languages on the platform. Hands-on features such as videos, simulators, the World Share Practice Forum and Guided Learning Pathways were the most favored features reported by users.  Some barriers and issues were reported that hinder realization of the full potential the OP platform. The first and foremost barrier is lack of awareness about the OP platform. Participants   128  reported that their health professional colleagues know little or nothing about the existence of OP, which leads to lower participation among those in the pediatric community. Also, from those who connect to the community, many are not aware of the variety of educational services that OP offers. Findings suggest that OP appears to act more like an educational platform rather than an online community of practice.  For example, it lacks individual and group interactions and collaborative activities, which is one of the main features of any CoP. Being primarily an English-based platform makes it hard for many non-native English speakers to fully benefit from its content and other features. Contextual differences between developed and developing countries also play an important role in utilization of the platform. For example, fewer Internet connections, low speeds and costs of Internet access, lack of related facilities and equipment, and lack of maintenance services are some of factors in developing countries that hinder access to and use of OP. Users would like to have access to a greater variety of educational topics and see more diversity among speakers. They would prefer the length of the videos, simulators and other features to be shorter to fit their restricted time. There were also some technical issues reported that discouraged some users such as the system logging out on its own after a few hours, not saving search results, not saving passwords or having an option for users to select “keep me logged in”, and the need to scan the entire Q&A section with no way to skip sections they don’t need. These technical issues reduce satisfaction among the users.  6.3. Limitations Findings of this study should be interpreted considering several limitations. The first limitation was the relative low number of participants compared to the overall number of registered users. Usually it is hard to recruit participants via a website especially when it comes   129  to busy health care professionals and pediatricians especially in developing countries. However, using in depth focus group and individual interviews, I have tried to overcome this restriction.  The second limitation was lack of any data to compare OP users and non-users in terms of learning new knowledge and doing their job better.    The third limitation was insufficient data to compare native English speaker and non-native English speakers in terms of benefits derived from the OP platform. Finally, ambitious OCoPs, such as OP, develop over time; whereas, this study evaluated OP at one point in its development. The features of a “mature” and highly interactive OCoP represent aspirations in a long-term developmental process, which is congruent with the life cycle of CoPs.  6.4. Conclusions Six research questions guided this study. They are listed below along with brief summaries of the answers.  6.4.1. RQ1. How well does OP cover topics of interest to the community? According to the participants in this study, OP has successfully addressed the domain of interest and satisfies users with the related educational content provided.  Both quantitative and qualitative data confirmed that OP is perceived a useful platform and the educational content is relevant to the field of pediatric practice.  6.4.2. RQ2. To what degree are knowledge, experiences and stories being shared across the community?  Participants in this study believed that OP as a community of practice contains a strong core group of knowledgeable experts.  However, what seems to be missing is the exchange of   130  knowledge and experience between and among the members. OP users need to improve users’ engagement in collaborative activities expected in a vibrant CoP. 6.4.3. RQ3. To what extent are members engaging in productive and sustained interactions via the OP platform?  OP users, despite being interested in connecting to one another, are either not aware of the possibility of interacting with others via the platform or do not have enough time to do so. There is no or very limited social interaction between the core group and members or among members. The core group is doing an excellent job in creating and sharing knowledge and experiences; however, more work is needed to facilitate connections between and interactions among OP members, as a fundamental component of a vibrant OCOP.   6.4.4. RQ4. How effective is OP in supporting the professional development of members? Professional development happens, in part, by connecting to the community (such as OP) and accessing the educational contents, services and other resources available in it. OP users do not seem to be benefitting from learning through the global social interactions that OP makes possible. There is evidence that some OP members make use of the platform to support “local” professional development activities, however, the promise of the platform to promote global, connected professional development has not yet been realized.  6.4.5. RQ5. What effect is OP having on health care practices across the community?  From participants perspective, being part of the OP community had positive effects on users’ clinical work and helped them to do their job better and with more confidence. Solving medical problems, delivering safer and better patient care were also reported as benefits of watching World Share Practice, simulators, videos and the guided learning pathway. Participants   131  in focus group discussions mentioned that after participating in OP they felt more comfortable at the bedside. 6.4.6. RQ6. Overall, how helpful is OP in delivering better care and in what way(s) has it fallen short? This study shows that OP has had a positive impact on pediatricians’ clinical practice and helped them deliver better care by providing the latest educational materials in the field of pediatrics and making them available to health care professionals. Despite being successful in providing the latest knowledge in the field and delivering it via a well-developed and accessible platform, OP misses a significant portion of its potential audience due to focusing on limited number of specialties /topics. Lack of social interaction in OP among members is the other barrier for fully achieving its goals. The lack of an organized plan for promoting the platform, lack of interactions between the core group and other members, lack of encouraging members to interact via the OP community and join groups and participate in collaborative activities are examples where OP has fallen short in achieving its ambitious goals. 6.5.  Recommendations Recommendations have been provided in three sections: for OPENPediatrics leadership, for those who wish to launch OCoPs, and for those who want to evaluate OCoPs. For OPENPediatrics leaders I categorized the recommendations based on the three components of Communities of Practice—domain, community and practice.     132  6.5.1. Recommendation for OPENPediatrics leadership Domain (members’ common interests) 1. OP leaders should consider adding relevant content from a wider range of specializations by systematically investigating users’ professional development needs.  2. OP leaders should review the resources on the platform to ensure the platform does not overly privilege members from developed countries at the expense of members from less developed countries in terms of accessibility and applicability of content in “local” contexts. 3. OP leaders should continuously assess the connectivity and cost challenges faced by those who wish to access OP resources that require high bandwidth or long downloads. This is especially relevant to videos and simulations.  4. OP leaders should consider reviewing the time it takes for users to engage with all of the educational resources and keep the time required as short as possible to better match the time users have to engage.  5. OP leaders should prepare guidelines for simulation tools and equipment including the proper way of using and maintaining them and to trouble-shooting common problems.   6. OP developers should consider adding some clear guidelines for navigating through the website. Such guidelines will reduce “learning curve” experienced by new users. 7. OP developers should consider their older users who are not as comfortable with new technologies. Having a plan to help those users may increase the potential benefits older users obtain from the website. 8. OP developers should consider reviewing the website for technical issues to make OP as easy to use as possible by “fixing” some of the issues reported by users. Website   133  developers should consider adding options of “remember me” or “keep me logged in” so that users will not need to enter their password each time they log in. The other option that needs to be added is “save search results” so users will not need to start all over again when they are using the platform next time. Adding an iPad app for the program to let the users use the educational content at their convenience would also be helpful.  9. OP content leaders should consider reviewing the content to make sure it meets educational expectations of those users who need more in-depth materials. Adding more videos and simulations could be very helpful as they are more popular ways of learning mentioned by users. Written statements/guidelines in the videos will increase learning the same topic and will lead to better understanding the topic especially among those whose first language is not English. OP content leaders should be thinking of adding new versions with other languages if possible, as this will add new members to the community and will help current users. Community (social interactions) 1- OP leaders should actively and continually promote the OP platform in order to add new members and increase interaction. They need to introduce the community at every opportunity including at international conferences and at annual meetings of pediatric specialty groups. 2- OP leaders should more actively facilitate social interactions in the OP community. They should encourage and facilitate group creation and between-member interactions.  3- OP leaders also should consider involving other well-known experts in the field of pediatrics in creating content and other educational resources.    134  4- OP leaders should consider using additional motivational mechanisms to facilitate and support members’ interactions toward knowledge creation and experience sharing.  5- OP leaders should facilitate and create opportunity for members to connect with the core group especially for those members who are willing to help develop and expand the OP community.  6- OP leaders should be facilitating discussions about the community itself with the members. They should create opportunities for members to express their opinions beyond periodic user surveys in regard to processes and practices, technology, different cultures, and motivations for participating in the community. 7- OP leaders should be facilitating in-person meetings for members. Even though OP is an online community, it would be very helpful to provide occasional opportunities for community members to meet face- to- face. These meetings will help them develop friendships and build trust. They will also help the core group and community developers directly hear about members’ educational needs. Practice (shared knowledge and knowledge tools) 1. OP leaders should facilitate collaborative practice among members. They should encourage, and support members interested in jointly creating knowledge tools and educational products such as guidelines, articles, videos, books. 2. OP leaders should consider designing small group projects, sponsored by the community, to create opportunity for members to work together and produce the resources for developing the practice such as cases, effective methods for practice, articles and lessons learned.   135  3. OP leaders should facilitate members’ interactions. Live chats and synchronous discussions, for instance, allow members to discuss educational problems, needs and start collaborative activities in order to address those issues 4. OP leaders should consider creating an annual report. Creating an annual community activities report, especially focusing on collaborative activities, will motivate members and will improve community engagement. 5. OP leaders should have plan for an annual assessment linked to the annual report. They should be continually assessing knowledge production and collaborative activities in the community in order to identify new effective strategies and technologies to keep the community vibrant and active.    6.5.2. Recommendations for those who wish to launch an OCoP While the concept of Communities of Practice is not new, creating Online Communities of Practice (OCoP) is a more recent development. The desirable features of Communities of Practice, such as in-person meetings, communications, sharing knowledge and experiences are a challenge to implement in a virtual environment. It is also a challenge for OCoP developers to keep the community vital and dynamic and help it grow over the time.  Learning from literature and evaluating OP as an online community of practice, I offer several recommendations for those who desire to launch an online community of practice. 1. When creating an OCoP, the first step is to have an accurate and precise purpose. 2. The next step would be legitimating the creation of the community. Explain to prospective users why the community is needed, where it fits in the broader landscape of professional development resources, and the ways in which it is unique.   136  3. Educating potential and interested members and conducting a need assessment research to identify potential problems, needs or interests of the community would be the third step. 4. Step four is to Create a core group and support collective activities, which are very important steps in creating a successful OCoP.  5. Identifying barriers and removing them is step five that helps community to stay active and effective 6. Step six is evaluating outcomes of the community in order to improve the activities. Surveys or other assessments could be utilized on an ongoing basis to identify emerging  7.  Keeping members’ engagement sustained is step seven and also a vital step in creating or managing any OCoP. The best way for that is to create a community performance plan, leverage technology, connect people to people, define roles and responsibilities, train and support community leaders, promote awareness about the community and its values, create meaningful recognitions and reward opportunities.   6.5.3. Recommendations for those who want to evaluate on OCoP There are few examples in the literature of evaluations of OCoP. Based on the general principles of Utilization-focused evaluation and my experience conducting this study, I offer several recommendations to others who plan to conduct similar evaluations.  1. One of the important steps in evaluating any OCoP is to involve as many stakeholders as possible. It is crucial to engage related stakeholders from the very beginning of the evaluation process especially when the evaluation approach is utilization-focused evaluation 2. Recruiting participants for evaluation research is the most challenging step.  The usual means of increasing participation such as reducing time for data collection to the   137  minimum necessary, providing rewards, sending multiple reminders and using multiple communication channels should all be employed. 3. Organizing online focus group on a global scale is very complex because of the barriers such as time zone differences and people’s technology problems. For projects like OP with a global scope, in order to get greater participation from a larger number of people, individual interviews might be a better choice than online focus groups.  6.5.4. Recommendations for Further Research  More research needs to be done about how to get people fully engaged in Online Communities of Practice, especially those that are global in nature. Based on my learning from conducting this study, I offer the following recommendations for further research: 1. It would be very helpful to investigate and evaluate effects of OP on learning and professional development in mini OPs (small local communities created around the OP community). This will help OP leaders understand if OP has had “spin-off” effects on learning and work improvements among mini OPs in different countries. Learning from those communities could also help OP leaders improve the social interactions among the OP global community. 2. To investigate OP’s effects on learning and professional development, the next research could be a comparison between those who use OP and those who do not. 3. Further research could be done on comparing the effectiveness of traditional lectures and web-based learning approaches. Understanding the relative effectiveness of these two styles might be helpful in making decisions about the best use of professional development platforms like OP.   138  4. Further research could investigate OP effects on learning and professional development among those whose first language is not English compared to those of native-English speakers. 5.  Further research could be conducted to explore effective strategies to help developing countries to get more out of the OP platform.  6. More research could be done to find out the best strategies to encourage and support members’ engagement, collaborative activities, and the core group interactions in the OP community.  7. It would be very effective if researchers employ individual interviews instead of online focus group discussions as it is very complicated to recruit participants via an online website.   6.6.  Closing comments/ final reflections  It has been four years since starting my master’s degree. I was extremely excited about my project, but I was also very stressed out. With English as my second language, academic life did not seem easy. My will and determination, however, were the driving forces that made me work harder. I probably studied twice as hard and wrote twice as long as everyone else. Writing my thesis has been the most demanding yet highly rewarding endeavor in my academic life here in Canada. Now that I look back, I am proud of what I have learned from this hard work, days and late nights reading and writing, and hours of editing.  Carrying out this study was a transformative process for me, a self-discovery experience of how much I could persist to achieve something I consider worthwhile. I am one of those people who love to learn and always seek to obtain more knowledge in and out of the classroom. I am   139  especially passionate about learning things that are related to my previous studies and work experiences in health education. I studied public health and worked in the health education department in Ministry of Health and Medical Education (Iran) as a health care professional for more than ten years. It was a time I realized that health professionals might make great health care clinicians due to their related education and experiences, but they wouldn’t necessarily make good teachers. Being an effective educator requires special preparation, which requires more than taking just one or two courses. In other words, health professionals need to learn how to teach people, especially adults, as their main audience. This problem was the reason many of our health education activities failed to achieve the desired outcomes. With this in mind, I decided to pursue a program in educational studies when I moved to Canada. While studying in my master’s, I was introduced to the OPENPediatrics platform as a global educational endeavor by Dr. Kissoon and was encouraged to think about its evaluation. This, in turn, led me to learn about how to evaluate educational activities. So, I started to take evaluation courses. I took two evaluation courses and three research methodology courses. While taking those courses I always had OP in my mind as an example of an educational program to be evaluated. Eventually, the evaluation of OP became the focus of my Master’s thesis.  Considering the fact that OP is a health education program, I thought working on the evaluation of OP could be a great opportunity for me to get experience evaluating educational programs.  I originally had no idea as to what OP was, but after spending a couple of years researching and writing about it I can now say I know more about OP than I ever could have hoped. I learned about how a global educational program is planned, launched and evaluated. It was very impressive that the main purpose for developing the OP platform was to help developing   140  countries that have a smaller health workforce and more health issues. I was very interested in finding out how and to what degree developing countries are benefiting from using this platform and how connecting to a global community such as OP would help them to work better and save more lives.  My favorite part of evaluating OP first was studying “The Commission on Education of Health Care Professionals for the 21st Century” and the fact that the need for health care professionals’ education that I was thinking about many years ago was actually the main reason for forming this commission. It was a joy when I realized I am in right path by pursuing educational studies. My second favorite was talking face- to-face with users while interviewing them or conducting online focus group discussions. It was a great chance for me to hear from them in person, especially from those who live in developing countries, about their experiences with and feelings about using OP. Evaluating OP helped me better understand online communities of practice, their technical and social aspects, the way they work, grow and improve practice or fade away due to problems and barriers they may encounter.  My least favorite part was when it became clear that the OP program--originally initiated to help those people in developing courtiers the most--is serving them the least. This is due, in part, because they are so busy dealing with a wide variety of daily clinical problems which are totally different from those in developed countries, that they cannot spend time using OP.  Other problems such as less English language proficiency, restricted time, higher workloads, lack of adequate equipment and or maintenance services, lack of Internet access also hinder them from fully benefiting from the OP platform.  To conclude, I feel that my Master’s journey has been a valuable opportunity to learn great lessons and get prepared now to apply all those lessons in my career as an adult education/ health   141  education specialist. The road toward success was not easy to navigate through especially when the job is working on a multi-institutional project, which mine was. It was easy to get lost along the way, delayed, and give in to distractions while working with two different organizations with different expectations for the research. It was so easy to get trapped in those paralyzing moments but with the help of my family and my wonderful academic advisors, I was able to deal with all issues and move forward. It was a long and difficult journey not just for me but also for my husband and son who supported me throughout the process. This experience was transformative as I came out with more knowledge, better understanding and more confidence in evaluating educational programs, which was my personal reason for carrying out this thesis.   142  References Adams, A. E., Nnawulezi, N. A., & Vandenberg, L. (2015). Expectations to change (e2c): a participatory method for facilitating stakeholder engagement with evaluation findings. American Journal of Evaluation, Online First Version, 36(2), 1–13. Alali, H., & Salim, J. (2013). Virtual communities of practice success model to support knowledge sharing behaviour in healthcare sector. Procedia Technology, 11(Iceei), 176–183. Bates, T. (2015). Online learning and distance education resources. Baylor, L. (2014). The impact of an online community of practice on preservice teachers’ perception of technological self-efficacy. Ann Arbor,  MI:Walden University. Beasley, S. W. (2013). The challenges facing training in pediatric surgery worldwide, Front. Pediatr. 1(24), 1–4. Boud, D., & Middleton, H. (2003). Learning from others at work: Communities of practice and informal learning. Journal of Workplace Learning, 15(5), 194–202. Boyd, P., & Lawley, L. (2009). 10: Becoming a lecturer in nurse education: the work-place learning of clinical experts as newcomers. Learning in Health and Social Care, 8(4), 292–300. Burnette, K., Ramundo, M., Stevenson, M., & Beeson, M. S. (2009). Evaluation of a web-based asynchronous pediatric emergency medicine learning tool for residents and medical students. Academic Emergency Medicine, 16, S46–S50. Cambridge, D., Kaplan, S., & Suter, V. (2005). Community of practice design guide. American academics and higher education. Carolan, I., Smith, T., Hall, A., & Swallow, V. M. (2014). Emerging communities of child-healthcare practice in the management of long-term conditions such as chronic kidney disease: qualitative study of parents’ accounts. BMC Health Services Research, 14(1), 292. Carolina, N., & Carolina, S. (2010). National shortage of pediatric specialists, (September). China Medical Board. (2010). Health professionals for the 21st century. Retrieved March 11, 2018, from Detlor, B. (2004). Towards knowledge portals: From human issues to intelligent agents. Springer Science & Business Media, B.V. Dicicco-bloom, B., & Crabtree, B. F. (2006). Making sense of qualitative research: The qualitative research interview. Medical Education, 40, 314–321.   143 Dubé, L., Bourhis, A., & Jacob, R. (2006). Towards a typology of virtual communities of practice. Interdisciplinary Journal of Information, Knowledge, and Managemen, 1, 69–93. Edgecombe, K., Wotton, K., Gonda, J., & Mason, P. (1999). Dedicated education units: 1. A new concept for clinical teaching and learning. Contemporary Nurse : A Journal for the Australian Nursing Profession, 8(4), 166–171. Eliot & Associates. (2005). Guidelines for conducting a focus group. Retrieved January 16, 2018, from Evance, C., Yeung, E., Markoulakis, R., & Guilcher, S. (2014). An Online Community of practice to support evidence-based physiotherapy practice in manual therapy. Journal of Continuing Education in the Health Professions, 4(34), 215–223. Evans, J. R., & Mathur, A. (2005). The value of online surveys. Internet Research (Vol. 15). Feilzer, M. (2010). Doing mixed methods research pragmatically: Implications for the rediscovery of pragmatism as a research paradigm. Journal of Mixed Methods Research, 4(1), 6–16. Fitzpatrick, R., & Boulton, M. (1996). Qualitative research in health care: I. The scope and validity of methods. Journal of Evaluation in Clinical Practice, 2(2), 123–130. Fletcher, R. H., Fletcher, S. W., Jiménez, V., Díaz De Salas, S., Mendoza, V., Porras, C., … Noor, K. B. M. (1997). Case study as a research method. Academy of Management Review, 5(2), 301–316. Floding, M., & Swier, G. (2011). Legitimate Peripheral Participation : Entering a community of practice. Reflective Practice: Formation and Supervision in Ministry, 31(Online), 193–204. Retrieved from Fred, P. ;, Renkl, A. ;, & Sweller, J. (2003). Cognitive load theory. In Educational psychologist (1st ed.). Sydney. Frenk, J., Chen, L., Bhutta, Z. A., Cohen, J., Crisp, N., Evans, T., … Zurayk, H. (2010a). Health professionals for a new century: Ttransforming education to strengthen health systems in an interdependent world. The Lancet, 376(9756), 1923–1958. Frenk, J., Chen, L., Bhutta, Z. A., Cohen, J., Crisp, N., Evans, T., … Zurayk, H. (2010b). Health Professionals for a New century. Washington, Dc: The Lancet. Fuller, A., Hodkinson, H., Hodkinson, P., & Unwin, L. (2005). Learning as peripheral participation in communities of practice: a reassessment of key concepts in workplace learning. British Educational Research Journal, 31(1), 49–68. Gaál, Z., Szabó, L., Obermayer-Kovács, N., & Csepregi, A. (2014). Exploring the role of social   144  media in knowledge sharing. The Electronic Journal of Knowledge Management, 13(3), 185–197. Gardner, K., Bridges, S., & Walmsley, D. (2012). International peer review in undergraduate dentistry: Enhancing reflective practice in an online community of practice. European Journal of Dental Education, 16(4), 208–212. George, M., & George, M. (2012). Teaching focus group interviewing: Benefits and challenges. American Sociological Association, 41(3), 257–270. Global Solution Networks. (2014). Openpediatrics: Lighthouse case study. Retrieved from Gotsch, A. R. (2001). Profession in the twenty-first century profession in the twenty-first century. U.S.A: Coalition of National Health Education Organizations (CNHEO). Grealish, L., Bail, K., & Ranse, K. (2010). Investing in the future: residential aged care staff experiences of working with nursing students in a “community of practice.” Journal of Clinical Nursing, 19(15–16), 2291–9. Green, J., & Thorogood, N. (2004). Qualitative methodology and health research. Qualitative Methods for Health Research, 3–26. Gunawardena, C. N., Beth, M., & Carol, H. (2002). A theoritical framework for building online Communities of Practice with social networking tools. Hall, B. H. (2017). Adoption of new technology adoption of new technology Bronwyn H . Hall Beethika Khan Working Paper. Massachusetts. Hancock, B., Windridge, K., & Ockleford, E. (2007). An introduction to qualitative research. Qualitative Research. Nottingham, UK: The NIHR RDS EM / YH. Henning, P. H. (2004). Everyday cognition and situated learning. Handbook of research on educational communications and technology, 143–168. Hew, K., & Hara, N. (2006). Identifying factors that encourage and hinder knowledge sharing in a longstanding online community of practice. Journal of Interactive Online Learning, 5(3), 297–316. Holloway, I., & Wheeler, S. (2010a). Qualitative research in nursing and healthcare (3rd ed.). Iowa, USA: Wiley-Blackwell. Holloway, I., & Wheeler, S. (2010b). Qualitative research in nursing and healthcare (3rd ed.). Iowa, USA: Wiley-Blackwell. Hsu, M., Ju, T. L., Yen, C., & Chang, C. (2007). Knowledge sharing behavior in virtual communities: The relationship between trust , self-efficacy , and outcome expectations, 65, 153–169. Jelenic, D. (2011). The importance of knowledge management in organizations – with emphasis on the balanced. Management, Knowledge and Learning International Conference 2011, 33–43.   145  Jiménez-zarco, A. I., González-gonzález, I., Saigí-rubió, F., & Torrent-sellens, J. (2015). The co-learning process in healthcare professionals: Assessing user satisfaction in virtual Communities of Practice. Computers in Human Behavior, 51, 1303–1313. Junaidu, S. (2008). Effectiveness of multimedia in learning & teaching data structures online. Turkish Online Journal of Distance Education, 9(4), 97–107. Kelsey, K. D., & Stafne, E. T. (2012). A model for evaluating eXtension Communities of practice. Journal of Extension, 50(5), 1–4. King, W., Chung, T., & Haney, M. (2009). Knowledge management and organizational learning. Omega, 4(2), 167–172. Kitzinger, J. (1994). The methodology of focus groups: The importance of interaction between research participants. Sociology of Health & Illness, 16(1), 103–121. Kroll, T., & Neri, M. (2009). Designs for mixed methods research. In S. Andrew & E. J. Halcomb (Eds.), Mixed methods research for nursing and the health sciences (First, p. 230). Malaysia: Wiley-Blackwell. Krueger, R., & Casey, M. (2001). Designing and conducting focus group interviews. In R. Krueger, C. Mary, J. Donner, J. Maack, & S. Kirsch (Eds.), Social Development Papers: Social Analysis Selected Tools and Techniques (pp. 4–23). Washington, D.C., USA. Kvale, S. (1996). Interviews: an introduction to qualitative research interviewing. SAGE Publications,Inc. Lave, J., & Wenger, E. (1991). Situated learning: legitimate peripheral participation. New York, USA: Cambridge University Press. Lave, J., & Wenger, E. (2002). Legitimate peripheral participation in communities of practice. Supporting Lifelong Learning. Lee, J. H., & Sadana, R. (2011). Improving Equity in health by addressing social determinants. world health organization. LeMay, A. (2009a). Communities of practice in health and social care. (A. LeMay, Ed.) (First). Wiley-Blackwell. LeMay, A. (2009b). Communities of practice in health and social care (First). West Sussex, UK: Wiley-Blackwell Publishing Ltd. Liamputtong, P. (2011). Focus group methodology : Introduction and history. In Focus group methodology: principles and practice (pp. 1–14). london: SAGE Publications Ltd. Lincoln, Y. S., & Guba, E. G. (1985). Naturalistic inquiry. California, USA: SAGE Publications, Inc. Malamut, M. (2013, October). Boston children’s and IBM team up. Boston Magazine, Boston Children’s and IBM team up. Retrieved from   146 Malterud, K. (2001). Qualitative research: Standards, challenges, and guidelines. Qualitative Research Series, 358(panel 2), 483–488. Mathur, S. C. (2011). Health professionals for twenty-first century. Health and Population, 34(4), 215–222. McKellar, K. A., Pitzul, K. B., Yi, J. Y., & Cole, D.C. (2014). Evaluating communities of practice and knowledge networks: A systematic scoping review of evaluation frameworks. EcoHealth, 11(3), 383–399. Meessen, B., Kouanda, S., Musango, L., Richard, F., Ridde, V., & Soucat, A. (2011). Communities of practice: The missing link for knowledge management on implementation issues in low-income countries? Tropical Medicine and International Health, 16(8), 1007–1014. Metaxiosis, K., Karrillo, F., & Yigitcanlar, T. (2010). Knowledge-based development for cities and societies: Integrated multi-level approaches. New Yourk: Information Science Reference. Michael, C., & Patton, Q. (2005). Encyclopedia of evaluation. In M. Sandra (Ed.), (pp. 430–434). Thousand Oaks: SAGE Publications,Inc. Morgan, D. L. (1996). Focus groups. Annual Review of Sociology, 22, 129–152. Moule, P. (2006). E-learning for healthcare students : Developing the communities of practice framework. Journal of Advanced Nursing, 3(54), 370–380. Nagle, B., & Williams, N. (2011). Methodology brief: Introduction to focus groups. Center for Assessment, Planning & Accountability. Retrieved  December 10th, 2018 from,  Ontario Centre of Excellence for Child and Youth Mental Health. (2013). Program evaluation toolkit. Ottawa, Canada: Ontario Centre of Excellence for Child and Youth Mental Health. Retrieved from OPENpediatrics. (2013). OPENPediatrics | The world’s best practices, shared worldwide. Retrieved April 8, 2018, from OPENpediatrics. (2015). OPENpediatrics. Retrieved April 8, 2018, from Parboosingh, J. T. (2002). Physician communities of practice: where learning and practice are inseparable. The Journal of Continuing Education in the Health Professions, 22(4), 230–6. Patel, R. (2007). Evaluation and assessment of the online postgraduate critical care nursing course. In K. Klaus, J. Warren, & L. Tze-Yun (Eds.), Medinfo 2007: Proceedings of the 12th World Congress on Health (Medical) Informatics; Building Sustainable Health Systems (pp. 1377–1381). IOS Press. Retrieved from;dn=783515848154814;res=IELHEA%3  147  E ISBN: 9781586037741 Patton, M. Q. (2000). Utilization-focused evaluation. In D. Stufflebeam, G. Madaus, & T. Kellaghan (Eds.), Evaluation Models. Viewpoints on Educational and Human Services Evaluation (2nd ed., pp. 245–238). Boston: Kluwer Academic Publishers. Patton, M. Q. (2008a). Toward more useful evaluations. In Utilization-focused evaluation (4th ed., pp. 23–89). Thousand Oaks, California: SAGE Publications. Retrieved from Patton, M. Q. (2008b). Utilization-focused evaluation, 24(2), 5–6. Pope, C., & Mays, N. (1995). Reaching the parts other methods cannot reach: An introduction to qualitative methods in health and health services research. BMJ (Clinical Research Ed.), 311(6996), 42–45. Preskill, H., & Jones, N. (2009a). A practical guid for engaging stakeholders in developing evaluation questions. NJ: Robert wood Johnson foundation, Preskill, H., & Jones, N. (2009b). A practical guid for engaging stakeholders in developing evaluation questions. NJ: Robert wood Johnson foundation. Ramírez, R., & Brodhead, D. (2013). Utilization focused evaluation a primer for evaluators. George Town, Malaysia: Southbound Sdn. Bhd. Ranmuthugala, G., Cunningham, F., Plumb, J., Long, J., Georgiou, A., Westbrook, J., & Braithwaite, J. (2011). A realist evaluation of the role of communities of practice in changing healthcare practice. Implementation Science, 6(1), 49. Ranmuthugala, G., Plumb, J., Cunningham, F., Georgiou, A., Westbrook, J., & Braithwaite, J. (2010). Communities of practice in the health sector: A systematic review of the peer-reviewed literature. Sydney, Australia: Centre for Clinical Governance Research- Australian Institute of Health Innovation. Retrieved from Ranmuthugala, G., Plumb, J., Cunningham, F., Georgiou, A., Westbrook, J., & Braithwaite, J. (2011). How and why are communities of practice established in the healthcare sector? A systematic review of the literature. BMC Health Serv Res, 11(273), 1–16. Ranse, K., & Grealish, L. (2007). Nursing students’ perceptions of learning in the clinical setting of the dedicated education unit. Journal of Advanced Nursing, 58(2), 171–9. Rindstedt, C., & Aronsson, K. (2012). Children’s intent participation in a pediatric community of practice. Mind, Culture, and Activity, 19(4), 325–341. Roeckelein, J. (2006). Elsevier’s dictionary of psychological theories. Amsterdam.   148  Ruth, C., & Mayer, R. (2011). e-Learning and the Science of Instruction. (T. Rebecca, Ed.) (third). San Francisco-United States of America: John Wiley& Sons, Inc. Sayer, L. (2014). Communities of practice, a phenomenon to explain student development in community nursing. Primary Health Care Research & Development, 15(4), 430–40. Scarso, E., Bolisani, E., & Salvador, L. (2009). A systematic framework for analysing the critical success factors of communities of practice. Journal of Knowledge Management, 13(6), 431–447. Scheffler, R., Cometto, G., Tulenko, K., Bruckner, T., Liu, J., Keuffel, E. L., … Campbell, J. (2016). Health workforce requirements for universal health coverage and the sustainable development goals. Human Resources for Health Observer Series, (17), 1–40. 978 92 4 151140 7 Scheffler, R., Liu, J., & Dal, M. (2008). Forecasting the global shortage of physicians: An economic- and needs-based approach. Bulletin of the World Health Organization, 86(7), 516–523. SEG Resaech Team. (2008). Understanding multimedia learning: Integrating multimedia in the K-12 classroom. Brainpop. Retrieved from Shannon-Baker, P. (2016). Making paradigms meaningful in mixed methods research. Journal of Mixed Methods Research, 10(4), 319–334. Shaw, J. A., Connelly, D. M., & Zecevic, A. A. (2010). Pragmatism in practice: Mixed methods research for physiotherapy. Physiotherapy Theory and Practice, 26(8), 510–518. Spedding, R., Jenner, R., Potier, K., Mackway-Jones, K., & Carley, S. (2013). Blended learning in paediatric emergency medicine: Preliminary analysis of a virtual learning environment. European Journal of Emergency Medicine : Official Journal of the European Society for Emergency Medicine, 20(2), 98–102. Spradley, J. P. (1979). The ethnographic interview. Wadsworth Publishing. Belmont, CA Stevenson, A. (Ed.). (2010). Oxford Ditionary of English (3rd ed.). London: Oxford University Press. Sweet, C. (2001). Designing and conducting virtual focus groups. Qualitative Market Research: An International Journal, 4(3), 130–135. Tanja, T., Lindgren, S., Wohlin, M., Lagervall, H., & Andersson, F. (2015). Educating health professionals for tomorrow’s global health care systems - the role of universities. Lund. Thomas, Fried, G., Johnson, P., & Stilwell, B. (2010). Sharing best practices through online communities of practice: A case study. Human Resources for Health, 8(1), 25. Thomas, M. (2017). ATLAS.ti [Software for Qualitative Data Analysis]. Berlin: Scientific   149  Software Development GmbH. Thrysoe, L., Hounsgaard, L., Dohn, N. B., & Wagner, L. (2010). Participating in a community of practice as a prerequisite for becoming a nurse - Trajectories as final year nursing students. Nurse Education in Practice, 10(6), 361–6. Thrysoe, L., Hounsgaard, L., Dohn, N. B., & Wagner, L. (2012). 2: Newly qualified nurses-experiences of interaction with members of a community of practice. Nurse Education Today, 32(5), 551–5. Tony, B., & Conner, M. (2015). The new social learning: Connect. collaborate. Work (Second edi). ATD press. Trottier, D. (2014). Using software for qualitative data analysis: Research outside paradigmatic boundaries. Emeraldinsight, 13, 51–72. Tsai, C. (2012). Understanding social nature of an online community of practice for learning to teach. Journal of Educational Technology & Society, 15(2), 271–285. Tsai, C., Laffey, J. M., & Hanuscin, D. (2010). Effectiveness of an online community of practice for Learning to teach elementary science. Journal of Educational Computing Research, 43(2), 225–258. Tucker, C. (2014, March). WHO: Global health worker shortage to have major impact in next decades. The Nation’s Health, p. 15. Turney, L., & Pocknee, C. (2005). Virtual focus groups: New frontiers in research. International Journal of Qualitative Methods, 4, 32–43. UNICEF. (2013). Committing to child survival: A promise renewed. Unicef Progress Report 2013. New York, USA: Division of Data, Research and Policy, UNICEF. Retrieved from Uys, L., & Middleton, L. (2015). 5: Internationalising university schools of nursing in South Africa through a Community of Practice, 115–123. Valaitis, R. K., Akhtar-Danesh, N., Brooks, F., Binks, S., & Semogas, D. (2011). Online communities of practice as a communication resource for community health nurses working with homeless persons. Journal of Advanced Nursing, 67(6), 1273–84. Walsh, A. (2015). 7: The effect of social interaction on mental health nurse student learning. Nurse Education in Practice, 15(1), 7–12. Wenger, E. (2000). Communities of practice and social learning systems. Organization, 7(2), 225–246. Wenger, E., McDermott, R., & Snyder, W. (2002a). Cultivating communities of practice: A guide to managing knowledge,. Harvard Business School Press, Cambridge, USA,. Wenger, E., McDermott, R., & Snyder, W. (2002b). Cultivating communities of practice: A guide to managing knowledge. Harvard Business School Publishing. Buston. Wiesenberg, F. (2000). A critical appraisal model of program evaluation in adult continuing   150  education. Canadian Journal of University Continuing Education, 26(1), 79–109. Wolbrink, T., & Burns, J. (2012). Internet-based learning and applications for critical care medicine. Journal of Intensive Care Medicine, 27(5), 322–332. Woodyatt, C. R., Finneran, C. A., & Stephenson, R. (2016). In-person versus online focus group discussions: A comparative analysis of data quality. Qualitative Health Research, 26(6), 741–749. Yin, R. K. (2011). Qualitative research from start to finish. Journal of Chemical Information and Modeling (Vol. 53). London and New York: Guilford Press. Zboralski, K., Salomo, S., & Gemuenden, H. G. (2006). Organizational benefits of communities of practice: A two-stage information processing model. Cybernetics and Systems, 37(6), 533–552.      151  Appendices Appendix A: Online questionnaire Online questionnaire Part 1: Demographic information Please fill in the blank space that best corresponds to you. 1. Please select your gender?      □ Male □ Female         □ Other 2. Please select your age range? □ 20-34 years   □ 35-49 years   □ 50-64 years    □ 65 years or older  3. Please select the professional background representing your current practice.   □Physician- Attending/Consultant □ Physician – Resident/Registrar □ Physician-Fellow □ Physician Assistant □ Surgeon- Attending/ Consultant □ Surgeon – Resident/Registrar □ Surgeon – Fellow □Nurse □ Nurse Educator □Nurse practitioner  □Other health care professional  □Child Life Specialist □Respiratory Therapist □Non- Health Professional □Health care Administrator □Medical Student □ Emergency Medical Technician □Clinical Officer □Laboratory Technician □Pharmacist □Physiotherapist □Biomedical Engineer    □ General Surgery □ Neurological Surgery   152  4-Please select the specialty area best representing your current practice.   □ Physical Medicine and Rehabilitation □ Allergy and Immunology □ Family Medicine □ Internal Medicine □ Neurology □ Orthopedic Surgery □ Thoracic Surgery □Urology  5. Please indicate your country of registration.  □ North America □ Europe □ Asia □ South America □ Africa □ Central America □ Australia Part 2: Questions related to the OPENPediatrics website 6. How long have you been using OPENPediatrics?   □ Two or more years □ Between one and two years □ Between 6 months and one year □ Between 3 and 6 months □ Less than 3 months □ Not sure 7- Approximately how many times in total have you visited OPENPediatrics?   □ Once □ 2-5 times □ 6-10 times □ More than 10 times □ More than 50 times □ More than 100 times □ Not sure   153  8- How frequently do you visit OPENPediatrics?  □ Daily □ Weekly □ Monthly □ Occasionally (less than once a month) □ Not sure □ Other (please specify) 9-What are common reasons for you to use OPENPediatrics? (Please select up to THREE of the options)  □ Learn basic information about a topic □Learn more in-depth information about a topic □Learn how to solve a medical problem □Learn how to deliver safer or more effective patient care □Review things I already know □Learn the latest advances or developments in an area □ I was assigned to watch the video by a supervisor □World Shared Practice Forum videos are part of the educational curriculum at my institution □ Other (please specify) 10- Where do you most commonly access OPENPediatrics?   □ Home □ Work (clinical setting) □ Work (non-clinical setting) □ During my commute  11- Which type(s) of internet connection do you most often use to access OPENPediatrics?   □ Work internet □ Home internet □ Cellular data network □ Other (please specify): □ Satellite    154  12- What device do you most commonly use to access OPENPediatrics?  □ Computer □Smartphone □ Tablet □ Other (please specify)  13- Do you use Internet Explorer as your main browser to access OPENPediatrics? □ Yes □ No 14- What technical barriers, if any, do you face in accessing the OP platform?  □ I have not encountered any technical problems using OPENPediatrics □ Problems with Internet connection □ Limited internet access □ I do not want to use data on my cellular phone plan □ Lack of adequate computers or other devices □ OPENPediatrics is not compatible with my device(s) □ Other 15- Which of the OPENPediatrics features you are aware of (Please select all that apply)  □ Videos □ simulators □ World Share Practice Forum □ Guided Leaning Pathway □ Geggel's Congenital Heart Disease Library □ Group 16- Have you ever used OPENPediatrics as a Group?  □ Yes □ No   155  17- Regarding OPENPediatrics Groups, which of the statements apply to you? (Please select all that apply) □ I have joined an OPENPediatrics Group □ I have browsed an OPENPediatrics Group □ I have created or been an administrator for an OPENPediatrics Group Please check the option that best describes your opinion on the following topics with regard to OPENPediatrics: Strongly agree Agree Uncertain Disagree Strongly disagree 18-The OPENPediatrics website is easy to use.      329- The OPENPediatrics website is visually appealing.      20- The resources on OPENPediatrics are up-to-date and reflect current best knowledge and evidence.      21- The resources on OPENPediatrics are relevant to my learning needs.      22- OPENPediatrics had positive impact on my clinical practice.      23- Overall, I consider OPENPediatrics to be a useful resource.      24- I plan to use OPENPediatrics in the future.      25- I would recommend OPENPediatrics to my peers.      26- OPENPediatrics videos are relevant to my clinical practice.        156  27- OPENPediatrics videos are up to date and reflect current best knowledge and evidence.      28- I find OPENPediatrics videos to be engaging.      29- I find the length of OPENPediatrics videos to be just right.      30- I will watch OPENPediatrics videos in the future.      31- OPENPediatrics World Shared Practice Forum videos are relevant to my clinical practice.      32- OPENPediatrics World Shared Practice Forum videos are up to date and reflect current best knowledge and evidence.      33- I will watch OPENPediatrics World Shared Practice Forum videos in the future.      34- I watched OPENPediatrics World Shared Practice Forum videos as part of a group? □ Yes □ No 35- OPENPediatrics Guided Learning Pathways are relevant to my clinical practice.      36- OPENPediatrics Guided Learning Pathways are up to date and reflect current best knowledge and evidence.      37- I will use OPENPediatrics Guided Learning Pathways in the future.        157     38- I did not complete an OPENPediatrics Guided Learning Pathway because … ( Select all that apply)  □ I have not enough time □ I am still working on the Guided Learning Pathway and intend to complete it □ I ran into technical barriers when using the Guided Learning Pathway □I did not find the content engaging □I was not required to complete the Guided Learning Pathway by my supervisor □I did not find the content relevant to my practice □Other (please specify)  39-I have used the OPENPediatrics simulators listed below in the last three months?  □ Mechanical Ventilation Simulator □ Peritoneal Dialysis Simulator □ None □ Hemodialysis Simulator  40- The OPENPediatrics simulators are relevant to my clinical practice. Strongly agree Agree Uncertain Disagree Strongly disagree      41- The OPENPediatrics simulators are up-to-date and reflect current best knowledge and evidence.       42- The OPENPediatrics simulators enable me to better perform key tasks.      43- I will use the OPENPediatrics simulators in the future.      44- Please rate the importance of the following items in terms of improving your OPENPediatrics experience: Very Important  Important  Uncertain Not Important Not at all important a) Offline browsing capability      b) Downloadable materials        158  c) Standalone mobile app      45- What OPENPediatrics features do you find most essential?    46- What do you find most valuable about OPENPediatrics?    47- What is the most frustrating thing about OPENPediatrics?           159  Appendix B: Focus group/ interview questions Engagement questions 1. How do you experience being part of OPENPediatrics’ community? Exploration Questions 2. When you started using OP, what value or benefits were you expecting to get from it? 3. To what degree has OP met your expectations; in what way(s) has it fallen short? 4. How well [and in what ways] have the topics included in OP addressed your professional development learning needs?  5. Which features of OP have been particularly useful to you [and can you provide an example]?  Exit question Is there anything else you would like to say about how OPENpediatrics affected your life in total?  


Citation Scheme:


Citations by CSL (citeproc-js)

Usage Statistics



Customize your widget with the following options, then copy and paste the code below into the HTML of your page to embed this item in your website.
                            <div id="ubcOpenCollectionsWidgetDisplay">
                            <script id="ubcOpenCollectionsWidget"
                            async >
IIIF logo Our image viewer uses the IIIF 2.0 standard. To load this item in other compatible viewers, use this url:


Related Items