Open Collections

UBC Graduate Research

An evaluation of the teacher education eportfolio project at the University of British Columbia Wright, Lucas 2009-07

Your browser doesn't seem to have a PDF viewer, please download the PDF to view this item.

Notice for Google Chrome users:
If you are having trouble viewing or searching the PDF with Google Chrome, please download it here instead.

Item Metadata

Download

Media
42591-Wright_Lucas_ADHE_590_Evaluation_2009.pdf [ 22.84MB ]
Metadata
JSON: 42591-1.0340066.json
JSON-LD: 42591-1.0340066-ld.json
RDF/XML (Pretty): 42591-1.0340066-rdf.xml
RDF/JSON: 42591-1.0340066-rdf.json
Turtle: 42591-1.0340066-turtle.txt
N-Triples: 42591-1.0340066-rdf-ntriples.txt
Original Record: 42591-1.0340066-source.json
Full Text
42591-1.0340066-fulltext.txt
Citation
42591-1.0340066.ris

Full Text

 AN EVALUATION OF THE TEACHER EDUCATION EPORTFOLO PROJECT AT THE UNIVERSITY OF BRITISH COLUMBIA by LUCAS WRIGHT A GRADUATING PROJECT SUBMITTED IN PARTIAL FULFILLMENT OF THE REQUIREMENTS FOR THE DEGREE OF MASTER OF EDUCATION IN THE FACULTY OF GRADUATE STUDIES (Adult and Higher Education) Approved by: Thomas Sork Supervisor Anne Schoelfield Second Reader July 28, 2009 Date Approved THE UNIVERSITY OF BRITISH COLUMBIA July 23, 2009 Introduction to ePortfolios in Teacher Education The use of portfolios, a purposeful collection of work as a means of assessment is not -new (Millman, 1999). Portfolios have been used in fields such as engineering, architecture, visual art and design to assess participants' competencies for decades (Greenburg, 2004). However, recently, a new kind of portfolio for assessment has emerged: ePortfolios. ePortfolios are defined by Lorenzo and Itellson (2005) as "a personalized, Web-based collection of work, responses to work, and reflections that are used to demonstrate key skills and accomplishments for a variety of contexts and time periods (p. 1)." This definition encapsulates a number of important features that are unique to this new form of portfolio. • An ePortfolio is like a physical portfolio, a collection of artifacts that demonstrates the acquisition of skills, notions and competencies; however, this collection is stored digitally. As Love, McKean, and Gathercoal, (2004) point out, the concept of digital storage is quite expansive and might include a portfolio made with programs such PowerPoint which are stored on CDs or memory drives and may also refer to ePortfolios that are stored digitally on the Internet and accessed and edited on any Internet connection. In this way, they are potentially larger and more accessible than traditional portfolios. • Lorenzo and Ittelson's definition also captures the nature of ePortfolios as a constantly growing and interactive document-as opposed to the more static paper portfolio form. 2 ) ePortfolios in Teacher Education Since the late 1990s, there has been an increase in the number of teacher education programs adopting ePortfolios (Millman, 1999). There are a number of factors behind this growth, including an increased emphasis on assessment and standard-driven evaluation of teacher training programs; an increased emphasis on authentic and collaborative forms of assessment in education (Wikersham & Chambers, 2006); the maturing of Internet technology; (Love et aI, 2004) and an increased access to powerful web publishing tools (Greenburg, 2004). In teacher education programs in North America, ePortfolios are often implemented as a means to provide evidence that a teacher candidate has developed skills and competencies that are required by national or provincial licensing or certifying bodies. In the United States, pre-service teacher education programs have used ePortfolios to provide evidence that teacher candidates have met the National Council for Accreditation for Teacher Education (NCATE) standards (Campbell, 2006). In British Columbia, teacher education programs at the University of British Columbia and at the University of Victoria incorporate ePortfolios as a way to provide evidence that teacher candidates have met the Standards for the Education, Competence and Professional Conduct of Educators in BC prescribed by the British Columbia College of Teachers (A. Scholefield, personal communication, September 5, 2009). ePortfolios are also employed in teacher education programs as constructivist learning tools intended to enable more collaborative assessment and facilitate media literacy and technological skill development (Greenburg, 2004). 3 l ' ePortfolios in Teacher Education at the University of British Columbia The teacher education program at the University of British Columbia offers pre-service teaching courses, to prepare teacher candidates for certification with the British Columbia College of Teachers (BCCT) (http://teach.educ.ubc.caJ). In accordance with the mandate of the BCCT, teacher candidates graduating in British Columbia are expected to meet the legislated standards of competency as outlined in the BC College of Teachers Standards for the Education, Competence and Professional Conduct of Educators in BC (Standards for Education Competence, 2008). After it has been deemed that a teacher candidate has completed their graduation requirements the BCCT must be confident that they have met the eight standards of competency. In cooperation with the BCCT the Teacher Education Office (TEO) will ensure that all teachers have met these standards before they are recommend to the accreditation with the college (UBC Attainment of Standards Report, 2006 ). When audited by the College the TEO intends to provide ePortfolios as evidence that these standards have been attained. In order to accommodate this the TEO has integrated ePortfolios into their program. ePortfolios were introduced program-wide in September 2006 and the WordPress platform was integrated as the ePortfolio practicum in 2007/2008. Previous to this project a number of teacher education cohorts developed ePortfolios as part of their overall assessment (Scholefield, Anne, Personal Communication, January 7, 2009). In the current program, teacher candidates upload digital artifacts and reflect on them in relation to UBC Attainment of Standards Report (ASR) which addresses the BC College of Teachers Standards for the Education, Competence and Professional Conduct of Educators in BC. (Scholefield, Anne, Personal Communication, March 17, 2009). 4 For each teacher candidate, the creation ofthe ePortfolio begins with the provision of a Computing Media Services (CMS) account. Teacher candidates are electronically sent the a user name and password for their ePortfolio account. Once the teacher candidate signs into their Web-based Wordpress account they can begin to upload artifacts and write reflections from their courses and school visits. In the final month of the program teacher candidates complete the EDUC 480 capstone course during which they received technical and theoretical support to review the concept of ePortfolios and their applicability in elementary and secondary education (htip://teach.educ.ubc.cal.) The primary objective of this course is to provide teacher candidates with opportunities to share their ePortfolios and to receive peer feedback. At the summation of the course the ePortfolios are graded on a pass/fail basis by the instructor and the artifacts designated by the teacher candidates to address the standards are automatically saved in an audit file (J.Yamamato, personal communication, October 12,2007). The "saved" audit file remains in UBC electronic archives for 5-years during which time it cannot be edited. Teacher candidates have access to their ePortfolio for one year after graduating from the program, however, they can export their entire ePortfolio to a free Web-based WordPress platform and can maintain their ePortfolios on this server indefinitely (A. Schofield, personal communication, May 9, 2009). Support Provided to the Teacher Candidates The Teacher Education Office (TEO), the Office of Learning Technology (OLT) and Computing Media Services (CMS) offer technical support for ePortfolio development to the teacher candidates and TEO instructors. In terms of teacher candidates, this support occurs through the ecoaching program. Approximately 50 teacher candidates are hired and trained at the beginning of each program to become ecoaches. The ecoaches work within their cohorts 5 assisting teacher candidates to develop their ePortfolios in WordPress through workshops and in-class assistance (J.Yamamato, personal communication, October 12, 2007). In addition, teacher candidates can attend drop-in workshops during which they can receive support developing their ePortfolios. The TEO in collaboration with the Office of Learning Technology has developed web resources to assist the teacher candidates learn about the WordPress Multi-user (WPMU) interface and develop their ePortfolios. These include a support blog with a step-by-step guide to setting up an ePortfolio using the WPMU platform and a collection of screencasts which guide teacher candidates through the features of the WPMU platform (http://efolio.educ.ubc.ca/cpaullefolio-supportL). The TEO also offers an ecoaching blog which is designed to share online resources and encourage interaction and collaboration between teacher candidates. In August, 2008 support for TEO instructors included a two-day workshop at the beginning of the term facilitated by the OLT and TEO. This workshop introduced instructors to the WordPressMU platform and provided them a forum to explore the concept of Web 2.0 and interactive Web-based platforms (http://wiki.elearning.ubc.calMoreThanSomethingShiny ). WordPressMU Platform WPMU is an open-source, Web-based blog and content management system (WordPress.org). It is employed as a platform for Web-design and a large WordPress blog community exists on the Internet. The WPMU platform was chosen by the TEO at the suggestion of the OL T at the University of British Columbia due to its adaptability as a platform, its interactivity and overall aesthetic (A. Scholefield, personal communication, March 5, 2008). 6 As well, WPMU is an open-source I platform and new plug-ins are being developed every day. This means the platform can be expanded and changed based on feedback and the changing needs of the TEO. Because WPMU is a blog platform rather than a proprietary ePortfolio platform, it has been adapted for these purposes. The WPMU platform and all the teacher candidates data are stored on the eMS servers ensuring that all the data collected is only accessible to the TEO. Project Evaluation Plan 2008/2009 Program Evaluator's Role with the Teacher Education Office I have worked with the TEO in a number of capacities with the ePortfolio project: as an ecoaching coordinator, a researcher and as a teaching assistant. I have led approximately 30 workshops on the WPMU platform and the process of developing an ePortfolio with teacher candidates. Through this experience with the TEO, I have built relationships with instructors, teacher candidates and ecoaches and have been able to develop a broad perspective with which to approach the evaluation process. Approach to Project Assessment This project assessment plan is an adaptation of Sork's question-based approach to program planning and evaluation. The question-based approach is a flexible approach to program evaluation and emphasizes socio-political as well as technical elements of the program evaluation process (Sork, 2000). In addition, I employ Sork's framework for the different levels I Open-source describes different types of software that makes the source code available to the public and has few restrictions on using and adapting the work. This typically leads to the development of different applications and functions for the program by users (http://en.wikipedia.org/wiki/Open sow-cej 7 . t of program planning in order to take into account the technical, ethical and socio-political dimensions of the process. I have adapted the generic questions that Sork (2000) proposes to the context ofteacher education and an Internet technology-based project. As I have developed answers for these questions, new questions have arisen and I have added these to the project as a new category. Overall, as is noted by Sork and Caffarella (1989), program planning is a non-linear process, thus I have frequently had to revise my evaluation plan based on the results of formative evaluations and suggestions made by stakeholders. The evaluation plan will be based around the following questions, adapted from Sork: • What contextual factors and learner characteristics will be especially important to take into account when developing the summative evaluation plan? • What is/are the primary purpose(s) of the evaluation? • What aspects of the WordPressMU platform will be evaluated? • What evidence will you collect to answer each question and how and when will it be collected 8 • What important socio-political issues you likely to encounter in evaluating the ePortfolio project and how will you address them? • What ethical aspects have you considered and how will you deal with them? • How will you evaluate online learning objects and learning resources provided for instructors, teacher candidates and ecoaches? • How can you make sure that the evidence that you collect is trustworthy? (i:e with minimal bias and distortion) • Who should receive the results of the evaluation and why? • How much time and money do you estimate it will take to carry out the evaluation plan that you have developed? • What could you eliminate from the plan to make it more feasible while still accomplishing the goal of assessing the worth or the value of the program? • How can Internet technology and the storage and aggregation capacities of ePortfolios be employed to collate and distribute the results of the evaluation? What contextual factors and learner characteristics will be especially important to take into account when developing the summative evaluation plan? Contextual Factors The emphasis of the program on Internet technology. Technology is at the centre of the TEO ePortfolio project and is among its broader goals to assist teacher candidates in developing digital literacy and to better integrate technology within the teacher education program (A. Scholefield, personal communication, January, 2009). Thus, this evaluation centres on the use of Internet technology and specifically examines the ePortfolio platform and digital learning objects developed as resources for teacher candidates, instructors and ecoaching resources. 9 The collaborative approach to program planning. The ePortfolio program began as collaboration between the TEO and the eMS. The OL T was recruited for advice and then as collaborators in the project's 2nd year. The evaluation plan will take this collaboration into account by ensuring that coordinators from all three units are involved in the development of the survey questions and the overall approach to evaluation. As well, by using Google Documents and Survey Monkey applications, coordinators from the involved units will have the opportunity to co-edit surveys and documents used in the evaluation throughout the entire process. The competing purposes of ePortfolios in the context of this project. There are two purposes that ePortfolios are generally used for within learning institutions. As proposed by Barrett and Wilkerson (2004), these include the use of ePortfolios as a constructivist learning tool and as means to formally evaluate teacher candidates. This formal evaluation is typically completed by external licensing bodies or the federal or provincial/state governments and assess whether teacher candidates have met the competencies required to be accredited as a teaching professional. This evaluation will address both of these areas. Teacher Candidate Characteristics Competencies and experience using Internet technology. The teacher candidates, ecoaches, and instructors participating in the evaluation have varying levels of experience and competencies using the Internet. In order to take into account these differences, a brief pre-test will be administered at the beginning of the program to assess participant's computer competencies. This data will be cross-tabulated with survey responses. In addition, participants will have the option to complete online or paper surveys. 10 1'\ In future years this computer competency/literacy survey can be completed by all teacher candidates at the beginning and end of each program year in order to examine the connections between teacher candidates' computer competencies and their evaluation of the WordPressMU platform and the support they have received from the TEO, OL T and eMS. Cohort differences. The teacher candidates are a diverse group, both in terms of the grade levels they wish to teach and the areas they will specialize in. In addition, there are cohorts in the elementary group that complete a two-year program rather than the majority of cohorts which attend for one year (J.Gettings, personal communication, January 7, 2008). The support teacher candidates receive and the extent that ePortfolios are integrated into different groups or cohorts is an important consideration in developing a program evaluation. One way to address this diversity is to ensure that the program evaluation identifies and examines cohort differences. Through examining these differences the stakeholders will be better able to provide adequate and appropriate support to teacher candidates across the Teacher Education program. Time constraints. Teacher candidates face serious time constraints which influence their ability to participate in program evaluation. In order to reduce barriers to participation amongst teacher candidates, the program evaluation will be limited to surveys. In addition, I have integrated the surveys into the teacher candidates' class-time. What is/are the primary purpose(s) o/the evaluation? • To determine teacher candidates' views of the ePortfolio project • To examine the extent that learning outcomes were met • To assess learners comfort using the WPMU platform. 11 , • • To evaluate the value of the ecoaching program and consider ways of enhancing it in future years • To assess the support provided to instructors and consider ways of supporting their learning about ePortfolios and incorporating what they learn into their instructional practice. • To evaluate the WPMU platform and consider changes to the platform necessary in future program years • To assess the support provided to teacher candidates in order to make continuous changes to the support provided by the TEO throughout the year What aspects a/the WordPressMU platform will be evaluated? The WPMU platform is integral to the ePortfolio program and because it is an adaptable platform, feedback can be used to assist in the selection of plug-ins that increase the functionality of the platform.2 The following dimensions of the WPMU platform will be evaluated. • The extent that the platform is intuitive to use for development of an ePortfolio • The functionality of the platform and the functions that could be added. • The security of the platform and the teacher candidates comfort with the security guidelines • The interactivity of the platform and the extent that teacher candidates used the interface for communication and as feedback tool What evidence will you collect to answer each question and how and when will it be collected? 2 Plug-ins are user created programs which change the layout and functionality of the WordPress platform (www.wordpress.com). 12 Surveys: Formative Teacher candidates. The main formative evaluation for teacher candidates will take place in two group sessions with the secondary teacher candidates mid-point in their program. The purpose of these sessions is to assess how far along teacher candidates have progressed in developing their ePortfolios and to identify areas of support that need to be enhanced/adjusted during the remainder of the term. Formative teacher candidate evaluation will also be used in order to examine changing views regarding the level of support provided as the program progresses (Appendix A). TEO instructors. The formative surveys for instructors will be completed as an email survey to all of the instructors. The purpose of these surveys will be to assess how far along teacher candidates have progressed in developing their ePortfolios and to identify areas of support that need to be enhanced/adjusted during the remainder of the term (Appendix B). The survey will also determine how many instructors have employed WPMU in their classes and what their objectives were in using the platform. Surveys: Summative Teacher candidates. The summative surveys will be completed during the final session of the EDUC 480 classes. These surveys will be mandatory for all teacher candidates, and will focus on both elementary one-year and secondary one-year cohorts. These surveys will not assess two-year teacher candidates. The survey will be administered in person and online (Appendix C). 13 • , . TEO instructors. Upon completion of the EDUC 480 courses, TEO instructors will complete a survey which will be emailedto them and completed online (Appendix D). Focus Groups Ecoaches. There will be one focus group held in July during the completion ofEDUC 480 for a select group of ecoaches. This group will be composed of ecoaches that I have worked more closely with during the term and will represent both the elementary and secondary cohorts. The focus group will discuss the value of ePortfolios as learning and assessment tools as well as the strengths and weaknesses of the training they were provided by the TEO. This will also be an opportunity for candidates to present their ePortfolios to the group. The purpose of this demonstration will be to assess the ecoaches' ability to use the WPMU platform and to gain an understanding of different ways that ePortfolios have been developed. The following questions are to be used as prompts during the focus group for the ecoaches: 1. How would you rate your overall experience developing an ePortfolio? 2. Talk about how easy or difficult you found the WordPressMU platform to use? 3. What challenges did you face providing support to the teacher candidates you worked with? 4. How did the teacher candidates you worked with view the ePortfolio program and the WordPressMU platform? 5. What suggestions do you have for improving the ePortfolio project? 6. What areas of the project did you find the most successful? 7. To what extent did you value the process of developing an ePortfolio? 14 • In addition to the questions the following prompts will guide ecoaches through the process of presenting their ePortfolios: 1. Demonstrate an aspect of your ePortfolio that you are proud of. 2. Demonstrate the area of your ePortfolio that caused you the most difficulty. 3. Demonstrate an aspect of your ePortfolio that you consider unique. TEo/eMS administrators. This focus group will consider ways in which the ePortfolio project has worked well and ways that it could be improved. The group will include people directly responsible for the ePortfolio project from the eMS, OLT and TEO. The following is a list of questions to be asked by the focus group facilitator to examine the administrations views ofthe ePortfolio program: 1. What is your overall view of the strengths and weaknesses of the project for 2008/2009? 2. How did the teacher candidates appear to receive the project as compared to last year? 3. What ways can the project be enhanced in future years based on your experience with the project this year? 15 , . TEO instructors. This focus group for the TEO instructors will be included in a staff meeting towards the end of term and will address the overall ePortfolio project. The following questions will be used by the facilitator to guide the process: 1. Did you receive adequate support instructing EDUC 480? 2. What is your assessment of the WordPressMU platform? 3. What challenges did you face in providing support to the teacher candidates you worked with? 4. How would you improve the ePortfolio project? 5. What areas of the project did you find the most successful? What important social-political issues are you likely to encounter in evaluating the ePortfolio project and how will you address them? Negotiating Limited Time The teacher candidates developing their ePortfolios have very little available time and this can make it difficult to implement project evaluations. The ePortfolio is not formally attached to any courses until the end of the program so there are limited opportunities for teacher candidates to explore ePortfolios conceptually or technically. Teacher candidates have to complete the ePortfolio on their own time until the five day EDUC 480 course held late at the end of the program. Similarly allocating class time to allow teacher candidates to complete the evaluation will be a challenge because I do not have any power to set classroom agendas. The decision whether 16 " to allow these evaluations to be completed during class-time will be negotiated between the TEO administration and the instructors. One way to ensure that the evaluations are implemented is to facilitate workshops in EDUC 480. In the process of assisting the teacher candidates to develop their ePortfolios with WPMU I can administer the survey. For many instructors these workshops would reduce the amount oftime they are required to spend assisting teacher candidates with the technological side of developing their ePortfolio and thus they should be more open to accept the evaluation time. However, this approach to data collection would reduce the number of teacher candidates surveyed, and potentially the overall validity of the survey. Bringing the Stakeholders to the Planning table In order to promote what Cervero and Wilson (2006) refer to as substantively democratic planning, it is important to "bring stakeholders to the table." However, the challenges that arise in this domain are again ones of limited time and of the limited power that I possess as a graduate student assistant to gather groups of stakeholders together in physical meetings. In lieu of this, I have been relying on Internet technology such as blogs, Google Documents and email to bring stakeholders to the 'virtual ' table. 17 What ethical aspects to this evaluation have you considered and how will you deal with them? Data Protection In choosing to use online platforms such as Survey Monkey, there are immediate concerns about data protection. In reviewing the Survey Monkey terms of service the company notes that it does not collect personally identifiable data such as email addresses, and names (surveymonkey.com). Survey Monkey has the option to collect IP address which could serve to identify participants completing the surveys online. However, this feature will be turned off throughout all surveys in order to ensure that no personally identifiable information is collected. In addition, the summative surveys will be encrypted. This is another layer of data protection that can help deal with some of the ethical aspects of data collection and retention. Ensuring the Focus Groups Confidentiality is Protected In the final evaluation report, the identity of participants will be kept confidential. All of the survey data will be confidential and the ecoaches and instructors participating in the focus groups will not be identified. However, in the case of the focus groups with TEO, OLT and eMS administrators, the parties will be named in the report. Due to the small numbers of administrators involved in the project it is neither possible nor important to conceal their identities. Voluntary Participation In order to ensure that all participation is as voluntary and the least coercive as possible, I have selected focus group participants whom I have been working with throughout the last two terms. In doing this I am attempting to focus on reciprocity rather than coercion. The ecoaches and instructors selected to participate in the focus groups are individuals who I have provided 18 with extra assistance with their classes or cohorts. The ecoaches that participate in the seminars will also be paid their e-coach wage for participating in the one-hour session. In regards to the teacher candidates, I have tried to reduce the potential for coercion by limiting the time they are required to spend being surveyed. Additionally, I have ensured that the teacher candidates are allocated class time to complete the surveys and if they are not able to complete them the surveys are available online. How will you evaluate online learning objects and learning resources provided for instructors, teacher candidates and ecoaches? Survey Questions Evaluating the Support Providedfor Teacher Candidates and ecoaches Questions about learning objects will be included in the summative evaluations. By integrating these questions within the summative survey teacher candidates will not be required to spend a large amount oftime completing surveys. The questions will be included in the summative teacher candidate survey. Google Analytic TEO Website Report In using the data collected by the Google Analytics program from the TEO website we can gain information about the site traffic patterns in regards to the resources offered (www.google.com/analytics). Figure I shows the different elements of web traffic that are analyzed by the Google Analytic platform. 19 This data can assist the TEO in examining the following questions: 1. How many unique visits have been received in the 2008/2009 academic year? 2. Which online resources received the most traffic? 3. How do the access patterns relate to the month and time in the academic year? 4. How much time was spent exploring online resources? Dashboard . tx~-~'~£';"*I 121,718 Y.1&!J~ 1.75 ~'IS~ 71 .39% ~UI\CI} ~.!£ :: Vl$ltCft;. OvervIew " S .. arcll Engill/.'9 ,r7~ « : ,¢Z%) 111 i1eferrill!l Sit ... 4i.21l5 f~O" ~%l - Iilrect 1~2~O \1i,$$.!!\) m ~hef r lMl~) r··:-~':-·--··"···"·~'~'··'·-·-·"''''--·'~·-'-·''· .  " ~,' ' ~, Mar 19, 2007 · May 8. 2007 '" 212 ,SO 3 I'_i!Jl~y!g;y~ 00:01 :27 ~9, Tim~ on Sir" 81 .01% ~J:!!':LY.wl! -~- --------'"-- --~-- -~----t Content Overview - , ,0.00) 9,"'% 165$~ IH I'" Fig 1: Google Analytics Screen Shot Sample Report 20 How can you make sure that the evidence you collect is trustworthy? (i.e., with minimal bias or distortion) • Including questions that evaluate the teacher candidate's computer competencies upon entering the program in order to cross tabulate this with their perceptions of the WPMU platform and the support they received. • Administering the surveys during the EDUC 480 class to help ensure that the majority of teacher candidates complete the survey and thus reduce the bias that might exist because of unequal sampling among the different cohorts and groups of teacher candidates • Ensuring that the evaluation collects data from different levels of participants: instructors, ecoaches, and administrators. • A focus on a number of approaches to gathering data. Including surveys and focus groups will ensure that the data collected explores the ePortfolio project from a number of different vantage points. Who should receive the results of the evaluation and why? TEO Administration, Project Coordinators The Special Project coordinator and other TEO faculty involved in the administration of WPMU will receive an electronic and a hard copy of the results of the evaluation. They will also receive access to the data archive. The TEO administration is a key stakeholder in this process and this data can assist them in making formative changes to the support provided to teacher candidates for the 200912010 academic year. This survey also provides sound data for the required reporting for Teaching and Learning Enhancement funding. In addition, the evaluation data can be used to inform changes to the ePortfolio project in future years. Finally, the 21 evaluation can be employed by the TEO to assess the success of the WordPressMU platform in order to consider ways that it can be modified appropriately in the upcoming years. aLT Project Coordinators The OLT is a primary stakeholder in the ePortfolio project. They will be provided with a digital summary of the data. This data can inform the development of other WordPress ePortfolio initiatives across campus. The data concerning the WPMU platform can also assist them in configuring the platform and utilizing plug-ins to add functionality to the platform. They will also receive access to the data archive. CMS Project Coordinators The CMS will receive digital results of the ePortfolio evaluation. They will also receive access to the data archive which can assist them in configuring the WPMU platform and determining ways in which the CMS can support the ePortfolio process. 22 How much time and money do you estimate it will take to carry out the evaluation plan that you have developed? Figure 2 shows the overall budget for the project evaluation. The primary expenses are hours paid to the planner to plan, collect and analyze the evaluation data. Result Distribution Subtotal Coffee and Muffins " Room Rental Computer Rental Sub-total Total $200 ~ $1(}.0 $40.00 $340 $ 1234 Fig 2: Evaluation Budget 23 What could you eliminate from the plan to make it more feasible while still accomplishing the goal of assessing the worth or value of the program? As this is a relatively low resource program evaluation, the costs should be covered by money allocated in the Teaching and Learning and Enhancement Fund grant. There are two ways in which the program evaluation can be scaled in order to reduce costs, if required: 1. Eliminating the Google analytics component of the evaluation. This component of the evaluation was developed primarily to assess online learning objects. As these objects are assessed through summative surveys this can be omitted in order to reduce costs. 2. To reduce survey costs, employ the built-in WPMU poll application rather than Survey Monkey. This would eliminate approximately, $19.00 per/month for three months. How can Internet technology and the storage and aggregation capacities of ePortfolios be employed to collate and distribute the results of the evaluation? Using Survey Monkey to Complete Online Surveys Survey Monkey, an online survey tool, will be used throughout the evaluation in order to more easily collect, analyze and distribute the results. In terms of collecting the data, using an online tool provides a more open means to access the survey. Survey Monkey also allows for the data for formative evaluations to be quickly collected compiled and distributed to the eMS, OLT and TEO and thus these units will be able to more quickly make suggested changes to their programs. Figure 3 shows a screenshot of the online survey utilized for this program evaluation. 24 Summative Teacher Candidate eFollo Survey ~ Thanks for completing this survey. The purpose of this survey Is to assess the eFolio program In order to enhance future program .. * 1. I hllve received approprinte support (documentlJtion, training etc) for using the WordPre.ss software J Strongly Agree .J Agree J Neutral J Disagree J Strongly Disagree * 2. Select the area that has ClIused you the most difficulty during the development of your efolio ...) Selecting artifacts J Developing refl~ctions J Navigating the WordPress platform .J Other (ple.se specify) Fig 3: Screenshot Survey Monkey online survey Archiving the Final Data Using the WordPressMU Platform In order to facilitate data sharing between stakeholders and to create a permanent archive of evaluation data that can be used in future years, the final data summary/report will be archived using the WPMU platform. This will entail organizing and uploading the raw data as well as the summary report to a WPMU site that I have developed. The security settings on this site will only allow the TEO administration access in order to protect the data. The Office of Learning Technology and Computing and Media services will be registered as users to allow them limited access. 25 Conclusion This evaluation plan for the ePortfolio program at UBC' s Teacher Education Office was developed around Sork's question-based approach to program evaluation, and examined the following aspects: the technical value of using WordPressMU as a platform; the support provided by the TEO and OLT to the faculty and teacher candidates; and the overall value of the program as expressed by teacher candidates, instructors and TEO and OLT administration. The data collected for this evaluation includes summative and formative program evaluations that surveyed the teacher candidates and instructors involved with the program, as well as focus groups and demonstration sessions held with a small group of ecoaches. The evaluation plan emphasizes collaboration, as a reflection of the collaborative nature of the ePortfolio program itself, and internet technology to collect and distribute information, which is also a key component of the program. The University of British Columbia Teacher Education program is at the cutting edge of ePortfolio development. Examining the successes and challenges that arose during this second year of the project, this evaluation will hopefully provide insights to assist the TEO in enhancing this project in future years as well as other teacher education programs that may wish to incorporate ePortfolios as collaborative platform and tool for archiving data. 26 Bibliography . Barrett H. & Wilkerson J. (2004). Conflicting Paradigms in Electronic Portfolio Approaches. Retrieved on Nov 2, 2009 from http://electronicportfolios.comlsystems/paradigms.html Campbell D & Cignetti P. & Melenyzer B. & Nettles D. & Wyman R. (2007). How to Develop a Professional Portfolio: a manual for teachers, Boston: Pearson Cervero, R.M. & Wilson A.L. (2006). Working the Planning Table: Negotiating democratically for adult, continuing and workplace education. San Francisco: Jossey-Bass. Greenburg G. (2004). The Digital Convergence: extending the portfolio model. Educase Review. Retrieved Nov 1, 2007 from: http://www.educause.edu/ir/library/pdf/erm0441.pdf Lorenzo G. & Ittelson, J. (2005) An Overview of e-Portfolios. Educause Learning Initiative, . Retrieved October 3,2007 from http//:www.educase.edu/irllibrary/pdf/ELI13001.pdf Love D., McKean G ,& Gathercoal P. (2004) Portfolios to Webfolios and Beyond: Levels of Maturation. Educase Quarterly. Vol. 27, No 2. Retrieved November 15 from http//:connect.educase.edu/lirbrary/ abstract/Portfoliosto W e bfolio/3 9864 Milman, N. (1999). Web-Based Electronic Teaching Portfolios for Preservice Teachers. In J. Price et al. (Eds.), Proceedings of Society for Information Technology and Teacher Education International Conference 1999 (pp. 1174-1179). Chesapeake, VA: AACE Sork, Thomas J., & Caffarella Rosemary S. (1989). Planning Programs for Adults. Merriam Sharan B., & Cunningham, Phyllis M. (Eds.) Handbook of Adult and Continuing Education. San-Francisco: Jossy Bass. Sork, Thomas J. (2000) Planning Educational Programs. Wilson, In A.L & Hayes (Eds.), Handbook of Adult and Continuing Education. San-Francisco: Jossey-Bass (735pp.) Standards of Education, Competence and Professional Conduct of Educators in British Columbia. (February 2008). Vancouver BC: British Columbia College of Teachers. Retrieved February 17,2009 from http: // www.bcct.ca!documents/ AboutU s/Standards/ edu _ stds. pdf The UBC Attainment of Standards Report. (February, 2006). Vancouver BC: UBC Faculty of Education, Teacher Education. Retrieved May 1, 2009 from http://educ.ubc.ca! about/ standards/ Wickersham L. ,& Chamber S. (2007). The Electronic Portfolio Journey: a year later Education, Vol. 127, No.3 27 Appendix A Teacher Candidate Evaluation (Formative) 28 Teacher Candidate Survey (Formative) 1.° Your input regarding this course will greatly improve future planning and course improvements. It will only take a few minutes to complete the survey. We greatly appreciate your time and input. Thanks for completing this survey. Check one or more responses. Write comments in the boxes. * 1. I experienced the WordPress interface to be intuitive and easy to navigate. o Strongly agree o Agree o Neutral o Disagree o Strongly disagree * 2. Check the statement that best describes your progress in developing your eFolio. o I have not logged into WordPress or begun collecting artifacts o I have begun collecting artifacts but I have not begun using the eFolio platform o I have begun collecting artifacts and completed the initial set-up of my eFolio o I have developed the basic structure of my eFolio and started adding artifacts o I have developed the structure of my eFolio and I have completed at least two artifacts * 3. I have received appropriate support (documentation, training etc) for using the Word Press software. o Strongly agree o Agree o Neutral o Disagree o Strongly disagree * 4. I have received sufficient support from my cohort's ecoach in developing my ePortfolio. o Strongly agree o Agree o Neutral o Disagree o Strongly disagree - < Teacher Candidate Survey (Formative) *- 5. Select the type of support that would be valuable for you in developing your ePortfolio this term. o Workshops (cohort-based) o The TEO blog o The TEO website (ePortfolio resources) o Workshops (drop-in) o Other (please specify) * 6. Select the area that has caused you the most difficulty during the development of your ePortfolio. o Selecting artifacts o Developing reflections o Navigating the WordPress platform o Other (please specify) * 7. In my courses faculty helped me make connections between the course content and my eFolio. o Strongly agree o Agree o Neutral o Disagree o Strongly disagree * 8. I received clear and timely communication about changes to the Education Word press site. o Strongly agree o Agree o Neutral o Disagree o Strongly disagree * 9. What is the name of your cohort? I I l • Appendix B Faculty Evaluation (Formative) 29 Faculty Survey (Formative) 1. Your input regarding this course will greatly improve planning and course improvements. It will only take a few minutes to complete the survey. We greatly appreciate your time and input. Thanks for completing this survey. Check one or more responses. Write comments in the boxes. * 1. I experienced the Word Press interface to be intuitive and easy to navigate. o Strongly agree o Agree o Neutral o Disagree o Strongly disagree * 2. I have received appropriate support (documentation, training etc) for using the Word Press software. o Strongly agree o Agree o Neutral o Disagree o Strongly disagree * 3. The ecoaches in the classes I instruct have been a valuable resource for the teacher candidates in developing their ePortfolios. o Strongly agree o Agree o Neutral o Disagree o Strongly disagree * 4. Select the area that appeared to cause the teacher candidates the most difficulty during the development of your ePortfolio. o Selecting artifacts o Developing reflections o Navigating the Word Press platform o Other (please specify) c, Faculty Survey (Formative) *- 5. I integrated ePorfolios into one or more of the courses I instruct. o Strongly agree o Agree o Neutral o Disagree o Strongly disagree * 6. I received clear and timely communication about changes to the Education Word press site. o Strongly agree o Agree o Neutral o Disagree o Strongly disagree 7. What areas of support could assist you in supporting the teacher candidates as they develop their ePortfolios? For example, drop in workshops. 1 :1 ( ~ .. Appendix C Teacher Candidate Evaluation (Summative) 30 ( .. Teacher Candidate Survey (Summative) 1. ' Thanks for completing this survey. Check one or more responses. Write comments in the boxes. * 1. I have received appropriate support (documentation, training etc) for using the WordPress software o Strongly agree o Agree o Neutral o Disagree o Strongly disagree * 2. Select the area that has caused you the most difficulty during the development of your eFolio o Selecting artifacts o Developing reflections o Navigating the Word Press platform o Other (please specify) * 3. In my courses faculty helped me make connections between the course content and my eFolio. o Strongly agree o Agree o Neutral o Disagree o Strongly disagree 4. Approximately how many hours did you spend developing your eFolio I I * 5. What is the name of your cohort? I I Teacher Candidate Survey (Summative) "6. Developing an eFolio increased my confidence using the Internet as an educational resource o Strongly agree o Agree o Neutral o Disagree o Strongly disagree o Other (please specify) 7. I received sufficient support from my cohort's ecoach in developing my eFolio o Strongly ag ree o Agree o Neutral o Disagree o Strongly disagree 8. I received clear and timely communication about changes to the Education Word Press server. o Strongly agree o Agree o Neutral o Disagree o Strongly disagree o Other (please specify) I 9. I will continue to develop my eFolio after graduating the Teacher Education program at UBC o Strongly ag ree o Agree o Neutral o Disagree o Strong ly disagree o Other (please specify) ( .. ' Appendix D Faculty Evaluation (Summative) 31 { , Faculty Survey (Summative) 1. Your input regarding this course will greatly improve future planning and course improvements. It will only take a few minutes to complete the survey. We greatly appreciate your time and input. Thanks for completing this survey. Check one or more responses. Write comments in the boxes. 1. I found the Word Press interace to be intuitive and easy to use. o Strongly agree o Agree o Neutral o Disagree o Strongly disagree o Other (please specify) 2. Select the area that has caused the most difficulty for the teacher candidates in you classes developing their ePortfolios. o Selecting arti facts o Developing reflections o Navigating the Word Press platform o Other (please specify) 3. I received appropriate support from the TEO concerning how to integrate eFolios in the classes I facilitate. o Strongly agree o Agree o Neutral o Disagree o Strongly aisagree o Other (please speci fy) - , Faculty Survey (Summative) "4. I find the ePortfolio project to be pedagogically valuable. o Strongly agree o Agree o Neutral o Disagree o Strongly disagree o Other (please specify) I .' 5. What type of support will assist you in integrating ePortfolios in your classes in future years? o Cohort workshops o Drop in workshops o Online learning objects o Other (please specify) 6. The teacher candidates attending my classes responded positivly to the ePortfolio assignment. o Strongly agree o Agree o Neutral o Disagree o Strongly disagree o Other (please specify) 7. Which online learning objects were the most valuable in learning about the concepts and technical components of developing an ePortfolio? D The TEO blog D Word Press screen casts D Frequently asked questions D Karen Brennan's ePortfolio guide D Other (please specify) -Faculty Survey (Summative) <s. I received clear and timely communication about changes to the Education Word Press site. o Strongly agree o Agree o Neutral o Disagree o Strongly disagree o Other (please specify) I 9. The ecoach assigned to my class provided adequate support to myself and the teacher candidates. o Strongly agree o Agree o Neutral o Disagree o Strongly disagree o Other (please specify) I 

Cite

Citation Scheme:

        

Citations by CSL (citeproc-js)

Usage Statistics

Share

Embed

Customize your widget with the following options, then copy and paste the code below into the HTML of your page to embed this item in your website.
                        
                            <div id="ubcOpenCollectionsWidgetDisplay">
                            <script id="ubcOpenCollectionsWidget"
                            src="{[{embed.src}]}"
                            data-item="{[{embed.item}]}"
                            data-collection="{[{embed.collection}]}"
                            data-metadata="{[{embed.showMetadata}]}"
                            data-width="{[{embed.width}]}"
                            data-media="{[{embed.selectedMedia}]}"
                            async >
                            </script>
                            </div>
                        
                    
IIIF logo Our image viewer uses the IIIF 2.0 standard. To load this item in other compatible viewers, use this url:
https://iiif.library.ubc.ca/presentation/dsp.42591.1-0340066/manifest

Comment

Related Items