UBC Library and Archives

Good Assessment Starts Before You Begin Friesen, Margaret; Buhler, Jeremy 2011-12-02

You don't seem to have a PDF reader installed, try download the pdf

Item Metadata


23303-Good Assessment Starts Before You Begin (Friesen, 2011).pdf [ 501.16kB ]
23303-Good Assessment Starts Before You Begin (Friesen, 2011).pptx [ 2.38MB ]
JSON: 23303-1.0077815.json
JSON-LD: 23303-1.0077815-ld.json
RDF/XML (Pretty): 23303-1.0077815-rdf.xml
RDF/JSON: 23303-1.0077815-rdf.json
Turtle: 23303-1.0077815-turtle.txt
N-Triples: 23303-1.0077815-rdf-ntriples.txt
Original Record: 23303-1.0077815-source.json
Full Text

Full Text

    December 2011  Page 1 Good Assessment Starts Before You Begin Speech to Academic Librarians in Public Service (ALPS), BC Library Association, Dec. 2, 2011  Margaret Friesen, Assessment Librarian, University of British Columbia Library Assisted by Jeremy Buhler, Josh Rose, Geralyn Sy, Assessment Office, UBC Library   “The demonstration of value is not about looking valuable; it’s about being valuable”   Oakleaf, M., 2010. The Value of Academic Libraries: A Comprehensive Research Review and Report, Chicago: Association of College and Research Libraries, p 140. http://www.ala.org/acrl/sites/ala.org.acrl/files/content/issues/value/val_report.pdf Outline  Before you begin – project management steps  Assessment goals and principles  Methods and tools  Assessment simplified – some tips Don’t have data? Don’t have the tools? Before You Begin Before starting an assessment project, consider using a project management methodology to focus your project and ensure a useful assessment outcome.  Purpose: why are you doing the assessment? Focus: what is the scope? what is not in scope? make it do-able, what will you do with the results?  Stakeholders: who is the subject? who are you reporting the assessment findings to?  Partners: who else needs to be involved?  Timelines: start/end dates (and everything in between)  Outcome: what is the assessment outcome? Product? Assessment goals and principles Why assess? Why do it? Primarily to improve services and demonstrate accountability of resources.  In other words, be valuable and then look valuable.  Other reasons:  Compare with others  Identify changing patterns  Market/promote/tell the story         December 2011  Page 2 Use data (evidence), not assumptions, to make decisions. (Steve Hiller, University of Washington Library, May 1, 2008 UBC Library site visit).  Definitions, principles, assessment context Definition: Library assessment is more than numbers, it is a structured process:  To learn about our communities  To respond to the needs of our users  To improve our programs and services Methods and tools – Where does data come from? What tools are used? A variety of sources, collected already by many libraries: statistics, big surveys, in-house surveys, usability studies, observation, focus groups, tracking tools, etc.  What is the difference between “assessment” and “research”? Megan Oakleaf lists them as:  Assessment Research strives to know "what is"  uses information to change status quo test hypotheses  observations of change correlation/causation among variables political context apolitical document observations prove/disprove ideas complete projects - even with flaws researchers can start over not always "perfect" studies  should be well planned, clear outcomes, appropriate methods, but "sometimes good enough"   Source: Oakleaf, M., 2010. The Value of Academic Libraries: A Comprehensive Research Review and Report, Chicago: Association of College and Research Libraries, pp 30-32. http://www.ala.org/acrl/sites/ala.org.acrl/files/content/issues/value/val_report.pdf         December 2011  Page 3 Oakleaf also articulates some principles for assessment programs. To “manage upward”, correlate strategy and assessment, correlate library data with institutional data and communicate outwards in language that resonates with the parent institution. Use their strategic plan wording, address their strategic directions and the library’s contribution to these directions.  Purpose of assessment: accountability In addition to using assessment activities to improve services, also apply assessment to projects that demonstrate accountability, show how the library adds value to the teaching and learning enterprise. (Town, J.S., 2011. Value, Impact, and the Transcendent Library: Progress and Pressures in Performance Measurement and Evaluation. The Library Quarterly, 81(1), pp.111-125. http://www.jstor.org/stable/10.1086/657445)  And consider the library’s value as place.  How does “place” affect student capacity to learn, including the library space beyond the library, “coming out from behind the desk”?  (Nitecki, D.A., 2011. Space Assessment as a Venue for Defining the Academic Library. The Library Quarterly, 81(1), pp.27-59. http://www.jstor.org/stable/full/10.1086/657446)    Let’s look at some examples of assessment projects, methods and tools carried out at the University of British Columbia Library recently.  Project: The LibQUAL story in 2010 The purpose was a total market survey of students and faculty to find out our users’ perceptions of UBC Library services. The clear message from our respondents was: “make it easy to find people, places, things; buy more content; in addition to groups spaces, provide more quiet space … and we also think it is important to have knowledgeable staff in the library to help us with our teaching and learning and research.” (see also https://circle.ubc.ca/handle/2429/23304/)  Project: Reference Services Purpose was to improve service delivery. The methodology was to track reference activity in detail in real time for a 2-week period, using Desk Tracker.  Scope: what’s going on at reference desks across the library branches?  Quantitative and qualitative data was collected and reported, using Excel and ATLAS.ti. Reference questions were categorized using 15 codes, and counted by frequency, sorted high to low. A surprising 17% of the questions fell into the category of “research”, requiring the assistance of skilled reference librarians with expert knowledge to answer them. ATLAS.ti was used to analyze these questions in more detail, to tell a story about the depth and breadth of this expertise (proof of worth, value, human capital).  Project: Teaching and Learning Program Purpose was to improve the delivery of the teaching and learning program. Scope was to find out who we were teaching/reaching? What we were teaching across the library system? Evidence gained might contribute to better decision making about better coordination/integration, identifying gaps in outreach, etc. Again, Desk Tracker was the tool used to track library teaching activities. Another tool, Nesstar, was used for further analysis of the data. It allows expert and novice alike to view and analyze data and publish tables and charts.  Project: Needs survey to plan a Research Commons space for graduate students and faculty The method was a survey, and the tool used to gather data from graduate students and faculty in the Faculty of Arts was the Vovici Survey Tool. Questions focused on potential facilities, hardware/software/equipment, services, among other variables. As in other projects, the need for librarian subject specialists was a strong finding. In addition, the needs expressed align with the university’s strategic directions in scholarly communications. The survey results provided a concrete assessment outcome: powerful evidence from graduate students about what they want to see in a Research Commons space.     December 2011  Page 4  The previous examples of assessment projects show methods and tools for gathering new data, using specific applications. But, what can we do if we don’t have the data? Assessment simplified: don’t have data? Try some guesses, “estimations”, using some numbers you might have already. Here are two examples of value/valuations.  Example 1. Value of Library Services (unit values from Cornell University, http://research.library.cornell.edu/value). What would it cost our users if they had to buy the books, instead of borrow them? Purchase online articles, instead of accessing them online via the library’s licenced e-resources? Pay for information services and information literacy classes?    Example 2. Value of Library Services to support new academic courses. What does it cost the library to support a new course with new book and journal purchases? New start-up Information literacy classes? Reference services for students/faculty members teaching/learning/research activities? Assessment simplified: don’t have the tools? Use Excel, learn Excel at the intermediate level and you will see lots of possibilities to learn about your users and tell the story in tables, charts, etc., including fancy radar and thermometer charts.  Communications: Back to being and looking valuable. Tell the story, using different methods and tools, of human capital, value added by the library, to many different audiences: to your own library staff for their learning, to the library administration for advocacy, to the Development Office for making the case with donors, to library managers for decision-making, to the public so that they can understand the value of the library to the learning enterprise.  Summary: We have offered some examples of assessment projects, goals, methods and tools – from big picture surveys to focused in-house investigations, from real time tracking to estimations, using sophisticated and readily available tools. Assessment takes time and effort, requires focus, can be done in small chunks, can provide evidence to improve services, ways to know our customers, opportunities to tell our own story.  Assessment is a positive experience:  when it is structured  when it is used to improve services and demonstrate value  when it is practical, continuous and effective.  You, too, can do at least a little assessment activity in your library.  We will be valuable and we will look valuable. Thank you.  Margaret Friesen, Assessment Librarian, UBC Library 


Citation Scheme:


Citations by CSL (citeproc-js)

Usage Statistics



Customize your widget with the following options, then copy and paste the code below into the HTML of your page to embed this item in your website.
                            <div id="ubcOpenCollectionsWidgetDisplay">
                            <script id="ubcOpenCollectionsWidget"
                            async >
IIIF logo Our image viewer uses the IIIF 2.0 standard. To load this item in other compatible viewers, use this url:


Related Items