1Getting Started: Applying ATLAS.ti and NESSTAR WebView to the LibQUAL+® Results at UBC Libraryby Margaret Friesen, Assessment Librarian,University of British Columbia Library®2The EnvironmentUBC Vancouver Campus9 libraries on campus, 5 off campus Off-site: 3 hospital libraries, Robson Square Library, UBC Okanagan (Kelowna) Barber(IKBLC)Koerner3• nearly 50,000 students• 3,500 faculty• 9,000 graduate students• 6,000 international students• 300 library staff4LibQUAL respondentsüTransition to Online programüLiaison servicesüTeaching and learning programsüPeopleüUndergrads – generally satisfactory? Grads/faculty – Information Control concerns? Place concerns5369 comments 126 separate codes3656 snippets (quotations)What and why?Exploring themes (ATLAS.ti)6ATLAS.ti® Workspaces Survey commentsCodingRelationshipsQuery toolQuery tool7The ProcessCoding the comments CommentsAdd codesFrustrated?iterative8The Process: analysing the codes- code frequencies -code familiesGROUNDED - DESCENDING COLLECTIONS FAMILY Koerner 242 ! 12print 37 Faculty 232 ! 13e-info res 12 Service 177 ! 17journals 13 Grad 164 ! A/V 15 Collections 136 ! Asian lang 2 positive 116 ! Books 73 InfoContPhys 98 ! Browsing 11 Gaps 97 ! Collections 136 Undergrad 77 ! Datafiles 1 negative 63 ! e-Books 7 etc e-jnls 81 TOTAL: 3656 Exhibits 2 Gaps 97 ILL/DD 25 Microforms 7 Missing 18 Newspapers 3 Preservation 4 Print jnls 28 RBSC 1 Reserves 8 Theses 3 Surprise 122 codes9The Process: Output to Theme TeamsQuery Report “17journals”13 (quotations) found for QueryCodes: [10e-res access] [11libweb][17journals][AppSci][Grad]….Quotation:…but sometimes I have difficulty finding proceedings…published in journals10What and Why? Theme Teams’ ProcessIdentify main concernsTheme teams (4) 1. Customer Services -behaviours/skills/expertise -teaching & learning Themes (top 4) 2. - Access to Information - Access to Collections 3. Collections and Gaps 4. Place and Physical Access Findability Findability Findability - physical Findability Education Education Education Education Visibility Visibility Visibility Accessibility Access to collections/ Access to information 11What and Why?The one big idea = FindabilityüResources (content, access to)üPeople (in person, expertise)üPlaces (physical access)üInformation (help in person, website)12Who? where? (NESSTAR WebView)* To begin. . . Click on the small plus sign beside The UBC Data Services Collection in the area on the the left. Streamlinesfinding/accessing/analyzing statistical informationAnyone can useSearch survey/variableAny statistics programUser defined variables(Surprise 2!)LayersSubsetsFiltersOutput*Networked Social Science Tools and Resources (NESSTAR) Norwegian Social Science Data Services NSD-web-based software to publish/share statistical datahttp://www.nsd.uib.no/nsd/english/index.html13Customizing the data and outputUser defined variablesLayers (UBC-V)Subsets (branch-Koerner)Filters (disc – sciences; users-ug’s)OutputData14The Process: 2 tools (ATLAS.ti, NESSTAR) + 4 theme teams= clear message from usersThank you !“make it easier to find”info/peopleresourcesplaces 449 Applying ATLAS.ti and Nesstar WebView to the LibQUAL+® Results at UBC Library: Getting Started Margaret Friesen University of British Columbia, Canada Abstract We asked survey respondents: How can the UBC Library serve you better? Please tell us! The respondents replied: By making it easier to FIND resources, people, places, and help. The University of British Columbia (UBC) Library participated in the LibQUAL+® survey for the first time in January/February 2007. The preliminary quantitative data from ARL (notebooks, including radar charts; LibQUAL+® Analytics-Institution Explorer; other worksheets and templates, etc.) were used and shared with over 200 library staff in open sessions. To get started with the qualitative analysis and to enhance the quantitative data, we used two analytical tools: ATLAS.ti and Nesstar WebView. We downloaded the “comments” from LibQUAL+® to ATLAS.ti and the SPSS files to Nesstar WebView. These tools enabled us to analyze the qualitative and quantitative data systematically, to expose and explore relationships between the qualitative and quantitative data, and to focus the results of the survey on specific user groups, places and services. Introduction: The Environment The University of British Columbia Vancouver campus sprawls over an area of 993 acres (402 hectares), encompasses 12 faculties, enrolls nearly 45,000 students at its Vancouver campus and nearly 5,000 at its Okanagan campus. Of the nearly 50,000 students, 9,000 are graduate students and 6,000 are international students. Over 3,500 faculty and 300 FTE library staff work at UBC. The library system is highly decentralized, with nine libraries on the Vancouver Point Grey site, four libraries off-site in Vancouver and one library at the UBC-Okanagan campus in Kelowna. UBC-O Library conducted its own LibQUAL+® survey in 2007, the results of which are not discussed in this paper. The two largest libraries are the Koerner Library (Humanities & Social Sciences, including government publications, maps, microforms, circulation) and The Irving K. Barber Learning Centre (Barber). Barber was in a state of being re-constructed during the 2007 survey period. At the time of the survey, it housed Art+Architecture+Planning, Science and Engineering, Rare Books & Special Collections, University Archives, the Automated Storage Retrieval System, and Circulation. In spring 2008, the newly renovated heritage core and a new wing were opened, including the Learning Commons, meeting rooms, multi-purpose classrooms, social spaces, café, and more. Other branch libraries are Asian, David Lam (management), Education, Law, Music, Robson Square, Woodward (life sciences), Xwi7xwa (First Nations) and three off-site hospital libraries: Biomedical Branch, Hamber, St. Paul's. A. General LibQUAL+® Results The preliminary quantitative data revealed that, in general, undergraduates were satisfied with library services, with some exceptions, but that the library did not meet the minimum service level for the “Information Control” dimension (collections, access to collections) for graduate students and faculty. A preliminary perusal of the “comments” survey question (qualitative data) revealed two divergent sets of perceptions. LibQUAL+® respondents commented positively on these issues:  the library's “transition to online” program (moving from print to electronic journals);  liaison services by subject librarians;  teaching and learning programs;  their many positive encounters with staff; and  the Interlibrary Loan/Document Delivery services (this was not a survey question, but the service received a noteworthy “write-in” vote of confidence). 2008 Library Assessment Conference 450 However, respondents frequently used the word “difficult” to describe their experiences in finding resources, people, places, and help:  resources are often hidden/invisible/lost, not where they are “supposed” to be;  service points are hidden, especially in the two largest branch libraries;  people are sometimes hard to find, subject specialists are too scarce;  policies and procedures are sometimes difficult to find online;  online help is hard to find or presented inconsistently;  some libraries are hard to find;  some places within libraries are hard to find; and  some content is hard to find. B. ATLAS.ti: Discovering “What” and “Why” ATLAS.ti enabled us to analyze the comments in a systematic way, to uncover patterns, to consolidate common threads, and to focus on the most important concerns. 1. The process: coding the comments We assigned 126 codes to the 369 comments received from respondents, expanding the individual ideas in the comments to over 3,600 snippets (parts of comments or subtopics). The coding scheme included the three LibQUAL+® dimensions, codes for all 22 core questions and demographics. In addition, we assigned free coding, using simple keywords from the respondents' own words or concepts that would be meaningful to librarians. The process of coding allowed for serendipitous discovery and was iterative. As we entered more codes, we discovered connections between codes, eliminated repetitious codes, and substituted better terms. 2. The process: analyzing the codes We sorted the codes by frequency of occurrence and streamlined the coding further by dropping codes, re-coding some comments/snippets, and adding others. The frequency of codes, sorted high to low, began to reveal the potential importance of common themes. For example, we were surprised by the number of concerns about the physical access to collections and the range of perceived gaps in collections. The analysis of the codes also pointed to a number of issues related to access, one of many signs that the collections gaps may have several causes, not only “real” gaps in holdings, but also findability gaps. Table 1. ATLAS.ti Codes - Frequency in Descending Order Demographics Issues Library Used Codes (#) Codes (#) Codes (#) Faculty (242) Services (177) Koerner - HSS (242) Grad (232) Collections (136) Woodward - LifeSci ( 64) SocSci ( 88) Positive (116) IKBLC - SciEng, FA ( 58) Humanities ( 77) Instruction (105) Educ ( 34) Undergrad ( 77) Physical access ( 98) Science/Math Gaps - Collections ( 97) HealthSci Reference Age 23-30* Library Web site Age to 22* E-journals AppSci Books Access *over 30 not coded Negative (63) Number of respondents: 755 Number of comments: 369 Number of codes: 126 Number of snippets: 3,656 Friesen 451 3. The Process: Creating Code Families Using the Code Manager in ATLAS.ti, we combined codes into code “families,” representing both broad and narrow concerns. For example, when combined, the codes in the "collections family" connected the specific detailed examples of collections gaps to the broader collections concerns. Table 2. Code Families Collections Family Codes # 12print 37 13e-info res 12 17journals 13 A/V 15 Asian lang 2 Books 73 Browsing 11 Collections 136 Datafiles 1 e-books 7 e-journals 81 Exhibits 2 Gaps 97 ILL/DD 25 Microforms 7 Missing 18 Newspapers 3 Preservation 4 Print journals 28 RBSC 1 Reserves 8 Theses 3 Total codes 584 Individual codes could "belong" to more than one family, creating the possibility of overlap and/or links between themes, e.g., the code “browsing” relates to the ”collections family” and “teaching and learning family” and the “physical access” family, depending on the context of the code. 4. Theme Teams Discover “What” and “Why” Three ATLAS.ti tools helped us to tailor the comments to specific audiences:  Code Manager: simplifies the process of coding, sorting the codes, revealing the most frequent codes;  Network Editor: enables understanding the connections between codes; and  Query Tool: enables sorting, combining codes and code families, creating “queries” and manageable reports (“query reports”). The 3,656 snippets of comments could now be assembled into manageable printed reports by broad themes. We called for volunteers to assist with the analysis of the “comments” and their associated codes. Thirty-two volunteers responded from across the library system. All three employee groups participated, librarians, library assistants, and “management and professional” staff (systems, circulation supervisors). The initial sorting of codes, creation of code families, and query reports indicated that four Theme Teams might be appropriate: 2008 Library Assessment Conference 452  Customer services (behaviours, skills, expertise, teaching & learning);  Access to information and access to collections;  Collections and gaps; and  Place and physical access. In addition, a few individual analysts also volunteered to look at the comments from specific user groups and disciplines. Some tailored query reports were also distributed to branch heads and standing committees (on public services, e-Library services, reference, and instruction). The four Theme Teams were asked to consider the following questions:  What are the 3-5 main themes/concerns?  Why are these concerns?  What issues might be addressed in the short term?  What issues cannot be addressed now (but might be explained/clarified)?  What issues might be addressed in the longer term (might require additional staffing or funding resources)?  Is this a collections gap?  Is this an access to collections question?  Is this a findability/navigation question?  Is this a policy/procedural issue?  Is this a physical access issue (signage, findability, arrangement, missing/misshelved)?  Is this a teaching and learning issue?  How can we best communicate the results from your team to the library administration, staff, users? The Theme Teams perused their respective reports and identified 14 dominant themes. When all teams assembled to share their stories and compare their findings, four main themes emerged, as follows. Table 3. Theme Teams Identify Main Themes (“What” and “Why”) Team 1 Customer Services -behaviours, skills, expertise -teaching & learning Team 2 Access to Information and Access to Collections Team 3 Collections and Gaps Team 4 Place and Physical Access Top 4 Themes: Findability Findability Findability Findability Education (teaching & learning) Education (teaching & learning) Education (teaching & learning) Education (teaching & learning) Visibility Visibility Visibility Accessibility Access to collections Access to information Finally, a common overarching word emerged. The one big idea was “findability.” LibQUAL+® respondents said: make it easier to find:  the people (in-person helpers, subject expertise);  information (about the resources, about contacts, about the places, about help on the Web site);  resources (the content, access to the content); and  places (the libraries and inside the libraries). In response, the Theme Teams recommended ways to address these service gaps, at least in part, either in the short term or longer term. C. Nesstar: Discovering “Who” and “Where” We turned to Nesstar WebView for a more detailed analysis of the quantitative data. This analysis would allow us to pinpoint more closely “who” was most concerned and “where”—which libraries, places, disciplines, or functions required attention the most. In LibQUAL+® terms, the advantages of using Nesstar WebView for this purpose include:  usability by novice and expert alike;  a choice of universal or limited access; Friesen 453  ability to search by survey or by survey variable;  ability to download any statistical program, SPSS, SAS, or other program;  ability to create user-defined variables;  ability to view and manipulate selected data only (layers, subsets, filters);  ability to customize output for specific audiences and purposes o view online tables, charts, pdf format o export data to spreadsheets, export pdf files; and  perhaps most important, with Web access, assessment teams can view the data online and create/manipulate the tables and charts in consultation with each other. 1. Exploring Data, Searching Surveys, and Variables We loaded the SPSS data for three datasets:  UBC Library (UBC-Vancouver and UBC- Okanagan);  UBC West; and  UBC CARL (Canadian Association of Research Libraries consortium). 2. Customizing Data: Browsing, Analyzing, Computing, Re-coding Since UBC Vancouver and UBC-Okanagan data were initially combined into one dataset, we created two new “user-defined” variables: UBC- Vancouver and UBC-Okanagan. The new variable “UBC-Vancouver” allowed us to isolate the data for UBC-V only. We explored the LibQUAL+® story in more detail through the application of layers, filters, subsets, and additional user-defined variables (re- coding), for example:  layers (survey, branch library);  filters (user group, disciplines);  subsets (the four largest branch libraries or benchmark libraries); and  user-defined variables (combined branches: Koerner, Barber). This is illustrated by the following two Nesstar tables: Table 4. Nesstar WebView Window User defined variables Data Key: Left column = survey data and variables, including user-defined variables, selecting layers, selecting row/column tables for output Center column = description, tabulation, analysis views Top right column = buttons to weight data, make graphs, create subsets, view/print/download output 2008 Library Assessment Conference 454 Table 5. Branch Libraries and Age Group (UBC-V Survey) Key: Layer = UBCV (University of British Columbia Vancouver survey) User-defined variable = “combined branches” Tabulation = LibQUAL+® variables: branch library, age of respondent Subset = selected 4 libraries 3. Customizing the Output Customized tables were viewed online or printed out in several ways:  previewing, viewing and printing PDF files in Nesstar WebView;  exporting tables to spreadsheets;  exporting tables as PDF files; and  downloading data and manipulating spreadsheets. D. Steps to Assessment These customized statistical reports and the Theme Teams' reports point us to the next stage of assessment planning and programming. Some possible investigations to pursue are as follows. 1. Why are respondents in the sciences using the Koerner Library (the “humanities & social sciences” library) in such high numbers? It's a long walk from most science teaching and lab classrooms to the Koerner Library. Is this usage a sign of increasing interdisciplinarity in research? Are science students and faculty using the specialized materials in Koerner for data services, map information, social science literature, browsing, attending information literacy classes, or meeting friends? Will this “gate count” change when Barber gets rediscovered by students and faculty? 2. Why are e-resources so difficult to find? Is the collections “gap” really a resource issue or is it perhaps a findability issue? Which resources are hidden? Does usage increase with better visibility on the Web site? 3. How can physical access be improved? How can we make the big small(er)? How can we overcome the complexities of a multi-branch, decentralized library system? Can the arrangement of materials be made more consistent between buildings? Can better signage and online information improve wayfinding? 4. How will the results of the next LibQUAL+® survey (2009) compare with the 2007 survey? Specifically, what difference will the opening of Friesen 455 Barber have made to perceptions of “library as place”? Conclusion In summary, use of both analytical tools, ATLAS.ti and Nesstar, and the Theme Teams' insightful interpretation of the data helped us to understand the LibQUAL+® story. The clear message from users was to make it easier for them to find people, information, resources, places (and inside the places). This directive informs our assessment plans and programs to improve customer services, the library Web site, access to collections, and to address gaps in our collections. —Copyright 2008 Margaret Friesen Nesstar: A Brief Description Nesstar (Networked Social Science Tools and Resources) http://www.nesstar.com Contact: Norwegian Social Science Data Services (NSD) http://www.nsd.uib.no/nsd/english/index.html Nesstar is a Web-based software system used to publish and share statistical data. The tools enable finding, browsing, visualizing and analyzing data online, as well as publishing various kinds of survey data. Nesstar is a complete metadata authoring tool (description of the various elements of the data resource, including documentation) and is DDI compliant (a metadata standard used for documenting datasets developed in European and North American agencies). http://www.nesstar.com/software/publisher.html Nesstar WebView is used to view data and metadata that have been published with Nesstar Publisher via a Nesstar Server. Nesstar WebView incorporates the following features: 1. searching and browsing  simple and advanced search  ability to browse data and accompanying documentation 2. analytical tools  display of descriptive statistics  crosstabulations  correlations  regressions  compute and recode  graphical representations of data in customizable forms  application of variable weights 3. data access  support for datasets to be downloaded in various statistical formats  subset functionality for customizing data according to users' needs http://www.nesstar.com/sofware/webview.html