UBC Theses and Dissertations

UBC Theses Logo

UBC Theses and Dissertations

Development of an instrument for evaluating customer interface of B-to-C websites Zhao, Xiaosong 2003

Your browser doesn't seem to have a PDF viewer, please download the PDF to view this item.

Item Metadata

Download

Media
831-ubc_2003-0128.pdf [ 3.5MB ]
Metadata
JSON: 831-1.0090905.json
JSON-LD: 831-1.0090905-ld.json
RDF/XML (Pretty): 831-1.0090905-rdf.xml
RDF/JSON: 831-1.0090905-rdf.json
Turtle: 831-1.0090905-turtle.txt
N-Triples: 831-1.0090905-rdf-ntriples.txt
Original Record: 831-1.0090905-source.json
Full Text
831-1.0090905-fulltext.txt
Citation
831-1.0090905.ris

Full Text

D E V E L O P M E N T OF A N I N S T R U M E N T F O R E V A L U A T I N G C U S T O M E R I N T E R F A C E OF B-TO-C WEBSITES by XIAOSONG ZHAO A THESIS SUBMITTED IN PARTIAL FULFILMENT OF THE REQUIREMENTS FOR THE DEGREE OF MASTER OF SCIENCE IN BUSNINESS ADMINISTRATION in THE F A C U L T Y OF GRADUATE, STUDIES DIVISION OF M A N A G E M E N T INFORMAITON SYSTEMS F A C U L T Y OF C O M M E R C E A N D BUSINESS ADMINISTRATION We accept this thesis as conforming to the required standard THE UNIVERSITY OF BRITISH C O L U M B I A January 2003 © Xiaosong Zhao, 2003 In presenting this thesis in partial fulfilment of the requirements for an advanced degree at the University of British Columbia, I agree that the Library shall make it freely available for reference and study. I further agree that permission for extensive copying of this thesis for scholarly purposes may be granted by the head of my department or by his or her representatives. It is understood that copying or publication of this thesis for financial gain shall not be allowed without my written permission. The University of British Columbia Vancouver, Canada Date " U L - , f . ^ > DE-6 (2788) Abstract This report describes the development of a web feature checklist that evaluates customer interfaces in the context of B-to-C (Business-to-Consumer) business. The associated study concentrated on the interaction between consumers and businesses, and the experimental design adopted as its framework the "7Cs" concept, which represents seven design elements to the customer interface introduced by Rayport, Jaworski, Griffin, and Dosiso in chapter four of "E-Commerce"(2000). The research was conducted in two phases. In phase one, a 113-item questionnaire pool was created by compiling issues proposed by the original 7Cs concept, in combination with items derived from existing guidelines, checklists and research works. Twelve respondents were then asked to categorize each item from the questionnaire pool into one of the "C" categories, using the card-sorting methodology discussed by Moore and Benbasat (1991). This process of sorting resulted in a 56-item checklist. In phase two, the checklist was applied to evaluate three websites selected and ranked by www.forbes.com. A non-parametric analysis showed that the ranking among the three websites by twelve respondents using the checklist were consistent with the ranking by Forbes. The test results proved that the checklist developed in this research can be an effective instrument for evaluating customer interface in B-to-C businesses. Limitations of the study and possible directions for future research are also addressed in this report. Keywords: Web Evaluation, 7Cs Framework, Checklist Development, Card Sorting. ii Table of Contents Abstract ii Table of Contents iii-iv List of Tables v List of Figures vi Acknowledgments vii Section 1.0 Introduction 1 Section 2.0 Literature review and research framework 3 2.1 Literature review 3 2.2 Framework of current research 5 2.3 Application of previous research to current framework 9 Section 3.0 Research methodology IV Section 4.0 Phase I: Construction of the checklist 19 4.1 Stage 1: Survey questionnaire design 19 4.2 Respondents 23 4.3 Stage 2: Sorting 24 4.4 Creation of the checklist ....25 Section 5.0 Description of the three websites 29 5.1 Description of www.redenvelop.com 30 5.2 Description of www.tiffany.com 37 5.3 Description of www.spawish.com 43 Section 6.0 Phase TJ: Instrument testing 49 6.1 Checklist testing design 49 iii 6.2 Evaluators 50 6.3 Testing objects 51 6.4 Test instruction 52 6.5 Testing result 52 6.5.1 Comparison of the average scores among evaluators 53 6.5.2 Non-parametric test result 54 6.6 Summary of the findings 58 Section 7.0 Discussion 60 7.1 Contribution 60 7.2 Limitation 61 7.3 Suggestions for future studies 61 Reference 64 Appendices 66 Appendix 1 Card sorting questionnaire 66 Appendix 2 Sorting result—Checklist 83 Appendix 3 Checklist testing questionnaire 87 Appendix 4 Inter-item Correlations within each 7C category 95 iv List of Tables Table 1 General Issues and Descriptions of 7Cs framework 7 Table 2 Gehrke and Turban's five determinants compared to categories within the 7Cs framework 10 Table 3 Liu and Arnett's critical factors to website success compared to categories within the 7Cs framework 11 Table 4 Customer decision process compared with categories within the 7Cs framework 11 Table 5 Jarvenpaa and Todd's consumer's concern compared with categories within the 7Cs framework 12 Table 6 Web application of Herzberg's Two-Factor theory compared with categories within the 7Cs framework 14 Table 7 Information system applications according to the CRLC model, compared with categories within the 7Cs framework 15 Table 8 Profile of respondents participating in the card sorting 23 Table 9 Sorting results 27 Table 10 Number of items of each category in the final checklist 29 Table 11 Profile of web evaluators 51 Table 12 Average scores among evaluators 53 Table 13 The mean rank of the total score and each of the 7Cs score among the three websites 56 v List of Figures Figure 1 Homepage of www.redenvelop.com 30 Figure 2 Homepage of www.tiffany.com 37 Figure 3 Homepage of www.spawish.com 43 Figure 4 Average score among evaluators of each C of the websites 54 vi Acknowledgements The author wishes to express her gratitude to Professor Izak Benbasat, Jai-Yeol Son and Paul Chwelos for their guidance and help in this thesis. Also, the author would like to acknowledge all the MSc and Ph.D. fellow students for their participation in the surveys. Last but not least, this project would not have been possible without the support of Dr. Susan Lazar and the advice, encouragement and friendship from Rushen Shi, Rachel Jen, Lynn li, Yang li, Hazel Yu, Dongmin Kim and Zhenhui Jiang. 1.0 I N T R O D U C T I O N With the rapid growth of the World Wide Web, shopping online has undergone remarkable acceleration during the past few years. Although the economic down-turn has put an end to many dot-Com companies, successful examples and survivors can still be found in the cyber world. While a company's success depends on many factors, customer interface is one of the major determinants for dot-com survivors. A website's interface is the storefront for an online company or organization. Its ability to attract customers' attention, induce them to complete purchases, and motivate them to return plays a crucial role for the online business. The current study focuses on the interaction between websites and consumers in the context of business-to-consumer electronic commerce. The objective of this study is to design and test an instrument using Raport and Jaworski's 7Cs framework (2000) for evaluating the interface of business-to-consumer websites. In this study, business-to-consumer e-commerce is identified as the process in which consumers select products from an e-commerce website, and then purchase those products, usually paying for the goods with a credit card. The general definition of Information System (IS) success is: "the extent to which a system achieves the goals for which it was designed" (Farhoomand and Drury, 1996, pp.45). In the context of electronic commerce, the success of an interface can be assessed by the ability of a website to provide quick answers to a number of questions that 1 confront their users, how it can support customers' decision-making, how the site attracts target segment customers and repels non-target customers, and how it induces customers to purchase and come back. Almost all business-to-consumer e-commerce websites are developed from pre-packaged modules. The usual process includes online ordering, a shopping cart listing the items the consumer has selected, a check-out that automatically calculates totals and shipping costs, and secure encrypted transfer of credit card information to a secure server. Beyond these functional features, however, successful websites are also designed to provide a comfortable and enjoyable experience to their potential customers, because the site should welcome them, not just support their purchases. During the current study, functional and design features identified in previous studies of e-commerce interfaces were subjected to a survey using the "card sorting" methodology described by Moore and Benbasat (1991), and sorted into a variety of categories. The resultant set of features and categories were then formed into a checklist, intended as an instrument to measure B-to-C websites. This paper begins, in Section 2.0, with a literature review and an introduction of the research framework of this study. Following the explanation of the study's research methodology in Section 3.0, the checklist construction process is described in detail in Section 4.0. Section 5.0 describes the interfaces of three sample websites. The testing process and results using the developed checklist on the three sample websites are 2 provided in Section 6.0. Section 7.0 discusses contributions made by this study and limitations to its scope. 2.0 L I T E R A T U R E R E V I E W A N D R E S E A R C H F R A M E W O R K 2.1 Literature review Website evaluations and usability studies have been an area of substantial research interest in recent years, but not all studies have involved solid frameworks or empirical evidence. A wide variety of "evaluation criteria" and "checklists" has been proposed in the relevant literature. Notably, Nielsen (1994) has designed a checklist of items to consider when developing computer interfaces. These checklist items, referred to as heuristics, are considered to be the standard performance indicators for e-commerce. The items included in the checklist are: • visibility of system status • correspondence between the system and the real world • user control and freedom • consistency and standards • error prevention • recognition rather than recall • flexibility and efficiency of use • aesthetic and minimalist design • help for users to recognize, diagnose, and recover from errors • help and documentation. Other individuals have engaged in conceptual discussions concerning what should be evaluated and how it should be evaluated, including "Usability Heuristics for the Web," 3 by Instone (1997), who adapts Nielsen's heuristics for the web environment. Instone's article expands on Nielsen's ten usability heuristics and reveals web interface principles. Microsoft Network has also been conducting usability research, and it has also produced a checklist which designers can use to ensure their web pages are effective (Keeker, 1997). A large number of other web interface checklists or criteria concentrating on various aspects of web sites, such as "what is reader-friendly"( Sullivan,1997), "ten things to avoid"( Sullivan, 1996), "Recipe for a successful website" (Shedroff, 1994) or "top ten mistakes in website design"(Nielsen, 1996) can be found both in the literature and on the web. Although few of these checklists and criteria are backed up by any conceptual models or empirical evidence, they have provided practical tips about how to make a website successful in cyber space. Aside from web evaluation criteria and checklists, research has also been conducted into how to make an effective web site, with various approaches to the issue. Some studies have directly examined the determining or critical factors of a successful website (e.g. Liu, Arnett, 2000; Gehrke, Turban, 1999); others assess consumer's purchasing behaviour and investigate consumers' concerns (Jarvenpaa and Todd, 1997), or how to support consumers' decision-making processes in the web environment (Helander, Khalid, 2000). Other research has attempted to identify web features that may influence user satisfaction (Zhang, Small, von Dran, Barcellos, 1999; 2000; Zhang and von Dran, 4 2001), and the application of information technology to business strategies (Ives and Learmonth, 1984). The above mentioned research, however, focuses either on the perspectives of consumers or on that of businesses, leaving a gap between consumers and businesses. In this study, Jaworski and Rayport's "7Cs" framework described in "E-Commerce" (2000) is adopted as a research framework. This framework has been selected not only because it provides a theoretical foundation of interface design that addresses the overall aspects of customer interface in the context of electronic commerce, but also because it bridges the gap by exploring the interaction between customer and business. 2.2 Framework of current research In chapter four of "E-Commerce," coauthored by Jaworski, Rayport, Griffin and Dosios (2000), a "7Cs" framework is introduced, representing seven major categories of the technology-mediated customer interface design. This 7Cs theory describes the interaction between web users and businesses from seven perspectives: Context, Content, Community, Customization, Communication, Connection and Commerce. • The Context of a website is "how" it is designed, its over-all look and feel, and the extent to which it focuses on graphics, design features and colours, financial goals, or a combination of these characteristics. 5 • Content is "what" is presented on the website. This includes "the form of the digital subject matter - text, video, audio, and graphics - as well as the domains of the digital subject matter, including product, service, and information offerings." (Jaworski et. al, 2000, pp.115) • Some websites provide the means for users of the same site to interact with each other, sharing their shopping and surfing experiences and their knowledge, and even to develop friendships. This builds a sense of Community in the cyber world. • Customization is defined as "the site's ability to tailor itself or to be tailored by each user" (Jaworski et. al, 2000, pp.115). This is the potential for users to choose their favourite content and to manage their accounts on-line, and also the site's ability to efficiently offer promotions according to each customer's profile. • While Community refers to user-to-user relationships, Communication refers to site-to-user communication: it is about how users submit requests to the site and how the site responds to the users. • Connection is defined as the links to the site from other sites. These include formal links, affiliated programs, banner ads, etc. 6 • Commerce refers to all the functional features related to the sales and services provided by the site. The examples and issues related to each of the 7Cs categories are described in Table 1 as follows: Table 1. General Issues and Descriptions of 7Cs framework 7Cs General Issues Examples and Descriptions Context Overall look and feel Functional, aesthetic or combined Relevance to the business's objectives Colour and background Consistent and appropriate Simple enough Section breakdown Clear and well-structured Easy to move back and forth 7 7Cs General Issues Examples and Descriptions Search engine Reasonable and helpful Products and services description Style of descriptions What multimedia is used Simple and clear Necessary and useful It is updated regularly Content Business description How the business, news and customer services are described Promotion Something special for the present Coupon and discount How the message is presented Functions, information, service Aids to prioritize needs Helpful to make decision Quality of the content Complete and accurate Types of community Message board/chat room Member-built site, etc. Community Sense of community Willing to exchange ideas, get information Group identity Benefit from community Influential, helpful, supportive and fun Customization Login/registration Keep personal profiles Provide free e-mail accounts Give personalized recommendations Short-cut information Customized e-mail contact Customized e-mail contact Layout and content of the screen People can chose their preferences Customized short-cuts provided Communication Ways to communicate Mass mailing/FAQ/newsletters Auto-reply mail Track order and delivery status Live chat or e-mail response Online dialogue Where to find contact information Customer service page Easy-to-find location 8 7Cs General Issues Examples and Descriptions Efficiency Delivery updated information Quick reply Page-loading time Credibility Personal contact always available Connection Link to other pages Links to sites that have what the customer is looking for Affiliate programs Structure Under separate frame Pop-up small frame Within original frame Easy to move back and forth Commerce Functions Login/registration Store customer information Shopping cart Discounts/coupons Payment methods Delivery options Save credit card information Check order and delivery status Credit approval Security Security statement Customer service Refund and cancellation Provide free service and useful information Registration form Necessary Excessive information is not required Personal contact Where to contact, in what ways 2.3 Application of previous research to 7Cs framework Most of the issues proposed by previous researchers can be fit into the 7Cs framework, as demonstrated above. For example, by concentrating on experts' recommendations and 9 by soliciting the opinions of end-users, Gehrke and Turban (1999) have found five categories of determinants for an effective website, from an e-commerce point of view. Among the five determinants, business content can be classified into "Content," page loading speed can cover the category of "Connection" and "Communication," navigation efficiency can be included within the category of "Context" and "Connection," security can be treated as an issue of "Commerce," and marketing/customer focus can cover the category of "Communication," "Community," "Commerce" and "Customization." The five determinants from Gehrke and Turban' research (1999) and the corresponding category within the "7Cs" framework are illustrated in Table 2.: Table 2. Gehrke and Turban's five determinants compared to categories within the 7Cs framework Determinants of an effective website Related 7C categories Page loading speed Connection, Communication Business content Content Navigation efficiency Context, Connection Security Commerce Marketing/customer focus Commerce, Customization, Community, Communication Liu and Arnett (2000) have conducted a survey of webmasters from Fortune 1000 companies with a research framework derived from previous studies. "A successful web site, in the context of EC, is one that attracts customers, makes them feel the site is trustworthy, dependable, and reliable and generates customer satisfaction" (Liu and Arnett, 2000). The research has revealed four factors that are the most significant to website success: information and service quality, system use, playfulness, and system design quality. Table 3 shows these factors and the related 7Cs categories. 10 Table 3. L iu and Arnett's critical factors to website success compared to categories within the 7Cs framework Critical factors to web site success in E C Related 7Cs categories Information and service quality Content, Context, Communication, Community, Commerce, Customization, Connection System use Connection, Commerce, Customization Playfulness Content, Context, Community, Customization System design quality Connection, Commerce Many researchers have attempted to model customers' behaviour while shopping, and they have investigated their concerns and the factors that can affect computer-users' decision-making. For example, Helander and Khalid's essay, "Modeling the customer in electronic commerce" (2000), identifies a set of five decisions that customers make when they go through a website: the decision to visit, the decision to navigate, the decision to buy, the decision to pay and the decision to keep. Helander and Khalid also suggest that "the artificial environment must be designed so that it can support customer decision-making" (2000). According to the five decision-making steps, Table 4 addresses the web features that affect customers' willingness to purchase, and the corresponding categories within the 7Cs framework. Table 4. Customer decision process compared with categories within the 7Cs framework Customer decision process Web features that affect customers' decisions Related 7Cs categories To enter the store Search engines, e-mail messages, links from the site Content, Context, Community, Connection To navigate Navigation facilities, content quality Content, Context, Connection To buy Price, product information, promotions Commerce, Content, Community, Communication, Community To pay Time to fill on-line form, ease of purchasing, credit card security Commerce, Customization To keep the merchandize Exchange/return policy, feedback information Commerce, Communication 11 Similarly, Jarvenpaa and Todd (1997) have studied consumer reactions to electronic shopping by interviewing 220 e-shoppers, and they have identified product perception, shopping experience, customer service, and consumer risk as major concerns for customers. The factors involved in the identified four categories and the related 7Cs categories are displayed in Table 5. Table 5. Jarvenpaa and Todd's consumer's concern compared with categories within the 7Cs framework Jarvenpaa and Todd's category Related factors Related 7Cs categories Product perception Quality, variety, price Content, Commerce Shopping experience Effort, compatibility, playfulness Commerce, Community, Content, Context, Customization Customer service Responsiveness, assurance, reliability, tangibility, empathy Content, Communication, Commerce Consumer risk Economic risk, social risk, performance risk, personal risk, privacy risk Content, Commerce, Community In addition to earlier studies of consumer shopping behaviour, other researchers have focussed on HCI (human computer interaction) to identify people's needs and the causes of their satisfaction, with the assumption that a successful website has to give users an enjoyable and comfortable experience, rather than simply compelling users to complete a purchase. Herzberg's hygiene-motivational theory (1966) about the workplace has been applied to web environments, to distinguish the features that contribute to the basic functionality of a website from other features that contribute to users' satisfaction and motivate their return to a website (Zhang, Small, von Dran, Barcellos, 1999; 2000; Zhang and von Dran, 12 2001). Hygiene factors provide the basic architecture and content of a website, and they make the website useful and serviceable; the absence of these factors will cause dissatisfaction among users. Therefore, hygiene factors are the predominant concern when consumers decide whether to shop electronically. Motivational factors, on the other hand, are the factors that will contribute to user satisfaction. They may add value to the functional aspects of design by appealing to aesthetic, cognitive and emotional preferences of users, and they can play a key role when consumers choose among different electronic stores. Identifying these factors may therefore help the business adopt effective strategies for various environments and different business focuses. For the hygiene-motivational factors applied to the web environment examined by the empirical research for "A Two Factor Theory for Website Design" and "Websites that Satisfy Users: A Theoretical Framework of Web User Interface Design and Evaluation" (Zhang, Small, von Dran, Barcellos, 2000), the 7Cs categories are seen to overlap each other. Details are shown in Table 6. 13 Table 6. Web application of Herzberg's Two-Factor theory compared with categories within the 7Cs framework Factor Theorized application to the web environment Related 7Cs categories Hygiene First impression or general appearance Context, Content Basic functions/ features that help to complete tasks Content, Connection, Commerce, Context Requirements for tasks Commerce, Communication Feedback or response Commerce, Communication, Connection Access restriction Content, Commerce Privacy and data confidentiality Content, Commerce Credibility of owners/designers and the website: trust and trustworthy Content, Community Web owners/designers' attitudes and perceptions Content Authority and availability of owner/designers Content, Commerce Navigation Context Motivation information-seeking tasks Content Quality of the information content Content Presentation/organization of information Context, Content Task completion Commerce, Community User control Customization, Content, Communication Knowledge or skills gained Content, Commerce Recognition by owners/designers of knowledge and skill levels Customization, Communication From the business strategy perspective, Ives and Learmonth (1984) have applied information system technology throughout the "Customer's Resource Life Cycle" (CRLC) in their work. They identify "the information system as a competitive weapon" to analyze and propose supplier-developed strategic information systems. CRLC is a 13-stage customer resource life cycle model, which customers experience when they acquire an organization's products or services. This model focuses on a firm's relationship with its customers and "how this relationship can be changed or enhanced by the strategic application of information system technology" (Ives and Learmonth, 1984). The strategic application of information system technology can provide firms with 14 competitive advantages. The categorizing of information system applications with each step in the CRLC model can also be classified into the 7 C s category as Table 7 below demonstrates: Table 7. Information system applications according to the CRLC model, compared with categories within the 7Cs framework Construct of CRLC Information system applications Related 7Cs category Establish requirements Provide aids to help customers identify their needs Display success stories from other customers Show the customers what products or services can be provided Content Community Content Specify Product appearance Product feature description Content Content Select a source Contact information Company history, description and updated information Alternative vendors if the product is temporarily unavailable Content Content Connection, Commerce Order Customers can place an order over the web The site will acknowledge customers' receipt of the orders Shopping cart is provided for adding and deleting items Customers are free to modify their orders before paying Product availability and delivery date is provided before the completion of the order Commerce Communication Commerce Commerce Commerce Authorize and pay for The web allows various methods of payment The site has an instant credit-checking function Security guarantee The site explains when the payment will be completed Commerce Commerce Content Content Acquire Customers can check their order status The shipment date will be confirmed The site provides a choice of delivery options Commerce Communication Commerce Text and accept Customers can try out and test the function of the product before placing an order Easy return policy Third party function verification Commerce Content 15 Construct of CRLC Information system applications Related 7Cs category Integrate into and manage inventory Monitor use and behaviour Upgrade if needed The site will upgrade its function with customers' needs Content Maintain On-line help, or information about how to get support with problems Communication Transfer or dispose Customers can upgrade the products on-line The site provides instructions on how to return/replace a product Content Commerce Account for The site makes it clear how much is spent on tax, products, shipment and packing The site provides details of each customer's account information (including previous purchases, returned/cancelled orders, coupons, free gifts etc.). Free and useful services for the customers Commerce Customization Content The 7Cs framework not only merges factors and structures from previous research with the research framework applied in this study, but it also includes pre-existing general guidelines and criteria in the questionnaire to which the websites will be subjected, and determines into which of the seven categories each of the guidelines and criteria can be placed. A checklist that as extensively as possible covers the range of electronic commerce website features, according to the 7Cs guideline, can be developed by aggregation and analysis of the opinions and findings from previous research and identification of related 16 issues. This checklist would then provide a more comprehensive view of the interaction between consumers and electronic business tools. 3.0 R E S E A R C H M E T H O D O L O G Y The goal of this research is to conduct a web feature checklist in the electronic commerce domain according to the 7Cs theory. This study was conducted in two phases: Phase I: Constructing a checklist for website evaluation according to the 7Cs theory Through an extensive literature review, 113 web evaluation items (see Appendix 1) were compiled from a number of resources to develop an initial questionnaire, then respondent subjects were asked to sort the items according to the seven categories defined in the 7Cs theory, using the "card sorting" methodology (Moore and Benbasat, 1991). Card Sorting is a technique for exploring how people group items, by developing structures that maximize the probability of users being able to find items. Normally, it starts with a list of all the items to be sorted, each recorded on separate index cards that are subsequently shuffled into random orders. The user(s) are then given the stack of cards and asked to divide the cards into piles, grouped the way they (the users) best see fit. 17 The current study did not literally use index cards. Instead, the "card sorting" concept was adopted to design a survey questionnaire form: the items were listed in random order and test subjects were asked to pick one category out of seven into which each of the items should be fit, and items that did not seem to fit into any category were left in the "none of the categories" section. The result was a shorter list of items representing each category and covering the scope of most web features within a certain length. Phase IP. Testing the effectiveness of the checklist After finishing the checklist design, the instrument's sufficiency and its effectiveness for evaluating websites were tested. Three web sites were selected from http://www.forbes.com/bow/b2c/main.ihtml for the evaluation test. These websites were ranked from the highest as "Forbes Favorites," one of the best sites as a "Forbes Best of the Web" pick, to one among the other sites, which have no special classification; in other words, the best, good and normal sites according to Forbes's criteria. Apart from the "Forbes Favorites" site, which was labelled by Forbes as the best website, the other two sites were chosen randomly from their classification. The success of each web site was measured with the checklist to see if the results corresponded to the ranking from the original resource (i.e. Forbes.com). 18 The two phases are described in detail in section 4.0 Construction of the checklist and section 6.0 Instrument testing. 4.0 P H A S E I: C O N S T R U C T I O N OF T H E C H E C K L I S T The objective of Phase I was to construct a checklist according to the 7Cs framework. A heuristic evaluation was completed to accomplish this. A heuristic evaluation (Nielsen and Molich, 1990; Nielsen 1994) is a type of expert evaluation "for finding the usability problems in a user interface design so that they can be attended to as part of an iterative design process. Heuristic evaluation involves a small set of evaluators examining the interface to judge its compliance with recognized usability principles." (Nielsen, 1994) Heuristic evaluations are well suited for the web because they can be easy, fast and inexpensive. Therefore, the issues identified in a heuristic evaluation have been used to generate a usability test for the current study. Construction of the checklist was carried out in two stages: Stage 1: a list of 113 items was constructed according to the description of the 7Cs framework and other resources. Stage 2: expert subjects were asked to fit each item into one of the 7Cs categories. 4.1 Stage 1: Survey questionnaire design 19 A list of 62 items that appeared to fit the 7Cs definitions were first collected as the initial questionnaire. Existing survey questions from several web checklists and evaluation criteria referenced by previous research (Zhang, Small, von Dran, Barcellos, 1999; 2000; Zhang and von Dran 2001) were also added to the list. The sources included: http://webreview .com/97/10/10/usabilitv/sidebar.html http://www.microsoft.com/workshop/management/planning/improvingsiteusa.asp http://www.useit.com/alertbox http://www.pantos.org/atw/35317.html http://www.capecod.net/schrockguide/eval.htm http://www.nathan.com/thoughts/recipe http://www.worldbestwebsites.com Other issues were extracted from theoretical research sources, according to the following guidelines : • Assessments of items that affect the decision to purchase electronically and the factors that drive repeated visits. ( Helander and Khalid 2000) • Items derived from five major categories (page loading speed, business content, navigation efficiency, security, marketing and customer focus), extracted from an 20 extensive literature review, and approved through the results of a consumers' survey. (Gehrke and Turban, 1999) • Four major factors associated with website success: quality of information and service, system use, playfulness and system design quality. (Liu and Arnett, 2000) • Twelve hygiene and motivational categories and 44 core features. (Zhang, von Dran, Small and Barcellos, 2000) • Based on a set of functional guidelines, seventeen factors considered very desirable or desirable for electronic stores. (Liang and Lai, 2000) • Five of the most important website quality features (security of data, ease of navigation, appropriate explanatory text, search tools, product and service price concerns) in electronic commerce. (Zhang and von Dran 2001) Items were added and compiled to create an instrument that would effectively cover the scope of electronic commerce interface, making the list of items a sufficient description of the subject. The original items were merged into the first questionnaire by deleting redundancies. Examples of redundant items are as follows: • The load time is appropriate to content, even on a slow dial-in connection. (http://www.pantos.org/atw/35317.html) • Will people be confused or frustrated by uncertainties regarding page-loading, audio loading, or download time? 21 (http://www.microsoftxoir^ A list of 193 items is the result of the compilation and extraction from the above-mentioned resources. Once the 193-item pool was created, items from various categories were then re-evaluated to merge and refine those which have similar meanings. Examples of similar and refined items are as follows: • The colour of the pages is consistent and appropriate, (constructed from 7C's definition) • The site is moderate in its use of color, (http://www.pantos.org/atw/35317. html) Refined as: The colour of the pages is consistent, simple and appropriate. The initial compilation left the following number of items in each category for a total of 113 items. The remaining items are listed in Appendix 1. The following shows the number of items left in each category: Context Content Community Customization Communication Connection Commerce 19 20 13 9 20 11 21 TOTAL 113 22 4.2 Respondents Seven MIS (Management of Information System) Masters students and five MIS Ph.D. students were asked to complete the sorting. A profile of the respondents was designed according to the findings of Bellman, et al.'s report in 1999 concerning consumers' attitudes towards electronic shopping. In this report, five consumer characteristics could predict 45 percent of the buying behaviour: 1. whether they look at product information on the web 2. their years of experience with on-line activities 3. their number of daily e-mails 4. whether they work on-line every week 5. whether they read news on-line at home The profile of expert respondents from the study are listed in Table 8. Table 8. Profile of respondents participating in the card sorting Subject Check product info, on the web (Y/N) Years of experience with on-line activities Number of daily e-mails Work on line every week (Y/N) Read news on-line at home (Y/N) 1 Y 6 10-12 Y Y 2 Y 5 Over 10 Y Y 3 Y 10 Over 10 Y Y 4 Y 5 8 Y Y 5 Y 5 15 Y Y 6 Y 5 3 Y Y 7 Y 5 5 Y Y 8 Y 13 60 Y Sometimes 9 Y 5 8-10 Y Y 10 Y 8 6-7 N Y 11 Y 7-8 50 Y N 12 Y 6 5 Y Y 23 This respondent profiles have been retained for further research and explanation of any unexpected results. 4.3 Stage 2: Sorting The goal of sorting was to construct seven categories for items, according to the 7Cs framework. From the original study of Moor and Banbasat (1991), there were two ways to do the sorting: One was to give the sorting judges clear labels of which constructs the items should be fit into; the other way was to ask the judges to provide their own labels for each construct. Since there were 113 items in the sorting questionnaire, it would have been difficult for the sorting respondents to create constructs if the construct labels were not provided. In this study, 7Cs categories were clearly defined before the sorting. The basic procedure was to ask the experts participating in the study to sort the items from the initial pool into separate categories. Items which had the most "inconsistent opinions" and which did not seem to fit into any of the categories were eliminated. Items were then examined and ambiguous items were eliminated. The final items left after the sorting clearly fit into particular categories, and formulated the checklist according to the 7Cs framework. The items were placed into a random order in a 113-item-survey. The survey was designed as a multiple-choice questionnaire, asking test subjects which category each item should be fit into. The latter included eight choices, seven of which were the 7Cs categories, and one denoted as "none of the categories." 24 Each respondent was asked to complete the questionnaire independently from the other subjects. At the end of the survey, an "additional opinions" section was provided for testers to express their special ideas and to suggest interface features that had been excluded from the list but which could be important to the commerce of the website. Prior to sorting items on the survey, test respondents read the standard definition of each category of the 7Cs from the original text of Rayport and Jaworski (2000), detailed descriptions of what each C encompasses, with examples, to ensure that the respondents fully comprehended the distinctions between the categories. 4.4 Creation of the checklist After the list of criteria for e-commerce web sites were sorted in Phase I of this study, the answer sheets from the twelve participating judges were examined. The analysis of each judge's answers is outlined in the following table in the example of items A and B listed below. Item Context Content Communication Connection Commerce Community Customization None Total A 1 0 0 0 10 1 0 0 12 B 0 0 3 4 4 1 0 0 12 After collecting the raw data, particular items were eliminated if there was a high level of agreement among the judges, identifying the items as belonging to "none of the categories." Because there are no established guidelines for determining a "good" level of agreement, the definition of "high agreement level" was set as eight identical 25 responses out of twelve opinions. It must also be emphasized that this procedure is more like a qualitative analysis than a strictly quantitative procedure. Examples of the items placed in the "none of the categories" list are given below. People will find new challenges as they gain experience. Context Content Communication Connection Commerce Community Customization None Total 0 0 0 0 0 0 0 12 12 Customers will gain confidence during transactions. Context Content Communication Connection Commerce Community Customization None Total 1 0 0 0 3 0 0 8 12 For the items that elicited differing responses regarding which category they should fall into, three factors can explain the disagreement: a. The description of the item is not clear enough b. The item was not clearly defined as belonging to any of the categories c. The item tends to cover more than one area Since the instrument being developed is a checklist within the 7Cs framework, items that do not clearly belong to any category within the framework are beyond the scope of the current research. In other words, items that have disagreement (less than eight identical responses out of twelve opinions) for any of the above mentioned reasons are beyond present research considerations. 26 The items with a high level of agreement (more than eight identical responses out of twelve opinions) from the judges on a particular category were kept. Examples are as follows. Customization: Web users can personalize the layout and content of their page and make their personal settings. Context Content Communication Connection Commerce Community Customization None Total 0 0 0 0 0 0 12 0 12 Context: The colour of the pages is consistent, simple and appropriate Context Content Communication Connection Commerce Community Customization None Total 11 1 0 0 0 0 0 0 12 Content: The site provides a clear description of the product Context Content Communication Connection Commerce Community Customization None Total 0 11 0 0 1 1 0 0 12 Based on the results of analysis, a list of 53 items that have high agreement regarding the category to which they belong was identified. Table 9 shows the original number of items in each category, and the total quantity after the items were sorted: 27 Table 9. Sorting results 7Cs Original length Reduced length Context 19 8 Content 20 8 Community 13 9 Customization 9 5 Communication 20 7 Connection 11 4 Commerce 21 12 Total 113 53 After they were sorted, the resulting items were compared with the definition and description of 7Cs in the original text, to discern if any important item had been overlooked on the new list. This was done to ensure the development of a list that can fully cover the scope of the 7Cs and best represent the assessment of these websites according to the concepts used in the 7Cs theory. After further examination, three items that are clearly defined in the text (e-Commerce, 2000) were found to be missing. The omissions consisted of one item within the category of Context and two items from Communication. Descriptions of the items are listed below, with the results of sorting. Context: The links are logically grouped and easy to move back and forth Context Content Communication Connection Commerce Community Customization None Total 6 1 0 4 0 1 0 0 12 28 Communication: The site offers FAQs Context Content Communication Connection Commerce Community Customization None Total 0 5 6 0 0 1 0 0 12 The load time is appropriate to content, even on a slow dial-in connection Context Content Communication Connection Commerce Community Customization None Total 3 0 5 1 0 0 0 3 12 After this inspection, the final sorting result of this research was conducted (see A p p e n d i x 2) and the number of items in each category is as follows. Table 10. Number of items of each category in the final checklist 7Cs Original length Reduced length Context 19 9 Content 20 8 Community 13 9 Customization 9 5 Communication 20 9 Connection 11 4 Commerce 21 12 Total 113 56 5.0 DESCRIPTIONS OF T H E T H R E E W E B SITES. Three websites were selected from http://www.forbes.com/bow/b2c/main.jhtml.as evaluation objects for phase JJ of this study. A detailed description of each site within the framework of 7Cs follows. 29 5 . 1 Description of www.redenvelope.com Figure 1. Homepage of www.redenvelop.com 1 3| HodCnvBlope - hitp //hyzxk| hnme r.hinaien mats B«c* Stop fl*«h '& _sd -^ J* Mom* Setftfi Fevontet Hutery Mai Rr* Diwajti Fleal.tum M UA* " AS*ei»\tt] Hip ' w w fedon-oope com •e/g*tj/cot)uct_<Juptaii/hc««>e i«cVnc"35385SflV_Se»t«rtC •99990233! 5697. "1031946733@«3@e*BV_Enone ) - ceck adcgeiekf micel rC0EN V E L O P E O C C A S I O N KCCICJCNT LIT-EST?II: S H O O I. Hi* mt ' K«S*:<.(| :-*«:*xi • *hrj> *<t>*:»ij - t,(*l:>m*. «*!,•(* • :-v t(r :; -,f . . . : jpfodLd 3r item t \"\ firciM tracking • stsaui RfidEnvaicfi* - conUct us - pnvar.y policy - shipping "~ ~ :<£ internet www.redenveIope.com is ranked as the favourite site according to Forbes. The analysis with the 7Cs framework of this site is described below. Context: RedEnvelope.com is designed to sell gifts varying from candles, kitchenware and flowers to jewellery and cigars. It combines aesthetics and functionality. On the aesthetic side, RedEnvelope.com is a highly visual site, with lots of photographs of products. On the functional side, it focuses on its core offering - gift selling. 30 A wide variety of products is crucial in this business. The website has to make people feel that there must be something that suits them. RedEnvelope.com's top level of options include "occasion," "recipient," "lifestyle" and "shops," under which are listed subcomponents of "for her," "for him," "birthday," "romantic," etc. Each of the sections is very well organized and structured. The top-level categories are at the top of every page; customers therefore find it very easy to move back and forth. The links of policy, seasonal products, company information and customer service are clearly displayed on the home page and on every other page. When browsing, a description of the section breakdown is included on top of the left side, making the customers' current location very clear. The background colour is white, and colourful pictures of every product are provided. The layout of each page is consistent, clean and simple. The site provides reasonable speed and reliable downloads of graphic information. Content: RedEnvelope.com is an online store for gifts. The site content focuses primarily on the company's products. Besides the colourful pictures of the products, the site also provides text information. There are no other multimedia descriptions of either the products or the services. 31 "Gift Reference" is provided in the "occasion " page to help customers make their decisions. For example, the explanations of "birth stone," "anniversary" and "birth flowers" give customers ideas about how to choose gifts for certain occasions. Beyond the product classifications on top of every page, there is also a shopping alternative in the option, "shop our catalogue," which allows customers to choose items from the full catalogue. Seasonal products are listed on the home page in a very distinctive area. They are updated from time to time. The search box is located at the top of right side of every page. If people know the product number, they can find the product directly, or alternatively they can go to the "refined search" page. In this page, people can search products by keyword, category, source code or price range. A link to the business description "about RedEnvelope" is located at the bottom of the homepage. Company information, including its history, contact information and policies, is described in plain text on that page. 32 A link to the customer service page can be found in every other page. Besides the contact information, there is a very organized table with a list of links concerning the assistance the page offers to customers. Communication: People can find the customer service link on top of every page, and contact information is provided on the service page. The customer service is available 24 hours a day, seven days a week. A chat room is provided within the customer service page, where customers can participate in a live chat if they have any questions while shopping. A help icon called "Personal Shopper" is also listed in the customer service page. When customers need help, they can simply click on that icon and use the pop-up window to chat. The live help service is provided any day of the week, 6 a.m. to 6 p.m. No FAQ can be found on this site. The site provides a "Gift Reminder" function in customers' personal accounts, which will automatically remind customers of their special occasions 14 days earlier. RedEnvelope.com keeps contact with customers mainly via e-mail. The site sends out a welcome e-mail soon after a new customer's registration is received, to confirm their 33 account ID and password. Also, there is an option while registering for customers to subscribe to the company's e-mail newsletter. In the customer service page, which can be found through the link on every page, RedEnvelope.com provides a contact e-mail address, a fax number, a physical address and a telephone number. Connection: RedEnvelope.com presents links to users as underlined and highlighted words and pictures. Affiliated programs can be found in the customer service page for those who are interested in joining the program. Registration and login pages are also provided. The links on the site are fast and correct. In the course of this study no system crash was found from the user's screen. Commerce: RedEnvelope.com provides login/registration functions for customers. It stores information for both the customer and the gift recipient. Credit card information and billing addresses are also saved. 34 The shopping bag is always at the top of every page. It is every easy to locate and check, and to delete and add items. The site has security and privacy statements under the customer service page. It uses a secure server (SSL) to encrypt all of the customers' personal information. The strong security measures prevent the loss, misuse and alteration of customer information once it is in the records. The site accepts various shopping and payment options. Customers can place orders online or over the phone, and they can pay by American Express, Visa, Mastercard and the Discover credit card, and by debit cards with a Visa or MasterCard logo. RedEnvelope.com has instant credit approval functions to check if the credit card is valid. The inventory is constantly updated in real time. When an item is ordered, customers will find instantly on the product information page whether or not it is in stock, and if not, when it will be. An order summary of tax, shipping and the item is calculated into a subtotal before check-out. After ordering, a customer receives two e-mail confirmations from RedEnvelope. First, the site confirms receipt of the order within a few minutes of ordering. Second, the customer is notified when the order ships. RedEnvelope.com offers three delivery options: FedEx, UPS and Airborne Express. 35 Customers can check their order history and delivery status via their personal accounts. Customer satisfaction is guaranteed by RedEnvelope.com. If, for any reason, a customer wants to exchange or return their purchase, the company will replace it with another selection or make a refund to the credit card used for the purchase. If the gift is returned because it is damaged, defective or not the correct product, RedEnvelope.com provides a prepaid USPS return label. Community: There are no special functions for community such as a "users' club," a "group chat room" or any kind of bulletin board. However, in the text description of goods and pictures of models, RedEnvelope.com presents a strong sense of lifestyle ranging from romantic to sports enthusiast, from daily life to special occasions. These identifiers could be interpreted by customers as a sense of community. Customization: RedEnvelope.com provides personal accounts for members. Members' profiles and gift reminders are included in each account. Members can log in to their account any time while they are browsing. They can check their order history and delivery status via their account records. 36 No customized page layout is provided. The web site's "Personal shopper" function can give customers live recommendations while they are shopping. 5.2 Description of www.tiffany.com Figure 2. homepage of www.tiffany.com 3| titfanv S Co hllp t/bVixk\ home chinaien com ???????????????? KM 3 J i3 a Ja 3 1 Back S(op Fteltt.h Horn* Stach Favortes H^oy < M*i !l V Ji Discw* Rftslaom Maswngw . .• :,i.V-»..:,p"-m^d*ADE47562DSO7*05aAC7B*C5e*19UCA?Mn>e -103194W5315OWW TIFFANY&CU A K B j r - r a r n » u i M U S I 'IF ' J '.lO-V1 N - ^^^^^^^^^^^^^^^^ • B B s M M : ' TlFFANY&CO. Weiccuet to Tiffany & Co., Am«nca's Mcuic rf dttttW '83? »viMf k> meweMM . CAWpFJuea j teKat rc tK i Manna www.tiffany.com is ranked as one of "Forbes Best Web Picks." The analysis of this site using the 7Cs framework is described below. Tiffany is a brand name; the store has been running since 1837. The company's focus on selling high-valued jewellery, luxury and romantic atmosphere is very important for this site. 37 Context: The Tiffany.com homepage consists of two versions, one in HTML and the other in Flash. The Flash version is recommended as an "enhanced" version of HTML. The Flash features pop up once the page starts. The design of the Flash page is more fancy, and it enables users to view the product picture and to zoom in and out to see every detail of the jewellery from different angles. Pictures from the flash site have better quality, making the site more elegant and luxurious, but the loading speed is slow. Both the site and the picture background are white, making the graphics of the jewellery very distinctive. The grey header complements the pattern well. The layout is consistent from page to page. The top level of the structure consists of links to a business description "About Tiffany," a guide to choose and care for jewellery, "Jewellery Expertise" and a design collection, "Tiffany Style." A site index is at the bottom of each page, giving a map of the structure of the whole website. However, some pages don't have a site map or "go back" link to locate the current position. It is not always easy to go back to the previous page or move to any other page as desired. Content: 38 Tiffany.com is a specialty store that focus on the jewellery business. It doesn't offer any discounts or promotions under any circumstances. The jewellery pictures appearing on the homepage are changed from time to time to show different styles. The site starts with the introduction of Tiffany, and a link to the site's shopping channel is listed on top of the homepage and all the other pages. A business description, "About Tiffany," is listed at the bottom of each page. Some extraordinarily expensive jewellery is also presented, though it is not available for on-line shopping. Tiffany.com displays those items with high quality pictures and with a contact number listed below to attract people's attention and to encourage purchases by other means. The search engine is found on the "shopping" page. The search criteria include categories and keywords. It is easy for customers to locate the products they are looking for. Besides the high quality graphics of the products on the consistently white background, there is a simple and clear text description and the product price for each item. 39 "Jewellery expertise" is aimed at teaching customers how to choose diamonds and pearls, and how to take care of the jewellery. This is a value-added service and can help customers to make decisions. A customer service link is found at the top of every page. FAQs and a customer on-line account is located through this page. Community: There is no "message board," "chat room" or any kind of member-build site provided in Tiffany.com. However, the luxury environment gives people a strong group identity. Customers' initial registration can also be made by phone or by visiting a registry consultant at a Tiffany store. This gives people the feeling that they are special and well cared for. Tiffany has its own credit bureau; people can apply for a Tiffany & Co. credit card. This may provide a strong sense of community and increase customer loyalty. Customization: Tiffany.com includes cookies to save users' information. Orders placed on Tiffany.com are limited to shipments within the U.S. and the U.K. People living outside these two countries cannot even browse the "shopping pages" once they specify their location the first time they browse the site. 40 Customers can set up their own online accounts when they make a purchase on Tiffany.com. The account information can subsequently be checked when a customer logs in with a registered e-mail address and password. The account retains records of the customer's complete order history and all of the items they were have ever been interested in online. No customer-personalized layout or content is provided in Tiffany.com. Tiffany.com sends an automatic reply to people who e-mail comments and suggestions to the site. Communication: A link to the customer service page is located at the bottom of each page. It is very easy to find. On the customer service page, there is a form to allow people to send their feedback to the site, which is convenient and easy for the user. Customers can also leave their names and e-mail addresses to be contacted directly. E-mail questions will solicit e-mail replies. There is also a link to FAQs from the customer service page. No mass-mailing is provided from the site. Except to confirm an online order or response to a sender's e-mail query, Tiffany.com does not contact visitors via their e-mail addresses. 41 The company contact number is a 1-800 number, posted at the bottom of each page. It is obvious and convenient. The service is available 24 hours a day, seven days a week. This gives the site an excellent level of company credibility. Connection: The links on the site are fast and correct. In the course of this study no system crash was found from the user's screen. If people choose an "enlarged view" of a particular product, a separate frame is popped up with a bigger picture. Because some product pictures are designed in Flash, browsers require high-speed connections for high quality pictures. There is no affiliation program within Tiffany.com. Commerce: Tiffany.com provides login/registration functions for customers. It stores customer information such as names, titles, e-mail addresses and phone numbers. On the customer service page, there is a link to a list of web policies, including statements regarding privacy and returns. 42 People must register and set up their own account before they can make online purch at Tiffany.com. The estimated time of delivery is displayed once the product is selected. Every purchase can be accompanied by a gift card. Customers who want to buy a product as a gift can add their message to the card. There are two shipping options: "ground" and "express." The subtotal of shipping, tax and product price is calculated before checking out. Besides Visa, Master, American Express, customers can also pay for their purchases using a Tiffany credit card. Tiffany.com has an instant credit approval function, and the credit card information, in addition to the customer's shipping and billing addresses, are all stored by the site. 5.3 Description of www.spawish.com Figure 3. homepage of www.spawish.com TJSfjiWisl. .;„.n S|M Vocation). Peisurul und CojiMiidlt; Gill CL-il.lit.ilt.-j litlp l.nmu thinoicn Linn mm a «* £** fcjdp • « • Back &JJ tfl d M J " iA- J %"*> httxit Home $%**.A f~^r >+t*«y -rf f>m 3 . J J Ed> D«cu*» Redeem /www tpawith com/ Ge ahead, mafca spa wish tifeatro come true? , , w » « . c . SpaWish.com I Gift Ceriifjcates < l +~ HowftWprts I 6f jrcJi Civ Jpa Dii ~4 l | OrawipeW** tfrtiateHi i 'About U | (BClMfckaH EXTRA SPECIAL. - PAMPERING GIFT BUNDLES - SOOTHING EYE MASKS, SCENTED BATH CRYSTALS, and more! » j.) ' • *< .->; • S E C « J » « , I ei**Pl*«-'ii°«*^<»1siawu..lai. - s. " S » »W>6Kt «S f ISflsi mm 4 3 SpaWish.com is designed to sell spa certificates across the United States. The buyer simply needs to decide how much money they want to spend, and the recipient will receive a personalized SpaWish certificate elegantly packaged in a silver envelope, along with a gift card and a printed directory of nationwide participating day spas. Then the recipient can choose any day, any spa and any kind of service or gift they desire. Context: Apart from its home page, which uses a simple purple tone on a white background, SpaWish.com has more than six colours in its menus bars, logos, forms and text. Various fonts and colours are also used in the site's text. Most links and services are listed on the left side of the pages, and the layout is consistent from page to page. However, although the menu bars and logos are at the same position on every page, the length of each page differs significantly. For example, the page titled "Drop A Hint" has a rather short layout, while the page for "Price, Services & Holidays" has five times the length and consists of long service descriptions and forms. The structure of this site is rather simple. There are altogether four sections on the home page: shopping, customer service, business partnerships, and membership programs. All four sections are listed in a menu bar on each page. 44 Content: The main commodity that SpaWish.com sells is gift certificates. Rather than searching for products, the search engine on this site is designed to locate participating spas within the U.S.; results can be sorted by spa name, city and zip code. The search box is at the top left of every page. Besides gift certificates, SpaWish.com also provides gift bundles, but there is no way to search for a bundle. All the gift bundles are displayed with pictures and prices on the "gift bundles" page. The company founder's picture is posted at the bottom of the homepage. There is no clear description of the company except the founder's short story about how she got the idea. SpaWish.com has different featured bundles on its homepage from time to time, but these do not necessarily have anything to do with seasonal promotions. There are also lists of "Gift Giving Occasions" and "Popular Services at Day Spas" to remind people when it is "time for a gift" and "what to give." Community: 45 "SpaWish loyalty rewards" is a membership program to accumulate points from purchases, which can be redeemed for free gifts. The site sends out bonus points to customers via e-mail during the holiday seasons. Apart from the "Testimonials" from some specific users on the homepage, there is no message board or chat room provided on this site. Users at the same site get very little contact among each other and the community influence is limited. Customization: SpaWish.com offers site visitors the opportunity to log-in and register themselves, but there are no customized lay-outs for each individual. Customers can check their order history via their account; it takes about four to six weeks for the website to update that information. No personalized recommendation is provided. Communication: There are a lot of forms to fill out in order to communicate via e-mail with the site. The design of the form is very detailed and people have to provide a substantial amount of 46 information, for example their specific needs and the particular occasions for which they are shopping. All the company information is gathered on the "Company Contact" page, which lists the contact number, company address, fax, office hours, FAQs, etc. Users who register as members of this site get promotional letters via e-mail from time to time. Connection: SpaWish.com has a lot of interest in establishing business with others. All the partnership programs and affiliate sales programs are offered in the "Business Service" page. People interested in joining the business can contact the company by filling out a form and submitting the information through the web. SpaWish.com has links to the websites of most participating spas. Under the "Spa Owners - Get listed" page, a form invites Spas that meet a certain standard to join the network. The company has also partnered with www.growerflowers.com. The link is a picture of a flower, displayed conspicuously on every page and leading to another separate website, which sells flowers. 47 Technically speaking, the connection link is fast and accurate; no system crash has been found. Commerce: Only members can purchase via SpaWish.com. Registration is required for each member. The registration form is very detailed, including company information, date of birth, gender, occupation, address and martial status etc. and each field of information is compulsory. Besides the registration form for members, SpaWish.com offers quite a few other forms such as a "Bridal Gift Questions" form, a "Business Gift Need" form, a "Gift Required Fields" form, a "Loyalty Program Registration" form, a "checkout" form etc. When people want to order, they have to go through the shopping cart page to view their orders, and they must click the link of "take me to the order form" to fill out another form. The order form is long and complicated. Although the shopping cart is on the top of every page, making it every easy to locate and check, people always have to complete the form before they can add or delete anything from the shopping cart. There are six choices of shipping methods, and each has a different price. There is a confirmation page with a description of the order before checking out. 48 Payment can be made either with a credit card or a "GiveAnything.com gift certificate." In addition to the credit card information on the payment page, the company requests very detailed information on the card holder, including their occupation and information in a short survey about "how did you learn about SpaWish." An order confirmation is sent by e-mail to customers after they complete an order. The return policy is only offered in the FAQ page, stating that customers can only receive credit instead of cash refunds. SpaWish.com uses Thawte SSL Web Server Certificates to offer secure communications by encrypting all data to and from the site. 6.0 P H A S E II: I N S T R U M E N T TESTING 6.1 Checklist testing design The object of Phase II was to test the usability of the checklist created from Phase I. According to the sorting results, a shorter list of 56 items was created (see Appendix 2), within which every item had a high level (eight out of twelve opinions) of consistency in the category it was attributed to by test respondents. A close examination of each item was also taken to make sure that no ambiguous or redundant items were included. 49 A testing questionnaire of the checklist was designed subsequently (see Appendix 3). Each item on the list invited assessments on a seven-point scale ranging from "extremely disagree" to "strongly agree," with point one "strongly disagree" and point seven "extremely agree," while making a field evaluation on live websites. The point of the scale was also the score of each item. Since no evidence was attributed for assigning different weights of importance to each item, the mean score of each question was used as the category score, and the sum of seven categories' score was used as the final score of the evaluation. 6.2 Evaluators Twelve of the study's participating MSc and Ph.D. students who had more than five years on-line activities experience were asked to be the evaluators of the usability testing. Among the evaluators, four were Ph.D. students and eight were MSc students. They were asked to indicate the degree of their agreement with each item through the checklist while they browsed and evaluated the sample websites. The information of the evaluators is listed in Table 11. 50 Table 11. Profile of web evaluators Subject Check product info, on the web (Y/N) Years of experience with on-line activities Number of daily e-mails Work on line every week (Y/N) Read news on-line at home (Y/N) 1 Y 6 10-12 Y Y 2 Y 5 Over 10 Y Y 3 Y 7 15 Y Y 4 Y 5 8 Y Y 5 Y 5 10 Y Y 6 Y 5 Over 10 Y Y 7 Y 5 5 Y Y 8 Y 13 60 Y Y 9 Y 5 8-10 Y Y 10 Y 8 6-7 N Y 11 Y 5 20 Y Y 12 Y 6 5 Y Y 6.3 Testing objects The checklist testing questionnaire was distributed to the twelve evaluators to evaluate three live websites: www.redenvelope.com www .tiff any.com www.spawish.com The three websites were selected from http://www.forbes.com/bow/b2c/main.ihtml. Forbes reviews over 3,500 sites in www.forbes.com. Editors of Forbes rate sites according to five criteria: Content, Design, Speed, Navigation and Customization. The purposes of these websites range from stock picking, obtaining mortgages, and pursuing passions like scuba diving or golf, to online shopping for ordinary commodities. 51 The three websites selected were all from the sub-division of "Gift," under the directory of "Luxury Shopping." According to Forbes's criteria, www.redenvelope.com is ranked the highest as a "Forbes Favourite," www.tiffany.com is ranked as one of the best sites as a "Forbes Best Web Pick," and www.spawish.com is among the other sites, which have no special classification. 6.4 Test instruction In the testing, evaluators were asked to browse each of the three websites for half an hour before answering questions. The order of the websites visited was randomly assigned to each participant. After visiting each website for half an hour, participants were asked to mark their answer on the checklist item-by-item while browsing the web page continually. To ensure consistent standards, evaluators were asked to do the three websites' evaluations consecutively one after another. To avoid influential bias, the rank of the websites from Forbes.com was concealed and every evaluator performed the evaluation independently from others. 6.5 Testing Results The average score resulting from the checklist designed in Phase I of this study is one system of measurement applied to these three websites, and a non-parametric analysis 52 was also conducted to test for any difference in effectiveness among these web sites. The evaluation results from the checklist were compared with the rankings by the third party (Forbes.com) to see if the checklist could be a fair instrument to measure the usability of electronic commerce websites. 6.5.1 Comparison of the average scores among evaluators The average scores of each C and total of the three websites among the twelve evaluators are listed below: Table 12. Average scores among evaluators 7Cs RedEnvelope.com Tiffany.com SpaWish.com Context 5.73 5.39 4.07 Content 5.91 5.05 3.74 Communication 4.66 4.03 4.14 Connection 5.10 4.75 5.15 Commerce 5.08 4.38 4.43 Community 3.20 2.56 2.58 Customization 2.92 2.42 2.05 Total 32.60 28.57 26.16 From the table above, RedEnvelope.com has the highest total score (32.60), compared to Tiffany.com, whose score is in the middle (28.57), and SpaWish.com, which has the lowest total score (26.16). Regarding particular Cs, the average scores of "Context,", "Content," and "Customization" of the three websites are still in the same order while "Communication," "Commerce," "Connection" and "Community" are in different orders. 53 Figure 4. Average score among evaluators of each C of the websites average score of each C of the three sample websites .com 6.5.2 Non-parametric test result Since only twelve subjects participated in the test, a non-parametric test was applied to identify any significant differences among subjects' responses according to the 7Cs criteria towards the three websites. Because the same respondents rated each of the websites, the samples are not independent. The "Friedman Test" was therefore conducted, with calculations based on ranks within each case. The score for each variable is ranked, and the mean ranks for the variables are compared. 54 In the current study, the Friedman test was used to test the null hypothesis that there is no difference among three of the websites. The alternative is that some websites tend to be preferred over others. If the significance level is lower than 0.05, the null hypothesis can be rejected. The test was performed for the total score of each of the three websites, as well as for each of the 7Cs. To determine the rank of the total score of each website, each subject's responses for all three sites were ranked, and the mean ranks were compared. The same operation was performed for the scores in each of the 7Cs. Table 13 displays the ranking result of both the total scores and each C of the Friedman test for each of the three websites. 55 CO O CO -H J3 g to s a £ m (Q- 3" "D ={; CD ^ t/) > 3 m co .O S >< 3 :^ « - <- CD § g . o o 3 3 < CD O T3 CD o o 3 H S3 a-H BT CO S3 s 0 —' —1 ro ro b 0 CO b CD 0 CD co 0 1^ CD 0 0 0 H o 5T o 0 —>• — ro ro b Ol b bi 0 —L co 0 -* CD 1^ 0 0 0 0 —L —L ro ro b CD LL b 0 01 co Ol 0 O 0 0 01 p -t^  _L _L ro LL co CD 4^  ro ro co 1^ CD co 0 O 0 CD X CD 3 O 3 3 c o' tu o' 13 o S3 © a a n a-© «5 re - J O o o ffi S3 0 ro ro —L ro co _L L CD ro co 00 —L 01 0 0 0 O o CD o 0 1^ —L L ro b 4^  CD CD ro co CO Ul co 01 O 0 0 CD —\ o ro o 3 as 1 n re a-0 CO -L -L ro LL co k l co co CD —L co ro CD ro O 0 0 3 c 3 0 co -L ro ro 'ro ro CD b CO 0 0 1^ 0 co ro 0 O 0 0 o I QJ. 6' 3 as For the total score, RedEnvelope.com has the highest mean rank of 2.67, indicating that it is the best website according to the 7Cs criteria. SpaWish.com is the worst, with a mean rank of 1.33, and Tiffany.com is in the middle with a rank of 2.00. The value of the chi-square statistic is 10.667, with a significance of 0.005. Therefore, the null hypothesis that there is no difference in the total score among the three websites can be rejected. Besides the analysis of the total score of each website, the Friedman Test was performed on the websites for each of the seven Cs. For Customization, Community, Connection and Communication, the differences among RedEnvelope.com, Tiffany.com and SpaWish.com are not significant. However, concerning the Context, Content and Commerce, the difference is significant (significance value lower than 0.05). The "Context" score for RedEnvelope.com (2.50) was the highest among the three, and Tiffany.com (2.42) was greater than that of SpaWish.com (1.08) [significant at 0.001]. From the test results for "Content," it is evident that there are strong differences in levels of preference among RedEnvelope.com, Tiffany.com and SpaWish.com, with a significance of 0.001. The score of RedEnvelope.com is the highest (2.75), Spawish.com is the lowest (1.17), and Tiffany.com is in the middle (1.17). The score order of "Commerce" is the same as the test result of "Total," "Context" and "Content," with RedEnvelope.com scoring 2.63, Tiffany.com at 1.75 and SpaWish.com earning 1.63 [significant at 0.024]. Friedman tests between each two websites were also performed for Customization, Community, Connection and Communication to see if there is any difference between 57 any two of the websites in these categories. Result shows no significant difference between any pairs. Reliability analysis was also conducted for each 7C category (see Appendix 4). Cronbach Alphas for the dimensions of Context, Content, Community, Customization and Commerce are 0.9079, 0.8968, 0.9516, 0.9648 and 0.7217 respectively, with all above well-acceptable level, i.e., 0.7 (Barclay, Thompson and Higgins, 1995). Cronbach Alphas for Communication and Connection are lower than 0.7, at 0.6173 and 0.6742, but still acceptable. An inter-item correlation test shows that some items are not significantly correlated. This can be explained by low variances among the responses to these items. For example, the correlations between three items (i.e., a. the site provides contact information on each page; b. the site provides customer service through e-mail contact; c. the site offers FAQ) and all other items within "Communication" category are very low. The reason could be that the three items are all "Yes or No" questions. Since all the three websites have those features, subjects tended to choose the same answers. While for other items, there exists a much larger variation. Therefore, the correlations between these items and other items are reportedly low. 6.6 Summary of the findings 58 The total scores show that RedEnvelope.com is the best website among the three, and Tiffany.com tends to be preferred over SpaWish.com. This result concurs with the ranking by Forbes.com. This indicates that the checklist developed in this study can be a potential measurement for website evaluation in B-to-C electronic business. Also, for all the significant differences found in the Friedman test for each of the seven "Cs,", which include Context, Content and Commerce, the order is the same as that of the total score rank among the three sample websites. This demonstrates that RedEnvelope.com is the best, SpaWish.com is the worst and Tiffany.com is between the others, according to these three aspects. It is also worth to mention that RedEnvelope.com had the highest score in all the categories. It is noticed that Forbes doesn't have the criteria of Commerce and Community to rate websites. This is the possible explanation about the insignificancy in the Community difference among the three sample sites. Differences are not significant for the categories of Connection, Communication and Customization among the sites as well. However, the theoretical separation of the seven dimensions doesn't mean that three websites should perform significant differently in every dimension. In the average test, the score rank of "Customization" among the three samples is in the same order as above, but this difference was assessed as being less significant [at 0.202], by the Friedman test. Also in the average test, SpaWish.com has a slightly higher average score (4.43) in "Commerce" than that of Tiffany.com (4.38); however the mean 59 rank order of the two is different in the Friedman test. This can be explained by an extraordinary low score given by one of the evaluators to Tiffany.com. 7.0 DISCUSSION 7.1 Contribution The major contribution of the study reported here is that it offers a 56-item-evaluation checklist which proved to be an effective measurement of B-to-C websites. This instrument development study consisted of two processes. The first created the instrument and the second tested it and got the desired results. The creation process included creating items according to the "7Cs" concept (Rayport and Jaworski, 2000), surveying known existing checklists, guidelines, research results, and compiling and extracting items from previous research into this "7Cs" framework. Compared to previous measurement instruments, this checklist was not only based on a strict framework, but it also intended to cover the overall range of customer interface perspectives in the context of B-to-C electronic commerce. Also, this checklist is classified into seven categories. When people evaluate websites with this instrument, it is very easy to identify the performance of the website in each perspective. If it is coordinated with the business strategy, this checklist can help companies analyze the advantages and weak points of their customer interfaces, and where and how to improve them. 60 7.2 Limitations This paper began with two surveys, each of which contains limitations. The main contribution of the first survey is to identify a group of items (see Appendix 2) belonging to each of the seven 7 C s categories. However, this is a conceptual result rather than a quantitative one. Twelve subjects have completed this survey, and the cut-off level has been set at eight out of the twelve. In other words, for each item, if eight out of the twelve respondents select the same category, the item is assigned to the category. Items below this cut off level have been eliminated. The calculation method has a primitive limitation: the sum of seven categories' score was used as the final score of the evaluation. However, there is no theoretical justification for summing the score of each category to determine the final score. Different weight might need to be assigned to each category in order to give a more accurate measurement. The aim of the second survey is to test the checklist to see if it is an effective measurement instrument. A field test was conducted among twelve evaluators. Evaluators specified their opinion about each item on the list in relation to three websites. The three commercial websites used in this study were chosen from a ranking list of www.forbes.com because Forbes was a generally recognized research company. The validity of the 7 C model was therefore tested by comparing our results with their ranking. However, it could be argued whether their ranking could reflect the true quality of websites. In this sense, our results is under challenge. The other limitation of the field 61 test is that because all the three sites are real commercial websites, evaluators were not able to make complete and accurate answers to some of the questions (e.g. "customer can track their order and delivery status") without making real purchases. Another limitation about this survey is that only twelve people participated in the test. Provided future resources are available, a study with a larger sampling can generate a more accurate assessment of the evaluation method. It has to be pointed out that although the checklist developed in this research is designed to evaluate all kinds of business-to-consumer websites, the analysis must be conducted under various business conditions. Theoretically, a website which includes more items from the checklist would have been the better website, but given the understandable reality that businesses have different sizes, operate in different marketing segments, and have different product lines and different marketing strategies, different aspects will be emphasized. 7.2 Suggestions for future studies As indicated above, the conclusions of the study are based on the responses of 12 subjects. Therefore, a survey with a larger sample size is expected to better prove the validity of the instrument. Also, comparing survey results with the opinion of more authorities with wider range of ranking criteria and focus is another way to improve the experimental design. 62 Furthermore, from the sorting process of our study, there were items that did not fall into any of the seven categories. Do they have any impact on the customer interface and in what way, under what circumstances? Would these items from some other constructions? Would those constructions function in parallel or in different hierarchy with the 7Cs construction? These are the questions expected to be answered in future studies. 63 References Barclay, D., Thompson, R. and Higgins, C. (1995) "The Partial Least Squares (Pis) Approach to Causal Modeling: Personal Compuer Adoption and Use as an Illustration," Technology Studies (2:2), pp. 285-309. Farhoomand, A.F. and Drury, D.H. (1996) "Factors influencing electronic data interchange success", Data Base 27 (1), pp.45-57. Gehrke, D. and Turban, E. (1999) "Determinants of successful Website Design: Relative Importance and recommendations for Effectiveness" Proceedings of the 32nd Hawaii International Conference on System Sciences Helander, M.G. and Khalid, H.M. (2000) "Modeling the customer in electronic commerce" Applied Ergonomics 31, pp.609-619. Herzberg, F. Work and the Nature of Man. Chapter 6, World publishing, NY, 1966, pp.71-91. Instone, K. (1997) Usability Heuristics for the Web, http://webreview.com/97/10/10/usability/sidebar.html Ives, B. and Learmonth, G.P. (1984) "The Information System as A competitive Weapon" Communication of the ACM , pp.1193-1201. Jarvenpaa, S.L. and Todd, P.A., (1997). "Consumer reactions to electronic shopping on the world wide web". Int. J. Electron. Commerce 1 (2), pp.59-88. Keeker, K. (1997) Improving Web Site Usability and Appeal: Guidelines Compiled by MSN Usability Research. http://www.microsoft.com/workshop/management/planning/improvingsitusa.asp Liang, T. and Lai, H. (2000) "Electronic Store Design and Consumer Choice: an Empirical Study" Proceedings of the 33rd Hawaii International Conference on System Sciences Liu, C. and Arnett, K.P. (2000) "Exploring the factors associated with web site success in the context of electronic commerce" Information & Management 38, pp.23-33. Moore, G.C. and Benbasat, I. (1991) "Development of an Instrument to Measure the Perceptions of Adopting an Information Technology Innovation" Information Systems Research 2:3, pp.192-222. Nielsen, J. (1994). "Enhancing the explanatory power of usability heuristics". Proc. ACM CHI'94 Conf. (Boston, MA, April 24-28), pp. 152-158. 64 Nielsen, J. and Molich, R. (1990). "Heuristic evaluation of user interfaces", Proc. ACM CHV90 Conf. (Seattle, WA, 1-5 April), pp.249-256. Nielsen, J. Ten Usability Heuristics. Available online: http://www.useit.com/papers/heuristic/heuristic list.html " Nielsen, J. Alertbox, http://www.useit.com/alertbox Rayport, J.F., Jaworski, B.J., Griffin, L. and Dosios, Y. e-Commerce, Chapter 4, McGraw-Hill/Irwin; 1st edition (November 3, 2000), pp.113-165. Shedroff, N. (1994) http://www.nathan.com/thoughts/recipe Sullivan, T. User Testing Techniques-A Reader Friendliness Checklist, http://www.pantos.org/atw/35317.html Zhang, P., von Dran, G. M., Small, R.V. and Barcellos, S. (1999) "Websites that Satisfy Users: A Theoretical Framework for Web User Interface Design and Evaluation" Proceedings of the 32nd Hawaii International Conference on System Sciences Zhang, P., von Dran, G. M., Small, R.V. and Barcellos, S. (2000) "A Tow Factor Theory for Website Design" Proceedings of the 33rd Hawaii International Conference on System Sciences Zhang, P. and von Dran, G. (2001) "Expectations and Rankings of Website Quality Features: Results of Tow Studies on User Perceptions" Proceedings of the 34th Hawaii international Conference on System Sciences 65 Appendix 1. Card Sorting Questionnaire Instruction Thank you for your participation in this survey. We have all together 113 questions in this questionnaire. All the items are descriptions of the features you can find on most e-commerce websites. Our subjects are going to be asked to make multiple choices on each of our questions. There are 8 choices: 1. Context 2. Content 3. Community 4. Customization 5. Communication 6. Connection 7. Commerce 8. None of the above Please to mark the answer for it the category that the item can be best fitted into. Before we start the survey, please read carefully about the definition of the 7Cs. The concept of each category is defined by B.Jaworski, J.Raport, L.Griffin and Y.Dosios. Context. The context of the website captures its aesthetic and functional look-and-feel. Some sites have chosen to focus heavily on interesting graphics, colours and design features, while others have emphasized more simple utilitarian goals, such as ease of navigation Content. Content is defined as all digital subject matter on the site. This includes the form of the digital subject matter—text, video, audio, and graphics—as well as the domains of the digital subject matter, including product, service, and information offerings. While context largely focuses on the "how" of site design, content focus on "what" is presented. Community. Community is defined as the interaction that occurs between site users. It does not refer to site-to-user interactions. User-to-user communication can occur between two users or between on user and many. Customization. Customization is defined as the site's ability to tailor itself or to be tailored by each user. When the customization is initiated and managed by the firm, we term it tailoring. When the customization is initiated and managed by the user, we term it personalization. 66 Communication. Comrnunication refers to the dialogue that unfolds between the site and its users. This communication can take three forms: site-to-user communication, user-to-site, or two-way communication. Connection. Connection is defined as the extent of the formal linkages between the site and other sites. Commerce. Commerce is defined as the sale of goods, products, or services on the site. Before we start the survey, please answer briefly the following questions about yourself: Name: 6. Do you get used to looking at product information on the web? 7. How many years of experience with on-line activities do you have? 8. Approximately, how many e-mails you have everyday? 9. Do you work on-line at work every week? 10. Do you read news on-line at home? 67 1. The search criteria designed reasonable and easy to find the product interest you Context • Content • Community • Customization • Communication • Connection • Commerce • None of the category • 2. The site design fits the stated scope, purpose, core activities and audience. Context • Content • Community • Customization • Communication • Connection • Commerce • None of the category • 3. The site provide free e-mail account Context • Content • Community • Customization • Communication • Connection • Commerce • None of the category • 4. The site clearly tell people when and how often content is updated Context • Content • Community • Customization • Communication • Connection • Commerce • None of the category • 5. The site makes all objects accessible at all times Context • Content • Community • Customization • Communication • Connection • Commerce • None of the category • 6. If the resource is to be removed, the site will state where it will be available Context • Content • Community • Customization • Communication • Connection • Commerce • None of the category • 7. People will be able to find out about others easily Context • Content • Community • Customization • Communication • Connection • Commerce • None of the category • 8. The site e-mail subscribers about change, new features and letters from other subscribers Context • Content • Community • Customization • Communication • Connection • Commerce • None of the category • 68 9. Is there a system of "help" for those requiring it? Context • Content • Community • Customization • Communication • Connection • Commerce • None of the category • 10. The document will stimulate the user's creativity or thinking Context • Content • Community • Customization • Communication • Connection • Commerce • None of the category • 11. Users should be in control with proactive assistance Context n Content • Community • Customization • Communication • Connection • Commerce • None of the category • 12. The links are well labelled and accurate Context • Content • Community • Customization • Communication n Connection • Commerce • None of the category • 13. When appropriate, each audience member can control the pace of sequences (for example, skip or replay sequences)? Context • Content • Community • Customization • Communication • Connection • Commerce • None of the category • 14. The site let user input their information in the integral part of the content Context • Content • Community • Customization • Communication • Connection • Commerce • None of the category • 15. The links to other site are useful and helpful Context • Content • Community • Customization • Communication • Connection • Commerce • None of the category • 16. The colour of the pages is consistent, simple and appropriate. 69 Context • Content • Community • Customization • Communication • Connection • Commerce • None of the category • 17. The site offer FAQ Context • Content • Community • Customization • Communication • Connection • Commerce • None of the category • 18. The site provides shopping cart Context • Content • Community • Customization • Communication • Connection • Commerce • None of the category • 19. The site offer discounts, freebies, coupons and other incentives Context • Content • Community • Customization • Communication • Connection • Commerce • None of the category • 20. The layout is consistent from page to page. Context • Content • Community • Customization • Communication • Connection • Commerce • None of the category • 21. There is a link back to the main page on each supporting page Context • Content • Community • Customization • Communication • Connection • Commerce • None of the category n 22. There is no "Under Construction" or any other similar signs during browsing Context • Content • Community • Customization • Communication • Connection • Commerce • None of the category • 23. The portion between multimedia or visual elements and textual information of this website is reasonable and suitable for the goal of the business Context • Content • Community • Customization • Communication • Connection • Commerce • None of the category • 24. The graphic, icons and/or multimedia part of the website is necessary and will strongly support the business activity 70 Context • Content • Community • Customization • Communication • Connection • Commerce • None of the category • 25. The links are logically grouped and easy to move back and forth. Context • Content • Community • Customization • Communication • Connection • Commerce • None of the category • 26. Is it accessible to readers with physical impairments? Context • Content • Community • Customization • Communication • Connection • Commerce • None of the category • 27. Are there indications of careless or hasty preparation, such as spelling or grammatical errors? Context • Content • Community • Customization • Communication • Connection • Commerce • None of the category • 28. The website overall is very appealing by its look and feel. Context • Content • Community • Customization • Communication • Connection • Commerce • None of the category • 29. The button linking to other page is very distinctive Context • Content • Community • Customization • Communication • Connection • Commerce • None of the category • 30. Customers can check their order history. Context • Content • Community • Customization • Communication • Connection • Commerce • None of the category • 31. How helpful is the "help" system? Context • Content • Community • Customization • Communication • Connection • Commerce • None of the category • 71 32. The interface support the social goals people will have while experiencing the site. Context • Content • Community • Customization • Communication • Connection • Commerce • None of the category • 33. Is the design so complex that it detracts from the content? Context • Content • Community • Customization • Communication • Connection • Commerce • None of the category • 34. People will find new challenges as they gain experience. Context • Content • Community • Customization • Communication • Connection • Commerce O None of the category • 35. There are charming and creative features to attract customers and to help them enjoy the visit. Context • Content • Community • Customization • Communication • Connection • Commerce • None of the category • 36. The level of challenge found in the site's content is appropriate for the core audience Context • Content • Community • Customization • Communication • Connection • Commerce • None of the category • 37. Keep the customer aware of promotions Context • Content • Community • Customization • Communication • Connection • Commerce • None of the category • 38. The homepage includes a "search input" box. Context • Content • Community • Customization • Communication • Connection • Commerce • None of the category • 39. The site provides a clear description of the product Context • Content • Community • Customization • Communication • 72 Connection • Commerce • None of the category • 40. Customers can check the delivery status Context • Content • Community • Customization • Communication • Connection • Commerce • None of the category • 41. The site provide various kind of delivery options Context • Content • Community • Customization • Communication • Connection • Commerce • None of the category • 42. The site provides clear and concise text information to describe the business, news and customer service Context • Content • Community • Customization • Communication • Connection • Commerce • None of the category • 43. The homepage groups all Corporate Information in One Distinct Area Context • Content • Community • Customization • Communication • Connection • Commerce • None of the category • 44. The homepage offer users a clear starting point for the main tasks they'll undertake when visiting the site Context • Content • Community • Customization • Communication • Connection • Commerce • None of the category • 45. The site use (approximately) standard link colors. Context • Content • Community • Customization • Communication • Connection • Commerce • None of the category • 46. The site suggests many payment alternatives Context • Content • Community • Customization • Communication • Connection • Commerce • None of the category • 47. The page length is appropriate to site content. 73 Context • Content • Community • Customization • Communication • Connection • Commerce • None of the category • 48. Customer has the sense of control in the process of transaction Context • Content • Community • Customization • Communication • Connection • Commerce • None of the category • 49. The site has mass mailing Context • Content • Community • Customization • Communication • Connection • Commerce • None of the category • 50. People can go into greater depth about topics if they desire. Context • Content • Community • Customization • Communication • Connection • Commerce • None of the category • 51. Information presented on each page is sufficient but not too much Context • Content • Community • Customization • Communication • Connection • Commerce • None of the category • 52. Page headings help people understand where they are located Context • Content • Community • Customization • Communication • Connection • Commerce • None of the category • 53. The home page shows some of the best or most recent content. Context • Content • Community • Customization • Communication • Connection • Commerce • None of the category • 54. Generate a confirmation page after consumers make purchases. Context • Content • Community • Customization • Communication • Connection • Commerce • None of the category • 55. The site provides login/registration function for the customer 74 Context • Content • Community • Customization • Communication • Connection • Commerce • None of the category • 56. There is a text alternative to the images Context • Content • Community • Customization • Communication • Connection • Commerce • None of the category • 57. The products and services offered by the site is updated Context • Content • Community • Customization • Communication • Connection • Commerce • None of the category • 58. The site provides some functions that can help users make the decision (price comparison, discounts, coupons and specials etc.) Context • Content • Community • Customization • Communication • Connection • Commerce • None of the category • 59. The site focus on products that have low delivery cost. Context • Content • Community • Customization • Communication • Connection • Commerce • None of the category • 60. Actions are reversible Context • Content • Community • Customization • Communication • Connection • Commerce • None of the category • 61. There are no obvious errors or misleading omissions in the document. Context • Content • Community • Customization • Communication • Connection • Commerce • None of the category • 62. The site give users a sense of group while they are participating in the shop Context • Content • Community • Customization • Communication • Connection • Commerce • None of the category • 75 63. Opinion of the group will influence user's life or decisions Context • Content • Community • Customization • Communication • Connection • Commerce • None of the category • 64. Provide no-cost/no-hassle return policy. Context • Content • Community • Customization • Communication • Connection • Commerce • None of the category • 65. The site provides contact information on each page Context • Content • Community • Customization • Communication • Connection • Commerce • None of the category • 66. Limit registration forms and the amount of requested information on them Context • Content • Community • Customization • Communication • Connection • Commerce • None of the category • 67. The structure enables the addition of new information Context • Content • Community • Customization • Communication • Connection • Commerce • None of the category • 68. The site is designed to meet individual audience needs (multiple developmental levels) Context • Content • Community • • Customization • Communication • Connection • Commerce • None of the category • 69. Users will go for help to the web group community if they have any problems Context • Content • Community • Customization • Communication • Connection • Commerce • None of the category • 70. The purpose of the website is clear. Context • Content • Community • Customization • Communication • Connection • Commerce • None of the category • 71. The site will give people something to talk about (for example, MSNBC has a daily chat about a specific news topic). Context • Content • Community • Customization • Communication • Connection • Commerce • None of the category • 72. Web users can personalize the layout and content of their page and make their personal settings Context • Content • Community • Customization • Communication • Connection • Commerce • None of the category • 73. Are there dead links? Broken CGI scripts? Functionless forms? Context • Content • Community • Customization • Communication • Connection • Commerce • None of the category • 74. The site provides a particular site for members or member-generated site Context • Content • Community • Customization • Communication • Connection • Commerce • None of the category • 75. The web can track and gather data in order to give appropriate recommendations Context • Content • Community • Customization • Communication • Connection • Commerce • None of the category • 76. The site dynamically adjusts content with various design layout according to my response Context • Content • Community • Customization • Communication • Connection • Commerce • None of the category • 77. Web users can benefit from being a member of the web group Context • Content • Community • Customization • Communication • Connection • Commerce • None of the category • 77 78. The value proposition (what's the relevance for me?) is clearly conveyed Context • Content • Community • Customization • Communication • Connection • Commerce • None of the category • 79. People will be able to represent themselves to others in their preferred way Context • Content • Community • Customization • Communication • Connection • Commerce • None of the category • 80. The site provide customer service through e-mail contact or live chat Context • Content • Community • Customization • Communication • Connection • Commerce • None of the category • 81. It is possible for users to communicate with people through the web community to develop friendship or individual interaction Context • Content • Community • Customization • Communication • Connection • Commerce • None of the category • 82. The site broadcast the news and event Context • Content • Community • Customization • Communication • Connection • Commerce • None of the category • 83. Users are willing to participate in the web group Context • Content • Community • Customization • Communication • Connection • Commerce • None of the category • 84. The site provides message broads, chat room or instant messaging for the members Context • Content • Community • Customization • Communication • Connection • Commerce • None of the category • 78 85. Customer can track their order status Context • Content • Community • Customization • Communication • Connection • Commerce • None of the category • 86. The site promotes a sense of group identification Context • Content • Community • Customization • Communication • Connection • Commerce • None of the category • 87. The load time is appropriate to content, even on a slow dial-in connection Context • Content • Community • Customization • Communication • Connection • Commerce • None of the category • 88. The domain name is easy to remember. Context • Content • Community • Customization • Communication • Connection • Commerce • None of the category • 89. There is no system crashes or "incorrect" message displayed on the users' screen Context • Content • Community • Customization • Communication • Connection • Commerce • None of the category • 90. The site provides feedback whenever possible. Context • Content • Community • Customization • Communication • Connection • Commerce • None of the category • 91. People will have an opportunity to learn? Context • Content • Community • Customization • Communication • Connection • Commerce • None of the category • 92. The site responds to each person as an individual Context • Content • Community • Customization • Communication • Connection • Commerce • None of the category • 79 93. Is the site English only or can speakers of other languages access the site in their languages? Context • Content • Community • Customization • Communication • Connection • Commerce • None of the category • 94. It is easy to go back to the main page from the linked page Context • Content • Community • Customization • Communication • Connection • Commerce • None of the category • 95. There is a convenient, obvious way to manoeuvre among related pages, and between different sections. Context • Content • Community • Customization • Communication • Connection • Commerce • None of the category • 96. The terms (especially the site's title and sections) adequately communicate the consequences of selection or action. Context • Content • Community • Customization • Communication • Connection • Commerce • None of the category • 97. It is easy to locate a particular page from any other page Context • Content • Community • Customization • Communication • Connection • Commerce • None of the category • 98. Essential instructions will appear before links and other interactive portions Context • Content • Community • Customization • Communication • Connection • Commerce • None of the category • 99. Users are informed of the type of file they are linking to (e.g., video, sound, text, etc.) Context • Content • Community • Customization • Communication • Connection • Commerce • None of the category • 80 100. The links are very reliable (are there inactive links or references to sites that have moved)? Context • Content • Community • Customization • Communication • Connection • Commerce • None of the category • 101. It is it easy to put, delete and check your items in the shopping cart Context • Content • Community • Customization • Communication • Connection • Commerce • None of the category • 102. The site provide a sense of security, you'll trust they will not give your information to any unauthorized parties Context • Content • Community • Customization • Communication • Connection • Commerce • None of the category • 103. The site offer instant credit approval so that you can shop right away Context • Content • Community • Customization • Communication • Connection • Commerce • None of the category • 104. The site offers customized layout and content according to some other similar people's profile Context • Content • Community • Customization • Communication • Connection • Commerce • None of the category • 105. Links provided primarily to resources rather than just lists of resources Context • Content • Community • Customization • Communication • Connection • Commerce • None of the category • 106. Easy method for exchange/return. Context • Content • Community • Customization • Communication • Connection • Commerce • None of the category • 81 107. There is an image map, index or table that can be used to navigate within the document. Context • Content • Community • Customization • Communication • Connection • Commerce • None of the category • 108. Trial purchase without being a member Context • Content • Community • Customization • Communication • Connection • Commerce • None of the category • 109. Customer will gain confidence during transaction Context • Content • Community • Customization • Communication • Connection • Commerce • None of the category • 110. The site store customer information such as credit card information, shipping address etc. Context • Content • Community • Customization • Communication • , Connection • Commerce • None of the category • 111. Customer can also order from the affiliate program of other websites. Context • Content • Community • Customization • Communication • Connection • Commerce n None of the category • 112. The site can automatically response to user's request. Context • Content • Community • Customization • Communication • Connection • Commerce • None of the category • 113. People will be able to express themselves freely (that is, safely and appropriately) Context • Content • Community • Customization • Communication • Connection • Commerce • None of the category • 82 Appendix 2. Sorting result-Checklist Context CTXT1. The colour of the pages is consistent, simple and appropriate. CTXT2. The layout is consistent from page to page. CTXT3. The portion between multimedia or visual elements and textual information of this website is reasonable and suitable for the goal of the business CTXT4. The website overall is very appealing by its look and feel. CTXT5. The design is not so complex that it detracts from the content CTXT6. There are charming and creative features to attract customers and to help them enjoy the visit CTXT7. The site use (approximately) standard link colors. CTXT8. The page length is appropriate to site content. CTXT9. The links are logically grouped and easy to move back and forth Content CTNT1. The search criteria designed reasonable and easy to find the product interest you CTNT2. The homepage includes a "search input" box. CTNT3. The site provides a clear description of the product CTNT4. The level of challenge found in the site's content is appropriate for the core audience CTNT5. The home page shows some of the best or most recent content CTNT6. There are no obvious errors or misleading omissions in the document CTNT7. The purpose of the website is clear. 83 CTNT8. The products and services offered by the site is updated Communication CMCA1. The "help" system is very helpful CMCA2. The site provides feedback whenever possible CMC A 3. The site has mass mailing CMCA 4. The site provides contact information on each page CMC A 5. The site broadcast the news and event CMCA 6. The site provide customer service through e-mail contact or live chat CMCA 7. The site can automatically response to user's request. CMCA 8. The site offer FAQ CMCA 9. The load time is appropriate to content, even on a slow dial-in connection Connection CONN1. CONN 2. CONN 3. CONN 4. The links to other site are useful and helpful There is a link back to the main page on each supporting page It is easy to locate a particular page from any other page The links are very reliable (there are no inactive links or references to sites that have moved) Commerce CMCE 1. CMCE 2. CMCE 3. CMCE 4. The site provides shopping cart The site offer discounts, freebies, coupons and other incentives The site suggests many payment alternatives The site will generate a confirmation page after consumers make purchases. 84 CMCE 5. CMCE 6. CMCE 7. CMCE 8. CMCE 9. CMCE 10. CMCE 11. CMCE 12. Community CMMU1. CMMU2. CMMU3. CMMU4. CMMU5. CMMU6. CMMU7. CMMU8. CMMU9. Provide no-cost/no-hassle return policy. The site focus on products that have low delivery cost. It is it easy to put, delete and check your items in the shopping cart The site provide a sense of security, you'll trust they will not give your information to any unauthorized parties The site offer instant credit approval so that you can shop right away The site offer trial purchase without being a member The site store customer information such as credit card information, shipping address etc. Customer can track their order and delivery status The site give users a sense of group while they are participating in the shop Opinion of the group will influence users' life or decisions Users will go for help to the web group community if they have any problems Web users can benefit from being a member of the web group It is possible for users to communicate with people through the web community to develop friendship or individual interaction The site will give people something to talk about (for example, MSNBC has a daily chat about a specific news topic). Users are willing to participate in the web group The site provides message broads, chat room or instant messaging for the members The site promotes a sense of group identification 85 Customization CUTM1. CUTM 2. CUTM3. CUTM 4. The site dynamically adjusts content with various design layout according to my response Web users can personalize the layout and content of their page and make their personal settings People will be able to represent themselves to others in their preferred way The site offers customized layout and content according to some other similar people's profile CUTM 5. The site responds to each person as an individual 8 6 Appendix 3. Checklist testing questionnaire Context 1. The colour of the pages is consistent, simple and appropriate. Do you agree with this statement? Strongly Disagree 1 2 3 4 5 6 7 Strongly Agree 2. The layout is consistent from page to page. Do you agree with this statement? Strongly Disagree 1 2 3 4 5 6 7 Strongly Agree 3. The portion between multimedia or visual elements and textual information of this website is reasonable and suitable for the goal of the business Do you agree with this statement? Strongly Disagree 1 2 3 4 5 6 7 Strongly Agree 4. The website overall is very appealing by its look and feel. Do you agree with this statement? Strongly Disagree 1 2 3 4 5 6 7 Strongly Agree 5. The design is not so complex that it detracts from the content Do you agree with this statement? Strongly Disagree 1 2 3 4 5 6 7 Strongly Agree 6. There are charming and creative features to attract customers and to help them enjoy the visit. Do you agree with this statement? Strongly Disagree 1 2 3 4 5 6 7 Strongly Agree 7. The site use (approximately) standard link colors. Do you agree with this statement? Strongly Disagree 1 2 3 4 5 6 7 Strongly Agree 8. The page length is appropriate to site content. Do you agree with this statement? Strongly Disagree 1 2 3 4 5 6 7 Strongly Agree 87 9. The links are logically grouped and easy to move back and forth Do you agree with this statement? Strongly Disagree 1 2 3 4 5 6 7 Strongly Agree Content 10. The search criteria designed reasonable and easy to find the product interest you Do you agree with this statement? Strongly Disagree 1 2 3 4 5 6 7 Strongly Agree 11. The homepage includes a "search input" box. Do you agree with this statement? Strongly Disagree 1 2 3 4 5 6 7 Strongly Agree 12. The site provides a clear description of the product Do you agree with this statement? Strongly Disagree 1 2 3 4 5 6 7 Strongly Agree 13. The level of challenge found in the site's content is appropriate for the core audience Do you agree with this statement? Strongly Disagree 1 2 3 4 5 6 7 Strongly Agree 14. The home page shows some of the best or most recent content Do you agree with this statement? Strongly Disagree 1 2 3 4 5 6 7 Strongly Agree 15. There are no obvious errors or misleading omissions in the document Do you agree with this statement? Strongly Disagree 1 2 3 4 5 6 7 Strongly Agree 16. The purpose of the website is clear. Do you agree with this statement? Strongly Disagree 1 2 3 4 5 6 7 Strongly Agree 17. The products and services offered by the site is updated 88 Do you agree with this statement? Strongly Disagree 1 2 3 4 5 6 7 Strongly Agree Communication 18. The "help" system is very helpful Do you agree with this statement? Strongly Disagree 1 2 3 4 5 6 7 Strongly Agree 19. The site provides feedback whenever possible Do you agree with this statement? Strongly Disagree 1 2 3 4 5 6 7 Strongly Agree 20. The site has mass mailing Do you agree with this statement? Strongly Disagree 1 2 3 4 5 6 7 Strongly Agree 21. The site provides contact information on each page Do you agree with this statement? Strongly Disagree 1 2 3 4 5 6 7 Strongly Agree 22. The site broadcast the news and event Do you agree with this statement? Strongly Disagree 1 2 3 4 5 6 7 Strongly Agree 23. The site provide customer service through e-mail contact or live chat Do you agree with this statement? Strongly Disagree 1 2 3 4 5 6 7 Strongly Agree 24. The site can automatically response to user's request. Do you agree with this statement? Strongly Disagree 1 2 3 4 5 6 7 Strongly Agree 25. The site offer FAQ Do you agree with this statement? Strongly Disagree 1 2 3 4 5 6 7 Strongly Agree 26. The load time is appropriate to content, even on a slow dial-in connection Do you agree with this statement? Strongly Disagree 1 2 3 4 5 6 7 Strongly Agree Connection 27. The links to other site are useful and helpful Do you agree with this statement? Strongly Disagree 1 2 3 4 5 6 7 Strongly Agree 28. There is a link back to the main page on each supporting page Do you agree with this statement? Strongly Disagree 1 2 3 4 5 6 7 Strongly Agree 29. It is easy to locate a particular page from any other page Do you agree with this statement? Strongly Disagree 1 2 3 4 5 6 7 Strongly Agree 30. The links are very reliable (there are no inactive links or references to sites that have moved) Do you agree with this statement? Strongly Disagree 1 2 3 4 5 6 7 Strongly Agree Commerce 31. The site provides shopping cart Do you agree with this statement? Strongly Disagree 1 2 3 4 5 6 7 Strongly Agree 32. The site offer discounts, freebies, coupons and other incentives Do you agree with this statement? Strongly Disagree 1 2 3 4 5 6 7 Strongly Agree 33. The site suggests many payment alternatives Do you agree with this statement? 90 Strongly Disagree 1 2 3 4 5 6 7 Strongly Agree 34. The site will generate a confirmation page after consumers make purchases. Do you agree with this statement? Strongly Disagree 1 2 3 4 5 6 7 Strongly Agree 35. Provide no-cost/no-hassle return policy. Do you agree with this statement? Strongly Disagree 1 2 3 4 5 6 7 Strongly Agree 36. The site focus on products that have low delivery cost. Do you agree with this statement? Strongly Disagree 1 2 3 4 5 6 7 Strongly Agree 37. It is it easy to put, delete and check your items in the shopping cart Do you agree with this statement? Strongly Disagree 1 2 3 4 5 6 7 Strongly Agree 38. The site provide a sense of security, you'll trust they will not give your information to any unauthorized parties Do you agree with this statement? Strongly Disagree 1 2 3 4 5 6 7 Strongly Agree 39. The site offer instant credit approval so that you can shop right away Do you agree with this statement? Strongly Disagree 1 2 3 4 5 6 7 Strongly Agree 40. The site offer trial purchase without being a member Do you agree with this statement? Strongly Disagree 1 2 3 4 5 6 7 Strongly Agree 41. The site store customer information such as credit card information, shipping address etc. Do you agree with this statement? Strongly Disagree 1 2 3 4 5 6 7 Strongly Agree 42. Customer can track their order and delivery status 91 Do you agree with this statement? Strongly Disagree 1 2 3 4 5 6 7 Strongly Agree Community 43. The site give users a sense of group while they are participating in the shop Do you agree with this statement? Strongly Disagree 1 2 3 4 5 6 7 Strongly Agree 44. Opinion of the group will influence users' life or decisions Do you agree with this statement? Strongly Disagree 1 2 3 4 5 6 7 Strongly Agree 45. Users will go for help to the web group community if they have any problems Do you agree with this statement? Strongly Disagree 1 2 3 4 5 6 7 Strongly Agree 46. Web users can benefit from being a member of the web group Do you agree with this statement? Strongly Disagree 1 2 3 4 5 6 7 Strongly Agree 47. It is possible for users to communicate with people through the web community to develop friendship or individual interaction Do you agree with this statement? Strongly Disagree 1 2 3 4 5 6 7 Strongly Agree 48. The site will give people something to talk about (for example, MSNBC has a daily chat about a specific news topic). Do you agree with this statement? Strongly Disagree 1 2 3 4 5 6 7 Strongly Agree 49. Users are willing to participate in the web group Do you agree with this statement? Strongly Disagree 1 2 3 4 5 6 7 Strongly Agree 50. The site provides message broads, chat room or instant messaging for the members 92 Do you agree with this statement? Strongly Disagree 1 2 3 4 5 6 7 Strongly Agree 51. The site promotes a sense of group identification Do you agree with this statement? Strongly Disagree 1 2 3 4 5 6 7 Strongly Agree Customization 52. The site dynamically adjusts content with various design layout according to my response Do you agree with this statement? Strongly Disagree 1 2 3 4 5 6 7 Strongly Agree 53. Web users can personalize the layout and content of their page and make their personal settings Do you agree with this statement? Strongly Disagree 1 2 3 4 5 6 7 Strongly Agree 54. People will be able to represent themselves to others in their preferred way Do you agree with this statement? Strongly Disagree 1 2 3 4 5 6 7 Strongly Agree 55. The site offers customized layout and content according to some other similar people's profile Do you agree with this statement? Strongly Disagree 1 2 3 4 5 6 7 Strongly Agree 56. The site responds to each person as an individual Do you agree with this statement? Strongly Disagree 1 2 3 4 5 6 7 Strongly Agree 93 Appendix 4. Inter-item Correlations within each 7C category CONTEXT CTXT1 CTXT2 CTXT3 CTXT4 CTXT5 CTXT6 CTXT7 CTXT8 CTXT9 CTXT1 1.000 CTXT2 0.793 1.000 CTXT3 0.750 0.713 CTXT4 0.837 0.664 CTXT5 0.519 0.359 CTXT6 0.673 0.525 CTXT7 0.140 0.250 CTXT8 0.741 0.765 CTXT9 0.551 0.473 Alpha = 0.9079 CONTENT CTNT1 CTNT2 CTNT3 CTNT4 CTNT5 CTNT6 CTNT7 CTNT8 CTNT1 1.000 CTNT2 0.527 1.000 CTNT3 0.656 0.509 CTNT4 0.738 0.550 CTNT5 0.614 0.434 CTNT6 0.699 0.405 CTNT7 0.650 0.562 CTNT8 0.464 0.159 Alpha = 0.8968 COMMUNICATION CMCA1 CMCA2 CMCA3 CMCA4 CMCA5 CMCA6 CMCA7 CMCA8 CMCA9 CMCA1 1.000 CMCA2 0.645 1.000 CMCA3 0.286 0.514 CMCA4 0.078 0.034 CMCA5 -0.119 0.096 CMCA6 0.183 0.116 CMCA7 0.197 0.291 CMCA8 0.207 0.097 CMCA9 0.334 0.067 Alpha = 0.6173 CONNECTION CONN1 CONN2 CONN3 CONN4 CONN1 1.000 CONN2 0.409 1.000 CONN3 0.272 0.093 1.000 CONN4 0.576 0.239 0.602 1.000 Alpha = 0.6742 1.000 0.808 1.000 0.661 0.567 1.000 0.746 0.791 0.665 1.000 0.104 0.175 0.008 0.335 1.000 0.650 0.705 0.437 0.633 0.241 1.000 0.600 0.610 0.257 0.518 0.317 0.571 1.000 0.738 1.000 0.746 0.769 1.000 0.667 0.802 0.682 1.000 0.590 0.632 0.578 0.543 1.000 0.609 0.641 0.655 0.621 0.412 1.000 -0.056 1.000 0.339 0.348 1.000 0.132 -0.118 0.412 1.000 0.337 -0.118 0.330 0.387 1.000 0.171 0.153 0.059 -0.059 -0.095 1.000 0.239 0.321 0.161 -0.185 0.000 0.427 94 > II o o o o o o o o o o o o o o o o o o o o o o o o o m m m m m m m m m m m r n - ' - ' - ' t D a J N O l O l ^ O M - ' r o -L o O O O O O O O O O O O - 1 o ro co o o ro - k co CD 4^  ro co co co oo oo j » . o -1 -"• CO O Ji. co co -L ro co -L s l Ol CD- - 1 p p p p p o o o o p ^ - 1 b b b ^ - c o L L ^ b b ^ - ' - b w - ' o i j i s i - ' c c - ' c n r o o c o c o j i - o c o c o - k c n o o - s i o o o o o o o o o o ^ Li^^corocoLi . L i.co<D cosioioocDJi.aicooo C O O ) - ' O S O ) C 0 4 k O O O O O O O O O O ; - ' -u c o b L K i w b j k b c o o i o i - ' - o o o o j i - C D O O U O - ' N l t D - ' O J O O O O O O O O - 1 j i . L L r o L i . c o J i . C O C D C 0 C 0 J i . C 0 J i . J i . 0 1 O 0 0 3 J i . C O J i . 0 0 0 o o o o o o o - k a) ro ro CO s | -L M v l co ro j i . co CD o o ro o J i . o O O O O O - ' c o ro b j i . b ro c o o si o o o vl -L CD c o c o o o o o o - k J i LL LL o O OO -L vl CD O j i . co co ro o o o o ro ro j i . b oo ro CD o co oo ro o o o ->• LL b b vl -L o o o ro o o - 1 j ^ b o o cn o COMMUNITY CMMU1 CMMU2 CMMU3 CMMU4 CMMU5 CMMU6 CMMU7 CMMU8 CMMU9 CMMU1 1.000 CMMU2 0.900 1.000 CMMU3 0.850 0.909 1.000 CMMU4 0.798 0.839 0.906 1.000 CMMU5 0.661 0.758 0.824 0.811 1.000 CMMU6 0.520 0.488 0.585 0.553 0.653 1.000 CMMU7 0.668 0.706 0.682 0.696 0.694 0.463 1.000 CMMU8 0.769 0.718 0.709 0.640 0.704 0.558 0.660 1.000 CMMU9 0.692 0.577 0.599 0.563 0.575 0.464 0.642 0.822 1.000 Alpha = 0.9516 CUSTOMIZATION CUTM1 CUTM2 CUTM3 CUTM4 CUTM5 CUTM1 1.000 CUTM2 0.908 1.000 CUTM3 0.896 0.950 1.000 CUTM4 0.875 0.926 0.918 CUTM5 0.779 0.759 0.758 Alpha = 0.9648 96 

Cite

Citation Scheme:

        

Citations by CSL (citeproc-js)

Usage Statistics

Share

Embed

Customize your widget with the following options, then copy and paste the code below into the HTML of your page to embed this item in your website.
                        
                            <div id="ubcOpenCollectionsWidgetDisplay">
                            <script id="ubcOpenCollectionsWidget"
                            src="{[{embed.src}]}"
                            data-item="{[{embed.item}]}"
                            data-collection="{[{embed.collection}]}"
                            data-metadata="{[{embed.showMetadata}]}"
                            data-width="{[{embed.width}]}"
                            async >
                            </script>
                            </div>
                        
                    
IIIF logo Our image viewer uses the IIIF 2.0 standard. To load this item in other compatible viewers, use this url:
http://iiif.library.ubc.ca/presentation/dsp.831.1-0090905/manifest

Comment

Related Items