Open Collections

UBC Theses and Dissertations

UBC Theses Logo

UBC Theses and Dissertations

Value-focused GAI network structure elicitation given a domain Ontology Al Baqui, Abeera Farzana 2007

Your browser doesn't seem to have a PDF viewer, please download the PDF to view this item.

Notice for Google Chrome users:
If you are having trouble viewing or searching the PDF with Google Chrome, please download it here instead.

Item Metadata

Download

Media
831-ubc_2007-0308.pdf [ 10.72MB ]
Metadata
JSON: 831-1.0052054.json
JSON-LD: 831-1.0052054-ld.json
RDF/XML (Pretty): 831-1.0052054-rdf.xml
RDF/JSON: 831-1.0052054-rdf.json
Turtle: 831-1.0052054-turtle.txt
N-Triples: 831-1.0052054-rdf-ntriples.txt
Original Record: 831-1.0052054-source.json
Full Text
831-1.0052054-fulltext.txt
Citation
831-1.0052054.ris

Full Text

V a l u e - f o c u s e d G A I n e t w o r k s t r u c t u r e e l i c i t a t i o n g i v e n a d o m a i n O n t o l o g y by A b e e r a F a r z a n a A l B a q u i B.A, Gettysburg College, 2005 A THESIS SUBMITTED IN PARTIAL FULFILLMENT OF T H E REQUIREMENTS FOR T H E DEGREE OF MASTER OF SCIENCE in THE FACULTY OF GRADUATE STUDIES (Computer Science) T H E UNIVERSITY OF BRITISH COLUMBIA October 2007 © Abeera Farzana Al Baqui, 2007 A B S T R A C T Making optimal decisions is important yet challenging. A decision maker has to take into account her preferences when she needs to make decisions. Often such preferences are over features in the world and exhibit certain structure. It is possible to exploit this structure by making assumptions of independence, to acquire a decision maker's preferences. However, a compromise must be observed while making these assumptions -too many independence assumptions means the preference model acquired is likely to be inaccurate, however too few assumptions yields an overly complex model. A Generalized Additive Independence (GAI) Network establishes a good compromise between the accuracy and the generality of the model. A GAI network represents a decision maker's preferences in terms of its structure and a set of utilities. Decision theory provides methods of obtaining both the structure and the utilities of a GAI network; however, these methods are too time consuming, error prone and therefore impractical. Several researchers have investigated methods for simplifying the elicitation procedures for GAI network utilities. However, elicitation of a GAI network structure has not received much attention. Value Focused Thinking (VFT) could be a promising solution to this problem, as it can be used to reduce the number of elicitations required to build a DM's GAI network structure as opposed to traditional decision theoretic methods such as standard gambles. VFT proposes that decision-making should start by decomposing a decision maker's values into additively independent objectives (these are the fundamental objectives). V F T shows how to acquire a decision maker's objectives and map attributes of a domain onto these objectives. We assume that the attributes of a domain are represented in an ontology (which specifies the vocabulary to describe the domain). The decision maker can specify how these attributes fulfill their fundamental objectives. It is tempting to build a system where non-expert decision makers, minimally trained in concepts of VFT, Ontologies and GAI networks, may express their preferences by 1) specifying objectives and 2) indicating attributes that fulfill these objectives thus creating a value tree. Once a decision maker's value tree is elicited, it is possible to build a corresponding GAI network. However, it is required that the structure of a decision maker's value tree adhere to the independence assumptions made for GAI networks. We set up an experiment to test whether the structure obtained from eliciting a decision maker's value tree in this manner follows GAI network independence assumptions. We tested this hypothesis in the real-estate domain and ii found that the resulting structure does not reflect the independence assumptions of a GAI network. We conclude by discussing implications and suggest changes to our original approach. iii T A B L E O F C O N T E N T S Abstract ii Table of Contents....................... . iv List of Tables vii List of Figures viii Acknowledgements iz Dedication Chapter 1. Introduction 1 Chapter 2. Background 4 2.1. History 4 2.2. Decision Theory 4 2.2.1. Axioms of Utility theory .-. 6 2.2.2. Utility Function 7 2.2.3. Standard Gamble Query 8 2.3. Structure of Preferences 8 2.3.1. Additive Independence 9 2.3.2. Comparison Gamble Query 9 2.3.3. Attribute dependence 10 2.3.4. Substitutes and Complements 10 2.3.5. implications of Additive Independence 11 2.3.6. Generalized Additive Independence 12 2.3.7. GAI Network 12 2.3.8. Algorithm for constructing a GAI network 13 2.3.9. Structural Elicitation of GAI Networks 14 2.4. Value Focused Thinking 14 2.4.1. Objectives 15 2.4.2. Attributes 16 2.4.3. Value Focused Thinking Process 16 2.5. Ontologies 18 2.5.1. Domain and Upper Ontologies -. 19 Chapter 3. Eliciting a GAI network based on Value Focused Thinking 20 3.1. Predictions from Theory 20 3.2. Non-expert Decision Makers 22 3.3. Real-Estate domain • 23 3.3.1. Apartment Ontology 23 Chapter 4. User Interface Test Bed 25 iv 4.1. Overview 25 4.2. Ontology Visualization 26 4.3. Value Tree Construction 27 4.3.1. Identifying objectives 28 4.3.2. Mapping attributes into Objectives 29 4.4. Value tree to GAINetwork 32 4.5. Measuring the quality of the GAI Network ; ! 34 4.5.1. Comparison Gamble. .34 4.5.2. Validating The GAI Network 36 Chapter 5. Experiment 38 5.1. Goal. 38 5.2. Hypotheses 38 5.3. Methodology 39 5.3.1. Pre-StudySo i 39 5.3.2. User Study Si ....40 5.4. Procedures 40 5.4.1. Training 40 5.4.2. Task 44 5.5. Data : 45 5.5.1. Collection '.' 45 5.5.2. Analysis 45 5.6. Discussion '. 50 Chapter 6. Conclusions and Future Work 55 Bibliography 57 Appendix A. List of Value Trees from User Study Participants 59 Appendix B. List of GAI networks generated from User Study Participants 68 Appendix C. List of Scatter Plots of Measured Dependencies between pairs of attributes and Distance between pairs of attributes of User Study Participants , 75 Appendix D. List of comments/suggestions/questions compiled from User Study Participants 81 Appendix E. Email for Pre-Study 83 Appendix F. Coded responses from Pre-study 84 Appendix G. Procedures and Instructions for Final User Study 92 Appendix H. Consent Form . 98 Appendix I. User Information 100 Appendix J. Validation Question Training 101 Appendix K. Post-study Questionnaire 104 Appendix L. Sample Receipt 105 AppendixM. UBC Research Ethics Board Certificates of Approval 106 Appendix N. Apartment Ontology in OWL 107 vi L I S T O F T A B L E S Table 1 : Total number of observations at each d(a,b) for a range of measure of dependence...47 Table 2: Frequency of participants with state of personalized value tree 50 vii LIST O F FIGURES Figure 1: Illustration for a Standard Gamble 8 Figure 2: Illustration for a Combination Gamble 10 Figure 3: Construction of a GAI net 13 Figure 4: Phases of decision-making using value-focused thinking 16 Figure 5: Comparison Gamble for testing independence between attributes Xi and Xg 21 Figure 6: Workflow Diagram 25 Figure 7: Sample Ontology Visualization 27 Figure 8: Possible default Value Tree in the Real Estate domain 28 Figure 9: GUI Section for adding an objective to the value tree 28 Figure 10: Personalized Value Tree 29 Figure 11: Adding an attribute to an objective 30 Figure 12: After adding an attribute to an objective 31 Figure 13: Attributes under objective highlighted in Ontology 32 Figure 14: Sample Personalized Value Tree 33 Figure 15: Sample Personalized GAI network 34 Figure 16: Sample Comparison gamble Question 37 Figure 17: First time Option A chosen for Comparison gamble Question 37 Figure 18: Scatter plot of all observed pairs of dependency between attribute pairs and distance between attribute pairs 46 Figure 19 : GAI Network of User 1, exemplifying a disconnected GAI Graph 47 Figure 20 : GAI Network of User 2, exemplifying a connected GAI Graph 48 Figure 21 : Scatter Plot clustering of all observed pairs of dependency between attribute pairs and distance between attribute pairs on disconnected GAI networks 48 Figure 22: Scatter Plot clustering of all observed pairs of dependency between attribute pairs and distance between attribute pairs on connected GAI networks..... 49 viii A C K N O W L E D G E M E N T S I would like to recognize the supervision and advice given by Dr. David Poole and Dr. Giuseppe Carenini during the writing and research for this thesis. I would also like to acknowledge all the assistance I received with Ruby on Rails and ActiveRDF from Eyal Oren. ix D E D I C A T I O N I dedicate this thesis to you, Paul - for his support, criticism and vision. Every conversation, discussion, argument and agreement regarding my work is finally on paper. Thank you for being my beacon for those dark days when light was nowhere to be found. Chapter!. I N T R O D U C T I O N Every day we are faced with making decisions. Some decision are simple and trivial such as "what do I wear to work today?" or "what do I make for dinner tonight?", whereas others have major consequences such as "which job offer should I accept?" or "which apartment do I want to live in?" Decisions with high stakes require a decision maker to contemplate, the pros and cons of every action and all manners of trade-offs, to obtain the optimal result. Often, when faced with such decisions, we take into account the state of the world around us, e.g. locations of places to commute and our own preferences, e.g. "I would like a dishwasher in my kitchen". When making decisions we make trade-offs, between our multiple preferences and the state of the world, to attain the most value from taking an action towards achieving that decision. Our preferences when making decisions are complex and multifaceted. They are expressed in terms of features in the world and properties of individuals (attributes) in the world. For instance, size of master bedroom and facing direction of windows are different properties for each individual apartment; however, the fact that an apartment has walls and doors are features in the world that are independent of any individual apartment. Theoretically, the assessment of preferences over every combination of values of attributes may be needed to completely elicit our value (utility) for an outcome; but in practice, the large size of the outcome set prevents such a procedure to be feasible. It would not be wise to ask ourselves when regarding an apartment, "do I prefer an apartment with south facing windows or an apartment with south-west facing windows?" and "do I prefer an apartment with south-west facing windows and a dishwasher?" and so on for all possible combinations of features and attributes in apartments. Fortunately, our preferences display an underlying structure [Fishburn. 1970, Keeney. 1976] which may be exploited to make elicitation more feasible. One such structural assumption is that of additive independence between attributes. A careful trade-off must be made when making independence assumptions about the structure of preferences. If too many independence assumptions are made, the preference model obtained is inaccurate. However, if too few independence assumptions are made, the model is overly complex and the initial problem of easing elicitation remains. Generalized additive independence (GAI) [Braziurias, et al. 2005] decompositions have recently gained attention because of their flexibility towards the accuracy and the complexity of a decision maker's preference model. GAI is a generalization of the additive model, where independence holds among certain subsets of attributes, rather than individual attributes. A GAI network is a graphical model, which represents a GAI decomposition. A GAI network 1 consists of a structure of the preferences and the utilities of attributes that make up the structure. Elicitation of a decision maker's preferences to build a GAI network using traditional decision theory tools (such as standard gamble queries) is quite impractical. While recent studies [Braziunas, et al. 2005, Gonzales, et al. 2004] have investigated methods of simplifying the utility elicitation of a GAI network, not much work has been done regarding simplifying elicitation of the GAI network structure. We propose value-focused thinking [Keeney. 1996] as an exploratory means to simplify elicitation of the GAI network structure. Value-focused thinking concentrates on obtaining a decision maker's values. For instance, when deciding which apartment to choose, a decision maker may consider the following objective "Minimize commute time". This general objective can be sub-divided to two or more specific fundamental objectives: "Minimize commute time to work", "Minimize commute time to school" etc. Value-focused thinking asks a decision maker to consider the objectives behind making a decision before considering the alternatives (outcomes). It allows a decision maker to create a value tree which is a hierarchically organized list of fundamental and means objectives. The creation of a value tree requires a decision maker to state explicitly which and how attributes in the domain fulfill certain objectives. Thus, to fulfill the objective "Minimize commute time to work", a necessary attribute might be close to bus stop. One paramount assumption in value-focused thinking is the necessity for a decision maker's value tree to be decomposable into additively independent fundamental objectives. Keeney has provided an efficient and systematic method of eliciting a decision maker's value tree. We briefly describe this process in Section 2.4. Thus, a promising way of acquiring a GAI network structure may be to use value-focused thinking to acquire a set of fundamental objectives and map sets of attributes under fundamental objectives as additively independent factors. The vocabulary describing the domain for which a decision is being made contains attributes that also exhibit an inherent hierarchical structure, namely in the form of an ontology. An ontology simplifies a decision maker's tasks in expressing her value tree in terms of attributes. A systematic breakdown of the domain into its sub-parts and their properties allows the decision maker to consider all aspects of the domain without overlooking any detail. It is therefore quite tempting to build a system for non-expert decision makers minimally 2 trained in the concepts of value-focused thinking and ontologies to express their preferences when making a decision. We may envision that such a system would allow a decision maker to specify her value trees in terms of her fundamental objectives; she would also indicate the attributes from a domain ontology that fulfill her fundamental objectives. A decision maker's value tree obtained in such a manner could be used to build the structure of a GAI network. However, for such a system to be built we need to ensure that the structure obtained from a decision maker's value tree adheres to all the independence assumptions inherent in the resulting GAI network. We built a system and set up an experiment in the real-estate domain to test the following hypotheses: • On preference Elicitation: It is possible to elicit the structure of a sound GAI network by having a minimally trained D M 1) create a value tree by following Value Focused Thinking and 2) map the domain attributes from an ontology into the fundamental objectives of the value tree (that are fulfilled) by using the interface developed. • On verification of the output of the elicitation procedure: It is possible to measure the quality of the obtained GAI network by asking a (minimally trained) D M to answer comparison gamble questions comparing two lotteries. This thesis is structured in the following manner. Chapter 2 discusses the history and background information for decision theory, preference structures and GAI networks, value-focused thinking and ontologies. Chapter 3 delves into our contribution and the details of our approach. Chapter 4 deals with the test-bed system built to test our hypotheses. Chapter 5 discusses the experimental procedure setup to test the above hypotheses and discusses our results. Finally, Chapter 6 wraps up this thesis by providing some suggestions for future work and the conclusion of our research. 3 Chapter!. B A C K G R O U N D 2.1. HISTORY Complex decision problems with multiple objectives have been faced since the beginning of v civilization. The history of decision analysis, however, is not very long. In 1738, Daniel Bernoulli used the concept of utility to explain the St. Petersburg paradox [Bernoulli. 1954]. This insight created the foundations of utility theory, a numerical measure describing the value of alternatives; and the notation of a utility function, the numerical measure itself. In the next century or so, utility theory was used to explain economic behavior. Some utilitarian philosophers used utility theory as a tool for constructing the theory of ethics. The theory did not gain much popularity, however, as it was not yet possible to measure a person's utility function. In the 1940s and 1950s, utility theory was placed on sound theoretical foundation. Game theory was developed to describe the behavior of rational people when engaging with others with conflicting goals. In 1944, John von Neumann and Oskar Morgenstern published The Theory of Games and Economic Behavior, which to this day counts as the most influential contribution in decision theory. A wave of new researchers and practitioners, from statistics and operations research community, ignited by this new theory of utility, developed approaches and tools to help Decision Makers (DMs) in difficult decisions. In contemporary times, decision theorists use artificial intelligence (AI) to apply and extend techniques developed in decision science (how ideal decisions should be made and optimal decisions can be reached) for addressing decision making under uncertainty, as reasoning about actions under incomplete information and scarce resources is central to solving difficult problems in AI. 2.2. DECISION THEORY Decision theory lies at the intersection of many academic disciplines - statistics, economics, psychology, game theory, operations research, and others. Assuming a set of axioms for rational behavior, it provides a theory for modeling user preferences and making optimal decisions based on these preferences. The following summary of main concepts is based on Fishburn. 1970, Keeney. 1976, von Neumann, et al. 1944. Definition 1. A decision maker (DM) is a person, organization or any other decision-making entity, empowered to make decisions concerning the problem at hand. 4 Definition 2. A decision is an irrevocable allocation of resources under control of the decision maker. Definition 3. Preferences describe a decision maker's relative valuations for possible states of the world, or outcomes. The valuation of an outcome may be based on the traditional attributes of money and time, as well as on other dimensions of value including pleasure, pain, life years and computational effort.* A D M has to select a single alternative (or action) a £ A from the set of available alternatives. Assume a set of attributes Xlt X2i —, Xn, each with finite domains. These define a set of outcomes X = X ... X Xn over which a D M has preferences. An outcome, X E X of the chosen action depends on the state of the world 9 E 0. The consequence function C '• A X 0 *-* X maps each action and world state into an outcome. D M preferences can be expressed by a value, or utility, function V ' X *-* WL that measures desirability of outcomes. The goal is to select an action a E A that leads to best outcomes. If the world state 6 is known, the set of outcomes is equivalent to the set of alternatives. When uncertainty over world states is quantified probabilistically, utility theory prescribes an action that leads to the highest expected utility (EU). The outcome space itself is defined in multi-dimensions. Most interesting problems fall in this category, and we explore one way of exploiting the structure in multidimensional outcome spaces to acquire a decision maker's preferences. A DM's preference for one object over another can be represented by one of the following three binary relations: strict preference, weak preference and indifference. • a > b means that the D M strictly prefers the object a to the object b. In other words, if a choice between a and b was offered to the D M , she would be disappointed if she had to select the object b. • a ^ b means that the D M weakly prefers the object a to the object b. That is, according to the D M the object a is at least as good as the object b. If the D M was offered a choice between the object a and b, she would not be disappointed if she was forced to take the object a. * Definitions from [Helsinki University of Technology (Systems Analysis Laboratory). 2005] 5 • a ~ b means that the D M is indifferent between the object a and b. In other words, if she was offered a choice between a and b, she would not be disappointed if she was subsequently forced to take either of the options. Decision theory is based on the axioms of probability and utility, as mentioned above. Where probability theory provides a framework for coherent assignment of beliefs with incomplete information, utility theory introduces a set of principles for consistency among preferences and decisions. Since we are concerned with a DM's preferences under uncertainty, we will focus more on utility theory rather than preference theory. Utility theory is based on a set of simple axioms or rules concerning choices under uncertainty, these axioms are discussed below [Fishburn. 1981]. 2.2.1. AXIOMS OF UTnjTY THEORY The first set of axioms concerns preferencesfor outcomes under certainty. • The axiom of orderability asserts that all outcomes are comparable, even if described by many attributes. Thus, for any two possible outcomes x andjy, either one prefers x toy or one prefers ^  to x, or one is indifferent between them. • The axiom of transitivity asserts that these orderings are consistent; that is, if one prefers x toy andy to z, then one prefers x to z. These axioms, together with two auxiliary axioms, ensure a weak preference ordering of all outcomes. This result implies the existence of a scalar value function V(x), which maps from all outcomes x into a scalar "value" such that one will always prefer outcomes with a higher it 1 » value . The second set of axioms describes preferences under uncertainty. They involve the notion of a lottery, an uncertain situation with more than one possible outcome. Each outcome has an assignable probability of occurrence. Definition 4. Lottery: Let Xt be an outcome and let X be a set of outcomes. Let P be a simple probability measure on X, thus P = (p(*i), p(.x2),..., p(*n)) where in a lottery p{x{) are probabilities of outcome xteX occurring, Le. p(x{) > 0 for all i = 1 , n and 2XiP(*t) = l • The axiom of monotonicity says that, when comparing two lotteries, each with the same 6 two alternative outcomes but different probabilities, a D M should prefer the lottery that has the higher probability of the preferred outcome. • The axiom of decomposability says that a D M should be indifferent between lotteries that have the same set of eventual outcomes and probabilities, even if they are reached by different means. For example, a lottery whose outcomes are other lotteries can be decomposed into an equivalent one-stage lottery using the standard rules of probability. • The axiom of substitutability asserts that, if a D M is indifferent between a lottery and some certain outcome (the certainty equivalent of the lottery), then substituting one for the other as a possible outcome in some more complex lottery should not affect her preference for that lottery. • The axiom of continuity says that, if one prefers outcome x toy, andjy to z, then there is some probability^ such that one is indifferent between getting the intermediate outcomey for sure and a lottery with ap chance of x (the best outcome) and (1 - p) chance of z (the worst outcome). It follows from accepting the axioms of utility that there exists a scalar utility function U(x), which assigns a number on a cardinal scale to each outcome x, indicating its relative desirability. Further, it follows that when there is uncertainty about x, preferred decisions d are those that maximize the expected utility E\U(X', d) \over the probability distribution for x. The consistency criteria embodied in classical decision theory can be stated as follows [Horvitz. 1988]: Given a set of preferences expressed as a utilityfunction, beliefs expressed as probability distributions, and a set of decision alternatives, a decision maker should choose that course of action that maximizes expected utility (EU). The power of this result is that it allows preferences for distributions of outcomes to be computed from preferences expressed for the individual outcomes. Thus, it may be used as a tool to help people think about complex choices by decomposing them into simpler choices. 2.2.2. UTILITY FUNCTION Definition 5. A utility function u:Jfi->l rationalizes a preference relation ^  on X if for every xt, Xj E X, u(x{) < u(xj) if and only if xt ^ Xj. If u rationalizes then this implies ^ is complete and transitive, and hence rational. Utility 7 functions enable a compact representation of preferences over outcomes. For describing preferences over n outcomes, n real numbers are required; the outcome with a greater utility is preferred to outcomes with smaller utilities. T o measure the utility of each outcome in Xwe can perform standard gamble queries. 2.2.3. STANDARD GAMBLE QUERY A standard gamble is a paired comparison between two alternatives where one alternative is a lottery between two possible outcomes and the other alternative is a sure outcome. Consider the standard gamble defined in Figure 1, where lottery L yields a/> chance of consequence x T (the best outcome) and a 1-p chance X*~ (the worst outcome) and x, which is a sure outcome. A decision maker's utility for outcome x can be obtained by measuring the probability^ at which the decision maker is indifferent between lottery L and the outcome x. Figure 1: Illustration for a Standard Gamble 2.3. STRUCTURE OF PREFERENCES A n attribute X1 has a set of possible values x\,x\ ... X™. A n outcome is defined by the values of each attribute in the attribute set {X-y, X2,... X3}. Since the number of outcomes is exponential in the number of attributes, specifying the utility of each outcome is infeasible in most practical applications. The assessment of preferences for every outcome compared to the best and the worst outcome is needed to elicit completely the DM's utility. The large size of the outcome set makes such a procedure impractical. Fortunately, preferences often have an underlying structure that can be exploited to reduce the elicitation burden. Several structures described in terms of different independence concepts have emerged from the multi-attribute utility theory community the most popular of which is the additive decomposition. A DM's preferences can be expressed concisely if it exhibits sufficient structure such as an additive decomposition. Additive independence is one structural assumption commonly used in practice. 8 2.3.1. ADDITIVE INDEPENDENCE Definition 6. Additive Independence: Let Li and L2 be any pair of lotteries and let p and q be their respective probability distributions over the outcome set. Then Xlt..., Xn are additively independent for £ if p and q having the same marginals on every Xt implies that both lotteries are indifferent, Le. Li £ L2 and L2 £ Li (or Li ~ L2 for short). [Bacchus, et al. 1995] illustrates additive independence on the following example: Let X = X x x X2where X x = {a1; b±} and X 2 = {a2, b2}. Let Li and L2 be lotteries whose respective probability distributions onXare^? and q. Assume p(ab a2) = p(ai, b2) = p(bi, a2) = p(bi, b2) = lA q(ai, a2) = q(bi, b2) = V2 and q(ah b2) = q(bi, a2) = 0. Then p and q have the same marginals on Xi and X 2, since p(ai) = q(ai) = V4, p(b,) = q(bi) = V4, p(a2) = q(a2) = 'A, p(b2) = q(b2) = fc. So, under additive independence, lotteries Li and L 2 should be indifferent. Given a complete ' utility function that can be derived using standard gambles, we can test dependence between attributes. However, comparison gambles make this task more feasible and direct. 2.3.2. COMPARISON GAMBLE QUERY A comparison gamble is a decision aid tool to adequately measure the dependence between two attributes .X^and X2 • A comparison gamble is a paired comparison between two alternatives where each alternative is a lottery between two possible outcomes. Consider two lotteries L1 and Z<2 a s defined in Figure 2. Lottery L1 yields ap chance at the outcome (x~r, y"1") and a 1-p chance at the outcome (xJ~, y"1") and lottery L2 yields equal 0.5 chances at each of (x'r, y"1") and (3c"1", y"•")*. In the case when p equals 0.5, note that both lotteries have an equal (i.e. 0.5) chance at either or x^ and also that both have an equal 0.5 chance at either y"1" or y"1". By definition then the marginal probability distributions on each of the attributes of X and Y are the same in * X"1", X , y"1" and y""" are used here as possible values for attributes. For attributes X and Y to be additively independent, this needs to hold for any pair of values for the attributes. 9 both the lotteries when p equals 0.5. Thus, if X and Y are additively independent the D M must be indifferent between lotteries and L2. This same indifference condition must hold if either or both of andy"1" are changed, because Lt and L 2 will still have the same marginal probability distributions on the two attributes. Figure 2: Illustration for a Combination Gamble Attributes X and Y are considered dependent when a D M is indifferent between lotteries and L2 when p 0.5. 2.3.3. ATTRIBUTE DEPENDENCE To measure the degree of dependence between two attributes, the probability^ at which a D M is indifferent between lotteries Lt and L2 is measured. For example, for the comparison gamble shown above, when a D M has the following preference Lj > L2, decreasing the probabilityp and asking the D M the same comparison gamble query might result in a preference of Lx ~ L2, thus the degree of dependence between the two attributes X and Y would corresponds to the new p value. 2.3.4. SUBSTITUTES AND COMPLEMENTS Attributes interact with each other in many ways. One interaction that attributes may have can be described as being substitutes of each other, i.e. one attribute's presence replaces the absence of another attribute in terms of value. For example, the attributes close to park and close to beach may behave as substitutes for a D M . Conversely, attributes may complement each other, i.e. the presence of both attributes is required for the D M to be satisfied. For example, the attributes facing South and has window are complementary attributes, as having only both would satisfy the DM's need for adequate lighting. This categorization of the interaction between attributes can be measured using a comparison gamble query. If a DM's preference is L-y ~ L 2 when^ > > 0.5 for the comparison gamble shown 10 above, this indicates that the attributes are substitutes, i.e. a D M would rather prefer a gamble where she has one or the other attribute. Correspondingly, whenp < 0.5 arid a DM's preference is Ly ~ L2 the attributes are complementary, i.e. the D M would rather gamble to obtain both the attributes then end up with one or the other attribute. 2.3.5. I M P L I C A T I O N S O F A D D I T I V E I N D E P E N D E N C E Under a strong independence assumption—specifically, that the D M is indifferent among lotteries that have same marginals on each attribute—u can be written as a sum of single-attribute sub-utility functions: it n i=l i=l This simple factorization exploits subutility functions U j ( X i ) = XiVi(x{), which themselves depend on local value functions V( and scaling constants X(. The assumed utility independence among attributes allows elicitation to proceed locally: specifically, the V>i can be elicited independently of other attribute values. Since each attribute is utUity independent, each attribute's best and worst levels can be determined separately. Since utility functions corresponding to <S are unique only up to positive affine transformation, it is customary to set the utility of the best value (xTj for the attribute to 1, and the utility of the worst value (x^j for the attribute to 0. A standard gamble query between x\ and a lottery between the two anchor outcomes, the best and worst levels of attribute X{: {p, xf; 1 — p, X~j~) is put to the D M . Eliciting the indifference for a specific p for this gamble will return the V j for the attribute as: u(x?) = puixj) + (1 - p)u(xt) and therefore, because of the additive form «, v{x*) = pvtixj) + (1 - p)Vi(xt) Thus, if a D M is indifferent for the given standard gamble, then = p. To elicit the global scaling constants, queries involving full outcomes are asked [Braziunas, et al. 2005]. Additive Independence simplifies the elicitation procedures yet compels the DM's preferences to satisfy very stringent constraints. This form of independence captures the fact that 11 preferences only depend on the marginal probabilities on each attribute, but rules out interactions between attributes, which are quite common in DMs preferences [Bacchus, et al. 1995], therefore such functional form cannot be applied in many practical situations. One-way out is to allow some limited forms of interactions between attributes; for instance, by separating the utility function into a sum of sub-utilities on sets of interacting attributes: this leads to generalized additive independence (GAI) decompositions. 23.6. GENERALIZED ADDITIVE INDEPENDENCE Definition 7. Generalized Additive Independence: Let Li and L2 lotteries and let p and q be their probability distributions over the outcome set. Let Zi,..v Zjtbc some possibly overlapping subsets of N=fl,..., nj such, that N = UJLj Zt and let XZl = {Xy. j E Z J . Then XZl„.., XZk are generalized additively independent for ^  if the equality of the marginals ofp and q on all XZ('s implies that Li ~ L2. [Gonzales, et aL 2004] The following functional form of the utility called a GAI decomposition can be derived from generalized additive independence: Let Zi,..., Zk be some subsets of N ={1,..., n} such that N = U ^ = 1 Z[ andXZi = \Xj\ j E Zj} are generalized additively independent (GAI) for *S iff there exist some real functions Ui.XZl X ... X XZ{ H» IR such that fc u(x) = ^ ^ U i ( x z . ) , f o r allx = (xx, ...,xn) E X i= l where Xz. denotes the tuple of components of x having their index in Z;. GAI decompositions allow great flexibility because they make less stringent assumptions on the kind of relations between attributes. Furthermore, the amount of questions required by the elicitation is manageable thanks to the GAI decomposition itself. A graphical structure called a GAI network can effectively represent GAI decompositions. 2.3.7. GAI NETWORK Definition & Generalized Additive Independence network (GAI network): Let Zlt ...,Zkbc some subsets of N = {1, . . . , n} such that N = u f = 1 Zt. Assume that ^  is representable by a GAI-decomposable utility u(x) = Tu=i Ui(xZlj, for all x 6 X. Then a GAI network representing u (•) is an undirected graph G = (V.E), satisfying the 12 following properties: 1. V = {XZl>...,XZk}; 2. {xZi,xZj)e Effi3Ztn Zj* 0. Nodes of V are called factors. Every edge (XZt, XZj) 6 E is labeled by XTlf = XZl^Zf, which is called a separator. The GAI network provides an additive decomposition of utility functions in situations where single attributes are not additively independent, but (possibly non-disjoint) subsets of attributes are [Fishburn. 1970]. The form of a GAI network is as follows. Assume a collection {Zlt... Z }^ of (possibly intersecting) factors such that Uj Zj = {1,..., n} and local subutility functions Ui over Xz,. Then u is decomposed as: u(X)= u1(XzJ + -+ uk(XZk) For any GAI decomposition, Definition 8 is explicit as to which factors should be created: these are simply the sets of variables of each sub-utUity. For instance, if ll(Xi,..., X7) = Ui(xi, X2, X3) + U2(X3, X4) + U3(X4, X5) + U4(x6, X7) then, as shown in Figure 3.a, factors are {Xi,X2,X3}, {X3,X4},{X4,X5}and {X6,X?}. CJ) a) Factors of a GAI Network b) Edges of a GAI Network Figured: Construction of a GAI net Property 2 of Definition 8 gives us a clue for determining the set of edges of a GAI network. The following section describes the algorithm to construct a GAI network. 2.3.8. ALGORITHM FOR CONSTRUCTING A G A I NETWORK The algorithm to construct the GAI network structure, given the local sub-utility functions, is 13 shown below: c o n s t r u c t s e t V = {XZl,..., XZk} ; f o r i e {1 . . . , k - 1} do f o r j 6 { i + 1. . . , k} do i f Z i n Zj ± 0 then add edge (Xz.r Xz.) to E end • done done Applying this algorithm on set V= {{X\,XL,XZ}, {Xs,Xa,}, (XA,XS}, {X6,X7}}, sets (X\,XL,XS} and {Xi,Xn} having a nonempty intersection, an edge would be created between these two factors. Similarly, edge {{X$,XA}, {XA,XS}} would also be added as XA belongs to both factors. Since the intersection between {Xe,Xj} and all the other sets in Freturns a null set, there is no edge created. Consequently the network in Figure 3.b is a GAI network, representing U^X\,..., X7) = Ui(Xi, XL, Xi) + UL(XS, XA) + U3(X4, Xs) + Ui(Xe, xf). Eliciting each local sub-utility function has been investigated by [Braziunas, et al. 2005]; however, simplifying the elicitation of the structure of the decomposition has not received much attention. 2.3.9. STRUCTURAL EUCITATION OF GAI NETWORKS Using traditional decision theoretic tools such as comparison gambles for pairs of attributes to determine the set of sub-utility functions to create the GAI network structure is infeasible and time-consuming. The number of comparison gamble queries would be exponential in the number of attributes, also the same comparison gamble query would be asked several times to elicit the accurate probability for measuring the degree of dependence between attributes. We propose value-focused thinking to be a viable approach for this problem as it decomposes a DM's values into additively independent objectives. 2.4. VALUE FOCUSED THINKING Value-focused thinking advocates thinking about values first and then evaluating the alternatives present that achieve those values [Keeney. 1996]. Keeney refers to value-focused thinkings an iterative approach between expressing values and choosing alternatives; however, the principle 14 is "values first". Keeney details the multiple uses of value-focused thinking, such as uncovering hidden objectives, guiding information collection, interconnecting decisions, guiding strategic thinking, evaluating alternatives etc. Value focused thinking provides a structured way to think about decisions and develop and support subjective judgments that are critical for good decisions. 2.4.1. OBJECTIVES Definition 9. Objectives are statements of something that one desires to achieve. Generally, objectives are characterized by three features: • Decision context: the activity being contemplated • Object: a goal this objective would achieve • Direction of preferences For example, in selecting an apartment one of the objectives maybe to MINIMIZE C O M M U T E TIME. For this objective, the decision context is apartment selection, the object is the amount of time taken to get from home to workplace and less time taken is preferable to more time taken. • Objective specification does not require the identification of a measure (for example commute time in hours per day) to indicate the level to which the objective is achieved. • An objective does not quantify the relative desirability of different levels of the object. ^ • Objectives can be divided into two classes: Fundamental objectives and means objectives. Definition 10. Fundamental objectives characterize an essential reason for interest in the decision situation. Definition 11. Means objectives are of interest in a decision context because they are a means to achieving fundamental objectives. For example, lower rent may appear to be an important objective, but it may be seen important only because it would allow a D M to improve her living standard, to pursue activities that represent fundamental interests. Thus, low rent could be seen as a means objective and increase living standard as a fundamental objective. 15 2.4.2. ATTRIBUTES An objective is measured in terms of a set of attributes. For instance, X^d) = X( indicates the level Xi, where JCj G {xf,... xf}, to which an objective is achieved in the outcome a for the attribute X^. For example, Availability of Fire alarm is one measure for the objective MAXIMIZE SAFETY. If an apartment has a fire alarm, then the level at which this objective is achieved for this attribute is 1 and an absence of a fire alarm indicates that the value of this attributes for achieving this objective is 0. 2.4.3. VALUE FOCUSED THINKING PROCESS Using value-focused thinking, a D M would have to come up with her fundamental objectives and concern herself with which attributes fulfill those objectives, thus reducing the complexity to a polynomial function, i.e. number of FO in value tree x number of A in the domain. During the elicitation of preferences, the D M specifies objectives and attributes that fulfill these objectives as a value tree. As shown in Figure 4*, decision-making using value-focused thinking involves four main phases. Frame Dec is ion S i t ua t i on •Define decision context identify objectives ^Create hierarchical model of objectives 'Specify attributes Preference Elicitation R e c o m m e n d e d Dec is ion Sens i t i v i t y Ana l ys i s Figure 4: Phases of decision-making using value-focused thinking ' Adapted from Helsinki University of Technology [Systems Analysis Laboratory], 16 For our purposes, we will only concentrate on the first phase of the process as a plethora of research has already been done for each of the other phases [Fishburn. 1970, Braziunas, et al. 2005, Bacchus, et al. 1995]. The process is iterative and quite often long and time consuming. In addition, the DM's preferences and objectives may change and develop during the process. The main purpose of the framing a decision situation phase is to create a better understanding of the decision problem. For example, answers to the following questions should become evident. • What is important, and relevant? • What are the objectives? • What information is available? Define decision context: by doing this and establishing the nature of the decision problem carefully, the treatment of the real problem can be ensured. A careful specification of the decision context is particularly relevant if several DMs or stakeholders are involved in the decision analysis process. Without a mutual agreement on the decision context, problems are likely to occur in the subsequent phases. Identifying objectives requires significant creativity. The most obvious way to identify objectives is to ask a decision maker to recapitulate the decision context, and then provide a written list of objectives. Several devices can be used to stimulate the identification. Creating a hierarchical structure of the objectives, often called a value tree, is to produce a deeper and more accurate and analytic understanding of the problem and a basis for quantitative analysis. Specification of attributes aims to measure the achievement of objectives in different decision alternatives. After the decision situation and real nature of the problem is established, objectives are identified. Relations between multiple objectives are analyzed and a hierarchical model is obtained. With a hierarchical model, relations between objectives are more easily understood. The model also creates a basis for further analysis. Attributes measure the extent to which different decision alternatives satisfy the stated objectives. Specification of attributes thus enables the comparison of the alternatives. The aim of the preference elicitation is to measure and estimate the DM's utilities over a set of 17 objectives. • Measuring preferences is not straightforward. It may be that the D M is not sure about her preferences, she is unable to state them or she is even unaware of them. Furthermore, the D M may act inconsistently and give conflicting statements about her preferences. • In most cases, the preference elicitation is an iterative process used to ensure the best possible estimates of the DM's preferences. • Knowing the DM's preferences, information about the attribute levels for different decision alternatives and the DM's value tree can be used to find the most preferred alternative given as a recommended decision. The aim of the sensitivity analysis is to explore how changes in the model influence the decision recommendation. • If a small change in one or several aspects of the value tree causes the recommended decision to change, the decision is said to be sensitive to those changes. • Recognizing the aspects to which the decision is sensitive enables the D M to concentrate on, or possibly reconsider the issues, which may cause changes in the decision. • Any part of the decision analysis process, from the identification of the decision problem to the evaluation of the preferences, can be subjected to the sensitivity analysis. As Figure 4 shows, after the sensitivity analysis the D M may return to earlier phases of the decision analysis process; new objectives may be identified, the value tree may be changed etc. Thus, sensitivity analysis is a central part of the decision analysis cycle. A value tree, mentioned above, is a hierarchical list of objectives. The degree to which objectives are achieved in-different decision alternatives is measured with attributes. Often these attributes are properties of entities that exist in some domain. These entities and attributes can typically be ordered hierarchically as an ontology. 2.5. ONTOLOGIES Attributes that measure the achievement of objectives are specified in a domain ontology. Definition 12. Ontology: A description of the concepts and relationships that can exist. 18 Ontologies typically specify the meanings and hierarchical relationships among terms and concepts in a domain. An ontology is made up of several elements: • Individuals: the basic or "ground level" objects • Classes: sets, collections, or types of objects • Properties: features, characteristics, pr parameters that objects can have • Relations: ways that objects can be related to one another Definition 13. An ontology language is a formal language used to encode an ontology. There are a number of ontology languages such as the Web Ontology Language (OWL). An O W L ontology may include descriptions of classes, along with their related properties and instances. O W L is designed for use by applications that need to process the content of information instead of just presenting information to humans. It facilitates greater machine interpretabdity of content by providing additional vocabulary along with a formal semantics. O W L currently has three sub-languages (also referred to as 'species'): O W L Lite, O W L DL, and O W L Full. These three increasingly expressive sublanguages are designed for use by specific communities of implementers and DMs. 2.5.1. DOMAIN AND UPPER ONTOLOGIES A domain ontology (or domain-specific ontology) models a specific domain, or part of the world. It represents the particular meanings of terms as they apply to that domain. For example the word floor has many different meanings. An ontology about the domain of architecture would model the "lower horizontal surface of a room" meaning of the word, while an ontology about the domain of mathematical computation would model the "floor function" meaning. An upper ontology (or foundation ontology) is a model of the common objects that are generally applicable across a wide range of domain ontologies. It contains a core glossary in whose terms objects in a set of domains can be described. There are several standardized/formal upper ontologies available for use, including Dublin Core, GFO, OpenCyc/ResearchCyc, SUMO, andDOLCEl. 19 Chapters. E L I C I T I N G A GAI N E T W O R K BASED O N V A L U E F O C U S E D T H I N K I N G 3.1. PREDICTIONS FROM THEORY In our approach, we aim to elicit the factors of the GAI network for a D M by asking her to describe her value tree in terms of fundamental objectives and attributes that fulfill these objectives. Attributes bundled under one objective represents attributes that make up a factor of the DM's GAI network. So, given the DM's value tree, a preference model is generated as a GAI network. We hypothesize that our approach is an effective means for eliciting a DM's GAI network and would complement the local utility elicitation of GAI networks [Braziunas, et al. 2005] for obtaining a DM's preferences. Since our approach relies on the concepts of value-focused thinking and the independence assumptions made when building a value tree and its corresponding GAI network, a discussion of the predictions made in the theory regarding these concepts is necessary. According to Keeney, the fundamental objectives in a value-tree should be additively independent. If a user defines their preferences using the simplest set of fundamental objectives, the theory for additive independence predicts that a GAI network built based on a value tree would exhibit these properties: • Attributes in disconnected factors are independent • Attributes in the same factor are dependent • As the number of edges between attributes in factors increases, the dependence between these attributes decreases (weak) For instance, if a GAI network as shown in Figure 3 (page 13) is built based on a value-tree, the decomposition implies 1) A^is additively independent from Xi,X2, X3,X4,Xs, 2) Xi is additively dependent to X& X3,X4,Xs, 3) The dependence between X2 and X3 is higher than the dependence between X2 and X4 and so on. To test these predictions, we can ask comparison gamble queries to test for independence and degree of dependence if any. For instance, a comparison gamble query between J^and X6 can be structured as shown in 20 Figure 5 where X"1" represents the best value for attribute Xand X~^~ represents the worst value. Given a comparison gamble between lotteries Lyand L2, the attributes Xy and X6 are additively independent when the D M is indifferent between Ly and L2 when p = 0.5. Similarly, attributes Xy and X6 would be dependent if a D M is indifferent between lotteries Lyand L2 whenp 0.5. Figure 5: Comparison Gamble for testing independence between attributes Xi andX6 To measure the degree of dependence between two attributes, we can elicit the value marked by p in Figure 5, which represents the probability at which a D M is indifferent between lotteries Ly and L2. For example, for the comparison gamble shown above, when a D M has the following preference Ly > L2 atp = 0.5, we can decrease the probability at p by half to 0.25 and ask the D M the same comparison gamble query. If the DM's preference is now Ly ~ L2, we can estimate the dependency between attributes Xy and X6 to be 0.25. This preference ordering demonstrates two key concepts: • Attributes Xy and X6 are dependent, therefore the factors could either be {Xy, X2, X3, X6} or {Xy, X6, X7} or Xy and X6 should be in a new factor. • Attributes Xy and X6 are complementary attributes, i.e. the D M prefers an outcome where both attributes are present. Consequently, if the preference ordering when p = 0.5 were as follows L2 > Ly, we can increase the probability p by half to 0.75 and ask the D M the same comparison gamble query. This time, the D M prefers an outcome where attribute Xy may act as a substitute for attribute X6, and vice versa if her preference is now Ly~L2. Consider the following scenario: a D M is queried a comparison gamble question (regarding the GAI network in Figure 3 (page 13)) about attribute Xy and X2; she yields a probability of 0.3. However, when a comparison gamble question about attribute Xy and X3 is posed, the D M yields 0.9. Albeit all three attributes belong to the same factor, the theory does not make any 21 predictions about the degree of dependence between the attributes. The factor can also be separated into two distinct factors where Xy and X2 belong in one,and Xx and X$ belong in another. We built our system to test our two main hypotheses: Hypothesis 1: On preference Elicitation: It is possible to elicit the structure of a sound GAI network by having a minimally trained D M 1) create a value tree by following Value Focused Thinking and 2) map the domain attributes from an ontology into the fundamental objectives of the value tree (that are fulfilled) by using the interface developed. Hypothesis 2: On verification of the output of the elicitation procedure: It is possible to measure the quality of the obtained GAI network by asking a (minimally trained) D M to answer comparison gamble questions comparing two lotteries. To test Hypothesis 1, we would need to measure the efficiency of the user interface in aiding the D M to build her value tree using value-focused thinking concepts; and the ability of the user interface in supporting the D M to specify attributes from a domain ontology under objectives built in the value tree. To test Hypothesis 2, we need to measure the quality of the GAI network structure by measuring the dependence between attributes in factors at differing number of edges. We employ the three properties of a GAI network assumed to comprise of additively independent factors, s . • • If two factors are disconnected, the probability between any two attributes in those factors should be 0.5 (i.e. dependency = 0). • If two attributes belong in the same factor, the probability should not be 0.5 (dependent). • Moreover, if two factors are connected, the probabdity between any two attributes in those factors should not be 0.5. 3.2. NON-EXPERT DECISIONMAKERS The motivation for our approach was to aid non-expert decision makers to express their preferences with a naive idea about fundamental and means objectives, attributes and entities within a domain ontology, and the fulfillment of objectives by means of attributes. We designed and implemented a test-bed system keeping in mind that simplicity in the user 22 interface with very little bloat would aid our users in using the system effectively. Because the concepts of value-focused thinking and ontologies are not every-day notions, a highly complicated system with many features would only strive to make a decision maker confused and inefficiently perform the tasks needed to express her value tree. 3.3. REAL-ESTATE DOMAIN We chose the real world problem of searching for an apartment to test our hypotheses. Al l the examples in the following chapters regarding attributes, objectives and ontologies will concern apartments. Real-estate is a relatively complicated test domain. Yet the domain is quite familiar to most DMs for which it makes sense to elicit a complex preference model. Modeling the domain ontology for apartments was aided by using a formal ontology such as MILO (Mid-level ontology) [http://www.cim3.net/. 2007]. Having an upper ontology at our disposal, especially when it consists of a formal upper layer (SUMO) enriched with concrete domain-independent 'middle layer' concepts (MILO), aided the creation of a domain specific ontology for apartments. It allowed us to focus on the content of the domain specific ontology without having to worry on the exact higher structure or abstract phdosophical framework that gives ontologies rigid backbones. 3.3.1. APARTMENT ONTOLOGY Publicly available apartment ontology publication sites/such as SWOOGLE, SchemaWeb and OntoSelect etc were not suitable for our purposes because of the following issues: Some have too "shallow" ontologies. For example, APARTMENT-UNIT entity and APARTMENT-BUILDING entity would be defined; however, the ontology did not delve much further into the domain in terms of subclasses or properties of these entities. Others have only partial ontologies, i.e. several entities would be defined in a hierarchy but relations between these entities (interactions between one entity and another) were absent. Fortunately, many aspects of the domain ontology we needed were found in the Mid-Level Ontology (MILO) as considerable research has been-done for this ontology. MILO is meant as a "middle" ontology between an Upper ontology, in this case the Suggested Upper Merged Ontology or commonly known as SUMO [Niles, et al. 2001], and domain ontology such as an apartment's ontology. Therefore, the best approach was to create a domain ontology of apartments based on M I L O and SUMO jointly. SUMO is a candidate for the IEEE standard for a formal upper ontology and was also a good starting point for the domain ontology. 23 An initial concept map was built based on significant elements of the domain that are necessary for expressing preferences over individuals within the domain of apartments. These elements were identified from popular apartment rental websites such as (http://www.rentsline.com/, http://www.rentbc.com/, http://apartmentguide.ca/, www.move.com etc). The domain ontology was built upon the concepts already defined in SUMO and MILO by importing these two ontologies. Significant entities of the domain such as FLOOR-TYPE that were missing in both MILO and SUMO were added into the ontology. Such entities including any sub-classes and relations between other entities that were identified as relevant within the domain, were introduced into the domain ontology. Individuals and relations between entities from the concept map that were absent were also added. For example, the basAppliance property was not present in either MILO or SUMO, it would be an owhDatatypeProperty of type Boolean for the entity APARTMENT UNIT-There were certain entities and relations already created in MILO or SUMO; however, the relation between these entities and relations were absent. These were added in the domain ontology, for example the hasApartmentUnit property between an APARTMENT-BUILDING and an APARTMENT-UNIT was created to describe that an apartment building H A D apartment units. Some entities such as FACILITIES, APPLIANCES etc which would aggregate all factilities (Parking-lot, Swimming Pool etc) and all appliances (bathtub, fridge etc) already defined in S U M O or MILO had to be created as subclasses of other entities. Certain entities such as ROOM were partially filled. Some subclasses of such as entities (LivingRoom, Office etc) were not defined and were also added. The apartment ontology that was created was a fuller and much richer version of the , intersection between the entities relevant to apartments in M I L O and SUMO. Extensive additions and modifications had to be made in order for all the relationships and entities to be fully embodied in the domain ontology. The complete apartment ontology O W L description is available in Appendix N . 24 Chapter 4. USER INTERFACE TEST BED 4.1. OVERVIEW This chapter overviews the user-interface test-bed that was used to test the two hypotheses. It also describes each part of the workflow diagram shown in Figure 6 in detail. I Default Value F Q g O , objectives AddO&iactem Remove Objectives Modify Objectives Mov* Aanbuta(s) Add Aflntxj!e(s) Remove Objective to 'roii^ Ontology'M AttnbuleW) from ObjecUVC<l) Ques-jons lasedon GAI O m a n OH Network M M M QW Netwoik/Value Tree by answering Figure 6: Workflow Diagram The design of the user interface for the test-bed is primarily composed of four parts. Part 1 entails the visualization of the domain ontology. Part 2 entails the visualization of the default value tree. Part 3 entails the modification of the default value tree - this can be done by a) adding objectives, b) modifying the current objectives, c) removing objectives, d) adding attributes from the ontology in part 1 that fulfill the objectives in the personalized value tree and e) modifying the state of the attributes, i.e. the presence (true) or absence (false) of the attribute fulfills the objective. Once the D M has created a personalized value tree, we convert the tree into a G A I network as explained in Chapter 3. Part 4 comprises of testing the validity of the G A I network structure obtained by asking comparison gamble queries over the attributes used to describe the DM 's value tree. The user interface was written for the web-platform using Ruby on Rails [Hansson. 2004]. The 25 chosen framework is quite adept at handling complicated user-interactions and is A J A X [Garrett. 2005] friendly. Therefore, every time a D M chose to perform an action on the value tree, a call to the server was seamless. Ruby on Rails' inherent Model-View-Controller design pattern allowed for the separation of the database and the interactions as well as the manipulations made to the data. This was necessary to keep the code clean and readable and the application versatile when it came to making changes. 4.2. ONTOLOGY VISUALIZATION The Act iveRDF library was used for accessing the O W L data. Act iveRDF can be used as a data layer in Ruby on Rails providing a very convenient Object-Relational Mapping (ORM) to relational databases. Design and implementation of this application was sped up considerably by the simplicity of design of this library. The domain ontology for apartments, in O W L format, was parsed into ntriples using the rapper utility. This was necessary since the ActiveRDF library only accepts ntriple formatted files for processing. Using ActiveRDF library, the ontology was processed into a tree format of entities, the relations between entities and the domain and range values of these relations, and the attributes of each entity. For the purposes of simplicity only :subPropertyOfand :subClassOf relations were processed for visualization. Using a JavaScript Tree Control library the ontology was visualized as shown in Figure 7. The D M is able to interact with all the leaf-level attributes only. The text of the non-allowed attributes and entities are italicized and grayed-out, thus enabling D M s to realize easily which attributes are interactive. These elements of the ontology are also static and non-draggable. 26 - 9 Apartment Balding - ® Has Apartment Unit - Si Apartment Unit - ® Has Room ® Has Balcony 9 Has Bath room ® Has Bedroom ® Has Den ® Has Foye r ® Has Hallway ® Has Kitchen ® Has Living R o o m ® Has Office - SI Room - ®Has Window - ® Window ® Faces East ® Faces North ® Faces South ® Faces West ® Has Door - ® Has Flooring ® Has Carpet ® Has Hardwood ® Has Linoleum ® Has Vinyl • ® Ha: Paint ® Has Painting ® H a s Wallpaper ® Has Window Cover ing • ® Has .Area Pleasure • ® Has Appliance * ® Has View - ® Cose To Facility - ® Close To Tr>eate ® Close To Movie Theatre ® Close To Live Theatre • ® Close To Mercantile Organization ® Close To Major Road Intersection ® Close To Parking Lot ® Close To Laundromat ® Close To Park ® Close To Beach ® Close To Tourist Attract ion - ® Close To Transit Terminal ® Close To Airport ® Close To Ferry Terminal 4.3. VALUE TREE CONSTRUCTION The user interface provides each D M with a default value tree that specifies a set of objectives as shown in Figure 8. The default value tree is intended to facilitate the D M in expressing her fundamental objectives in the domain. The D M can look at the objectives and decide that the list is all encompassing of her own preferences and move on to specifying attributes that fulfill these objectives. 27 Q ^ Minimize commute time Q Maximize a healthy environment # „ Maximize the comfort of the indoor living space # ^ Maximize the comfort of the outdoor living space $ # Maximize energy efficiency Q & Maximize proximity to nature $ # Maximize ability to entertain Q 4 Maximize safety Figure 8: Possible default Value Tree in the Real Estate domain A D M may also decide that the objectives in the default value tree only partially represents her preferences and can modify the ones that need to be changed. She may realize, while looking at the given value tree, that one or more of her objectives are missing and thus concentrate on identifying those. We argue that the default value tree reduces the mental effort of identifying a DM's objectives yet the interface allows her to modify and personalize the value tree to reflect her preferences if need be. However, a D M could also remove all the objectives and start from a blank value tree. The user interface in addition to enabling a D M to identify her objectives, allows the D M to specify attributes that fulfill these objectives. When any action to modify the value tree is taken, a message describing that action is provided to the D M by means of a pop-up window on the screen. 4.3.1. IDENTIFYING OBJECTIVES A D M may add objectives to the value tree by specifying the statement in the text box shown in Figure 9. For example, if one of the user's objectives was to MAXIMIZE STORAGE SPACE, then to add this objective the D M would type this statement in the text box and click the Add button shown in Figure 9. Add an objective: A d d » Figure 9: GUI Section for adding an objective to the value tree Once added, the value tree will look as shown in Figure 10. To modify an objective in the value tree, a D M must click on the pencil icon [ » ] directly left of the objective label. For example, if 28 the D M wanted to modify the objective MAXIMIZE THE COMFORT OF THE OUTDOOR LIVING SPACE then he must click on the pencil icon [ * ] next to this objective. When clicked, the objective label becomes an in-line editing text box with two options " O K " and "cancel". The D M may now input the new objective label, such as MAXIMIZE THE COMFORT OF LIVING SPACE in the in-line text box. Once the D M has completed modifying the objective label, he may click on " O K " to save the changes or "cancel" to reset the changes. © 4 Minimize commute time $ 4 Maximize a healthy environment O « Maximize the comfort of the indoor living space Q 4* Maximize the comfort of the outdoor living space Q 4? Maximize energy efficiency Q 4 Maximize proximity to nature © 4 Maximize ability to entertain Q 49 Maximize safety © 4 Maximize storage space Figure 10: Personalized Value Tree To remove an objective from the value tree the D M must click on the red-cross [ O] icon, which is to the left of the objective label. For example, i f the D M wishes to remove the objective MAXIMIZE THE COMFORT OF THE INDOOR LIVING SPACE, he must click on the red-cross [ O] icon left of this objective. This will pop up a message asking the D M to confirm the removal of the objective from the value tree. 4.3.2. MAPPING ATTRIBUTES INTO OBJECTIVES To specify attributes that fulfill objectives, a D M must select the attribute from the ontology on the right-hand side and drag the attribute under the objective. For example, the objective MINIMIZE COMMUTE TIME may be fulfilled by the attribute Close To Bus Terminal. The D M must select the attribute Close To Bus Terminal and drag the attribute on the highlighted box for MINIMIZE COMMUTE TIME, as shown in Figure 11. 29 . . . . . ® Close To Bus Terminal © i Minimize commute time Ci ., Maximize a healthy environment © Close To Park © _. Maximize the comfort of living space © .. Maximize energy efficiency © Has Air Conditioning is false © .. Maximize proximity to nature © Close To Beach © .. Maximize ability to entertain © „ Maximize safety © Close To Major Road Intersection is false © „ Maximize storage space ® dose To Facility - ® Close To Tteate © Close To Movie Theatre ® Close To Live Theatre - ® Close To Mercenttle Organization - ® Close To Retail Store ® Close To Drug Store ® Close To Grocery Store ® Close To Wholesale Store ® Close To Shopping Mall ® Close To Major Road Intersection ® Close To Parking Lot ® Close To Laundromat ® Close To Park ® Close To Beach ® Close To Tourist Attraction - ® Close To Transit Terminal ® Close To Airport ® Close To Ferry Terminal ® Close To Railway Terminal Figure 11: Adding an attribute to an objective If an attribute is not dragged under any objective, then the attribute is not added to any objective in the value tree. Once the attribute is added to the objective, the attribute shows up in green under the objective label as shown in Figure 12. The color green indicates that the value of the attribute should be true to fulfill the objective. For example, to fulfill the objective minimize commute time the D M would like to have an apartment close to the bus terminal. However, i f a D M wants to specify that an attribute satisfies an objective when its value is false, she has to add the attribute to the objective and then double click the attribute. For instance, to maximize energy efficiency, the D M would like the apartment to have no air conditioning. The D M must add the attribute Has Air Conditioning under the objective MAXIMIZE ENERGY EFFICIENCY then double clicks on the attribute. The attribute will change to Has Air Conditioning is false in red, specifying that when the apartment does not have air conditioning, the objective MAXIMIZE ENERGY EFFICIENCY is fulfilled. Note, the D M can add the same attributes to several objectives. For instance, the attribute Close to Park may fulfill the objectives MAXIMIZE A HEALTHY ENVIRONMENT, as well as MAXIMIZE PROXIMITY TO NATURE as shown in Figure 12. 30 © tt Minimize commute time © C l o s e T o B u s T e r m i n a l © it Maximize a healthy environment © C l o s e T o P a r k © o Maximize the comfort of the indoor living space © J Maximize the comfort of the outdoor living space © ., Maximize energy efficiency © Has Air Conditioning is false © ., Maximize proximity to nature © C l o s e T o B e a c h © C l o s e T o P a r k © „ Maximize ability to entertain © J Maximize safety © C l o s e T o M a j o r R o a d I n t e r s e c t i o n i s f a l s e © 4 Maximize storage space • ® Dose To Facility - $ ctese Tc ® Close To Movie Theatre ® Close To Live Theatre - « Close To Percentile Organization - <& ro Retail Store ® Close To Drug Store ® Close To Grocery Store ® Close T o Wholesale Store ® Close T o Shopping Mall ® Close T o Pubs ® Close T o Restaurants ® Close T o Major Road Intersection ® Close T o Parking Lot ® Close T o Laundromat ® Close T o Park ® Close T o Beach ® Close T o Tourist Attraction - <B Close To Transit Terminal ® Close To Airport ® Close To Ferry Terminal ® Close T o Railway Terminal - <£ Close To Bus Stop Figure 12: After adding an attribute to an objective A D M may also move an attribute from one objective to another by dragging the attribute label under the source objective to under the label of the destination objective. For example, the value tree shown in Figure 12 illustrates that the objective MAXIMIZE A HEALTHY ENVIRONMENT has the attribute Close To Park. If the D M decides that this attribute is more suitable to fulfill the objective MAXIMIZE PROXIMITY TO NATURE, then the D M can drag the attribute Close To Park from the objective MAXIMIZE A HEALTHY ENVIRONMENT to MAXIMIZE PROXIMITY TO NATURE. A D M may also decide at a later point that an attribute added under an objective, does not really fulfill the corresponding objective and can remove it by clicking on the [©] red cross icon to the left of the attribute label under that objective. For example, if a D M were to decide that being close to a beach did not really fulfill the objective of maximizing the proximity to nature due to the large crowds of people that gather during summer time, the D M would click on the red cross icon on the left of the attribute label Close To A Beach which would pop-up a message asking to confirm whether the D M really wishes to remove the attribute Close To A Beach from the objective MAXIMIZE PROXIMITY TO NATURE. One useful feature in the user interface is a means to easily locate the position of all attributes under an objective in the ontology by clicking on the objective label thus highlighting the objective and all the attributes under the objective in the Ontology to its right. For example, if the D M had a value tree as shown in Figure 13, the D M would click on the objective label MAXIMIZE A HEALTHY ENVIRONMENT to highlight the attributes Has Gymnasium, Has Swimming Pool, Close To Swimming Pool, Close To Gymnasium, Close To Baseball Field, 31 Close To Park, Close To Beach in the ontology. ,®.ClvSe.lQ.U9Undromat ;®.CJft5e.Io.RarkL, [® Close To Beachj "®" Close'ToTourist Attraction • ® Close To Transit Terminal ® Close To Airport ® Close To Ferry Terminal ® Close To Railway Terminal - ® ® Close To Bus Terminal i ® Close Ta Educational Organization - ® Close To Sports Ground r®.ClQSS.T«.S.<5Ecer.Rfi)d.., i® Close To Baseball Field! ®" Close To'Basketb'alf Court ® Close To Bank - •;;te.i«.aacte4aji|i'a t®. .ciose.T.o. Swim roJaqPpol • i® Close To Gymnasium: - ® ® Has Grounds ® Has Laundry Facility ® Has Library ® Has Park ® Has Parking Garage ® Has Parking Lot i® Hfls.fjyjnassiumL..., i® Has Swimming Pool; Figure 13: Attributes under objective highlighted in Ontology 4.4. VALUE TREE TO GAI NETWORK i© ., Maximize a healthy environment; i © Has Gymnasium ! © Has Swimming Pool • © Close To Swimming Pool I © Close To Gymnasium j © Close To Baseball Field • © Close To Park i © Close To Beach © 0 Maximize the comfort of living space © j> Maximize proximity to nature © „ Maximize ability to entertain © 4 Maximize safety © 4* Maximize storage space Once the D M is satisfied with her value tree, she would click on the search [K • ] icon. The value tree is converted into a GAI Network, where the objectives are factors and the attributes to fulfill each of the objectives are the attributes for each factor' *. The GAI network is created using a graphing library from Ruby called GRATR. For example, the value tree as illustrated in Figure 14 is converted into the GAI network shown in Figure 15 (this GAI network is not available for the D M but is visualized for the purposes of this thesis). * For more information, see Chapter 3. 32 I 1 7 4 : Minimize commute time a s ! : Close T o Grocery Store a-,-. Close T o Railway Terminal a 7,: Close T o Bus Terminal a„ : Close T o College I17,: Maximize a healthy environment a 7 ! : Close T o Park a l n C l o s e T o Baseball Field a|„ 7: Close T o Swimming Pool Am-. Close T o Gymnasium a,: Has Balcony I, 7 6:Maximize the comfort of the indoor living space a^,: Has Dishwasher a,7: Faces South a : : : Has Hard wood a, |: Has Kitchen a,: Has Balcony I,—: Maximize the comfort of the outdoor living space a,,.,: Has Park a , u : Has Grounds I 1 7 8 : Maximize energy efficiency a f f : Has Air Conditioning is false a,7: Faces South I,7»: Maximize proximity to nature a 7 4: Close T o Beach a 7 1: Close T o Park Iu„:Maximize ability to entertain a,,*: Has Swimming Pool a,: Has Balcony IiX,:Maximize safety a7„: Close T o Major Road Intersection is false a J 7:Has Security Alarm The library takes in an array of attributes for each factor and creates the edges based on the common attributes. For example, for the objectives MAXIMIZE T H E COMFORT OF INDOOR LIVING SPACE and MAXIMIZE ENERGY EFFICIENCY from the Value Tree illustrated in Figure 14, the common attribute Faces South causes an edge to be created between the nodes Factor 178 and Factor 176 in Figure 15. Factors that have no common attributes show up as independent nodes such as Factor 174, Factor 177 and Factor 181. 33 Figure 15: Sample Personalized GAI network 4.5. MEASURING THE QUALITY OF THE GAI NETWORK Once a D M expresses her value tree, and a GAI network is generated, a set of questions are created to validate how well the GAI network represents the DM's true preferences. We are interested in measuring the degree of independence between attributes within objectives in the value tree. To measure this independence value we propose to the D M a comparison gamble question as generated from the algorithm below. 4.5.1. COMPARISON GAMBLE The comparison gamble consists of paired comparisons in which the D M must choose between two alternatives (See Section 2.3.2). . 34 f o r d i s t a n c e = D M I N to D M A X Randomly ( a v o i d i n g p o s s i b l e reuse'of a t t r i b u t e s ) p i c k two a t t r i b u t e s A K , A j such as d i s t a n c e ( f a c t o r ( A K ) , , f a c t o r (Aj.) ) = d i s t a n c e Propose the comparison gamble Option A ) 50* ( A K - b e s t , A j - b e s t ) : 50 ( A K-wbrst, A j -worst) Option B) 50 ( A K - b e s t , A j - w o r s t ) : 50 ( A K-worst, A j-best) We assume that D M A X is the maximum possible distance between two attributes in the GAI Network, so for the GAI graph in Figure 15, D M A X is 3. We assume that D M I N is the minimum possible distance between two attributes in the GAI Network. In cases where the graph is fully connected and has no independent factors, D M I N is 0: the shortest distance between two attributes is when they belong in the same factor. Whereas in a GAI graph, as shown in Figure 15 D M I N is -1, due to the presence of non-connected factors. Given a pair of attributes (AK,AJ) in factors Ii and I2, the distance between the attributes in AK and Aj is measured as the smallest number of the edges it takes to traverse from factor Ii to factor I2. For example, distance between attribute 37 and attribute 73 in Figure 15 is 2, not 3 even though there exists a path from two factors which have attributes 37 and 73 where the distance would be 3. Independent factors (factors that are not connected to any other factors) are a special case; with distance from all other factors being equal to -If. For example, distance from attributes 70 and 47 in Factor 174 to any other attributes are -1. The algorithm iterates from D M I N to D M A X , and searches through the graph for a pair of attributes where the shortest distances between them equal distance, then proposes the comparison gamble question as a paired comparison between two alternatives. In our case, the two alternatives are a lottery between two possible outcomes. One alternative, option A, initially provides the D M with the following outcomes: a 50% chance of being in a situation where the D M will receive the best of both attributes and a 50% chance of being in a situation where the D M will receive the worst of both attributes. Consequently in the next alternative, option B, the same outcomes are always provided for the relevant question. A 50% chance of being in a situation where the D M will receive the best of attribute Aj and the worst of * This probability value is referred to as p in the comparison gambles described below. t -1 was used to denote no path. It was a design decision made for simplifying implementation. 35 attribute AK and a 50% chance of being in a situation where the D M will receive the best of attribute AK and the worst of attribute Aj. If the D M states she is indifferent between options A and B when thep probability is 50, the attributes in the pair are consideredindependent. However, if the D M is not indifferent we keep asking questions to achieve a narrower dependency estimate by adjusting the probability/) in the comparison gamble. For instance, if the D M prefers B (the attributes are substitutes), we make A more desirable by increasing the probability atp to the midpoint of [50-100] which is 75. Now the D M may prefer A, therefore to get closer to identifying the indifference the probability at p is decreased to the midpoint of [75-100] which is 63. If the D M is not indifferent, depending on the choice, we will know that the indifference is between either 50 and 63 or 63 and 75. The D M is asked the same question with a different probability ax.p until she claims to be indifferent or the difference between the maximum and the minimum probabilities in the range is 1. We claim that our estimate for indifference between attributes in a pair corresponds to an estimate for dependency between the attributes in the pair (the further the probability/) is from 0.5 the more the two attributes are dependent)*. We validate the GAI network based on the observed dependence values of the attributes in the pair of each comparison gamble question proposed. 4.5.2. VALIDATING THE GAI NETWORK A comparison gamble question asked in the user interface's validation part is shown in Figure 16. Option A is a choice with two outcomes: either a good outcome (for e.g., presence of dishwasher and facing south) or a bad outcome (i.e. absence of dishwasher and not facing south). * See Section 3.1 for more details. 36 Please choose one of the 2 options or "I'm Indifferent" if you are indifferent between Option A and Option B. Option A Option B You have a You have a B 50% chance of getting an apartment that faces I I south and has dishwasher and a and B 50% chance of getting,an apartment that does I I not face south and has no dishwasher. Which option do you prefer? I QptJonA"! 1 Option B | I I'm lndifferenT| If you would like to answer this question again, please dick —> here. If you would like to skip this question and answer another one, please dick —> here. 50% chance of getting an apartment that faces south and has no dishwasher 50% chance of getting an apartment that does not face south and has dishwasher. Figure 16: Sample Comparison gamble Question Please choose one of the 2 options or "I'm Indifferent" if you are indifferent between Option A and Option B. Option A Option B You have a You have a U 25% chance pf getting an apartment that faces I I south and has dishwasher M and a and a B 75% chance of getting an apartment that does I I not face south and has no dishwasher. Which option do you prefer? I Option A | I Option B | ! I'm Indifferent | If you would like to answer this question again, please click —> here. If you would like to skip this question and answer another one, please dick —> here. 50% chance of getting an apartment that faces south and has no dishwasher 50% chance of getting an apartment that does not face south and has dishwasher. Figure 17: First time Option A chosen for Comparison gamble Question Option B is an intermediate in desirability between the good and bad outcomes of Option A, i.e. presence of dishwasher and absence of a south facing apartment and vice-versa. DMs are asked to choose between the gambles Option A and Option B or whether they are indifferent given the current set of circumstances. If the D M chooses Option A, the probability of the good outcome in Option A decreases to make Option B more worthwhile as shown in Figure 17, and vice versa. This probability is varied until the D M is indifferent between the two alternatives. The D M can express indifference in two ways, by choosing the "I'm Indifferent" button or by choosing the options until the difference between the two probabilities in Option A is 1. If a D M makes a mistake, she can click the link to answer the question again. The D M may also skip a question for whatever reason if they are uncomfortable with the question or choose not to answer it by clicking on the link for skipping that question. 37 5.1. GOAL Chapters. EXPERIMENT We performed a series of experimental studies to test our approach for eliciting a D M ' s G A I network. W e wil l first discuss the set of hypotheses which we are testing. Then we wil l describe in detail the experimental study we ran to test our approach; this will include a discussion of the methodology employed, the participants involved, the procedures carried out, the data collected and analyzed. In conclusion, the results are discussed and some future work is proposed. 5.2. HYPOTHESES We have two major hypotheses and each of these hypotheses requires several sub-hypotheses. They are detailed below: Hypothesis 1 O n preference Elicitation: It is possible to elicit the structure of a sound G A I network by having a minimally trained D M 1) create a value tree by following Value Focused Thinking and 2) map the domain attributes from an ontology into the fundamental objectives of the value tree (that are fulfilled) by using the interface developed. Hypothesis 1.1 A D M would understand the concepts of Value-Focused thinking from minimal training. (10 minutes*) Hypothesis 1.2 A D M would understand the concepts of the hierarchical nature of the , domain represented as an ontology from minimal training. (10 minutes*) Hypothesis 1.3 The default value tree would help the D M in building her personalized value tree. Hypothesis 1.4 A D M is capable of building her personalized value tree by identifying fundamental objectives autonomously. Hypothesis 1.5 The G A I network obtained from the mapping is sound. Hypothesis 1.5.1 Attributes in the same factor are additively dependent while attributes in disconnected factors are additively independent. * See Appendix E . 38 Hypothesis 1.5.2 The degree of dependency between pairs of attributes decreases proportionally to the distance d between the pairs of the attributes, where given two attributes (a,b) in the pair, distance d between the attributes denotes the shortest number of edges spanned between the containing factors of a and b. Hypothesis 2 O n verification of the output of the elicitation procedure: It is possible to measure the quality of the obtained G A I network by asking a (minimally trained) D M to answer comparison gamble questions comparing two lotteries. Hypothesis 2.1 A D M understands the comparison gamble questions comparing the two lotteries given minimal training (10 minutes). Hypothesis 2.2 A D M can answer many comparison gamble questions in sequence without losing performance/accuracy/consistency in her answers. 5.3. METHODOLOGY We have designed a user study to verify Hypothesis 1, the key hypothesis on which our preference elicitation approach relies. However, we needed to perform a pre-study to validate the quality of the default value tree, which was needed for the user-study. The methodologies for both studies are described below. We call the pre-study, So and the actual user study Si. 5.3.1. PRE-STUDY So A study, So, with twenty-eight participants, was conducted to validate the quality of the default value tree. This was necessary so that participants for study Si could begin with a value-tree that was general enough that it would not require too-much thought when the participants begin. In So, participants were sent a survey through email to answer the following question: ... Suppose you were searching for an apartment. You want the place that you are searchingfor to fulfill certain goals and objectives you have, such as, you want to reduce the amount of time it takesfor you to commute to either work/schoolfrom home - so you want to "minimize commute time". ... If you could please take a look at thefollowing list and let me know whether something should be removed (which number) something should be added (which number) the wording needs to be changed so that it is clear immediately what is being talked about (which number) ... The goal is to find out whether I've been able to capture the majority of things people 39 care about when searching/or a new place... This is the list of fundamental objectives: Minimize commute time Maximize a healthy environment Maximize the comfort of the indoor living space Maximize the comfort of the outdoor living space Maximize energy efficiency Maximize proximity to nature Maximize ability to entertain Maximize safety Based on their answers, the most popular remarks and the most general objectives were compiled to form a default value tree as shown in Figure 8*. 5.3.2. USERSTUDYSI In S i , the goal of the study was to evaluate the effectiveness of our approach in eliciting a DM 's preferences expressed as a G A I network. Two pilot studies were performed to validate the format of the user study and the final user study was conducted using ten participants. The participants for the pilots were not remunerated and were friends of the author. The participants for the study S i included graduate students, two females and all between the ages of 20 - 40. The procedure for the study S i is detailed belowf. 5.4. PROCEDURES Each participant was asked to sign a consent formf. After that, she was asked to fill out a brief questionnaire where demographic information such as age, gender etc were collected§. 5.4.1. TRAINING Participants needed training before the tasks in the user study could be performed. The training comprised of two parts. Training Part 1 began with participants being given a brief * See Section 4.3. t See Appendix G . f See Appendix H . § See Appendix I. , 40 introduction of the research question and relevant concepts such as Value Tree, Objectives, Attributes and Ontology, which were explained in the first person using a sample domain for office desks t (these exact terms and phrases were used). When objectives are organized hierarchically, they form a value tree. On the left side is a value tree. I have been provided with a value tree to begin with. lean personalize this value tree to reflect my preferences. My objectives can be fulfilled by things in the real world, i.e. things such as 'made of wood', 'wide keyboard drawer' etc. We call these elements in the real world, attributes. . Most often attributes belong to a class that is a set of objects, known as entities such as 'desk', 'drawer'etc. These entities and attributes often have a hierarchical representation in the real world - we call this domain ontology. Each participant was then asked to train on the desk domain for each of the following actions* • Adding an objective • Modifying an objective • Removing an objective • Adding an attribute to an objective • Changing the attribute value under an objective from true to false • Moving an attribute from one objective to another • Removing an attribute from an objective Once the participants completed this set of actions, Training Part 1 concluded. For Training Part 2f, each participant was given a brief introduction to the concept of a standard gamble question: You will be asked to determine probabilities that make you indifferent between pairs of outcomes. Outcomes can be certain or uncertain. In addition, they can be gains or losses.... Here is a sample question. Please determine the value ofp that would make you indifferent between OPTION A and OPTION B. OPTION A: to enter a lottery in which you have a p chance of winning afree coffee and a muffin and a(l-p) chance of winning nothing. * For more details on each action, see Chapter 4. t See Appendix J. 41 OPTIONB: to win a free coffee. Please write down all the values ofp that you consider and your corresponding preferred outcome. p 1-P I prefer ... 0 1 OPTION B 1 0 OPTION A 0.5 0.5 It was explained to DMs, when one preferred muffins and coffee, given a probability of 0 for p, i.e. winning nothing, the rational D M would choose Option B; although, a probability of winning both muffin and coffee, i.e. p is 1, causes a rational D M to choose Option A. However, when the probability of winning a free coffee and a muffin is 0.5, what option would the D M choose? When asked this question, participants proceeded to answer the simple standard gamble over a lottery and a sure outcome. When a participant would choose Option B, the p value would be increased to make Option A more favorable; however, when Option A was chosen the p value was decreased. The participants were given increments/decrements of 0.1 at each step of the process. When a participant would change their choice from the previous answer, the process would be concluded and the p value would be recorded as the arithmetic mean between the last p value and the second-to-last p value. This p value was the probability at which the participant was indifferent between the two options. Following this step, participants were asked to answer a set of three standard gamble questions (comparing a lottery and a sure outcome), followed by a set of three comparison gamble questions (comparing two lotteries) in the office desk domain on paper. The simple gamble questions had the following format: For which value ofp would you be indifferent between OPTION A and OPTION B? Again, please write down in the table all the values,ofp that you consider and your corresponding preferred outcome. p 1-P I prefer... 0.5 0.5 42 OPTION A 1-p Desk with 3 drawers and 3 shelves © i no drawers or OPTION B Desk with 1 drawer and 1 shelf© p that makes you indifferent _ Participants were given a simple visualization of the standard gamble in the form of a table where the first column represented probability, the second and third column represented the two alternatives. The rows of the table usually represented a lottery in the alternative. The participants were asked to answer these questions on their own, however assistance was provided when asked and answers were verbally confirmed to represent what the participant was trying to express. Therefore, for the above question, if a participant's indifference probability was at 0.75, it could be that the participant preferred some storage space rather than gamble and end up with none. The comparison gamble questions followed the following format: Let us focus on the desk domain. You are searchingfor a new desk. You are trying, to come up with the best balancefor your storage needs. Assume that a large desk is best suitablefor your needs, a medium desk will be a tradeoff and a small desk is going to be difficult to work with. The grid on the left hand side now has an extra column for OPTION B. The percentages for OPTION B, however, are always the same, 50%. Consider two aspects of a desk. One is the number of shelves andthe other is the size ofthe desk. For which value ofp would you be indifferent between OPTION A and OPTION B? Again, please write down in the table all the values ofp that you consider and your corresponding preferred outcome. P 1-p I prefer... 0.5 0.5 43. O P T I O N A O P T I O N B P Large desk with 3 shelves © \ "I Small desk with 3 shelves 1-p Small desk with no shelves © 1 I Large desk with no shelves r — i p that makes you indifferent Once again, the participants were asked to answer these questions on their own and answers were verbally confirmed to make sure that participants were comfortable with the format of the questions and understood what it meant when particular options were preferred. Once the participant had completed Training Part 2, she was introduced to the comparison gamble feature of the user-interface. This procedure is detailed in Section 4.5.2. 5.4.2. TASK Having completed both training parts, each participant was provided with the following scenario: _. '. Suppose, you are relocating to Vancouver, British Columbia Canada. You have preferences when it comes to renting an apartment. You are given a fixed budget of $1000for rent. For the purposes ofthe study, you do not need to consider minimizing your costs as an objective. Participants were then asked to create their own value tree in the domain of apartments. After having completed this part, they were asked to proceed to the validation part of the interface. The validation of the participant's generated G A I network based on the Value tree created in Part 1 took a maximum of fifteen minutes. D M s were asked to answer comparison gamble questions starting with an attribute pair that were from factors that were not connected, to pairs of attributes within the same factor and then increased the distance between the pairs of attributes to the maximum possible, after which this cycle repeated until the maximum amount of time had been reached. D M s were allowed to skip any question they felt too uncomfortable to answer or did not understand its phrasing. Once they had completed both parts, the study was concluded. Participants were finally provided with a questionnaire to express any 44 suggestions, comments or questions they may have*. The study was concluded by asking participants to sign a receipt for the payment of remuneration. 5.5. DATA Each participant's value tree, generated GAI network and scatter plots based on the answers obtained from the questions asked in Part 2 of the study are present in Appendix A. Appendix B. and Appendix C. 5.5.1. COLLECTION Each participant's scatter plot shows the relationship between the distance connecting a pair of attributes A . and Aj in the GAI network and the measure of dependence between them. The dependence between a pair of attributes A and Aj is determined by the probability p, where 0 < p < 1, that makes the D M indifferent between the two lotteries in the comparison gamble questions as described in Section 4.5.1. If p was 0.5, it indicates that the pair of attributes is additively independent. If 0.5 < p < 1, the pair of attributes are dependent as substitutes, and if 0 < p < 0.5 the attributes are complements. The dependence in the comparison gambles was normalized to a number between 0 and 1, which consequently removed the information of the dependence classification, i.e. whether they were substitutes or complements. The observed dependencies between each pair of attributes were then normalized into the measure of dependenceMD = \0D — 0.50|/0.50, where M D is the measure of dependence and O D is the observed dependenceThe analysis of this data is discussed in more detail in the following section. 5.5.2. ANALYSIS The measure of dependence between attribute pairs (obtained in the validation phase) and the distance between attribute pairs in the GAI network was plotted as shown in Figure 18. Figure 18 also shows the interpolation-line for all the observed points in the scatter plot. The size of the dots on the plot represents the number of observations made for that measure of dependence at that distance. Based on the analysis of the data, we had to remove one * See Appendix K . 45 participant from our results and analysis as it was clear from the value-tree and the GAI network generated that the participant did not grasp the task. In total, ten participants' data has been analyzed and discussed below. 41 3 ja i.ooH 0.80 H ra c-§.oso-o i ra a i >-K OI < S .E 0.40-c O) TJ C OI a. O) 0.20 H o O) ra g o.oo H 130 Bin Counts • 15 • 10 • 5 • 0 i 25 20 same factor 1 . 2 3 4 no path D i s t a n c e b e t w e e n pa i r s of a t t r ibutes (a,b) in GAI G r a p h = d(a,b) Figure 18: Scatter plot of all observed pairs of dependency between attribute pairs and distance between attribute pairs Table 1 shows the same information presented in Figure 18, aggregating the ranges of the measure of dependence in a tabular format, which allows for precise reading. Please note there was only one participant (User 10*) who had answered one question where the distance between the pairs of attributes was 4. * See User 11 at Appendix A. for further details 46 Distance between pairs of attributes (a,b) In GAI Graph = d(a,b) Total same factor 1 2 3 4 no path Range of Measure of Dependence between pair of attributes (a,b) in GAI Graph 0.8 -1.0 Number of Observations 15 2 2 0 0 5 24 % within Distance between pairs of attributes (a,b) in GAI Graph = d(a,b) 17.90% 14.30% 16.70% 0.00% 0.00% 5.60% 11.50% 0.6 - 0.8 Number of Observations 12 1 2 0 0 7 22 % within Distance between pairs of attributes (a,b) in GAI Graph = d(a,b) 14.30% 7.10% 16.70% .0.00% 0.00% 7.80% 10.60% 0.4-0.6 Number of Observations 10 5 3 2 0 13 33 % within Distance between pairs of attributes (a,b) in GAI Graph = d(a,b) 11.90% 35.70% 25.00% 28.60% 0.00% 14.40% 15.90% 0.2 - 0.4 Number of Observations 24 3 2 3 1 23 56 % within Distance between pairs of attributes (a,b) in GAI Graph = d(a,b) 28.60% 21.40% 16.70% 42.90% 100.00% 25.60% 26.90% 0.0 - 0.2 Number of Observations 23 3 3 2 0 42 73 % within Distance between pairs of . attributes (a,b) in GAI Graph = d(a,b) 27.40% 21.40% 25.00% 28.60% 0.00% 46.70% 35.10% Total Number of Observations 84 14 12 7 1 90 , 208 Table 1: Total number of observations at each d(a,b) for a range of measure of dependence . After further analysis of the data, the observations were grouped into two categories, which represented the nature of the GAI network generated from the value tree: disconnected GAI graphs and connected GAI graphs. Five participants had disconnected graphs*, i.e. graphs with factors that have no edges between them as shown in Figure 19. f Factor 25 ^ L / ' Factor.32 \ V 79,89,78. J ' V 5,55,56,76, J / Factor 34 \ f Factor 30 \ I 15,16,17,18,19, J V 59,67,82,74, y V 23,24,30,34, / ' Factor 33 \ 40,41,44,45i48, 1 y** 50,52,53,49,112, J V. 63, / f Factor 27 \ \7,6,11,12,37^/ Figure 19: GAI Network of User 1, exemplifying a disconnected GAI Graph * User 1, User 3, User 5, User 6, User 7 and User 9 [See Appendix B. and Appendix C. for further details] 47 The other five participants had connected graphs*, where there were a minimum of seven factors and at least one edge between two factors were present, as shown in Figure 20. f Factor 49 \ V 41,37;33, J f Factor.42 \ V 57,73,74, J ^Factor 4 8 ^ ^Factor.^ TX V 79,89. J Figure 20: GAI Network of User 2, exemplifying a connected GAI Graph JO 19 3 JB I 14-o U l 0.60 A <u ra a i * -K 41 < ja o S .£ 0.40-c 41 " O c 4* C L 41 o 41 ra 41 0.20 H 0.00 H 1.00- • Bin Counts • • 15 io • • • 5 0.80- • • £ 20 0 T" T T same factor 1 2 3 4 no path Distance between pairs of attributes (a,b) in GAI Graph = d(a,b) Figure 21: Scatter Plot clustering ofall observed pairs of dependency between attribute pairs and distance between attribute pairs on disconnected GAI networks * User 2, User 4, User 8, User 10 and User 11 [See Appendix B. and Appendix C. for further details] 48 JI 1.00-1 ti •S 1 c a. t> « oi *s ai < A O at c c 01 " O c 4) a. oi XI I*-o w 3 (A 19 4> 0.60-0.40 -4 ooo H Bin Counts >20 • 5 • 15 . ' 0 I 10 same factor no path D i s t a n c e b e t w e e n pa i r s o f a t t r ibutes (a,b) in GAI G r a p h = d(a,b) Figure 22: Scatter Plot clustering ofall observed pairs ofdependency between attribute pairs and distance between attribute pairs on connected GAI networks Figure 21 shows a scatter plot of the measure of dependence between attribute pairs and the distance between attribute pairs for all disconnected GAI graphs. It also shows the interpolation-line for all the observed points in that case. Figure 22 shows a scatter plot of the measure of dependence between attribute pairs and the distance between attribute pairs for all connected GAI graphs also showing the interpolation-line for all the observed points in that case. Based on these figures, we notice that DM's with disconnected GAI nets followed the trend we predicted in Hypothesis 1.5.2; however, connected GAI nets did not follow any pattern. The observations were grouped into another category, which represents the nature of the personalized value tree with respect to the default value tree. If the personalized value tree had all the objectives as the default value tree and no new ones were added or default ones removed or modified it was classified as an unchanged default value tree, otherwise the value tree was classified as changed default value tree. Table 2 shows the frequency of participants whose personalized value tree was either changed or unchanged. There was no significant relationship among the distance between pairs of attributes and the measure of dependence between them. 49 Nature of Personalized value tree Frequency Percentage Changed Default Value Tree 7 70.00% Unchanged Default Value Tree 3 30.00% 10 100.00% Table 2: Frequency of participants with state of personalized value tree 5.6. DISCUSSION Let us now discuss the results in terms of how they provide evidence for confirming or invalidating each hypothesis stated in Section 5.2. Hypothesis 1 On preference Elicitation: It is possible to elicit the structure of a sound GAI network by having a minimally trained D M 1) create a value tree by following Value Focused Thinking and 2) map the domain attributes from an ontology into the fundamental objectives of the value tree (that are fulfilled) by using the interface developed". Hypothesis 1 is not verified by the above results: we discuss each of its sub-hypotheses for deeper insight into this claim. Hypothesis 1.1 claims, "A D M would understand the concepts of Value-Focused thinking from minimal training. (10 minutes)". Every participant was given a very brief introduction to the concept of a Value Tree, Objectives and Attributes that fulfill objectives. It was also stressed to each participant that the default value tree provided did not necessarily reflect her objectives and was encouraged to add, remove and modify objectives from the value tree as she saw fit. Each participant was prompted to ask questions regarding any concept that they felt uncomfortable about during training. However, based on the design of the study and the results obtained it cannot be ascertained whether participants understood the concepts of Value Focused Thinking. A compiled list of all the comments made regarding the value tree is shown below: Quite clear and intuitive (User 2) The tree covers pretty much what I need when I searchfor an apartment (User 9) Well Selected Objectives for the Value Tree (User 5) Good Objectives but a strong tendency to use the default value tree (User 1) Perhaps provide verbal suggestions instead (User 1) Perhaps a couple more default categories to get me thinking (too few makes me have to think about groupings) (User 6) 50 / did not use many of the default Value tree concepts. I wasn't thinking in terms of maximization as much as fulfilling different requirements of like the apartment will be used for parties, work, relaxation, cooking etc. (User 7) Some general observations based on these comments are that some participants (2, 9, 5) were comfortable with the value tree and some (1,6) were not. Table 2 shows that 70% participants' personalized value tree was different from the default value tree. This may indicate that most participants' personalized value tree was not well reflected in the default value tree; however, this may also indicate understanding of the concepts explained about Value Focused Thinking during training and thus personalization was easily done to reflect the participants' preferences. Hypothesis 1.2 states, "A D M would understand the concepts of the hierarchical nature of the domain represented as an ontology from minimal training. (10 minutes*)" This hypothesis is confirmed. A l l participants showed a clear understanding of the concepts of domain ontology, entities and attributes introduced during training. There were no questions asked and no negative comments obtained during the study regarding the hierarchical structure of an ontology. Comments from participants after the completion of the study regarding the ontology are as follows: Clear and intuitive (User 2) Needs more options and be broader (User 9) Odd to have to add all the leaves when really, suppose, all I wanted was a window (not caring which way it faced) (User 1) Do changes in the ontology affect a user's value tree? (User 2) Maybe the ontology should have other types of attributes besides boolean (i.e. true and false) such as quantity or quality / thresholds (User 3) Felt incomplete, such as no "bus stop "just "bus terminal" etc (User 5) Some concepts such as doors and sinks are essential yet others are optional such as "close to park"; difficult to combine both. (User 7) Again based on these comments some observations were made. Based on comment 1, it is clear that participant 1 was clearly able to grasp the concept of the hierarchical nature of the real world domain of apartments. Participant 1, however, wanted less entities and attributes to reduce the clutter and confusion present when first introduced to the ontology about apartments, whereas participants 3, 5 and 9 were less than thrilled with the limitations present 51 in the ontology regarding the depth and the type of attributes present. The interface does not allow for any attributes that are not boolean, for example a quantity or a quality threshold etc such as number of bedrooms or quality of neighborhood. Participant 3 indicated that he was unable to express his preferences fully because of this missing feature. Absence of certain attributes in the ontology was another complaint that reflected itself in the usability of the interface for expressing preferences by participant 5. Participant 7 also mentioned a means to classify the significance of the attributes to a D M so that essential attributes such as Has Sink could be identified sooner over non-essential attributes such as Close To Park. Participant 5 mentioned, while performing the tasks for the study that the concept of a hierarchy was strange for the value tree since the interface allowed one level for the objectives in the value tree. He was observed trying to create an objective beneath another and when asked what he was trying to do, the participant stated he would like to organize his objectives hierarchically. Hypothesis 1.3 claims, "The default value tree would help the D M in building her personalized value tree." As seen from the above comments regarding the value tree, some participants (1,2, 4, 5,7, 8 andlO) personalized their default value tree whereas participants 3, 6, and 9 were comfortable with the default value tree enough to express their preferences without having changed them. For instance, participant 1 may have grasped the concepts of objectives but was challenged in isolating his own value tree from the default value tree (as can be seen from the comment above) even though he did make modifications to his value tree. It is unclear whether D M s can identify their fundamental objectives without prompts from decision analysts and perform the task autonomously and thus Hypothesis 1.3 cannot be confirmed. Hypothesis 1.4 states, " A D M is capable of building her personalized value tree by identifying fundamental objectives autonomously." Our study was not designed to provide clear evidence for either confirming or invalidating this hypothesis. W-e have obtained full value trees from each user but cannot conclude that these value-trees are structured so that they do not violate any independence assumptions. Hypothesis 1.5 assumes "The G A I network obtained from the mapping is sound." To confirm or invalidate this hypothesis we need to delve a little bit deeper into its two sub-hypotheses. Hypothesis 1.5.1 states, "Attributes in the same factor are additively dependent while attributes in disconnected factors are additively independent." This hypothesis is based on G A I network literature [Braziunas, et al. 2005, Bacchus, et al. 1995]. The cluster of observations under the 52 same factor variable is hypothesized to be additively dependent, while those under the no path variable are hypothesized to be additively independent. A T-test was run on these two clusters and the significance obtained was 0.02. This hypothesis can thus be confirmed. Hypothesis 1.5.2 states "The degree of dependency between pairs of attributes decreases proportionally to the distance d between the pairs of the attributes, where given two attributes (a,b) in the pair, distance d between the attributes denotes the shortest number of edges spanned between the containing factors of a and b.". This hypothesis is not verified, as is clearly illustrated in Figure 18. The function increases and then reaches a peak at distance 2 and then decreases sharply to distance 3 after which it levels out. This does not represent any pattern theorized, in decision theory or graph theory literature. Since not all sub-hypotheses for Hypothesis 1 were confirmed or invalidated, further study and research is needed for verification of Hypothesis 1. Hypothesis 2 On verification of the output of the elicitation procedure: It is possible to measure the quality of the obtained GAI network by asking a (minimally trained) D M to answer comparison gamble questions comparing two lotteries." This hypothesis claims we can measure how sound a GAI network is based on the DM's answer about comparison gamble questions of attribute pairs. This hypothesis can be investigated in two-fold. Hypothesis 2.1 claims, "A D M understands the comparison gamble questions comparing the two lotteries given minimal training (10 minutes)." Once participants received some training about the interface, they were given some constructive training about the structure of a standard gamble question comparing a lottery and a sure outcome and a comparison gamble question comparing two lotteries. A sample domain of desks was employed for this purpose. A cursory inspection of the answers was verbally confirmed from all participants when each section was completed. However, some participants (1,7) stated they had difficulty in answering some of the questions while performing the study. Structure of questions during validation was difficult. Would there be any other way of posing them? (User 1) Hard part is answering comparison gamble queries especially when one item is mandatory and one is superfluous such as 'sink' and 'near a park' (User 7) This can be taken in two ways. The participants understood the format of a comparison gamble question; however, questions that were proposed were challenging to answer. This is highly likely as can be seen from the comment above. Participants understood the consequence of 53 answering such questions but were unaware of their true preferences in these regards. Alternatively, the participants did not understand the format of comparison gamble questions and were unable to answer them correctly. There is some evidence that this hypothesis is not valid. Hypothesis 2.2 asserts, "A D M can answer many comparison gamble questions in sequence without losing performance/accuracy/consistency in her answers." The design of the experiment hindered us when trying to confirm or invalidate this hypothesis. This analysis indicates that both hypotheses require further research. We may need to design specific experiments to test each hypothesis separately. 54 Chapter 6. CONCLUSIONS AND FUTURE WORK This thesis explored a means of eliciting a DM's preferences in terms of a GAI decomposition. We delved into utility theory and the inherent structure of preferences; mentioned the independence assumptions made when preferences are structured; and described the background behind our decision in choosing the GAI decomposition to represent a DM's preferences. We further explained our motivation behind building a GAI network and the absence of methods for the elicitation of the GAI network structure. We proposed value-focused thinking as a viable approach to this problem and suggested using domain ontologies, which structure the vocabulary used to describe the world, in conjunction. We experimentally evaluated a framework to support decision-making according to both a DM's objectives and a complex preference profile over a domain ontology in which the decision is contextualized. Our study reported that significant work in this area needs done and we suggest some viable options to consider that we were unable to do due to constraints in time and feasibility. We have identified several aspects in which further research and study needs done to refine our approach and its evaluation. The ontology presented to the DMs was trimmed out of all attributes that were not boolean. Further studies where attributes of all types are present may yield different results, as a D M will not feel limited in expressing her preferences. We believe that using value focused thinking to elicit a DM's preferences should be explored further. DMs were only asked to consider their fundamental objectives when creating their value tree; however, if the value tree was more complete in terms of means and fundamental objectives, it is likely that the DMs would better understand the relevant concepts and be better able to express their preferences. Perhaps, the attainment of a default value-tree can be done using a more thorough study. Some DMs suggested a longer value tree would be more useful to express their preferences, as they would find it easier to revise a large value tree than expanding a small one. However, one D M also suggested a list of fundamental objectives provided was distracting and caused him to think less about his own needs and goals when it came to searching for an apartment. It seems a possible solution that should be explored is to allow the D M to choose whether to start from an empty Value Tree, a small default one or a large default one. 55 We also believe providing the D M s with some guidance when eliciting a personalized value tree is necessary. This can be done by asking natural language questions based on the actual statement provided for a fundamental objective or while attributes are being added to the fundamental objectives. Perhaps after the validation part of the study, D M s could be given a rating of how well their answers compared to their value tree and give them an opportunity to re-arrange the value tree, or provide intelligent suggestions based on the answers provided as to what the predicted value tree looks like. During elicitation of the generated GAI network, there could be an added step included where the objectives that make up the value tree and the attributes that fulfill these objectives are verified, giving D M s a chance to revise their value tree by providing an alternate means of visualizing the value tree. For example if a D M adds an objective MINIMIZE NOISE POLLUTION and places attributes such as Close To Major Road Intersection is false, Has Garbage Disposal and Close to Grocery Store under this objective, a fulfillment of this objective by means of these attributes could be assured. A possible solution may involve asking natural language questions where the D M is asked whether an attribute placed under an objective truly fulfills the DM's goal. Therefore, dependent attributes are left alone while attributes in dispute are identified as not withholding the DM's preferences. The D M is given intelligent suggestions to modify the value tree accordingly, possibly improving elicitation. We believe supporting the integration of a DM's value with a domain ontology for eliciting a GAI network structure for the DM's preferences is an approach worth further research and consideration. 56 BIBLIOGRAPHY FISHBURN, P.C. 1970. Utility theory for decision making. Wiley, New York. KEENEY, R. 1976. Decisions with Multiple Objectives. Cambridge University Press, Cambridge. BRAZIUNAS, D. A N D BOUTILIER, C. 2005. Local Utility Elicitation in GAI Models. In Proceedings of the 21th Annual Conference on Uncertainty in Artificial Intelligence (UAI-05), Anonymous A U A I Press, Arlington, Virginia, 42. GONZALES, C. A N D PERNY, P. 2004. GAI Networks for Utility Elicitation. In Principles of Knowledge Representation and Reasoning, Whistler, British Columbia, Canada. KEENEY, R. 1996. Value-Focused Thinking. Harvard University Press, Cambridge. BERNOULLI, D. 1954. Exposition of a New Theory on the Measurement of Risk. Econometrica 22. V O N N E U M A N N , J. A N D M O R G A N S T E R N , 0.1944. Theory of Games and Economic Behavior. Princeton University Press, Princeton. FISHBURN, P.C. 1981. Subjective expected utility: A review of normative theories. Theory and Decision. H O R V I T Z , . 1988. Decision theory in expert systems and artificial intelligence. International journal of approximate reasoning. B A C C H U S , F. A N D GROVE, A. 1995. Graphical Models for Preference and Utility. In Uncertainty in Artificial Intelligence. Proceedings of the Eleventh Conference , Anonymous Morgan Kaufmann Publishers, San Francisco. IEEE STANDARD UPPER O N T O L O G Y W O R K I N G GROUP, 2005. Suggested Upper Merged Ontology (SUMO), http://ontology.teknowledge.com/. NILES, I. A N D PEASE, A. 2001. Towards a standard upper ontology. In FOIS '01: Proceedings of the international conference on Formal Ontology in Information Systems, Ogunquit, Maine, USA, Anonymous A C M Press, New York, NY, USA. HANSSON, D .H . 2004. Ruby on Rails, www.rubyonrails.org. 57 GARRETT, J.J. 2005. Ajax: A New Approach to Web Applications. http://www.adaptivepath.com/ideas/essays/archives/000385.php. HELSINKI UNIVERSITY OF T E C H N O L O G Y [SYSTEMS ANALYSIS LABORATORY].2005. Value Tree Analysis Theory, http://www.mcda.hut.fi/value_tree/theory/. 58 Appendix A. LIST OF VALUE TREES FROM USER STUDY PARTICIPANTS U S E R l I25: Minimize commute time aw: Close To Bus Terminal a^ : Close To College a7K: Close To Railway Terminal I27: Maximize the comfbrc of the indoor living space a7: Has Bedroom a6: Has Bathroom an: Has Kicchen a,2: Has Living Room a,7: Has Air Conditioning Maximize proximity to culture aw: Close To Movie Theatre a67: Close To Pubs | are: Close To Public Library a74: Close To Beach Iu: Maximize beauty of setting a,: Has Balcony aH: Has Mountain View a,6: Has Ocean View a76: Close To Airport is false I„: Is near to and includes useful things a^ ,: Has Dishwasher a41: Has Dryer aM: Has Fridge a,,: Has Mailbox a^ : Has Shower a„: Has Stove aQ: Has Toilet a„: Has Washer a4s: Has Sink a n l: Has Laundry Facility aH: Close 1 o Grocery Score Iu: Maximize comfort of Ixdroom an: Faces hast a1(: Faces North a,7: Faces South a1(j: Faces West a(<: Has Door a2,: Has Linoleum is false-ay: Has Vinyl is false aw: Has Window Covering aM: Has Large Area I,,: Maximize ability co entertain IM: Maximize proximity co nature [JT: Maximize energy efficiency IM: Maximize the comfort of the outdoor space lyf Maximize a healthy envinximent 1^ : Maximize safety 59 USER 2 I41: Minimize comniute rime a :^ Close l o Bus Terminal a^ ,-. Close To College I42: Maximize a healchy environment Has Tree View a73: Close To Park a74: Close To Beach I4J: Maximize the comfort of die indoor livingspace a l l 2: Has Laundiy Facility a,,: Has Kitchen a7: Has Bedroom a(,: Has Bathroom a,: Has Balcony a13: Has Living Room aM: Has Fridge , a4„: Has Sink a„: Has Telephone Line a41i: Has Shower » a c : Has Toilet a46: Has Oven a,,: Has Stove a :^ Has Tree View aw: Has Washer a :^ Has Hardwood I4,: Maximize energy efficiency' aw: Has Chimney is false I4S: Maximize safety a_v: Has Chimney is false a47: Has Security Alarm l 4 v : Minimize coses a4 l: Has Dryer is false a-: Has Air Conditioning is false rtj3: Has Medium Area I„: Maximize nearness to useful places (err) aM: Close T o C ioce'ry Store a :^ Close To Restaurants a,,7: Close To Pubs •\ l a l: Close To Soccer Field aw: Close To Movie Theatre I„: Maximize comfort of lied room a,9: Has Door A U : Has Hardwood a„: Has Medium Area a16: Faces North am: Has Window Covering lv: Maximize che comfort of che outdoor liv space l,v Maximize proximity to nature Iw: Maximize ability to entertain 60 US E R 3 [,,: Minimize commucc time Close To Bus Terminal Close lo Grocery Store Close To Restaurants Maximize a healthy environment a74: Close I o Beach a75: Close To Park a7,: Close 1 o 1 ourist Attraction is false 1^ : Maximize the comfort of the indoor livingspace ' a,,: Has Office a^ : H as H a rdwood a,: Has Bathroom a7: Has Bednxim an: Has Kicchen a,: Has Foyer a2J: Has Linoleum is false a24: Has Vinyl is false aw: Has Wallpaper is false aM: Has Large Area a4,: Has Dryer •Au: Has Fridge a^ : Has Shower a v: Has Stove a a: Has Toilet a„: Has Washer a46: Has Oven I,n: Maximize checomf'otT o f chcoucdoor livingspace a5: Has Balcony Iw: Maximize energy efficiency a^ : Close To Bus Terminal [61: Maximize ability to entertain a,s: Faces North a„: Has Mountain View a,6: Has Ocean View a v: Close To Movie Theatre la: Maximize safety a^ : Close To Major Road Intersection is false a47: Has Security- Alarm a,i5: Has Parking Garage I„: Maximize proximity to nature 61 U S E R 4 lM: Minimize commute time a„: Close To Movie Theatre a67: Close To Pubs a l d o s e l'oGrocery Store a6K: Close To Res ta lira ncs a74: Close To Beach a7,: Close To Park a^: Close l'o Bus Terminal aw: Close l'o College a107: Close To Swimming Pool au w: Close ToGyivmasium all)f,: Close To Bank IH: Maximize a healthy environment Iw: Maximize proximity to nature a,6: Has Ocean View 1^: Maximize ability to en terrain I71: Maximize safety a47: Has Security Alarm I72: Maximize comfort of bedroom a2l: Has Carpet al7: Paces South I7,: maximize the facilities in kicchen a16: Has Oil Paint aw: Has Chimney a^ ,: Has Dishwasher a^: Has Fridge a^: Has Oven a,,: Has Scove a„: Has Washd-ay Has Large Area a4v: Has Sink I7,: maximize che comfort in living room a2(,: Has Oil Paine a4,: Has Fireplace a„: Has Telephone Line a,: Has Balcony aly: Has Door a2,: Has Carpet' aw: Has Chair I—: maximize good neighborhood a,,,: Has Parking Garage a,,6: Has Parking Lot I7S: maximize the other useful feature in theaparcmenc :ii,„: Has Swimming Pool a45: Has Mailbox 1^ : maximize che features in che apaicmenc a,: Has Balcony a7: Has Bedroom -i a u: Has Kicchen a,2: Has Living Room a,,: Has Office aM: Has Large Area a^: Has Shower a a: Has Toilet IK,: Maximize che eomforc of che indoor living space l,i: Maximize cheaimforc of che outdoor livingspace IJQ: Maximize energy efficiency 62 USER 5 ly,: Minimize commute time a^ : Close To College 1^ : Maximize ability to entertain a,2: Has Living Room a„: Has Medium Area a^ : Has Large Area L,X: Maximize safety L ,^: Maximize proximity to a good gym aKH: Close To Gymnasium llaf Maximize proximity to a park with a track a7J: Close To Park a8,: C lose To H igh School a,(l2: Close To Soccer Field a,,,,: Close To Baseball Field I101: Maximize proximity to cheap produce stores a6y (Jose To Grocery Store l l n 2 : Maximize proximity to a big grocery store a4J: Close To Gaxcry Store . aM: Close To Wholesale Store lloi: Is iii an urban area with services and things to aw: (.'.lose To Movie Theatre a67: Close To Pubs aw: Close To Restaurants USER 6 I l20: Minimize commute time a^ ,: Close To Major Road Intersection aw: Close To College a-^ : Close To Bus Terminal lln: Maximize the comfort or' the indoor livingspace a^ : Has Large Area a,,: Has Air Conditioning A U : Has Hardwood aly: Has Door aM: Has Kitchen a,2: Has Living Room a72: Close To Laundromat a74: Close To Beach I11>s: Maximize proximity to a bus route a^ ,: Close To Major Road Intersection Iui6: B'g kitchen a u: Has Kitchen I107: Strong water pressure lm: Room tor secure bike storage lufc>: Is above ground Ii, 0 : Is relatively new I,,,: Hardwood floors an: Has Hardwood IU 2: Well insulated IMJ: Access to climate control 1,14: Doesn't require too many major purchases to make aw: Has Stove a^ : Has Fridge [,„: Maximize a healthy environment In6: Maximize the comtbrt or" the indoor livingspace I i r : Maximize the comrort ot'the outdoor livingspace 1IIS: Maximize energy efficiency IMy: Maximize proximicy to nature I l26: Maximize ability to entertain ' a,,: Has Telephone Line a v: Close To Movie Theatre Close To Pubs a7J: Close To Park a u c: Close To Soccer Field Ii2X: Maximize a healthy environment IiW: Maximize the comfort or the outdoor livingspace I|X): Maximize energy efficiency I,,,: Maximize pawimity to nature IIJJ: Maximize safety 63 I,„: Minimize eommuce time aw: Close lo College a,,,: Close l'o Major Road Incerseccion a6V- Close To Grocery Score a66: Close To Shopping Mall I l v ): Maximize convenience a^ : Has Dishwasher a„: Has Washer a4): Has Diyer au: Close l'o Drug Score a^ : Close l'o Major Road Incerseccion a71: Close To Parking Loc a,u: Has Laundiy Facility a,,,: Has Parking Garage a,,6: Has Parking Loc I,„: Satisfies essencials ot indoor living a,,: Has Kicchen a,: Has Balconv a7: Has Bedioom a12: Has Living Room a„: Has Den a2l: Has Carpet a,,: Has Medium Area a4s: Has Oven a42: Has Fan a„: Has Telephone Line an: Has Small Area is false aX): Has Window Covering a^ : Has Fridge a^ : Has Shower a„: Has Stove a a: Has'Toilet a4s: Has Sink Satisfies Comfortable living a,7: Has Air Conditioning a „: H as Mou n ta i n View a4,: Has Fireplace aw: Has Ocean View USER 7 . a^ : Has Tree View a,7: Faces Souch a,,: Faces Last a^ Close To Restaurants a^ : Close To Public Library l l x ) : Maximize abilicv co entertain a,: Has Balcony a|2: Has Living Room a1(: Has Carpet a n: Has Hardwood a^ : Has Dishwasher a^ : Has Oven a42:HasFan af l: Has Mouncain View a^ ,: Close To Major Road Intersection a71: Close To Parking Lot aw: Has Door a21i: Has Painting I,4I: Allow space for work a„: Has Office a„: Has Den I,42: Maximize ease of maintenance aw: Has Wallpaper a n: Has Hardwood is false [,4J: Maximize Healthy Living a,iw: Close ToGymnasium allI7: Close ToSwimmihg Pool a„: Close To Park a74: Close To Beach a m : Has Grounds a,,-: Has Gvmnasium a,,,,: Has Swimming Pool I144: Maximize a healthy environment 1,4,: Maximize the wniroit of the indoor living space 1,^: Maximize the comfort of the outdoor livingspace I,^ : Maximize energy efficiency I!4S: Maximize proximity to nature Ii 4 l ): Maximize safety 64 USER 8 Minimize navel rajuiremencs aw: Close lo Movie Theacre am: Close l'o Live Theacre a a: Close To Drug Store aM: Close To Grocery Store a^ : Close To Pubs a,,,: Close l'o Res can ran cs a^ : Close To Public Library •a-,: Close To Beach • a7V- Close To Park a^ : Close To College : \ m : Close To Bank 1,^ : Maximize che comfort or" che indoor livingspace a7: Has Bedroom a,,: Has Kicchen a6: Has Bach room a12: Has Living Room a„: Has Office a2V-Has Linoleum a2]: Has Carpec a^ Has Window Coveting a y: Has Large Area Ayr-. Has Air Condicioning a19: Has Door a-Qi Close l'o Major Road Incerseccion is false Maximize the comforc of che outdoor livingspace a5: Has Balcony a„: Has Mountain View a,6: Has Ocean View a^ : Has Tree View a71: Close l'o Parking Loc is false a m : Has Grounds 1,^: Maximize energy efficiency . aM: Has Window Covering Maximize safecy a,„: Has Dooc I i W : Maximize serviceability a4l: Has Dryer Atf. Has Dishwasher a^ : Has Fridge a^ : Has Oven a4,: Has Sink a^ : Has Shower a51: Has Telephone Line a^ : Has Toilet a„: Has Washer a4,: Has Maillx>x a m : Has Laundry Facility Minimize commute cime li a ): Maximize a healthy environment lu,: Maximize proxiniicy co nature li f f l: Maximize ability coencercain 65 USER 9 lu, : Minimize commute time a6,: Close l o Grocery Store aM: Close To Wholesale Store aM: Close To Shopping Mal l a^: Close l o Bus Terminal a 1 0 6: Close l o Bank I 1 6 4 : Maximize a healthy environment a7 4: Close To Beach a^: Close T o Park a,6: Has Ocean View I,n: Maximize the comfort ot the indoor livingspace a,: Has Balcony a6: Has Bathroom a7: Has Bedroom a,,: Has Kitchen a l 2: Has Living Room a,7: Faces South a^: Has Hardwood a w : Has Wallpaper au: Has Largc Area a.j,: Has Dishwasher a4,: Has Dryer 84,: Has Oven a^: Has Shower a4i): Has Sink a,,: Has Stove a,,: Has Telephone Line a^: Has Toilet lm: Maximize the comfort ot the outdoor livingspace . 1 aT4: Close To Beach aTJ: Close T o Park a w : Has Aerobic L.xercise Device Irxt Maximize safety a^: Has Security Alarm l , 7 l : Maximize energy efficiency l r a : Maximize proximity to nature Ii 7 , : Maximize ability to entertain 66 USER 10 Iv: Minimize commute time a*: Close To College a^ : Close To Bus Terminal aU6: Has Parking Lot I,u: Maximize proximity to nature aw: Has Tree View a7,: Close To Park a74: Close lo Beach a,,,: Has Parking Garage I,,: Maximize the comfort of the indoor livingspace a6: Has Bathroom a7: Has Bedroom au: Has Kitchen a12: Has Living Room a16: Faces North a,7: Faces South a21: Has Carpet a„: Has Window Covering a,,: Has Medium Area aw: Has Chimney a^ ,: Has Dishwasher a4,: Has Dryer a4,: Has Fireplace a^ : Has Fridge a45: Has Mailbox a46: Has Oven a4s: Has Shower a4y: Has Sink AV: Has Scove a^ : Has Toilet a„: Has Washer a^ : Has Tree View I,,: Maximize energy efficiency a,7: Faces South aly: Has Door a74: Close lo Beach Ii4: Maximize safety a,-. Has Balcony is false aly: Has Door a71: Close To Parking Lot is false aU6: Has Parking Lot is false Maximize comfort of guests a,v: Has Door au: Has Large Area a4,: Has Fireplace aQ: Has Toilet a76: Close To Airport a-ry: Close To Bus Terminal aU6: Has Parking Lot I,6: F.asy access to en terra in men c aw: Close To Movie Theatre a7J: Close To Park a7,: Close To Tourist Attraction aw: Close To Museum 67 Appendix B. LIST OF GAI NETWORKS GENERATED FROM USER S T U D Y PARTICIPANTS U S E R I f Factor 25 > \ f Factor 32 \ V 79,89,78, J ' V5,55.56,76, J f Factor 30 \ [ \ 59,67,82,74,J \ Factor 34 \ 15,16,17,18,19, J 23,24,30,34, / ' Factor 33 \ 40,41,44,45,48. J 50,52,53,49,112, / . V 63, / I f Factor 27 ^ ^7,6,11,12,37^' User74 G A I Graph U S E R 2 6 8 U S E R 3 69 U S E R 5 U S E R 6 /^•Factor 120 \ f Factor 122 X ' Factor 126 ^ V 70,89,79, • ) I 34,37,22,19,11, J \ User805 G A I Graph (^  51,59,67,73,102^/ 70 U S E R 7 71 U S E R 9 72 U S E R 10 73 74 Appendix C. LIST OF SCATTERPLOTS OF MEASURED DEPENDENCIES BETWEEN PAIRS OF ATTRIBUTES AND DISTANCE BETWEEN PAIRS OF ATTRIBUTES OF USER STUDY PARTICIPANTS USERl ti 1.0CM 0.80 H 0.60 H 0> ** 3 .a I I k o VI 01 9" a ) a i < . o O S . £ 0.40-c 01 T3 C 01 a. 0) o 01 41 0.20-o.oo H 27( 7), 27(11) 33(49), 33(41) 33( 52), 33( 48) 33( 53), 33( 45) 30( 74), 30( 82) 34(30), 34(18) 34( 30), 34( 34) T" T 34(30), 27(12) 27(11), 33(53) 30( 82), 32( 55) C 34(23), 30(74) 32(76), 34(16) C 30(59), 34(17) T same node 1 2 3 4 no path D i s t a n c e b e t w e e n pa i r s o f a t t r ibutes (a,b) in GAI G r a p h = d(a,b) 75 U S E R 2 ja 1.00-ra ja £ 0.80-ra C •§. 0.60-01 It ai *-B ° % g ja o o .E 0.40-c 01 XI c 01 CL 01 ~° 0.20-n-o 01 3 in ra s o.oo H 43(46),43( 44) O o 43(112), 43(49) 43(49), 51(16) 49(41), 49(33) 51 (30), 43(48) 4 9 ( 3 7 M 3 C 5 ) O 43(11), 45(39) 43( 6), 50(63) C 50( 59), 43( 53) same node —r no path D i s t a n c e b e t w e e n pa i r s o f a t t r ibu tes (a,b) in GAI G r a p h = d(a,b) U S E R 3 ja i.oo-3 £ 0.80 H ai ra oi •-2 ° 01 < ja (3 2 .E 0.40-c 01 CL 01 0.20 H o 01 i» 3 IA ra « 57(34), 57(46) 0.00 H O 57( 48), 57( 50) 57(41), 56(73) 57(13), 56(75) 8 57( 6), 56(73) 61(,56),62(47)g 57( 6), 61(59)° no patli D i s t a n c e b e t w e e n pa i r s o f a t t r ibu tes (a,b) i n GAI G r a p h = d(a,b) 76 U S E R 4 in <u *• 3 JX i O i n k_ "«5 to n ^ o u .= 0) a. oi o 41 a 01 S 1.00- 72(17), 79(52) O 7^9( 7), 79(11) 73( 26), 79( 7) 72(17), 79(12) o 0.80-64(67), 64(107) O 73( 40), 73( 34) 75(19), 73(34) 73(53), 72(17) O 78(118), 79( 5) O 64(74),72( 21) 0.60-2S(rST)JS(38) 73( 53), 75( 38) O 75(51), 73(53) 73(46), 72(17) O 73(40), 72(17) 71(47)/73("53T-O 78(118), 72(21) 0.40-64(59), 64(106) O 75(43), 79(11) O 0.20-0.00-73( 49), 73( 53) o O 79( 5), 64(107) r same node 1 2 3 4 no path D i s t a n c e b e t w e e n pa i r s o f a t t r ibu tes (a,b) in GAI G r a p h = d(a,b) U S E R 5 JI I 1.00-^ 0.80 H 0.60 H IS c a. oi n oi *-l l 41 < S . £ 0.40-c 01 XI 01 •a o 41 19 01 g o.oo-102(63) ,102(65) 100(102) ,100(103) O 101 (63),104(72) 97(12),97(34) O 0 111(22),104(72) 104(67),104(59) O 104(74),104(68) O g104(68),104(67) 104(74),104(59) 97(33),104(59) 101 (63),100(85)8 111(22),104(67)8 same node 1 2 3 4 no path D i s t a n c e b e t w e e n pa i r s o f a t t r ibu tes (a,b) in GAI G r a p h = d(a,b) 77 U S E R 6 ja « in oi 1.00-0.80 H c - = 0.60-oi ra oi *-t% oi < ja © S .E 0.40-c oi 01 "O O 01 ra « 0.20-o.oo H 120(79) ,120(89) O 122(19),122( 34) o 122(11),126(67) O O 126(102),122(11) ni 22(12) ,122(19) °122(12),122(11) &I26(73),126(59) O 122(37) ,126(73) O 120(70) ,122(12) g 126(53) ,122(34) 122( 34),122( 37) O 126( 51 ),126(67) 122(37),122(22) O 120(70) ,122(22) O 126( 51 ),120(79) ' 122(12),126(102) 8 120(89) ,126(59) 122(19) ,122(37) O O 122(22),120(79) same node 1 2 3 4 no path D i s t a n c e b e t w e e n pa i r s o f a t t r ibutes (a,b) i n GAI G r a p h = d(a,b) U S E R 7 ja ra tn OI ja I ra a., 1.00-0.80 A 0.60-C CL oi n ai *-s < ja Q S .£ 0.40-01 CL oi o 01 ra oi 0.20-135(30),135( 32) 33(63) ,133(66) 135( 51 ),143(74) 143(118),135(32) 133(66),14>rr5) o o o — 133( 89),139( 21) 142( 29),135( 51 ) J # r t i 16),141 (13) 133(63),141(13) Q 0 143(111J,136(15) O O 139(28),135(30) 134(41),135( 8) same node — r no patli D i s t a n c e b e t w e e n pa i r s o f a t t r ibu tes (a,b) i n GAI G r a p h = d(a,b) 78 U S E R 8 LOCH |158(53) ,158(112) | |150(82) ,150(59) | |150(74),150(60)| O 152(19) ,152(70) 4*50162) ,150(106) | A 2 M58( 53),158(51) j154(30),153( 5) j )152(37),153(57) )|50(59),153( 5) | j158(53),153(111)| O ]150(74),152( 7)P |152(12),158(53) T no path same node D i s t a n c e b e t w e e n pa i r s o f a t t r ibutes (a,b) in GAI G r a p h = d(a,b) U S E R 9 1.00-0.80 H c-S. 0-60 H at ai « < .a (3 2 . £ 0.40 H 0.20-o.oo H 165(46),165(52) 166( 36),164( 56) o o 166(36),163(106) 165( 22) ,165(12) 166(73),165( 7 ) 0 same node — r no patli D i s t a n c e b e t w e e n pa i r s o f a t t r ibu tes (a,b) i n GAI G r a p h = d(a,b) 79 U S E R 10 80 Appendix D. LIST OF COMMENTS/SUGGESTIONS/QUESTIONS COMPILED FROM USER STUDY PARTICIPANTS DEFAULT V A L U E TREE • Quite clear and intuitive (User 2) • The tree covers pretty much what I need when I search for an apartment (User 9) • Wel l Selected Objectives for the Value Tree (User 5) • Good Objectives but a strong tendency to use the default value tree (User 1) • Perhaps provide verbal suggestions instead (User 1) • Perhaps a couple more default categories to get me thinking (too few makes me have to think about groupings) (User 6) • I did not use many of the default Value tree concepts. I wasn't thinking in terms of maximization as much as fulfilling different requirements of like the apartment will be used for parties, work, relaxation, cooking etc. (User 7) ONTOLOGY • Clear and intuitive (User 2) • Needs more options and be broader (User 9) • Odd to have to add all the leaves when really, suppose, all I wanted was a window (not caring which way it faced) (User 1) • Do changes in the ontology affect a user's value tree? (User 2) • Maybe the ontology should have other types of attributes besides boolean (i.e. true and false) such as quantity or quality / thresholds (User 3) • Felt incomplete, such as no "bus stop" just "bus terminal" etc (User 5) • Some concepts such as doors and sinks are essential yet others are optional such as "close to park"; difficult to combine both. (User 7) 81 INTERFACE • Easy to use interface (User 2, User 7) • Slick (User 3) • Good (User 5) • Improve slow updating (User 2) • It can be improved, quite slow updates (User 4) • Perhaps have the tree not expanded/condensed by default, with smaller font sizes for "less" important attributes. (User 6, User 10) • Suggest collapsing ontology to make navigation easier (User 5) O T H E R • Structure of questions during validation was difficult. Would there be any other way of posing them? (User 1) • Hard part is answering comparison gamble queries especially when one item is mandatory and one is superfluous such as 'sink' and 'near a park' (User 7) 82 J Appendix E. E M A I L FOR P R E - S T U D Y Suppose you were searching for an apartment. You want the place that you are searching for to fulfill certain goals and objectives you have, such as, you want to reduce the amount of time it takes for you to commute to either work/school from home - so you want to "minimize commute time". I want to compile a list of high-level objectives that are most common amongst people. If you could please take a look at the following list and let me know whether • something should be removed (which number) • something should be added (which number) • the wording needs to be changed so that it is clear immediately what is being talked about (which number) and just provide a reason for it (you can have any sort of reasoning behind your decision - after all your choices are personal). The goal for this is to find out whether I've been able to capture the majority of things people care about when searching for a new place and if I have missed anything MAJOR! This is the list of fundamental objectives: • • Minimize commute time • Maximize a healthy environment • Maximize the comfort of the indoor living space • Maximize the comfort of the outdoor living space • Maximize energy efficiency • Maximize proximity to nature • Maximize ability to entertain • Maximize safety 83 Appendix F. C O D E D RESPONSES F R O M PRE-STUDY USER1 , Hi Primary researcher, I think all the points except number are valid and should remain on the list. I'm not sure what you mean by 7. Maximize a healthy environment. ' Changing the wording might make what you are trying to get at clearer, hope this helps and hope all is well with you, U s e r l . U S E R 2 . _ _ • • 9. Minimize rent The reason is I think obvious :).... Cheers, -- User 2 U S E R 5 hey Primary researcher, Here are my thoughts-something should be removed (which number )—none something should be added (which n u m b e r ) — t h e neighborhood is really important to me when looking for a place the wording needs to be changed so that it is clear immediately what is being talked about (which number) —3 & 4, these are a little vague Let me know if you need more info :) User 3 • U S E R 4 . Minimise rent? U S E R S • One of the most critical criteria for an apartment (for me) is the utilities and appliances available: washer/dryer, dishwasher, modern bathroom, stove, oven, dual-basin sink in the kitchen, modern electrical power with abundant three-prong outlets, high-speed internet access, etc etc. Then there are the structural concerns: how old is the building? How much of a hazard does it present in an earthquake or fire? How well- insulated is it? (If you're paying your own utilities then cost to heat the space is a major issue. A modern, properly-insulated building will be much cheaper to heat) How clean is the building? Does the building have a flat roof? (A flat roof gets hotter in the summer, and can leak more easily. The famous "leaky condominiums" had flat,roof panels that al lowed water to run down the outside face of the walls. This let water seep into the walls, and the walls began to get moldy and rotten. This is expensive to fix, and poses a health hazard.) Structure also effects item #3: what is the temperature indoors? Do you have your own thermostat? How good is the lighting (both electrical and natural), d i rect ion/number of windows, (e.g. is there a south-facing window to get nice natural light?) Then of course there is PRICE. How much is the rent? Is it a month-to-month rental or a lease? Is the landlord/ lady a reasonable person? (If they're a pain in the butt it can ruin an otherwise fantastic apartment.) Are utilities included? Parking? As with most practical problems there are a huge number of factors. 84 Hope this helps -User 5 . USER 6 Minimize cost USER? I would add the following, Minimize the cost Living near people with the same ethnic background Minimize distance to everday utilities (more important for ethnic minorities, such access to halal/kosher meat) USER 8 ; • Hi Primary researcher, In my opinion, you have covered all the factors that I can think of !! I am curious to know if there could be any more additions .. so it would be great if you could also compile pplz answers .. and send it around :) I! User 8 • USER 9 __ Hi Primary researcher, It looks really good, I would recommend the following: Number 1: The impression I got was that this included commute to work time only. Either another entry needs to be,added, or this one needs to be expanded, as there are other "work like" things that one might want to be minimize their distance to, such as grocery shopping, day care, etc. Number 6: In addition to wanting to be close to nature people might also have other priorities: close to nightlife, a particular neighborhood, restaurants, their family, etc. Finally, I'm not sure I understand the difference between 2 and 3&4.. That's all I can think of, good luck. User 9 USER 10 . You forgot cost! If you can smoke, have pets, babies, etc. USER 11 • I'd removed #5 since I don't think that there is a great variation in how energy efficient apartments are and it's hard to tell without having lived there for a while. For that factor it seems much more important what you put inside and you travel time and mode relative to conveniences. In the same vein, I'd added a new value which is proximity to non-work conveniences like supermarkets, bus routes, a good bar, and friends. This could perhaps be bundled in with proximity to nature. Cheers, 85 User 11 USER 12 • Hi Primary researcher, 2. Might need to be clarified or removed. It seems very similar to either 5 or 6, but I don't understand why people would want their house to be "in a healthy environment" other then making the house more efficient (5), having an outdoors (4) or being close to nature (6). +9. While certainly not important for some people, Maximizing proximity to schooling or day care might be important. You might factor this into commute time. User 12 USER 15 . . . . . . Hi Primary researcher. Sounds like an interesting study. Except for the commute time, I found your questions difficult to actually consider -- they're almost meta-questions. Or maybe it's that when thinking about places, I use a combination of analytical decision making as well as gut-based decision making. I would actually consider neighbourhood vibrancy over commute time, generally, as long as the commute time is within reason. I lived in the West End which had a longer commute time than, say, South Granville, but I preferred: the neighbourhood feeling (busy vs quiet, arty vs sterile) the diversity of the neighbourhood (ethnic mix, age mix) the easy availability and quality of day-to-day shopping needs (e.g., green grocers, super market) But none of those are captured directly by your questions. And I think there are also more negatives than just commute time. You haven't incorporate rent/cost, size. Have you thought of asking several people why or why they wouldn't want to live in certain areas, ensuring to include some outlying areas. You could then ascertain the order of their preferences by varying different conditions: if Cloverdale is too far, would that change if there was a nearby SkyTrain station and a quick route into UBC? If rent is too high, would that change if they found a real bargain? But I wonder if you'd only find out that different people have different preferences. After all, some people like living in the burbs, and others like living in Yaletown. Places I wouldn't want to live in! Actually, this sounds pretty complicated. Have you looked at the community planning literatures? I'm sure somebody must have done something about people's preferences. Anyways, good luck with your study. User 13. USER 14 Hey Primary researcher, While I'm not sure if these are things that can be maximized per se, they're still definitely things I anticipate considering when I'm looking for my next place. One, ability to keep pets. Not sure that can be maximized since it's just yes or no, but perhaps you can work that in somewhere. Two, desirable neighborhood. E.g. even a wonderful apartment in Surrey is still in Surrey. Regardless of 86 how it affects my commute, there's no way I'm living in Surrey. It's simply too far from what I considering to be the interesting parts of the Vancouver area. Aside from that, it looks good. I didn't see the actual dollar cost of rent, but I assume that's factored in somehow. Good luck and if you need any volunteers later, just let me know. Thanks! -User 14 USER 15 you missed the cost of the apartment....you want to minimize cost. . USER 16 _ _ _ _ This is the list of fundamental objectives > 1. Minimize commute time > I want commute time to be within a range, like under 1 hour. My current commute is 2 minutes, so when I look for a new place, I don't want another minimum commute. > 1. Maximize a healthy environment 2. Maximize the comfort of the indoor living space > I don't understand what this means. The comfort would be based on too many things that are not necessarily a part of the apartment. Some want carpet, some want wood floors. TV, kitchen size, windows, furniture and many other things go into comfort level, and I don't see how you can express that all in some maximization formula. > • 1. 2. Maximize the comfort of the outdoor living space 3. Maximize energy efficiency 4. Maximize proximity to nature > Proximity to what type of nature? Some want the beach, some want the mountains, some want parks, or forests. Similarly, you should have a proximity to city things, like grocery stores, restaurants, bars, shopping, movie theaters, etc. > 1. Maximize ability to entertain 2. Maximize safety > Also, you don't have standard apartment features. # of bedrooms, # of bathrooms, washer/dryer, kitchen, patio, balcony, yard, basement or not, apartment or house, roommates or not, furnished or unfurnished, length of lease, pets, which utilities are included, price range. Not everything can be expressed in terms of minimizing and maximizing. USER1? What about general cost? Or bedroom count? Or proximity to a major metropolitan area? What does 1. mean? And how is 2. or 3. quantified when searching for a place to live? I can understand a basic space criteria, of "comfort" throws me. User 17 . USER 18 . add Minimize cost clarify if "healthy environment" means "my local environment is healthy for me" or "it contributes to the health of the planet, or at least minimizes damage" 87 add Minimize distance to {friends, family, bf/gf} add Minimize distance to hobbies (e.g. skiing) add Maximize access to {libraries, shopping, bus line, bank, restaurants} USER 19 Hi Primary researcher, Two I would add are: -maximize (or minimize?) proximity to friends and family -minimize costs (factoring in utilities, etc) Good luck with the study... Cheers, User 19. USER 20 ..• . Hi Primary researcher, I would probably look for somewhere that has some recreational facility (such as pool or/and gym) as well. I think number 7 would refer to such things as party rooms, movie rooms or satellite dishes. Moreover I think it's better to change number 8 to 'maximize safety and security'. I think safety mostly refer to having for example a good fire alarm system. Cheers, User 20 ; USER 21 . hi, my name is Jacob, and im looking for an apt. the list isnt in order of importance in my opinion, but, it seems good, i guess if i were to say what is important to me, i would say: minimize cost (factoring in salary), based on these factors: location: commute safety close to friends closeness to public transportation (i.e. walking distance to subway) city (some people want to be close to the city) beach (some people want to be close to the beach) stores (some people want groceries to be convenient) some people want to be near certain things and i dont know how to phrase that space: size (spaciousness, whether everything Jacob owns will fit inside) interior quality (floors, walls, windows) rooms (kitchen, bathroom, balcony, garage, laundry room) view (I LOVE THIS ONE!!! obviously., hehe) higher floors usually have better views yard, garden, whether it's in a nice complex, deck or patio parking building amenities (gym, daycare, pool, doorman) exterior appearance (architecture, paint job) 8 8 ok, that's all i can think of. i know this is not in the format that you asked for, but maybe you can get some ideas from this if you want, but, the list you have covers most of these issues. good luck with your survey! (im making * * * * * * respond to you, so let me know if he doesnt) User 21 USER 22 Hi, One thing that is important for my wife and is the proximity to services (laundry, bank, clinic, mall, etc.), but this might fall under "Minimize commute time" already. I do think of "commute time" as being to school/work and back to home, and these others as something separate. Hope this help, User 22 USER 25 . Hi, I am somewhat confused about number 2. Do you mean maximize eco-friendliness, or a healthy living environment? Best, User 23 USER 24 ; , Hi, How about: quiet environment allow pets non-smoking sunlight (important in Vancouver) room mates? 3 & 4 probably don't need the phrase: "the comfort ofthe". User 24 USER 25 hi Primary researcher, I think people might get confused about #2, some people might have different definitions of a healthy environment. Also, #7, do you mean entertain guests, or entertain yourself? Just a suggestion, maybe you could provide an example with each objective. For example: Minimize commute time (You want a place that is near public transportation) Maximize a healthy environment (You don't want to live near dirty, run-down streets with lots of homeless people) etc... User 25 USER 26 89 Primary researcher, 1. Minimize commute time Seems self-explanatory to me. 2. Maximize a healthy environment This seems to be a little too general. Does it include, say, the amount of natural light indoors (conflict with #3); the pollution-noise and otherwise-outside (conflict with #4)? 3. Maximize the comfort of the indoor living space 4. Maximize the comfort of the outdoor living space 5. Maximize energy efficiency 6. Maximize proximity to nature Seems to be related (at least, to many people) with #7. 7. Maximize ability to entertain This perhaps should be stated in terms similar to #6: proximity to cultural/shopping hot-spots, or some such. 8. Maximize safety Thanks a lot (in advance) Hope this is helpful. Good luck with your study. USER 2? Hi Primary researcher, Here are my two cents... Add as #1: Minimize distance to the places where family (kids, parents) and friends live Put "Maximize Safety" as #2. It's not a good idea to live in certain areas of the city. There should be "Minimize commute time by car" AND "Minimize commute time by public transit", because commute time by car and by transit are often drastically different. Public transit is very helpful for teens, elderly people, and people working close to SkyTrain. Commute time is less important than healthy environment. Noone wants to to have a highway exit or a SkyTrain station in their backyard. I'm not sure whether "Comfort of the outdoor living space" includes insfrastructure like every-day-use municipal parks. I assume it does and the "Proximity to nature" refers to less often used wilderness like provincial park. , All in all, my list looks like "Minimize distance to family/friends" "Maximize safety" "Maximize a healthy environment" "Minimize commute time by public transit" "Minimize commute time by car" "Maximize the comfort of the indoor living space" "Maximize the comfort of the outdoor living space" "Maximize energy efficiency" "Maximize proximity to nature" "Maximize ability to entertain" Hope that helps, User 27 USER 28 9 0 Hi Primary researcher, I think that you missed a major objective that is rent/price. That is probably most often the most important objective. I think that point 2 is vague. I am not sure how many people think explicitly about 5, they are more likely to deliberate over room layout (which I guess falls under 3) or different means of commuting than to think about 5. Also proximity to friends/family might be important to some people. I am not sure if any of your points addresses proximity to shopping places. Cheers, User 28 91 Appendix G. PROCEDURES AND INSTRUCTIONS FOR FINAL USER S T U D Y Except for the first participant, click on Thank You for Your Time from the apartments interface to begin a new instance ofthe study when the user arrives S T A R T Have them sign the consent form Read The goal of this experiment is to explore a method for acquiring people's preferences. To start the study, I'd like to show you the sample interface. You will be learning how to interact with it. I N T E R F A C E T R A I N I N G Read Suppose I want to buy a desk..I,have preferences for my desk. For example, I would like an Has L - Shape or a desk with a keyboard-drawer. Often such preferences accomplish objectives that my desk will fulfill in my life, workplace, home etc. Show sample interface Read . For e.g., one of my objectives maybe to maximize aesthetics of my desk or improve ergonomics of my desk. When objectives are organized hierarchically it forms a value tree. On the left side is a value tree. I have been provided with a value tree to begin with. I can personalize this value tree to reflect my preferences. . My objectives can be fulfilled by things in the real world, i.e. things such as 'made of wood', 'wide keyboard drawer' etc. We Call these elements in the real world, attributes. Most often attributes belong to a class that is a set of objects, known as entities such as 'desk', 'drawer' etc. These entities and attributes often have a hierarchical representation in the real world — we call this domain ontology. On the right side is a domain ontology for desks. Do you have any questions? 92 PART 1: VALUE TREE AND DOMAIN ONTOLOGY Read Let us begin part 1. This interface allows me to represent my preferences in terms of objectives and attributes that accomplish those objectives. What you see in front of you is Part 1 of the training interface — this allows me to express my value tree. ' Suppose I would like to maximize my workspace, maximize the ergonomics of desk, minimize costs and maximize the aesthetics of my desk. Let's add the first objective to my value tree. How to add an objective to the value tree? To add an objective, I click on the text box or the label "Add an objective". Type my objective. M A X I M I Z E ERGONOMICS OF D E S K Click "Add" to add the objective to the value tree below. How about you do it now? Add the objective: M A X I M I Z E W O R K SPACE How to modify an objective in the value tree? I. have decided I want to improve the ergonomics of my workspace. To modify an objective, click on the pencil icon next to the objective, in this Case the M A X I M I Z E ERGONOMICS OF D E S K objective is what I wish to change. ' An inline-editing textbox will appear where the objective was. I can modify the objective to be IMPROVE ERGONOMICS OF D E S K Once I am done modifying the objective, I can click 'ok' to save the objective or 'cancel' to not make any modifications. I'll click'ok'and save the changes. How about you do it now? Modify the objective: M A X I M I Z E W O R K SPACE to be M A X I M I Z E STORAGE SPACE How to remove an objective in the value tree? To remove an objective from the value tree, click on the red cross [©] icon next to the objective you wish.to remove. Suppose I decided that space is no 93 longer a constraint in my decision making and would like to remove the objective: M A X I M I Z E STORGE SPACE A message will ask me to confirm whether I choose to remove the objective. Clicking 'OK ' will remove the objective from the value tree. Clicking 'cancel' will not remove the objective from the value tree. I'll click okay to remove the objective. How about you do it now? Remove the objective: MINIMIZE COSTS Suppose the objective: IMPROVE ERGONOMICS OF D E S K may be fulfilled by the following attributes: Has Wide Keyboard Drawer, Has L -Shape and Made of wood. Let's add these attributes. How to add an attribute to fulfill an objective? Drag the attribute from the ontology tree. I'll choose Has Wide Keyboard Drawer from the Ontology on the right hand side. Drop the attribute under,. the objective: IMPROVE ERGONOMICS OF DESK. When I hover with a selected attribute over an objective, a highlighted box will appear around the objective. • ., If the attribute is dropped in this box, the attribute will be added under that objective. -If the attribute is not dropped in such a box, the attribute will not be added to any objective. • , How about you do it now? Add the attribute Has L - Shape to the objective: IMPROVE ERGONOMICS OF DESK. Add the attribute Made of wood to the objective: IMPROVE ERGONOMICS. OF DESK. How to move an attribute from one objective to another? Say I decided that if the desk is Made of wood it would fulfill the' aesthetic objective of My desk rather than the ergonomic so L would like to move the attribute from one objective to another. I drag the attribute label Made of wood from the objective: IMPROVE ERGONOMICS OF D E S K to the objective: M A X I M I Z E AESTHETICS OF DESK. ' If the attribute is not dragged to any other objective's; highlighted box, the attribute will return to the original objective. , ' , 94 How about you do it now? Drag the attribute Has L - Shape from the objective: IMPROVE ERGONOMICS OF D E S K to the objective: M A X I M I Z E AESTHETICS OF DESK. How can I observe the relative positions ofthe attributes, under an objective, in the Ontology? Often an ontology can be quite large and difficult to navigate. To see where the relative positions of the attributes, under an objective, is in the domain ontology we have added this feature. . Click the objective label: M A X I M I Z E AESTHETICS OF D E S K to see where the relative positions of the attribute Has L — Shape, Made of wood is in the ontology. This will highlight the attribute in the ontology tree. How can I reset the highlighting in the Ontology? Clicking on the objective label that is already highlighted will remove the highlighting. Go ahead and click on the objective: IMPROVE ERGONOMICS OF D E S K to reset the highlighting for the objective. M A X I M I Z E AESTHETICS OF D E S K and highlight the attributes ofthe new objective clicked. . ' How about you do it now? Click on M A X I M I Z E AESTHETICS OF D E S K to highlight the attributes in the ontology. Suppose to IMPROVE ERGONOMICS OF DESK, the attribute Has L -Shape is necessary. Since the attribute, Has L - Shape is already under one objective it is easy to find its position in the ontology as it is highlighted. Add the attribute Has L - Shape to the objective IMPROVE ERGONOMICS OF DESK. However, to M A X I M I Z E AESTHETICS OF D E S K I don't want to get a desk made of plastic. Add the attribute made of plastic to the objective M A X I M I Z E AESTHETICS OF DESK. • When I double click on this attribute it changes color from green to red and now says made of plastic is false which means that not having this attribute 95 would specifically fulfill the corresponding objective M A X I M I Z E AESTHETICS OF DESK. How to remove an attribute from an objective? Say I then decided the aesthetics of my desk does not really depend on whether it is made of wood. Therefore, I would like to remove the attribute Made of wood from the objective: M A X I M I Z E AESTHETICS OF DESK. Click the red cross [©] icon next to the attribute label of Made of wood under the objective: M A X I M I Z E AESTHETICS OF DESK. A message will ask me to confirm whether I choose to remove the attribute ... from that objective. Clicking 'OK ' will remove the attribute from the value tree. • Clicking'cancel'will not remove the attribute from the value tree. I'll click okay to remove the attribute'from the objective. How about you do it now? Remove me attribute Has Wide Keyboard Drawer from the,objective: M A X I M I Z E AESTHETICS OF DESK. , . Do you have any questions? PART 2 : VALIDATION Read . To validate the value tree created in part 1, we will ask some questions in part 2, which will allow us to understand the structure of my true preferences. Before we move onto part 2 — here is some training to get you used to the structure of these questions. . • • • Hand the training document and read The questions are in the comparison gamble format, which is a paired comparison. You must choose between two alternatives or state you are indifferent between the two alternatives. , When the user has handed the training, read To validate the value tree created in part 1, we will ask some questions in part 2, which will allow us to understand the structure of my true preferences. I will do the first one and you can do the next one. Do you have any questions? 9 6 Click on Thank You for Your Time from the desk interface to reset the training module when the new user arrives S C E N A R I O Let us begin the study _____ Read „ Suppose, you are relocating to Vancouver, British Columbia Canada. You have preferences when it comes to renting an apartment. You are given a fixed budget of $1000 for rent. For the purposes of the study you do not need to consider minimizing your costs as an objective. Start the program Read The following interface is,provided for you to express your preferences in terms of your objectives and the attributes in the apartment's domain that fulfill these objectives. On the left hand side is the default value tree that specifies a set of objectives. You do not have to stick with this set of ' objectives. You are free to modify, add or remove any of these objectives as you see fit. Point out part 1 (Mapping) ' This is Part 1 of the interface. During this part 1, if you have any questions please do not hesitate to ask. Once you have completed this section, please proceed onto the next part-Part 2, the Validation. C O N C L U D I N G S T U D Y Thank them, Pay, and Have them sign receipt 97 Appendix!. U S E R INFORMATION UBC THE UNIVERSITY OF BRITISH COLUMBIA Department of Computer Science 2366 Main Mall Vancouver, B.C., V6T 1Z4 Value-Focused Preference Model Construction Given A Domain Ontology Age: • 19 • 20-30 0 31-40 • 41-50 • 51 - 60 • 61 or below and above Gender: • Male • Female Occupation: Have you ever rented an apartment? • Yes • No Have you ever purchased a desk? • Yes • No 100 Appendix J. VALIDATION QUESTION TRAINING Parti You wil l be asked to determine probabilities that make you indifferent between pairs of outcomes. Outcomes can be certain or uncertain. In addition, they can be gains or losses. T H E R E IS N O RIGHT OR W R O N G A N S W E R . It all depends on your preference and your attitude towards risk. Here is a sample question. Please determine the value of p that would make vou indifferent between O P T I O N A and O P T I O N B. O P T I O N A : to enter a lottery in which you have a p chance of winning a free coffee and a muffin and a (1-p) chance of winning nothing. O P T I O N B: to win a free coffee. A simple table like this can help your assessment by trying different values of p until you find one that makes you indifferent between OPTION A and OPTION B . Please write down in the table all the values of p that you consider and your corresponding preferred outcome. For example, I prefer OPTION B , i f the value of p is 0, because I would rather get a free coffee than nothing. However, I prefer OPTION A , i f the value of p is 1, because I would rather get a free coffee and a free muffin as there is no risk. p 1-P I prefer ... 0 1 O P T I O N B 1 o O P T I O N A 0.5 0.5 When you are done, write down the value of p: . , To help you visualize your choices, the pairs of outcomes wi l l be shown in a grid. The two outcomes are separated onto the left and right hand side. If an outcome is a lottery, a branching specifies probabilities and rewards of each lottery prize. For example, the previous example would be visualized as: , O P T I O N A O P T I O N B P Free coffee and muffin © Free coffee © 1-p Nothing ® 101 Part 2 Let us focus on the desk domain. You are searching for a new desk. You are trying to come up with the best balance for your storage needs. Assume that the desk has the exact same dimensions for all alternatives. For which value ofp would you be indifferent between OPTION A and OPTION B? Please write down in the table all the values of p that you consider and your corresponding preferred outcome. Again, remember, in this study T H E R E A R E N O RIGHT OR W R O N G A N S W E R S , it all depends on your preferences and your attitude towards risk. 0.5 0.5 I prefer ... OPTION A 1-p Desk with 3 drawers and 3 shelves © Desk with no drawers or shelves © OPTION B Desk with 3 shelves © p that makes you indifferent For which value ofp would you be indifferent between OPTION A and OPTION B? Again, please write down in the table all the values of p that you consider and your corresponding preferred outcome. 0.5 0.5 I prefer ... OPTION A 1-p Desk with 3 drawers and 3 shelves © Desk with no drawers or shelves © OPTION B Desk with 3 drawers © m p that makes you indifferent For which value of p would you be indifferent between OPTION A and OPTION B? Again, please write down in the table all the values ofp that you consider and your corresponding preferred outcome 0.5 i2_ 0.5 I prefer ... OPTION A 1-p Desk with 3 drawers and 3 shelves © Desk with no drawers or shelves ® OPTION B . Desk with 1 drawer and 1 shelf © • F p that makes you indifferent 102 Part 3 Let us focus on the desk domain. You are searching for a new desk. You are trying to come up with the best balance for your storage needs. Assume that a large desk is best suitable for your needs, a medium desk wil l be a tradeoff and a small desk is going to be difficult to work with. The grid on the left hand side now has an extra column for OPTION B. The percentages for OPTION B, however, are always the same, 50%. Consider two aspects of a desk. One is ihe number of shelves and the other is the size of the desk. For which value ofp would you be indifferent between OPTION A and OPTION B? Please write down in the table all.the values of p that you consider and your corresponding preferred outcome. Again, remember, in this study T H E R E A R E N O RIGHT OR W R O N G ANSWERS, it all depends on your preferences and your attitude towards risk. 0.5 J_v_ 0.5 I prefer... OPTION A 1-p Large desk with 3 shelves © Small desk with no shelves © OPTION B 50% 50% Small desk with 3 shelves Large desk with no shelves p that makes you indifferent For which value of p would you be indifferent between OPTION A and OPTION B? Please write down in the table all the values of p that you consider and your corresponding preferred outcome. 0.5 J_v_ 0.5 I prefer ... OPTION A 1-p Medium desk with 3 shelves © 1 1 Small desk with no shelves © OPTION B 50% 50% Small desk with 3 shelves Medium desk with no shelves r—r p that makes you indifferent For which value of p would you be indifferent between OPTION A and OPTION B? Please write down in the table all the values of p that you consider and your corresponding preferred outcome. 0.5 J_v_ 0.5 I prefer OPTION A OPTION B P Large desk with 3 shelves © 50% Medium desk with 3 shelves © 1 1 1 1 1-p Medium desk with no shelves 50% Large desk with no shelves 1 1 1 1 p that makes you indifferent 103 Appendix K. POST-STUDY QUESTIONNAIRE THE UNIVERSITY OF BRITISH COLUMBIA Department of Computer Science 2366 Main Mall Vancouver, B.C., V6T 1Z4 Value-focused preference model construction given a domain ontology Do you have any suggestions/comments/questions about the Default Value Tree provided? Do you have any suggestions/comments/questions about the Ontology provided? Do you have any suggestions/comments/questions about the interface? Other suggestions/comments/questions: 104 Appendix L. SAMPLE RECEIPT University of British Columbia Department of Computer Science have received the sum of $ _, as payment for my participation in the "Value-focused preference model construction given a domain ontology" user study. Signature Date U B C 105 AppendixM. UBC RESEARCH ETHICS BOARD CERTIFICATES OF r APPROVAL The University of British Columbia Office of Research Services Behavioural Research Ethics Board Suite 102, 6190 Agronomy Road, Vancouver, B.C. V6T123 CERTIFICATE OF APPROVAL- MINIMAL RISK RENEWAL PRINCIPAL INVESTIGATOR: Giuseppe Carenini DEPARTMENT: UBC/Science/Computer Science UBC BREB NUMBER: H05-80581 HNSTITUTION(S) WHERE RESEARCH WILL BE CARRIED OUT: Institution Site U B C Other locations where the research will be conducted: N/A Vancouver (excludes U B C Hospital) CO-INVESTIGATOR(S): Blair Tennessy Jeanette L . Bautista SPONSORING AGENCIES: Natural Sciences and Engineering Research Council of Canada (NSERC) - "Multimedia Generation and Interactive Visualization of Preferential Choice" PROJECT TITLE: Multimedia Generation and Interactive Visualization of Preferential Choice EXPIRY DATE OF THIS APPROVAL: September 10, 2008 IAPPROVAL DATE: September 10,2007 The Annual Renewal for Study have been reviewed and the procedures were found to be acceptable on ethical grounds for research involving human subjects. Approval is issued on behalf of the Behavioural Research Ethics Board 106 Appendix N. A P A R T M E N T O N T O L O G Y IN O W L <?xml version="l.0"?> <!DOCTYPE owl [ <!ENTITY MILO "http://www.es.ubc.ca/~albaab01/thesis/MILO.owl#"> <!ENTITY dc "http://purl.Org/dc/elements/l.l/">. <!ENTITY xsd "http://www.w3.Org/2001/XMLSchema#"> <!ENTITY SUMO "http://www.cs.ubc.ca/~albaab01/thesis/SUMO.owl#"> <!ENTITY APRT "http://www.cs.ubc.ca/~albaab01/thesis/APRT.owl#"> <!ENTITY r d f "http://www.w3.org/1999/02/22-rdf-syntax-ns#"> <!ENTITY owl "http://www.w3.Org/2002/07/owl#"> <!ENTITY r d f s "http://www.w3.Org/2 0 00/01/rdf-schema#"> ]> <rdf:RDF xmlns:MILO="http://www.cs.ubc.ca/~albaab01/thesis/MILO.owl# xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:xsd="http://www.w3.org/2001/XMLSchema#" xmlns:SUMO="http://www.cs.ubc.ca/~albaab01/thesis/SUMO.owl# 1! xmlns:APRT="http://www.cs.ubc.ca/~albaab01/thesis/APRT.owl# l l xmlns:rdf="http://www.w3.org/1999/02/22-rdf-syntax-ns#" xmlns:owl="http://www.w3.org/2002/07/owl#" xmlns:rdfs="http://www.w3.org/2000/01/rdf-schema*" xml:base="http://www.cs.ubc.ca/~albaab01/thesis/APRT.owl" 107 > <owl:Ontology rdf:about="http://www.cs.ubc.ca/~albaab01/thesis/APRT.owl"> <rdfs:label>APRT</rdfs:label> <owl:imports rdf:resource="http://www.cs.ubc.ca/~albaab01/thesis/MILO.ow l"/> <owl:imports rdf:resource="http://www.cs.ubc.ca/~albaab01/thesis/SUMO.ow l"/> </owl:Ontology> <owl:Class rdf:about="#Appliance"> <rdf s : subClassOf >•' .<owl:Class rdf:about="&SUMO;StationaryArtifact"> </owl:Class> </rdfs:subClassOf> </owl:Class> <owl:Class rdf:about="#Balcony"> </owl:Class> <owl:Class rdf:about="#BusTerminal"> <rdfs:subClassOf> <owl:Class rdf:about="#TransitTerminal"> </owl:Class> </rdfs:subClassOf> </owl:Class> <owl:Class rdf:about="#Carpet"> <rdfs:subClassOf> <owl:Class rdf:about="#FloorType"> </owl:Class> </rdfs:subClassOf> </owl:Class> <owl:Class rdf:about="#Den"> 108 </owl:Class> <owl:Class r d f : a b o u t = " # F a c i l i t y " > <rdfs:subClassOf> <owl:Class rdf:about="& SUMO;StationaryArtifact"> </owl:Class> </rdfs:subClassOf> </owl:Class> <owl:Class rdf:about="#FerryTerminal"> <rdfs:subClassOf> <owl:Class rdf:about="#TransitTerminal"> </owl:Class> </rdfs:subClassOf> </owl:Class> <owl:Class rdf:about="#FloorType"> <rdfs:subClassOf> <owl:Class rdf:about="&SUMO;Artifact"> </owl:Class> </rdfs:subClassOf> </owl:Class> <owl:Class rdf:about="#Foyer"> • </owl:Class> <owl:Class rdf:about="#Good-View"> <rdfs:subClassOf> <owl:Class rdf:about="#ViewAttribute"> </owl:Class> </rdfs:subClassOf> </owl:Class> <owl:Class rdf:about="#Gymnasium"> <rdfs:subClassOf> <owl:Class rdf:about="&MILO;SportsFacility"> </owl:Class> </rdfs:subClassOf> 109 </owl:Class> <owl:Class rdf:about="#Hallway"> </owl:Class> <owl:Class rdf:about="#Hardwood"> ' <rdfs:subClassOf> ,<owl:Class rdf:about="#FloorType"> .</owl:Class> </rdfs:subClassOf> </owl:Class> <owl:Class rdf:about="#Linoleum"> <rdfs:subClassOf> <6wl:Class rdf:about="#FloorType"> </owl:Class> </rdfs:subClassOf> </owl:Class> <owl:Class rdf:about="#LivingRoom"> <rdfs:subClassOf> <owl:Class rdf:about="&SUMO;Room"> </owl:Class> </rdfs:subClassOf> </owl:Class> <owl:Class rdf:about="#Mquntain-view"> <rdfs:subClassOf> <owl:Class rdf:about="#Good-View"> '</owl:Class> </rdfs:subClassOf> </owl:Class> <owl:Class rdf:about="#No-View"> <rdfs:subClassOf> <owl:Class rdf:about="#ViewAttribute"> </owl:Class> </rdfs:subClassOf> 110 </owl:Class> <owl:Class rdf:about="#Ocean-view"> <rdfs:subClassOf> . <owl:Class rdf:about="#Good-View"> </owl:Class> </rdfs:subClassOf> </owl:Class> <owl:Class rdf:about="#Office"> <rdfs:subClassOf> <owl:Class rdf:about="& SUMO;Room"> </owl:Class> </rdfs:subClassOf> </owl:Class> <owl:Class rdf:about="#TransitTerminal"> <rdfs:subClassOf> <owl:Class r d f : a b o u t = " # F a c i l i t y " > </owl:Class> </rdfs:subClassOf> </owl:Class> <owl:Class rdf:about="#Tree-view"> <rdfs:subClassOf> <owl:Class rdf:about="#Good-View"> </owl:Class> </rdfs:subClassOf> </owl:Class> <owl:Class rdf:about="#ViewAttribute"> <rdfs:subClassOf> <owl:Class rdf:about="& SUMO;RelationalAttribute"> -, </owl: Class> </rdfs:subClassOf> </owl:Class> <owl:Class rdf:about="#Vinyl"> 111 <rdfs:subClassOf> <owl:Class rdf:about="#FloorType"> </owl:Class> </rdfs:subClassOf> </owl:Class> <owl:Class rdf:about="&MILO;AerobicExerciseDevice" <rdfs:subClassOf> <owl:Class rdf:about="#Appliance"> </owl:Class> ' </rdfs:subClassOf> </owl:Class> <owl:Class rdf:about="&MILO;ApartmentBuilding"> </owl:Class> <owl:Class rdf:about="&MILO;ApartmentUnit"> </owl:Class> <owl:Class rdf:about="&MILO;Chair"> <rdfs:subClassOf> <owl:Class rdf:about="#Appliance"> </owl:Class> </rdfs:subClassOf> </owl:Class> <owl:Class rdf:about="&MILO;Chimney"> <rdfs:subClassOf> <owl:Class rdf:about="#Appliance"> </owl:Class> </rdfs:subClassOf> </owl:Class> <owl:Class rdf:about="&MILO;Door"> </owl:Class> <owl:Class r d f : a b o u t = " & M I L O ; E d u c a t i o n a l F a c i l i t y " > <rdfs:subClassOf> <owl:Class r d f : a b o u t = " # F a c i l i t y " > 112 </owl:Class> </rdfs:subClassOf> </owl:Class> <owl:Class rdf:about="&MILO;FanDevice"> <rdfs:subClassOf> <owl:Class rdf:about="#Appliance"> </owl:Class> .</rdfs:subClassOf> </owl:Class> <owl:Class rdf:about="&MILO;Fireplace"> <rdfs:subClassOf> <owl:Class rdf:about="#Appliance"> </owl:Class> </rdfs:subClassOf> </owl:Class> <owl:Class rdf:about="&MILO;Floor"> </owl:Class> <owl:Class rdf:about="&MILO;Mailbox"> <rdfs:subClassOf> <owl:Class rdf:about="#Appliance"> </owl:Class> </rdfs:subClassOf> </owl:Class> <owl:Class rdf:about="&MILO;Monument"> <rdfs:subClassOf> <owl:Class • r d f : a b o u t = " # F a c i l i t y " > </owl:Class> </rdfs:subClassOf> </owl:Class> <owl:Class rdf:about="&MILO;Oven"> . <rdfs:subClassOf> <owl:Class rdf:about="#Appliance"> 113 </owl:Class> </rdfs:subClassOf> </o'wl: Class>. <owl:Class rdf:about="&MILO;Paint"> </owl:Class> <owl:Class rdf:about="&MILO;ParkingLot"> <rdfs:subClassOf> <owl:Class r d f : a b o u t = " # F a c i l i t y " > </owl:Class> </rdfs:subClassOf> </owl:Class> <owl:Class rdf:about="&MILO;PerformanceStage <rdfs:subClassOf> <owl:Class r d f : a b o u t = " # F a c i l i t y " > </owl:Class> </rdfs:subClassOf> </owl:Class> <owl:Class rdf:about="&MILO;SecurityAlarm"> <rdfs:subClassOf> <owl:Class rdf:about="#Appliance"> </owl:Class> </rdfs:subClassOf> </owl:Class> <owl:Class rdf:about="&MILO;SportsFacility"> <rdfs:subClassOf> <owl:Class r d f : a b o u t = " # F a c i l i t y " > </owl:Class> </rdfs:subClassOf> </owl:Class> <owl:Class rdf:about="&MILO;SportsGround"> <rdfs:subClassOf> <owl:Class r d f : a b o u t = " # F a c i l i t y " > 114 </owl:Class> </rdfs:subClassOf> </owl:Class> <owl:Class r d f : about="&MILO;Stove"> <rdfs:subClassOf> <owl:Class rdf:about="#Appliance"> </owl:Class> </rdfs:subClassOf> </owl:Class> <owl:Class rdf:about="&MILO;SwimmingPool"> <rdfs:subClassOf> <owl:Class r d f : a b o u t = " # F a c i l i t y " > </owl: Class>. </rdfs:subClassOf> </owl:Class> <owl:Class rdf:about="&MILO;TelephoneLine"> <rdfs:subClassOf> <owl:Class rdf:about="#Appliance"> </owl:Class> </rdfs:subClassOf> </owl:Class> <owl:Class rdf:about="&MILO;Toilet"> <rdfs:subClassOf> <owl:Class rdf:about="#Appliance"> </owl:Class> </rdfs:subClassOf> </owl:Class> <owl:Class rdf:about="&MILO;Wallpaper">. </owl:Class> <owl:Class rdf:about="&MILO;WashingDevice"> <rdfs:subClassOf> <owl:Class rdf:about="#Appliance"> 115 </owl:Class> </rdfs:subClassOf> </owl:Class> <owl:Class rdf:about="&MILQ;Window"> </owl:Class> <owl:Class rdf:about="&MILO;WindowCovering"> </owl:Class> <owl:Class rdf:about="&SUMO;Artifact"> </owl:Class> <owl:Class rdf:about="&SUMO;DirectionalAttribute"> </owl:Class> <owl:Class rdf:about="&SUMO;EducationalOrganization"> <rdfs:subClassOf> <owl:Class r d f : a b o u t = " # F a c i l i t y " > </owl:Class> </rdfs:subClassOf> </owl:Class> <owl:Class rdf:about="&SUMO;MercantileOrganization"> <rdfs:subClassOf> <owl:Class r d f : a b o u t = " # F a c i l i t y " > </owl:Class> </rdfs:subClassOf> </owl:Class> <owl:Class rdf:about="&SUMO;RelationalAttribute"> </owl:Class> <owl:Class rdf:about="&SUMO;Room"> </owl:Class> <owl:Class rdf:about="&SUMO;StationaryArtifact"> </owl:Class> <owl: C l a s s r d f : about=" &.owl; Thing"> </owl:Class> <owl:ObjectProperty rdf:about="#hasApartmentUnit"> 116 <rdfs:domain> <owl:Class rdf:about="&MILO;ApartmentBuilding </owl:Class> </rdfs:domain> <rdfs:range> <owl:Class rdf:about="&MILO;ApartmentUnit"> </owl:Class> </rdfs:range> </owl:ObjectProperty> <owl:ObjectProperty rdf:about="#hasRoom"> <rdfs:domain> <owl:Class rdf:about="&MILO;ApartmentUnit"> </owl:Class> </rdfs:domain> <rdfs:range> <owl:Class'rdf:about="&SUMO;Room">' </owl:Class> </rdfs:range> </owl:Obj ectProperty> <owl:Obj ec t P r o p e r t y rdf:about="#hasWindow"> <rdfs:domain> <owl:Class rdf:about="&SUMO;Room"> </owl:Class> </rdfs:domain> <rdfs:range> <owl:Class rdf:about="&MILO;Window"> </owl:Class> </rdfs:range> </owl:ObjectProperty> <owl:DatatypeProperty rdf:about="#closeToAirport" </owl:DatatypeProperty> 417 <owl:DatatypeProperty rdf:about="#closeToArtSchool"> </owl:DatatypeProperty> <owl:DatatypeProperty rdf:about="#closeToBank"> </owl:DatatypeProperty> <owl:DatatypeProperty r d f : a b o u t = " # c l o s e T o B a s e b a l l F i e l d " > </owl:DatatypeProperty> <owl:DatatypeProperty rdf:about="#closeToBasketballCourt"> </owl:DatatypeProperty> <owl:DatatypeProperty rdf:about="#closeToBeach"> </owl:DatatypeProperty> <owl:DatatypeProperty rdf:about="#closeToBusStop"> </owl:DatatypeProperty> • <owl:DatatypeProperty rdf:about="#closeToBusTerminal"> </owl:DatatypeProperty> <owl:DatatypeProperty rdf:about="#closeToBusinessSchool"> </owl:DatatypeProperty> <owl:DatatypeProperty rdf:about="#closeToCollege"> </owl:DatatypeProperty> <owl:DatatypeProperty rdf:about="#closeToDaySchool"> </owl:DatatypeProperty> <owl:DatatypeProperty rdf:about="#closeToDrugStore"> </owl:DatatypeProperty> . <owl:DatatypeProperty rdf:about="#closeToEducationalOrganization"> </owl:DatatypeProperty> <owl:DatatypeProperty r d f : a b o u t = " # c l o s e T o F a c i l i t y " > <rdfs:domain> <owl:Class rdf:about="&MILO;ApartmentBuilding"> </owl:Class> </rdfs:domain> <rdfs:range> <rdfs:Datatype rdf:about="&xsd;boolean"/> </rdfs:range> 118 </owl:DatatypeProperty> <owl:DatatypeProperty rdf:about="#closeToFerryTerminal"> </owl:DatatypeProperty> <owl:DatatypeProperty rdf:about="#closeToGraduateSchool"> </owl:DatatypeProperty> <owl:DatatypeProperty rdf:about="#closeToGroceryStore"> </owl:DatatypeProperty> <owl:DatatypeProperty rdf:about="#closeToGymnasium"> </owl:DatatypeProperty> <owl:DatatypeProperty rdf:about="#closeToHighSchool"> </owl:DatatypeProperty> <owl:DatatypeProperty rdf:about="#closeToJuniorCollege"> </owl:DatatypeProperty> <owl:DatatypeProperty rdf:about="#closeToLaundromat"> </owl:DatatypeProperty> <owl:DatatypeProperty rdf:about="#closeToLawSchool"> </owl:DatatypeProperty> <owl:DatatypeProperty rdf:about="#closeToLibrary"> </owl:DatatypeProperty> <owl:DatatypeProperty rdf:about="#closeToLiveTheatre"> </owl:DatatypeProperty> <owl:DatatypeProperty rdf:about="#closeToMajorRoadIntersection"> </owl:DatatypeProperty> <owl:DatatypeProperty rdf:about="#closeToMedicalSchool"> </owl:DatatypeProperty> <owl:DatatypeProperty rdf:about="#closeToMercentileOrganization"> </owl:DatatypeProperty> <owl:DatatypeProperty rdf:about="#closeToMovieTheatre"> </owl:DatatypeProperty> <owl:DatatypeProperty rdf:about="#closeToMuseum"> 119 </owl:DatatypeProperty> <owl:DatatypeProperty rdf:about="#closeToPark"> </owl:DatatypeProperty> <owl:DatatypeProperty rdf:about="#closeToParkingLot"> </owl:DatatypeProperty> <owl:DatatypeProperty rdf:about="#closeToPostSecondarySchool"> </owl:DatatypeProperty> <owl:DatatypeProperty rdf:about="#closeToPrivateSchool"> </owl:DatatypeProperty> <owl:DatatypeProperty rdf:about="#closeToPublicLibrary"> </owl:DatatypeProperty> <owl:DatatypeProperty rdf:about="#closeTpPublicSchool"> </owl:DatatypeProperty> <owl:DatatypeProperty rdf:about="#closeToPubs"> </owl:DatatypeProperty> <owl: DatatypeProperty r d f : about="#closeToRai.lwayTerminal"> </owl:DatatypeProperty> <owl:DatatypeProperty rdf:about="#closeToRestaurants"> </owl:DatatypeProperty> <owl:DatatypeProperty rdf:about="#closeToRetaIlStore"> </owl:DatatypeProperty> <owl:DatatypeProperty rdf:about="#closeToSchool"> </owl:DatatypeProperty> <owl:DatatypeProperty rdf:about="#closeToSecondarySchool"> </owl:DatatypeProperty> <owl:DatatypeProperty rdf:about="#closeToShoppingMall"> ' </owl:DatatypeProperty> <owl:DatatypeProperty rdf:about="#closeToSoccerField"> </owl:DatatypeProperty> <owl:DatatypeProperty rdf:about="#closeToSportsFacility"> </owl:DatatypeProperty> 120 <owl:DatatypeProperty rdf:about="#closeToSportsGround"> </owl:DatatypeProperty> <owl:DatatypeProperty rdf:about="#closeToSwimmingPool"> </owl:DatatypeProperty> • <owl:DatatypeProperty rdf:about="#closeToTheate"> </owl:DatatypeProperty> <owl:DatatypeProperty rdf:about="#closeToTouristAttraction"> </owl:DatatypeProperty> <owl:DatatypeProperty rdf:about="#closeToTransitTerminal"> </owl:DatatypeProperty> <owl:DatatypeProperty rdf:about="#closeToUniversity"> </owl:DatatypeProperty> <owl:DatatypeProperty rdf:about="#closeToVocationalSchool"> </owl:DatatypeProperty> <owl:DatatypeProperty rdf:about="#closeToWholesaleStore"> </owl:DatatypeProperty> <owl:DatatypeProperty rdf:about="#facesEast"> <rdfs:domain> <owl:Class rdf:about="&MILO;Window"> </owl:Class> </rdfs:domain> <rdfs:range> <rdfs:Datatype rdf:about="&xsd;boolean"/> </rdfs:range> </owl:DatatypeProperty> <owl:DatatypeProperty rdf:about="#facesNorth"> <rdfs:domain> <owl:Class rdf:about="&MILO;Window"> </owl:Class> </rdfs:domain> <rdfs:range> <rdfs:Datatype rdf:about="&xsd;boolean"/> </rdfs:range> </owl:DatatypeProperty> <owl:DatatypeProperty rdf:about="#facesSouth"> 121 <rdfs:domain> <owl:Class rdf:about="&MILO;Window"> </owl:Class> </rdfs:domain> <rdfs:range> <rdfs:Datatype rdf:about="&xsd;boolean"/> </rdfs:range> </owl:DatatypeProperty> <owl:DatatypeProperty rdf:about="#facesWest"> <rdfs:domain> <owl:Class rdf:about="&MILO;Window"> </owl:Class> </rdfs:domain> . <rdfs:range> <rdfs:Datatype rdf:about="&xsd;boolean"/> </rdfs:range> </owl:DatatypeProperty> <owl:DatatypeProperty rdf:about="#hasAerobicExerciseDevice"> </owl:DatatypeProperty> <owl:DatatypeProperty rdf:about="#hasAirConditioning"> </owl:DatatypeProperty> <owl:DatatypeProperty rdf:about="#hasAppliance"> <rdfs:domain> <owl:Class rdf:about="&MILO;ApartmentUnit"> </owl:Class> </rdfs:domain> <rdfs:range> <rdfs:Datatype rdf:about="&xsd;boolean"/> </rdfs:range> </owl:DatatypeProperty> <owl:DatatypeProperty rdf:about="#hasAreaMeasure"> <rdfs:domain> <owl:Class rdf:about="&SUMO;Room"> </owl:Class> </rdfs:domain> <rdfs:range> <rdfs:Datatype rdf:about="&xsd;boolean"/> </rdfs:range> 122 </owl:DatatypeProperty> <owl:DatatypeProperty rdf:about="#hasBalcony"> </owl:DatatypeProperty> <owl:DatatypeProperty rdf:about="#hasBathroom"> </owl:DatatypeProperty> <owl:DatatypeProperty rdf:about="#hasBathtub"> </owl:DatatypeProperty> <owl:DatatypeProperty rdf:about="#hasBedroom"> <rdfs:subPropertyOf rdf:resource="#hasRoom" /> </owl:DatatypeProperty> <owl:DatatypeProperty rdf:about="#hasCarpet"> </owl: DatatypeProperty>. <owl:DatatypeProperty rdf:about="#hasChair"> </owl:DatatypeProperty> <owl:DatatypeProperty rdf:about="#hasChimney"> </owl:DatatypeProperty> <owl:DatatypeProperty rdf:about="#hasClpset"> </owl:DatatypeProperty> . <owl:DatatypeProperty rdf:about="#hasDen"> </owl:DatatypeProperty> <owl:DatatypeProperty rdf:about="#hasDiningRoom"> </owl:DatatypeProperty> <owl:DatatypeProperty rdf:about="#hasDishwasher"> </owl:DatatypeProperty> <owl:DatatypeProperty rdf:about="#hasDoor"> <rdfs:domain> <owl:Class rdf:about="&SUMO;Room"> </owl:Class> </rdfs:domain> <rdfs:range> <rdfs:Datatype rdf:about="&xsd;boolean"/> </rdfs:range> </owl:DatatypeProperty> <owl: DatatypeProperty r d f : about="#hasDre'ssingRoom"> 123 </owl:DatatypeProperty> <owl:DatatypeProperty rdf:about="#hasDryer"> </owl:DatatypeProperty> <owl:DatatypeProperty rdf:about="#hasFacility"> <rdfs:domain> <owl:Class rdf:about="&MILO;ApartmentBuilding"> </owl:Class> </rdfs:doraain> <rdfs:range> <rdfs:Datatype rdf:about="&xsd;boolean"/> </rdfs:range> </owl:DatatypeProperty> <owl:DatatypeProperty rdf:about="#hasFan"> </owl:DatatypeProperty> <owl:DatatypeProperty rdf:about="#hasFireplace"> </owl: DatatypeProperty>. <owl:DatatypeProperty rdf:about="#hasFlooring"> <rdfs:domain> <owl:Class rdf:about="&SUMO;Room"> </owl:Class> </rdfs:domain> <rdfs:range> <rdfs:Datatype rdf:about="&xsd;boolean"/> </rdfs:range> </owl:DatatypeProperty> <owl:DatatypeProperty rdf:about="#hasFoyer"> </owl:DatatypeProperty> <owl:DatatypeProperty rdf:about="#hasFridge"> </owl:DatatypeProperty> <owl:DatatypeProperty rdf:about="#hasGrounds"> </owl:DatatypeProperty> <owl:DatatypeProperty rdf:about="#hasGymnasium"> </owl:DatatypeProperty> <owl:DatatypeProperty rdf:about="#hasHallway"> </owl:DatatypeProperty> 124 <owl:DatatypeProperty rdf:about="#hasHardwood"> </owl:DatatypeProperty> <owl:DatatypeProperty rdf:about="#hasKitchen"> </owl:DatatypeProperty> <owl:DatatypeProperty rdf:about="#hasLargeArea"> </owl:DatatypeProperty> <owl:DatatypeProperty rdf:about="#hasLaundryFacility </owl:DatatypeProperty> <owl:DatatypeProperty rdf:about="#hasLibrary"> </owl:DatatypeProperty> <owl:DatatypeProperty rdf:about="#hasLinoleum"> </owl:DatatypeProperty> <owl:DatatypeProperty rdf:about="#hasLivingRoom"> </owl:DatatypeProperty> <owl:DatatypeProperty rdf:about="#hasMailbox"> </owl:DatatypeProperty> <owl:DatatypeProperty rdf:about="#hasMediumArea"> </owl:DatatypeProperty> <owl:DatatypeProperty rdf:about="#hasMountainView"> </owl:DatatypeProperty> <owl:DatatypeProperty rdf:about="#hasOceanView"> </owl:DatatypeProperty> <;owl: DatatypeProperty r d f : about="#hasOf f ice"> </owl:DatatypeProperty> <owl:DatatypeProperty rdf:about="#hasOilPaint"> </owl:DatatypeProperty> <owl:DatatypeProperty.rdf:about="#hasOven"> </owl:DatatypeProperty> <owl:DatatypeProperty rdf:about="#hasPaint"> <rdfs:domain> <owl:Class rdf:about="&SUMO;Room" </owl:Class> </rdfs:domain> 125 <rdf s : range> <rdfs : Datatype r d f : about="&xsd;boolean"/> </rdfs:range> </owl:DatatypeProperty> <owl:DatatypeProperty rdf:about="#hasPainting"> <rdfs:domain> <owl:Class rdf:about="&SUMO;Room"> </owl:Class> </rdfs:domain> <rdfs:range> <rdfs:Datatype rdf:about="&xsd;boolean"/> </rdfs:range> </owl:DatatypeProperty> <owl:DatatypeProperty rdf:about="#hasPark"> </owl:DatatypeProperty> <owl:DatatypeProperty rdf:about="#hasParkingGarage"> </owl:DatatypeProperty> <owl: DatatypeProperty r d f : about="#hasParkingLot">. </owl:DatatypeProperty> <owl:DatatypeProperty rdf:about="#hasSecurityAlarm"> </owl:DatatypeProperty> <owl:DatatypeProperty rdf:about="#hasShower"> </owl:DatatypeProperty> ' <owl:DatatypeProperty rdf:about="#hasSink"> </owl:DatatypeProperty> <owl:DatatypeProperty rdf:about="#hasSkylight"> </owl:DatatypeProperty> <owl:DatatypeProperty rdf:about="#hasSmallArea"> </owl:DatatypeProperty> <owl:DatatypeProperty rdf:about="#hasSportsFacility"> </owl:DatatypeProperty> <owl: DatatypeProperty r d f : about="#h,asStove"> </owl:DatatypeProperty> <owl:DatatypeProperty rdf:about="#hasSwimmingPool"> </owl:DatatypeProperty> 126 <owl:DatatypeProperty rdf:about="#hasTelephoneLine"> </owl:DatatypeProperty> <owl:DatatypeProperty rdf:about="#hasToilet"> </owl:DatatypeProperty> <owl:DatatypeProperty rdf:about="#hasTreeView"> </owl:DatatypeProperty> <owl:DatatypeProperty rdf:about="#hasView"> <rdfs:domain> <owl:Class rdf:about="&MILO;ApartmentUnit"> </owl:Class> </rdfs:domain> <rdfs:range> <rdfs:Datatype rdf:about="&xsd;boolean"/> </rdfs:range> </owl:DatatypeProperty> <owl:DatatypeProperty rdf:about="#hasVinyl"> </owl:DatatypeProperty> <owl:DatatypeProperty rdf:about="#hasWallpaper"> <rdfs:domain> <owl:Class rdf:about="&SUMO;Room"> </owl:Class> </rdfs:domain> <rdfs:range> <rdfs:Datatype rdf:about="&xsd;boolean"/> </rdfs:range> </owl:DatatypeProperty> <owl:DatatypeProperty rdf:about="#hasWashBasin"> </owl:DatatypeProperty> <owl:DatatypeProperty rdf:about="#hasWasher"> </owl:DatatypeProperty> • <owl:DatatypeProperty rdf:about="#hasWatercolorPaint"> • </owl:DatatypeProperty> <owl:DatatypeProperty rdf:about="#hasWindowCovering"> <rdfs:domain> <owl:Class rdf:about="&SUMO;Room"> </owl:Class> 127 </rdfs:domain> <rdfs:range> <rdfs:Datatype rdf:about="&xsd;boolean"/> </rdfs:range> </owl:DatatypeProperty> <owl:DatatypeProperty rdf:about="#closeToVocationalSchool"> <rdfs:subPropertyOf> <owl:DatatypeProperty rdf:about="#closeToSchool"> </owl:DatatypeProperty> </rdfs:subPropertyOf> </owl:DatatypeProperty> <owl:DatatypeProperty rdf:about="#closeToLawSchool"> <rdfs:subPropertyOf> <owl:DatatypeProperty rdf:about="#closeToGraduateSchool"> </owl:DatatypeProperty> </rdfs:subPropertyOf> </owl:DatatypeProperty> . ' <owl: DatatypeProperty r d f : about="#,closeToSportsFacility"> <rdfs:subPropertyOf> <owl:DatatypeProperty rdf:about="#closeToFacility"> </owl:DatatypeProperty> </rdfs:subPropertyOf> </owl:DatatypeProperty> <owl:DatatypeProperty rdf:about="#closeToBusTerminal"> <rdfs:subPropertyOf> <owl:DatatypeProperty rdf:about="#closeToBusStop"> </owl:DatatypeProperty> </rdfs:subPropertyOf>' 128 </owl:DatatypeProperty> <owl:DatatypeProperty rdf:about="#hasCarpet"> <rdfs:subPropertyOf> <owl:DatatypeProperty rdf:about="#hasFlooring"> </owl:DatatypeProperty> </rdfs:subPropertyOf> </owl:DatatypeProperty> <owl:DatatypeProperty rdf:about="#closeToGymnasium"> <rdfs:subPropertyOf> <owl:DatatypeProperty rdf:about="#closeToSportsFacility"> </owl:DatatypeProperty> </rdfs:subPropertyOf> 1 </owl:DatatypeProperty> <owl:DatatypeProperty rdf:about="#closeToPublicSchool"> <rdfs:subPropertyOf> <bwl:DatatypeProperty rdf:about="#closeToSchool"> </owl:DatatypeProperty> </rdfs:subPropertyOf> </owl:DatatypeProperty> <owl:DatatypeProperty rdf:about="#closeToTheate"> <rdfs:subPropertyOf> <owl:DatatypeProperty rdf:about="#closeToFacility"> </owl:DatatypeProperty> </rdfs:subPropertyOf> </owl:DatatypeProperty> <owl:DatatypeProperty rdf:about="#hasMountainView"> <rdfs:subPropertyOf> 129 <owl:DatatypeProperty rdf:about="#hasView"> </owl:DatatypeProperty> </rdfs:subPropertyOf> </owl:DatatypeProperty> <owl:DatatypeProperty rdf:about="#closeToMercentileOrgariization"> <rdfs:subPropertyOf> <owl:DatatypeProperty rdf:about="#closeToFacility"> </owl:DatatypeProperty> </rdfs:subPropertyOf> </owl:DatatypeProperty> <owl:DatatypeProperty rdf:about="#closeToCollege"> <rdfs:subPropertyOf> <owl:DatatypeProperty rdf:about="#closeToUniversity" </owl:DatatypeProperty> </rdfs:subPropertyOf> </owl:DatatypeProperty> <owl:DatatypeProperty rdf:about="#closeToPubs"> <rdfs:subPropertyOf> <owl:DatatypeProperty rdf:about="#closeToMercentileOrganization"> </owl:DatatypeProperty> </rdfs:subPropertyOf> </owl:DatatypeProperty> <owl:DatatypeProperty rdf:about="#hasSmallArea"> <rdfs:subPropertyOf> <owl:DatatypeProperty rdf:about="#hasAreaMeasure"> </owl:DatatypeProperty> 130 </rdfs:subPropertyOf> </owl:DatatypeProperty> -Cowl: DatatypeProperty r d f : about="#closeToPublicLibrary <rdfs:subPropertyOf> <owl:DatatypeProperty rdf:about="#closeToLibrary"> </owl:DatatypeProperty> . , </rdfs:subPropertyOf> </owl:DatatypeProperty> ' <owl:DatatypeProperty rdf:about="#hasPark"> <rdfs:subPropertyOf> <owl:DatatypeProperty rdf:about="#hasFacility"> </owl:DatatypeProperty> </rdfs:subPropertyOf> </owl:DatatypeProperty> <owl:DatatypeProperty rdf:about="#hasAirConditioning"> <rdfs:subPropertyOf> <owl:DatatypeProperty rdf:about="#hasAppliance"> </owl:DatatypeProperty> </rdfs:subPropertyOf> </owl:DatatypeProperty> <owl:DatatypeProperty rdf:about="#closeToBeach"> <rdfs:subPropertyOf> <owl:DatatypeProperty rdf:about="#closeToFacility"> </owl:DatatypeProperty> </rdfs:subPropertyOf> </owl:DatatypeProperty> <owl:DatatypeProperty rdf:about="#hasMediumArea"> 131 <rdfs:subPropertyOf> <owl:DatatypeProperty rdf:about="#hasAreaMeasure"> </owl:DatatypeProperty> </rdfs:subPropertyOf> </owl:DatatypeProperty> <owl:DatatypeProperty rdf:about="#hasOceanView"> <rdfs:subPropertyOf> • <owl:DatatypeProperty rdf:about="#hasView"> </owl:DatatypeProperty> </rdfs:subPropertyOf> </owl:DatatypeProperty> <owl:DatatypeProperty rdf:about="#nasGrounds"> <rdfs:subPropertyOf> <owl:DatatypeProperty rdf:about="#hasFacility"> </owl:DatatypeProperty> </rdfs:subPropertyOf> </owl:DatatypePrdperty> <owl:DatatypeProperty rdf:about="#closeToGraduateSchool" <rdfs:subPropertyOf> <owl:DatatypeProperty rdf:about="#closeToUniversity"> </owl:DatatypeProperty> </rdfs:subPropertyOf> </owl:DatatypeProperty> <owl:DatatypeProperty rdf:about="#hasWatercolorPaint"> <rdfs:subPropertyOf> <owl:DatatypeProperty rdf:about="#hasPaint"> </owl:DatatypeProperty> </rdfs:subPropertyOf> 132 </owl:DatatypeProperty> <owl:DatatypeProperty rdf:about="#hasLaundryFacility"> <rdfs:subPropertyOf> <owl:DatatypeProperty rdf:about="#hasFacility"> </owl:DatatypeProperty> </rdfs:subPropertyOf> </owl:DatatypeProperty> <owl:DatatypeProperty rdf:about="#hasLinoleum"> <rdfs:subPropertyOf> <owl:DatatypeProperty rdf:about="#hasFlooring"> </owl:DatatypeProperty> </rdfs:subPropertyOf> </owl:DatatypeProperty> <owl:DatatypeProperty rdf:about="#hasMailbox"> <rdfs:subPropertyOf> <owl:DatatypeProperty rdf:about="#hasAppliance"> </owl:DatatypeProperty> </rdfs:subPropertyOf> </owl:DatatypeProperty> <owl:DatatypeProperty rdf:about="#hasDishwasher"> <rdfs:subPropertyOf> <owl:DatatypeProperty rdf:about="#hasAppliance"> </owl:DatatypeProperty> </rdfs:subPropertyOf> </owl:DatatypeProperty> <owl: DatatypeProperty r d f : about="#closeToMo'vieTheatre"> <rdfs:subPropertyOf> 133 <owl:DatatypeProperty rdf:about="#closeToTheate"> </owl:DatatypeProperty> </rdfs:subPropertyOf> </owl:DatatypeProperty> <owl:DatatypeProperty rdf:about="#closeToDaySchool"> <rdfs:subPropertyOf> <owl:DatatypeProperty rdf:about="#closeToSchpol"> </owl:DatatypeProperty> </rdfs:subPropertyOf> </owl:DatatypeProperty> <owl:DatatypeProperty rdf:about="#closeToGroceryStore"> <rdfs:subPropertyOf> <owl:DatatypeProperty rdf:about="#closeToRetailStore"> </owl:DatatypeProperty> </rdfs:subPropertyOf> </owl:DatatypeProperty> <owl:DatatypeProperty rdf:about="#closeToPark"> <rdfs:subPropertyOf> <owl:DatatypeProperty rdf:about="#closeToFacility"> </owl:DatatypeProperty> </rdfs:subPropertyOf> </owl:DatatypeProperty> <owl:DatatypeProperty rdf:about="#closeToRailwayTerminal"> <rdf's: subPropertyOf > <owl:DatatypeProperty rdf:about="#closeToTransitTerminal"> </owl:DatatypeProperty> </rdfs:subPropertyOf> 134 </owl: DatatypeProperty>. <owl:DatatypeProperty rdf:about="#hasLibrary"> <rdfs:subPropertyOf> <owl:DatatypeProperty rdf:about="#hasFacility"> </owl:DatatypeProperty> </rdfs:subPropertyOf> </owl:DatatypeProperty> <owl:DatatypeProperty rdf:about="#closeToMedicalSchool"> <rdfs:subPropertyOf> <owl:DatatypeProperty rdf:about="#closeToGraduateSchool" </owl:DatatypeProperty> </rdfs:subPropertyOf> </owl:DatatypeProperty> <owl:DatatypeProperty rdf:about="#closeToRestaurants"> <rdfs:subPropertyOf> <owl:DatatypeProperty rdf:about="#closeToMercentileOrganization"> </owl:DatatypeProperty> </rdfs:subPropertyOf> </owl:DatatypeProperty> <owl:DatatypeProperty rdf:about="#hasToilet"> <rdfs:subPropertyOf> <owl: DatatypeProperty. r d f : about="#hasAppliance"> </owl:DatatypeProperty> </rdfs:subPropertyOf> </owl:DatatypeProperty> <owl:DatatypeProperty rdf:about="#hasOven"> 135 <rdfs:subPropertyOf> <owl:DatatypeProperty rdf:about="#hasAppliance"> </owl:DatatypeProperty> </rdfs:subPropertyOf> , . </owl:DatatypeProperty> <owl:DatatypeProperty rdf:about="#hasFireplace"> .<rdfs:subPropertyOf> <owl:DatatypeProperty rdf:about="#hasAppliance"> </owl:DatatypeProperty> ' </rdfs:subPropertyOf> </owl:DatatypeProperty> <owl:DatatypeProperty rdf:about="#closeToDrugStore"> <rdfs:subPropertyOf> <owl:DatatypeProperty rdf:about="#closeToRetailStore" </owl:DatatypeProperty> / </rdfs:subPropertyOf> </owl:DatatypeProperty> <owl:DatatypeProperty rdf:about="#hasFan"> <rdfs:subPropertyOf> <owl:DatatypeProperty rdf:about="#hasAppliance"> </owl:DatatypeProperty> </rdfs:subPropertyOf> </owl:DatatypeProperty> <owl:DatatypeProperty rdf:about="#hasTreeView"> <rdfs:subPropertyOf> <owl:DatatypeProperty rdf:about="#hasView"> </owl:DatatypeProperty> </rdfs:subPropertyOf> • 136 </owl:DatatypeProperty> <owl:DatatypeProperty r d f : abo.ut="#closeToTouristAttraction"> <rdfs:subPropertyOf> <owl:DatatypeProperty rdf:about="#closeToFacility"> </owl:DatatypeProperty> </rdfs:subPropertyOf> </owl:DatatypeProperty> <owl:DatatypeProperty rdf:about="#closeToMaj orRoadIntersection"> <rdfs:subPropertyOf> <owl:DatatypeProperty rdf:about="#closeToFacility"> </owl:DatatypeProperty> </rdfs:subPropertyOf> </owl:DatatypeProperty> <owl:DatatypeProperty rdf:about="#hasParkingGarage"> <rdfs:subPropertyOf> <owl:DatatypeProperty rdf:about="#hasFacility"> </owl:DatatypeProperty> </rdfs:subPropertyOf> </owl:DatatypeProperty> <owl:DatatypeProperty rdf:about="#closeToMuseum"> <rdfs:subPropertyOf> <owl:DatatypeProperty rdf:about="#closeToEducationalOrganization"> </owl:DatatypeProperty> </rdfs:subPropertyOf> </owl:DatatypeProperty> 137 <owl:DatatypeProperty rdf:about="#closeToSwimmingPool"> <rdfs:subPropertyOf> <owl': DatatypeProperty r d f : about="#closeToSportsFacility" </owl:DatatypeProperty> </rdfs:subPropertyOf> </owl:DatatypeProperty> <owl:DatatypeProperty rdf:about="#closeToWholesaleStore"> <rdfs:subPropertyOf> <owl:DatatypeProperty rdf:about="#closeToMercentileOrganization"> </owl:DatatypeProperty> </rdfs:subPropertyOf> . </owl:DatatypeProperty> <owl:DatatypeProperty rdf:about="#closeToBaseballField"> <rdfs:subPropertyOf> <owl:DatatypeProperty rdf:about="#closeToSportsGround"> </owl:DatatypeProperty> </rdfs:subPropertyOf> </owl:DatatypeProperty> <ow.l: DatatypeProperty r d f : about="#hasChimney"> <rdfs:subPropertyOf> <owl:DatatypeProperty rdf:about="#hasAppliance"> </owl:DatatypeProperty> </rdfs:subPropertyOf> </owl:DatatypeProperty> <owl:DatatypeProperty rdf:about="#hasSportsFacility"> <rdfs:subPropertyOf> 138 <owl:DatatypeProperty rdf:about="#hasFacility"> </owl:DatatypeProperty> </rdfs:subPropertyOf> </owl:DatatypeProperty> <owl:DatatypeProperty rdf:about="#closeToLiveTheatre"> <rdfs:subPropertyOf> <owl:DatatypeProperty rdf:about="#closeToTheate"> </owl:DatatypeProperty> </rdfs:subPropertyOf> </owl:DatatypeProperty> <owl:DatatypeProperty rdf:about="#closeToPrivateSchool" <rdfs:subPropertyOf> <owl:DatatypeProperty rdf:about="#closeToSchool"> </owl:DatatypeProperty> </rdfs:subPropertyOf> </owl:DatatypeProperty> <owl:DatatypeProperty rdf:about="#closeToUniversity"> <rdfs:subPropertyOf> <owl:DatatypeProperty rdf:about="#closeToPostSecondarySchool"> </owl:DatatypeProperty> </rdfs:subPropertyOf> </owl:DatatypeProperty> <owl:DatatypeProperty rdf:about="#closeToPostSecondarySchool"> <rdfs:subPropertyOf> <owl:DatatypeProperty rdf:about="#closeToSchool"> </owl:DatatypeProperty> 139 </rdfs:subPropertyOf> </owl:DatatypeProperty> <owl:DatatypeProperty rdf:about="#closeToBusStop"> <rdfs:subPropertyOf> <owl:DatatypeProperty rdf:about="#closeToTransitTerminal"> </owl:DatatypeProperty> </rdfs:subPropertyOf> </owl:DatatypeProperty> <owl:DatatypeProperty rdf:about="#closeToEducationalOrganization"> <rdfs:subPropertyOf> <owl:DatatypeProperty rdf:about="#closeToFacility"> </owl:DatatypeProperty> </rdfs:subPropertyOf> </owl:DatatypeProperty> <owl:DatatypeProperty rdf:about="#hasStove"> <rdfs:subPropertyOf> <owl: DatatypeProperty. r d f : about="#hasAppliance"> </owl:DatatypeProperty> </rdfs:subPropertyOf> </owl:DatatypeProperty> <owl:DatatypeProperty rdf:about="#hasGymnasium"> <rdfs:subPropertyOf> <owl:DatatypeProperty rdf:about="#hasSportsFacility" </owl:DatatypeProperty> </rdfs:subPropertyOf> </owl:DatatypeProperty> 140 <owl:DatatypeProperty rdf:about="#closeToSoccerField"> <rdfs:subPropertyOf> <owl:DatatypeProperty rdf:about="#closeToSportsGrdund"> </owl:DatatypeProperty> </rdfs:subPropertyOf> </owl:DatatypeProperty> <owl:DatatypeProperty rdf:about="#hasChair"> <rdfs:subPropertyOf> <owl:DatatypeProperty rdf:about="#hasAppliance"> </owl: DatatypeProperty>. </rdfs:subPropertyOf> </owl:DatatypeProperty> <owl:DatatypeProperty rdf:about="#closeToJuniorCollege"> <rdfs:subPropertyOf> <owl:DatatypeProperty rdf:about="#closeToPostSecondarySchool"> </owl:DatatypeProperty> </rdfs:subPropertyOf> </owl:DatatypeProperty> <owl:DatatypeProperty rdf:about="#closeToHighSchool"> <rdfs:subPropertyOf> <owl:DatatypeProperty rdf:about="#closeToSchool"> </owl:DatatypeProperty> </rdfs:subPropertyOf> </owl:DatatypeProperty> <owl:DatatypeProperty rdf:about="#hasParkingLot"> <rdfs:subPropertyOf> 141 <owl:DatatypeProperty rdf:about="#hasFacility"> </owl:DatatypeProperty> </rdfs:subPropertyOf> </owl:DatatypeProperty> <owl:DatatypeProperty rdf:about="#closeToShoppingMall"> <rdfs,: subPropertyOf > <owl:DatatypeProperty rdf:about="#closeToMercentileOrganization"> </owl:DatatypeProperty> </rdfs:subPropertyOf> </owl:DatatypeProperty> <owl:DatatypeProperty rdf:about="#closeToBusinessSchool"> <rdfs:subPropertyOf> <owl:DatatypeProperty rdf:about="#closeToGraduateSchool" </owl:DatatypeProperty> </rdfs:subPropertyOf> </owl:DatatypeProperty> <owl:DatatypeProperty rdf:about="#closeToSchool"> <rdfs:subPropertyOf> <owl:DatatypeProperty rdf:about="#closeToEducationalOrganization"> • </owl:DatatypeProperty> </rdfs:subPropertyOf> </owl:DatatypeProperty> <owl:DatatypeProperty rdf:about="#hasShower"> <rdfs:subPropertyOf> <owl:DatatypeProperty rdf:about="#hasAppliance"> </owl:DatatypeProperty> 142 ' </rdfs:subPropertyOf> </owl:DatatypeProperty> <owl:DatatypeProperty rdf:about="#hasLargeArea"> <rdfs:subPropertyOf> <owl:DatatypeProperty rdf:about="#hasAreaMeasure" </owl:DatatypeProperty> </rdfs:subPropertyOf> </owl:DatatypeProperty> <owl:DatatypeProperty rdf:about="#hasWasher"> <rdfs:subPropertyOf> <owl:DatatypeProperty rdf:about="#hasAppliance"> </owl:DatatypeProperty> </rdfs:subPropertyOf> </owl:DatatypeProperty> <owl:DatatypeProperty rdf:about="#hasFridge"> <rdfs:subPropertyOf> <owl:DatatypeProperty rdf:about="#hasAppliance"> </owl:DatatypeProperty> </rdfs:subPropertyOf> </owl: DatatypeProperty>. ,<owL: DatatypeProperty r d f : about="#closeToAirport"> <rdfs:subPropertyOf> <owl:DatatypeProperty rdf:about="#closeToTransitTerminal"> </owl:DatatypeProperty> </rdfs:subPropertyOf> </owl:DatatypeProperty> 143 <owl:DatatypeProperty rdf:about="#hasDryer"> <rdfs:subPropertyOf> <owl:DatatypeProperty rdf:about="#hasAppliance"> </owl:DatatypeProperty> </rdfs:subPropertyOf> </owl:DatatypeProperty> <owl:DatatypeProperty rdf:about="#hasAerobicExerciseDevice"> <rdfs:subPropertyOf> <owl:DatatypeProperty rdf:about="#hasAppliance"> </owl:DatatypeProperty> </rdfs:subPropertyOf> </owl:DatatypeProperty> <owl:DatatypeProperty rdf:about="#closeToParkingLot"> <rdfs:subPropertyOf> <owl:DatatypeProperty rdf:about="#closeToFacility"> </owl:DatatypeProperty> </rdfs:subPropertyOf> </owl:DatatypeProperty> <owl:DatatypeProperty rdf:about="#hasVinyl"> <rdfs:subPropertyOf> <owl:DatatypeProperty rdf:about="#hasFlooring"> </owl:DatatypeProperty> </rdfs:subPropertyOf> </owl:DatatypeProperty> <owl:DatatypeProperty rdf:about="#closeToSecondarySchool"> <rdfs:subPropertyOf> <owl:DatatypeProperty rdf:about="#closeToSchool"> 144 </owl:DatatypeProperty> </rdfs:subPropertyOf> </owl:DatatypeProperty> <owl:DatatypeProperty rdf:about="#closeToRetailStore"> <rdfs:subPropertyOf> , <owl:DatatypeProperty rdf:about="#closeToMercentileOrganization"> </owl:DatatypeProperty> </rdfs:subPropertyOf> </owl:DatatypeProperty> <owl:DatatypeProperty rdf:about="#hasHardwood"> <rdfs:subPropertyOf> <owl:DatatypeProperty rdf:about="#hasFlooring"> </owl: DatatypeProper'ty> </rdfs:subPropertyOf> </owl:DatatypeProperty> <owl:DatatypeProperty rdf:about="#hasSecurityAlarm"> <rdfs:subPropertyOf> <owl:DatatypeProperty r d f :about="#hasApp.liance"> </owl:DatatypeProperty> </rdfs:subPropertyOf> </owl:DatatypeProperty> <owl:DatatypeProperty rdf:about="#hasSink"> <rdfs:subPropertyOf> <owl:DatatypeProperty rdf:about="#hasAppliance"> </owl:DatatypeProperty> </rdfs:subPropertyOf> </owl:DatatypeProperty> 145 <owl:DatatypeProperty rdf:about="#hasTelephoneLine"> <rdfs:subPropertyOf> <owl:DatatypeProperty rdf:about="#hasAppliance"> </owl:DatatypeProperty> </rdfs:subPropertyOf> </owl:DatatypeProperty> <owl:DatatypeProperty rdf:about="#closeToLaundromat"> <rdfs:subPropertyOf> <owl:DatatypeProperty rdf:about="#closeToFacility"> </owl:DatatypeProperty> </rdfs:subPropertyOf> </owl:DatatypeProperty> <owl:DatatypeProperty rdf:about="#closeToSportsGround"> <rdfs:subPropertyOf> <owl:DatatypeProperty rdf:about="#closeToFacility"> </owl:DatatypeProperty> </rdfs:subPropertyOf> </owl:DatatypeProperty> <owl:DatatypeProperty rdf:about="#hasOilPaint"> <rdfs:subPropertyOf> <owl:DatatypeProperty rdf:about="#hasPaint"> </owl:DatatypeProperty> </rdfs:subPropertyOf> </owl:DatatypeProperty> <owl:DatatypeProperty rdf:about="#closeToBank"> <rdfs:subPropertyOf> <owl: DatatypeProperty r d f : about="#closeToFa'cility"> 146 </owl:DatatypeProperty> </rdfs:subPropertyOf> </owl:DatatypeProperty> <owl:DatatypeProperty rdf:about="#hasSwimmingPool"> <rdfs:subPropertyOf> <owl:DatatypeProperty rdf:about="#hasSportsFacility"> </owl:DatatypeProperty> </rdfs:subPropertyOf> </owl:DatatypeProperty> <owl:DatatypeProperty rdf:about="#closeToArtSchool"> <rdfs:subPropertyOf> <owl:DatatypeProperty rdf:about="#closeToSchool"> </owl: DatatypePropert.y> </rdfs:subPropertyOf> </owl:DatatypeProperty> <owl:DatatypeProperty rdf:about="#closeToLibrary"> <rdfs:subPropertyOf> <owl:DatatypeProperty ' rdf:about="#closeToEducationalOrganization"> </owl:DatatypeProperty> </rdfs:subPropertyOf> </owl:DatatypeProperty> <owl:DatatypeProperty rdf:about="#closeToBasketballCourt"> <rdfs:subPropertyOf> <owl:DatatypeProperty rdf:about="#closeToSportsGround"> </owl:DatatypeProperty> </rdfs:subPropertyOf> </owl:DatatypeProperty> 147 < o w l : D a t a t y p e P r o p e r t y r d f : a b o u t = " # c l o s e T o F e r r y T e r m i n a l " > < r d f s : s u b P r o p e r t y O f > < o w l : D a t a t y p e P r o p e r t y r d f : a b o u t = " # c l o s e T o T r a n s i t T e r m i n a l " > < /owl:DatatypeProperty> < / r d f s : s u b P r o p e r t y O f > </owl:DatatypeProperty> < o w l : D a t a t y p e P r o p e r t y r d f : a b o u t = " # c l o s e T o T r a n s i t T e r m i n a l " > < r d f s : s u b P r o p e r t y O f > < o w l : D a t a t y p e P r o p e r t y r d f : a b o u t = " # c l o s e T o F a c i l i t y " > < / owl:DatatypeProperty> < / r d f s : s u b P r o p e r t y O f > </owl:DatatypeProperty> </rdf:RDF> When an element is defined in an ontology that has been imported it is referred to by appending the symbol defined in the header of the O W L file for the ontology to the name of the entity. For instance, the APARTMENT-UNIT entity in MILO is referred to as r d f : about=" &MILO; A p a r t m e n t U n i t " . The imported SUMO and MILO ontologies can be found at http://ontology.teknowledge.eom/#download 148 

Cite

Citation Scheme:

        

Citations by CSL (citeproc-js)

Usage Statistics

Share

Embed

Customize your widget with the following options, then copy and paste the code below into the HTML of your page to embed this item in your website.
                        
                            <div id="ubcOpenCollectionsWidgetDisplay">
                            <script id="ubcOpenCollectionsWidget"
                            src="{[{embed.src}]}"
                            data-item="{[{embed.item}]}"
                            data-collection="{[{embed.collection}]}"
                            data-metadata="{[{embed.showMetadata}]}"
                            data-width="{[{embed.width}]}"
                            data-media="{[{embed.selectedMedia}]}"
                            async >
                            </script>
                            </div>
                        
                    
IIIF logo Our image viewer uses the IIIF 2.0 standard. To load this item in other compatible viewers, use this url:
https://iiif.library.ubc.ca/presentation/dsp.831.1-0052054/manifest

Comment

Related Items