UBC Theses and Dissertations

UBC Theses Logo

UBC Theses and Dissertations

A driver visual attention model Lim, Clark C. 1997

Your browser doesn't seem to have a PDF viewer, please download the PDF to view this item.

Item Metadata

Download

Media
831-ubc_1997-0566.pdf [ 8.41MB ]
Metadata
JSON: 831-1.0050244.json
JSON-LD: 831-1.0050244-ld.json
RDF/XML (Pretty): 831-1.0050244-rdf.xml
RDF/JSON: 831-1.0050244-rdf.json
Turtle: 831-1.0050244-turtle.txt
N-Triples: 831-1.0050244-rdf-ntriples.txt
Original Record: 831-1.0050244-source.json
Full Text
831-1.0050244-fulltext.txt
Citation
831-1.0050244.ris

Full Text

A DRIVER VISUAL ATTENTION MODEL by C L A R K C. L I M B.A.Sc . (Civil Engineering), The University of British Columbia, 1993 A THESIS S U B M I T T E D IN P A R T I A L F U L F I L M E N T O F T H E R E Q U I R E M E N T S F O R T H E D E G R E E O F M A S T E R O F A P P L I E D S C I E N C E in T H E F A C U L T Y O F G R A D U A T E S T U D I E S D E P A R T M E N T O F CIVIL E N G I N E E R I N G We accept this thesis as conformation to the required standard T H E U N I V E R S I T Y O F BRITISH C O L U M B I A O C T O B E R , 1997 © Clark C. L i m , 1997 In presenting this thesis in partial fulfilment of the requirements for an advanced degree at the University of British Columbia, I agree that the Library shall make jt freely available for reference and study. I further agree that permission for extensive copying of this thesis for scholarly purposes may be granted by the head of my department or by his or her representatives. It is understood that copying or publication. of this thesis for financial gain shall not be allowed without my written permission. Department of ^/^( / . £ h O ) i ^ € 'Chr'nj The University of British Columbia Vancouver, Canada Date DE-6 (2/88) A B S T R A C T This thesis proposes a driver visual information acquisition model which gathers information based on a selective process so events such as distractions can be modelled. This model contains visual information gathering capabilities and visual attention mechanisms based on subjective and objective factors. A s the research focused on applicability, the model's framework was designed to be integrated as a component processor within a microscopic computer traffic simulation. The model determines visual attention using two mechanisms: internal and external focusing. The Internal Focusing Mechanism is a proactive attention director. This subjective-based mechanism directs the driver's attention to a general direction such that information relevant to the current task is actively searched for based on the driver's expectancy. The External Focusing Mechanism is a reactive attention director based on the characteristics of the objects within the driver's visual field. External control allows for distractions to be modelled, since irrelevant information may objectively demand higher attention than information relevant to driving. For each visible object, these two control mechanisms determine its Attention Demand Value ( A D V ) . Visual information from the object with the highest A D V is then acquired. The A D V also plays a role in determining the information processing time, amount of attention allocated to driving, and whether visual information is acquired through the foveal or peripheral vision. With the use of this model and its input of various internal and external variables, it is hoped that a variety of driver types with varying visual abilities (e.g. age-related, intoxicated) can be simulated within visually detailed environments. K e y W o r d s : visual attention, information acquisition, attention demand value, driver behavior, perceptual object ii T A B L E O F C O N T E N T S ABSTRACT ii TABLE OF CONTENTS iii LIST OF T A B L E S viii LIST OF FIGURES ix ACKNOWLDEGEMENTS ; xii 1 INTRODUCTION 1 1.1 The Driver Environment 2 1.2 Focus on Human Factors 4 1.3 Problem Identification 5 1.4 Research Purpose and Scope 7 1.5 Thesis Structure 8 2 DRIVER MODEL Literature Review 9 2.1 A Review of Driver Behaviour Models 9 2.2 Taxonomic Models 10 2.2.1 Trait models 10 2.2.2 Task Analysis Models 11 i i i 2.3 Functional Models 12 2.3.1 Mechanistic Models 12 2.3.2 Adaptive Control Models 12 2.3.2.1 Servo-Control Models 13 2.3.2.2 Information Flow Control Models 13 2.3.3 Motivational Models 15 2.3.3.1 Risk Compensation Models 16 2.3.3.2 Risk Threshold Models 16 2.3.4 Cognitive Process Models 18 2.4 Summary 19 3 CONCEPTUAL FRAMEWORK O F A DRIVER BEHAVIOUR MODEL 21 3.1 Driving Task Hierarchy 21 3.1.1 Transport Behavioural Levels 21 3.1.2 Driving Task Framework 23 3.2 Components of a Driver Behaviour Model 27 3.2.1 Driver Behavior Process at an Intersection 27 3.2.2 Process Supervisor (Tactical Level Driver Behavior) 29 3.3.3 Long-Term Memory 29 3.3.4 Short-Term Memory 32 3.3.5 Visual Processor 32 3.3.6 Decision Processor and Driver Motor System 35 3.4 Driver Behaviour Model Characteristics 37 3.4.1 Processor Time Delays 37 3.4.2 Learning Mechanism 40 iv 3.4.3 The Fallible Driver 41 3.5 Summary 42 4 A DRIVER VISUAL Attention MODEL 44 4.1 Scope and Philosophy 44 4.1.1 Information Flows 44 4.1.2 Visual Attention 45 4.1.3 Perceptual Objects and their Attributes 46 4.1.4 Physical Limitations in Visual Information Acquisition 48 4.2 Details of the VISA Model 50 4.2.1 Model Components 50 4.2.2 Visual Focusing (Object Selection) 51 4.2.2.1 External Focusing Mechanism ..52 4.2.2.2 Internal Focusing Mechanism 54 4.2.3 Determination of Attention Demand Values 54 4.2.4 Information Acquisition and Processing Delays 56 4.2.4.1 Perception Time Delay 56 4.2.4.2 Identification Time Delay 57 4.2.4.3 Emotion Time Delay ; 58 4.2.4.4 Multiple-Glancing 59 4.2.5 V I S A Model Algorithm 61 5 Model Validation Process 64 5.1 Model Verification, Calibration, and Validation 64 v 5.2 ADV Relation Calibration 66 5.2.1 External Focusing Mechanism 67 5.2.2 Internal Focusing Mechanism 69 5.2.3 Driving Task Determination 70 5.2.4 A D V Equation 70 5.3 Data Collection System 71 6 RECOMMENDATIONS 72 6.1 Model Improvements 72 6.2 Validation 73 7 Summary and CONCLUSION 74 7.1 Overview 74 7.2 Potential Applications 76 BIBLIOGRAPHY 78 APPENDIX A - Driving Simulator 87 A . l Simulators as Diver Behaviour Data Collection Apparatus 87 A . 1.1 Advantages 88 A . 1.2 Disadvantages/Reservations 88 A.2 Current Simulators 88 A.2.1 I O W A Driving Simulator 89 A.2.2 Northeastern Virtual Environments Driving Simulator 92 vi A.2.3 Driver Training System (DTS™) 94 A.3 Simulator Requirements for VISA Model Validation 95 A.3.1 Visual Environment Simulation 95 A.3.2 Auditory Simulation 96 A.3.3 Vestibular Simulation 97 A.3.4 Driver Measurements 98 A.3.4.1 Physical Monitoring 99 A.3.4.2 Bioelectric Monitoring 99 A.3.4.3 Driving Environment Monitoring 100 vii LIST O F T A B L E S Table 3.1 Driving Task Hierarchy 25 Table 3.2 Stages of Perception-Reaction Time 38 Table 3.3 Similarities Between the Components of a Driver Model and Digital Computer 42 Table 3.4 Characteristics of a Fallible Driver Model 43 Table 5.1 Objective External Variable Measurement Parameters 68 Table 5.2 Subjective External Variable Measurement Parameters 68 Table 5.3 Internal Variable Measurement Parameters 69 Table 5.4 Driving Task Variable Measurement Parameters 70 viii LIST O F F I G U R E S Figure 1.1 Visual Classification of the Driver 3 Figure 2.1 Summary of Driver Behaviour Model Types 9 Figure 2.2 Approaching an Intersection based on an Information Flow * Control Model 14 Figure 2.3 Wilde's Risk Compensation Model 17 Figure 3.1 Hierarchical Structure of Problem Solving Tasks in Traffic and Transportation 22 Figure 3.2 Driving Task Hierarchy 24 Figure 3.3 Hierarchical Structure of Driving as a Social Agent, Transportation Consumer, and Road User 26 Figure 3.4 Driving Task Hierarchy at an Intersection 28 Figure 3.5 Framework of the Process Supervisor 30 Figure 3.6 Framework of the Long-Term Memory 31 Figure 3.7 Framework of the Short-Term Memory 33 Figure 3.8 Framework of the Visual Processor 34 ix Figure 3.9 Framework of the Decision Processor and Driver Motor System 36 Figure 3.10 Timeline of Perception-Reaction Time 39 Figure 4.1 The Visual Field as Seen by the Left Eye of Ernst Mach 47 Figure 4.2 Potential Fields-of-View of the Driver in a Vehicle 50 Figure 4.3 Foveal/Peripheral Information Acquisition Logic 55 Figure 4.4 Median Driver Reaction Time to Expected and Unexpected Information 58 Figure 4.5 Driving Attention Allocation Logic 60 Figure 4.6 VISA Model Flowchart 63 Figure 5.1 Verification, Calibration, and Validation Process 65 Figure A.1 The Iowa Driving Simulator Motion Platform 89 Figure A.2 Automobile Interior 90 Figure A.3 Quad Screen Driver Performance Monitor 90 Figure A.4 Iowa Driving Simulator Control Room 91 Figure A.5 Northeastern Virtual Environments Driving Simulator HMD Setup 92 x Figure A.6 Steering and Pedal Array 93 Figure A.7 Virtual Environments Screen Rendering 94 Figure A.8 9-Degrees of Freedom Motion Carriage 98 xi A C K N O W L D E G E M E N T S I owe a big thanks to my thesis advisor Dr. Tarek Sayed for his enthusiasm, patience and bottomless pool of ideas in this research. His invaluable insights and constant encouragement guided me through my work. I would also like to express my appreciation to Dr. Francis Navin for providing direction to my research and always reminding me to think imaginative and in the "big picture". He also provided funding for my trip to the 1996 T R B Night Visibility and Driver Behavior symposium which proved valuable to my work. Financial support was made possible through the B C Ministry of Transportation and Highways graduate co-op research program. During my stay there, Mr. Richard Voyer supervised my work providing much appreciated insight, criticism and encouragement, and helped to expand my understanding of design standards. Dr. Robert Dewar of Western Ergonomics Inc. provided initial comments and encouragement that helped direct the scope of the research. Dr. Ronald Mourant of the Northeastern University provided encouraging feedback on my work and fed my interest in virtual reality and driving simulators with his vast knowledge of these subjects. Special thanks to my friends, colleagues, and co-workers for their encouragement, advice, and patience throughout this work: Albert Lo, Gerry Froese, T i m Murphy, Paul DeLeur, everyone at G V R D Strategic Planning, and my good friends at Living Hope Fellowship. Without the help that Norman Stang has provided from the beginning of this research, I could not have completed this thesis as such. Our talks were always insightful and challenging, continually leading to new ideas and approaches. I would like to thank him for his time, lessons on " C " , vast knowledge of driver behaviour, and most of all, friendship. Finally, I would like to express my sincerest love and thanks to my family who have faithfully supported me in everything I do. T o me they have always been my example of commitment, happiness, and unconditional love. xii I N T R O D U C T I O N In British Columbia, there was a reported number of 534 fatalities, 50,000 injuries, and 180,000 property damage claims due to motor vehicle accidents in 1995 (ICBC, 1996). The direct cost attributed to these accidents is approximately $2.2 billion and does not account for the indirect cost associated with the pain, loss of time, and other intangibles that resulted from these accidents. More pre-retirement years of life are lost to road accidents than from the two leading diseases: cancer and heart disease (Evans, 1991). When categorized as to the cause of these accidents, a significant percent can be attributed to a problem of driver inattention. In a survey of several studies on the cause of accidents, Zaidel et al (1978) found inattention, in its broad sense (i.e. improper lookout, misperception and distractions, etc.), to be a factor in 25-50% of all accidents. These findings were remarkably consistent across several studies despite the lack of consistency in the definition of driver inattention and the wide variety of methods used for investigation. In a comprehensive analysis of 1982 accident data from the U.S . National Accident Sampling System, Sussman et al (1985) determined that 38% of drivers that had been involved in accidents failed to initiate any pre-crash avoidance manoeuvre, suggesting that they were unaware of the impending crash. v A n d in the classic Indiana Tri-Level Study of the Causes of Traffic Accidents, Treat et al. (1979) found that recognition errors definitely contributed to 41% of all accidents and probably involved in as many as 56%. These types of errors were far more prevalent than driver decision or performance errors. O f these errors, improper lookout (e.g. lack of proper visual search, "looked but didn't see") was cited as the most frequent driver error, contributing to 18-23% of accidents, followed by speeding (8-17%), inattention (e.g. 1 attention lapses, 10-15%), improper evasive action (5-13%) and internal distraction (6-9%). Recently, technologies such as in-vehicle navigation have been popularized as systems that will increase vehicular safety and efficiency. The use of more advanced computational technology in the car is promising, and to be expected in this computer age. However, the successful development of such systems requires a better understanding of the fundamental characteristics of driving. A more fundamental question will be whether these systems actually benefit the driver or will cause more risks of inattention. There have been studies recently that point towards the use of cellular phones as increasing driver inattention. Regardless of the advance of technologies or improvement of driver skill, the interaction between driver and driving environment is important issue that needs more research. 1.1 The Driver Environment In any given section of road, a driver is showered with a continuous array of visual objects of assorted shapes, sizes, colours and movement. The driver must make use of the information presented and navigate through the maze of objects in order to reach his or her desired destination safely and efficiently. In the point of view of the driver, visual objects can be categorized into three main groups: road users, road environment, and vehicle environment, as shown in Figure 1.1. 2 Figure 1.1 Visual Classification of the Driver Road users consist of people and vehicles in motion such as automobiles, cyclists, and pedestrians. These road users come in a variety of ages, mobility skills, and behaviour moving at varying speeds and directions. The road environment consists of the actual road and its geometry, delineation, signs, and signals. It also contains peripheral objects such as lamps, street furniture, buildings, trees, commercial signs, and natural scenery that do not necessarily aid road users in their use of the road. The vehicle in which the driver is situated also has various dials, meters, and accessories. A s well, the vehicle itself accommodates visual impediments such as the dashboard, hood, and roof pillars, while providing visual aids in the form of rear and side-view mirrors to increase the driver's available field of view. Within this environment, there can also be improper behaviour and unnecessary demands imposed on the driver. Drivers can be intoxicated or performing illegal and dangerous 3 manoeuvres, pedestrians may jay-walk unexpectedly across the road, and cyclists may weave erratically in and out of traffic, being temporarily hidden in a vehicle's blind spot. Visual objects such as flashing commercial signs or large billboards can deter drivers from attending to their task of driving. And finally, these events can occur in wet, icy, or foggy conditions. The driver environment is fundamentally a very complex one and is unfortunately taken for granted within the frequent chore of everyday driving. But in the event of conflicting decisions and actions, it can be an unforgiving environment with serious and sometimes fatal consequences. It is a desire of the transportation practitioner to reduce the potential of such conflicting events by providing designs and operations that accommodate considerations of safety, efficiency, and convenience to both drivers and non-drivers. But the immense number of factors that influence driver behaviour make it difficult for the transportation practitioner to analyze. Therefore, there is a need to enhance our understanding of the relationship between the driver and driving environment and then apply this understanding appropriately. 1.2 Focus on Human Factors Halsey (1941) identified the need for transportation practitioners to understand the interrelationship between transportation engineering and human factors stating that "the traffic engineer must begin his work with a careful study of human nature" and "in order to be a successful engineer, he [or she] has to know almost as much about psychology as he [or she] does about engineering." These statements are no less significant today as they were since the beginning of this profession. 4 For the purpose of this thesis, drivers can be regarded as machines that react to information they gather and process. Rockwell (1972) stated that "it is significant that without knowledge of the information-acquisition side of driving, no real development in driving theory such as information processing, anticipation, and learning can be developed." This statement holds true to the fact that the driver's cognitive processing and reaction to the driving situation is dependent on the input of information. In terms of information gathering, the driver generally requires the use of only three of his five physical senses: visual, auditory, and vestibular. And of these three, the most important without a doubt is the visual perception as it is commonly estimated that over 90% of driving information is visual (Rockwell, 1972). Therefore, it can be concluded that a key step in better understanding the driver is to understand the visual information acquisition process of the driver. 1.3 Problem Identification The driver is continuously confronted with a vast number of visual stimuli. Even though visual information is captured in the form of a two-dimensional images on the retina, the driver's visual system has to organize these "visual blobs" into meaningful perceptual objects. At the same time, the driver can be regarded as a single-channel information processor (Hulbert, 1972) with the ability to consciously attend to only one thing at a time (AASHTO, 1990). The driver is able to shift attention quickly from one source to another thereby mimicking a multi-channel information processor. Therefore, the driver must choose just one perceptual object from the whole group at any given instance. This requires the driver's visual system to have a selection mechanism. Additionally, depending on the perceptual object and the information that it carries, a certain amount of time is needed to identify and obtain the necessary information for appropriate action. 5 The driver's visual system also exhibits a delay or fixation duration in receiving information. The process of visual information acquisition can be considered in two levels: 1) object selection, where a certain visual object is selected out of a whole array of visual objects, even before it is known exactly what the selected object is (the object can be considered a stimulus at this level of which information about the object is limited to the basic visual characteristics of size, shape, contrast (also frequency of contrast change) and speed), and 2) object identification and information acquisition, where the selected stimuli is identified based on a list of prior known objects, and using this past knowledge, information is selectively acquired from the selected object. In other words, known objects would require relatively little information acquisition as most of the information about these objects are already known. On the other hand, "new" objects would require more information acquisition as the driver would be visually learning about this object. As information acquisition takes time, gathering visual information from "new" objects takes more time than known objects. This process of driver visual information acquisition can be summed up in two questions: Perceptual Object Selection: 1) What factors make a driver look at or attend to a particular object? Perceptual Object Identification and Information Acquisition Duration: 2) What factors determine the length of time a driver attends to an object? It is hoped that a thorough examination of these questions will provide better insight to the driver's visual attention process, why certain visual cues are selected or reversely missed, reasons for variations in driver visual fixation times, and ultimately, a better understanding of the driver. 6 1.4 Research Purpose and Scope On this basis, the purpose of this thesis is two-fold: Firstly, to summarize previous research on driver behaviour models and visual information acquisition; A n d secondly, to detail the development of a driver visual information acquisition model which facilitates visual constraints and gathers information based on a selective process such that distractions and "looked, but failed to see" events can be modelled. This model is designed to be eventually incorporated for use in computer driver simulation models to enhance the visual information selection abilities of these models by providing the use of a broader range of variables pertaining to the driver's visual system. In order to provide an adequate context for the visual model, a conceptual framework for a driver behaviour model is also discussed. Overall, it is desired to develop a driver visual attention model that is comprehensive enough to be used in a wide range of studies requiring visual sensitivity (e.g. road design, in-vehicle navigation systems research) and detailed enough to model a variety of driver conditions (e.g. age-related, intoxicated) within varying driving environments. If driver attention can be modelled, it could be used to as a tool that can provide a new dimension of information relating to a good portion of automobile accidents. This tool may also be able to quantify some measure of safety of an roadway design or vehicle environment. One possible measure is inattention or distraction to the driving task. A s the validation of such a model would require extensive empirical data, the scope of this research is limited to the development of a rudimentary driver visual attention model. 7 1.5 Thesis Structure Chapter one provides an overview of the thesis and its structure. A literature review summarizing the main concepts and terminology of driver behaviour models are discussed chapter two. Chapter three describes the conceptual framework of a driver behaviour model that was developed to set the context of this research work. Chapter four presents the details and processes of the proposed visual attention model. The validation process that the proposed model requires is discussed in chapter five. Recommendations for further research is outlined in chapter six and chapter seven summarizes the findings and conclusion of the thesis. The bibliography and appendix on driving simulators follows at the end of this thesis. 8 2 DRIVER M O D E L L I T E R A T U R E R E V I E W This chapter discusses the findings of literature reviews pertaining to driver behaviour models. These findings set out the groundwork upon which this thesis is based. 2.1 A Review of Driver Behaviour Models Driver psychology has been the subject of a wide range of work in human factors research worldwide. Many models of driving behavior, ranging from motivational to mechanistic types, have been developed. Michon (1985) identified a general two-way classification (behavior or motivation oriented, and taxonomic or functional) to define seven basic types of driver behavior models: Task Analysis, Trait, Mechanistic, Adaptive Control (Servo-Control and Information Flow Control), Motivational, and Cognitive (process) models (Figure 2.1). Taxonomic Functional Input-Output (Behavioural) Task Analyses Mechanistic Models Adaptive Control Models - Servo-Control - Information Flow Control Internal State (Psychological) Trait Models Motivational Models Cognitive (Process) Models Figure 2.1 Summary of Driver Behaviour Model Types (Source: Michon, 1985) 9 Taxonomic models are, in essence, an inventory of facts that are organized in sequential relations (before, while, after) and can be considered as predecessors to knowledge-based expert systems. These models can be very sophisticated such that various complex driving manoeuvres can be incorporated. One of the major limitations of these models is that dynamic relations cannot be expressed between the elements and empirical connections are at best correlative (Michon, 1985). But the nonexistence of dynamic links gives taxonomic models the flexibility to incorporate more driver skills relatively easily. On the other hand, functional models are identified by the dynamic linkages between model elements. These dynamic relations are equally important in describing the model as the elements that embody these models. Functional models have the benefit of being in algorithmic forms that are more easily adaptable into computer simulations. But the drawback of these models is that without the detailed database of facts that characterize taxonomic models, it is difficult to develop complex behaviour models. The following sections identify and briefly describe some of the driver behaviour models that were reviewed. It is beyond the scope of this report to discuss these models in detail. Michon (1985) and Ranney (1994) provide excellent reviews and summaries of driver behavior models and their evolution. 2.2 Taxonomic Models 2.2.1 Trait models Trait models show correlations between driver characteristics and driving actions. For example, a driver with smaller reaction times may have a lower accident rate. This correlation does not describe the mechanism by which the two factors are related. Trait 10 models do not offer insight in the actual processing involved in the performance of complex tasks, but they are valuable tools in tracing the overall stage of the learning process. Fleishman's factorial model (1975) is a trait model that is based on perceptual, cognitive, and motor skills which result from the combination of a small number of elementary traits (e.g. reaction speed, spatial orientation). 2.2.2 Task Analysis Models Driving task analysis models are essentially a description of facts about the driving task (task requirements), the behavioural requirements (performance objectives), and the ability requirements (enabling objectives) for performing that task (Michon 1985). Task analysis models provide an exhaustive breakdown of driving activities but do not address the dynamic relations between tasks. A good example of a task analysis driver model is from the work of McKnight and Adams (1970). This task analysis consisted of 45 major tasks (including 9 off-road tasks such as vehicle maintenance) which is composed of more than 1700 elementary tasks. A n example excerpt is taken from Task 42.0: "Negotiating On-Ramps and Off-Ramps": Task 42-123 [Driver] observes a general on-ramp/main roadway configuration 42-1231 Looks to see if on-ramp feeds into right side of main roadway or left side (speed lane) of main roadway 42-1232 Looks to see if acceleration lane is provided at end of on-ramp 42-1233 Looks for exit off-ramps or deceleration lanes which cross over or share continuing portions of the entrance ramp 42-1234 Evaluates effects of on-ramp/main roadway configuration on available merging distance and probable merging pattern 11 At times task analysis can seem to be "splitting hairs", while other times too vague. The McKnight/Adams model does not always specify how the driver chooses a task in a given situation, or whether one task can interrupt another, or how two task might be performed simultaneously (especially if they require conflicting actions). This is in the case of discriminating between the tasks to "observe pedestrians and playing children" and "ignore activity on the sidewalk that has no impact on driving" (Reece and Shafer, 1993). 2.3 Functional Models 2.3.1 Mechanistic Models Mechanistic models describe traffic as an aggregate whole - traffic likened to flow of fluid and Shockwaves. These models are mathematical descriptions of congestion and car following. They cannot be used to specify the actions of individual drivers as they model the behaviour of cars and not the driver. Examples of mechanistic models are by Greensberg (1959) and Edie and Foote (1960). Drew (1968) gives a good description of mechanistic models and how they relate to other types of modelling approaches (e.g. human factors, microscopic). 2.3.2 Adaptive Control Models Adaptive control models can be considered a level above mechanistic models in that they introduce the driver as a feedback control mechanism. Herman et al. (1959) initially introduced this concept into a car following model by assuming that drivers aim at minimizing the speed difference with the car in front of them. There are two kinds of adaptive control model types: servo-control and information flow control. 12 2.3.2.1 Servo-Control Models Servo-control models have mainly been used to describe driver at the operational level tasks such as steering and lane changing. These models represent a network of switches which are toggled by appropriate input signals. This design approach allows these models to be easily incorporated into machine control such as robotics. A good example of a servo-control model is the STI compensatory driver model by McRuer et al. (1977). 2.3.2.2 Information Flow Control Models With the advent of the digital computer, driver models were made in computational environments that demanded explicit detail, especially to temporal issues such as driver reaction time delays and event detection time. These computer models, or simulations, allow for testing of the driver models in various situations and are usually designed for practical use such as traffic flow analysis. A n early driver simulation was made by Kidd and Laughery (1964) that incorporated basic driver visual abilities and vehicle dynamics (Figure 2.2). Their work may be considered as a dynamic form of task analysis as the model covered a fair number of situations. Another information flow control model is the DRIver-Vehicle Effectiveness Model ( D R I V E M ) by Wolf and Barrett (1978) of Honeywell Inc. for N H S T A . It was developed to simulate the performance of a driver and vehicle in a number of high-accident-likelihood situations. The model determines, through numerous trials using Monte Carlo techniques, whether or not an "accident" occurs and an overall crash probability is then computed. 13 Decelerate or Acce lerate > t No • Y e s * Has angle x > v \ c h a n g e d ? . / — Y e s - * . Maintain S p e e d Mainta in Speed Brake (Stop) Figure 2.2 Approaching an Intersection based on an Information Flow Control Model (Source: Kidd and Laughery, 1964) 14 D R I V E M has a detailed detection submodel that determines the direction of looking from driver eye fixation distributions. The probability of detection is computed for each visual fixation as a function of variables such as object size, object angular location relative to driver's line of sight, existence of clear view to the object, luminance contrast of the object relative to the background, the state of light adaptation of the driver's eye, and change in headway relative to the vehicle ahead. The results of the detection submodel is fed into a manoeuvre decision submodel where either a null response or one of three evasive manoeuvres can be made (braking, steering, or braking and steering). Once a manoeuvre decision has been made, the driver control and vehicle dynamics submodel generates the resulting actions of both driver and vehicle. Information flow control models can be elaborate and impressive feats of programming but are still conceptually passive simulations that are data driven - they do not model driver mentality or intelligence (no learning or realistic priority interrupts). Many have good vehicle dynamics modelling capabilities but lack cognitive driver perception and decision. 2.3.3 Motivational Models Motivational models are theories of human cognitive activity during driving. Van der Molden and Botticher reviewed several of these models (1987). The models generally describe mental states such as "intentions," "expectancy," "perceived risk," "target level of risk," "need to hurry" or "distractions." These states are combined with perceptions in various ways to produce actions. These models do not concretely show how to represent driving knowledge, how to perceive traffic situations or how to process information to obtain actions. They are just comprised of elements of mentality. 15 Example types of motivational models are risk compensation and risk threshold. They differ mainly in the manner at which a perceived level of risk is evaluated, the control variable for the quality of driving performance. 2.3.3.1 Risk Compensation Models A n example of a risk compensation model is derived from Wilde's Risk Homeostasis Theory (1982) which states that the level of accepted subjective risk is more or less a fixed personal parameter (Figure 2.3). This implies that the driver's risk control behavior can only be influenced by affecting the level of his or her perceived risk. Wilde's theory states that accident countermeasures will fail (e.g. anti-lock brakes) because drivers will compensate the loss of risk from the countermeasure in another area (e.g. speeding, driving more aggressive). This compensation mechanism characterizes this theory as more of an economic theory rather than a psychological one. Taylor's (1964) "risk-speed compensation model's" basic tenant is that the larger the perceived risk, the lower a driver's chosen speed. In mathematical terms, the product of perceived risk and speed is constant. Taylor's model is purely descriptive and makes no effort to explain the internal processes of the driver. Even more importantly, it remains unclear what in fact is the effective stimulus that determines the level of risk perceived. 2.3.3.2 Risk Threshold Models The balance between subjective, perceived safety (S), and objective, physically or statistically determined safety (O), are the two mechanisms in Klebelsberg's (1977) risk theory to road users. He postulates that individual road users differ in their personal balance between S and O. This theory states that unsafe situations occur when a road user judges situations safer than they actually are (S>0). On the other hand, if S<0, then 16 there is a surplus safety margin. The ideal situation is when S=0 (perceived safety equals actual safety). In Naatanen and Summala's (1974) risk threshold model, perceived risk in traffic (R) depends on both the subjective probability of a hazardous event (P) and the subjective importance of the consequences (B) of the event and, more specifically, on the product of these two factors: R=PxB. Under normal driving conditions, R is perceived to be effectively equal to zero - the threshold of risk perception. Only when this threshold is exceeded is there any behavioural attempts to reduce the prevailing risk level. Naatanen and Summala's model contrasts with Wilde's theory as their model indicates that influencing people by education, campaigns or enforcement are not effective and that traffic safety will improve only from better vehicles and roads. 2.3.4 Cognitive Process Models The previous models approach driving from an adaptive control of behavior. Cognitive process models approach driving from an adaptive control of thought and are considered the future of driver behaviour modelling. Anderson's (1983) method employs a rule-based or production system approach ("IF. . .THEN") to modelling drivers. His framework combines the benefits of explicit knowledge and action descriptions of task analysis with the adaptive control dynamics of information flow control, and the principles underlying motivational models that provide behavioural purpose. Production systems are intrinsically flexible and can deal with may levels of specification, from general to detail, and can be interfaced with other types of mechanisms, algorithms and continuous adaptive control circuits without loss of generality. 18 A n example of a production system representing a pedestrian crossing a road is shown as follows (Newell and Simon, 1972): IF traffic-light red T H E N stop IF traffic-light green T H E N move IF move and left-foot-on-pavement T H E N step-with-right-foot IF move and right-foot-on-pavement T H E N step-with-left-foot Production systems have proven to be tremendously flexible and offer a formal basis for artificial intelligence, linguistics, and cognitive psychology. Using such an approach, a "language" could be created to handle any situational case and used to build a library of knowledge or skill. The only drawback of production systems is the extensive amount of knowledge information required. A reasonable model of the driver that incorporates all three levels of road user performance control (strategic, tactical, operational) could require between 10,000 and 50,000 productions ( " IF . . .THEN" statements). 2.4 Summary Driver behaviour models have aided transportation researchers and practitioners in understanding as well as appreciating the importance of the driver's mind in analyzing traffic operations and safety. The systematic and detailed approaches in the development of these models have provided clues to many of the factors that are considered to be influential to the driver's cognitive process. Typical of any theoretical model, each of these models have their strength and weaknesses and generally can be only used in specific areas of applicability such as safety or guidance research. 19 But to be most useful, a driving model must be detailed and complete. A detailed model must state specifically what decisions must be made, what information is needed, and how it will be used. A complete driving model must address all aspects of driving (Reece and Shafer, 1993). The next phase in this thesis is to develop a framework for such a driver model. 20 3 C O N C E P T U A L F R A M E W O R K O F A DRIVER B E H A V I O U R M O D E L The scope of this research is the development of a driver visual attention model. But before this can be attempted, the framework for a general driver behavior model that satisfies certain practical and theoretical criteria must be established to set the general context for which the visual model is to be incorporated. A n understanding of the other components of the driver is required i f the information needs of the driver is to be understood. A n d as the driver is not infallible, the framework must allow for driver imperfections and error so events such as distractions and mistakes can be modelled. 3.1 Driving Task Hierarchy 3.1.1 Transport Behavioural Levels In our current society, human mobility is embedded in a social and technological environment. The task of travelling from one place to another can be described at various degrees of behavioural levels (Figure 3.1) and defined as a problem to be solved. In this context, a descriptive framework can be defined which allows the specification of a number of basic tasks that together constitute the set of relations between people and the environment in which they attempt to satisfy their mobility needs (Michon, 1985). A s a psycho-biological organism, a person travels from one location to another in order to satisfy a basic need than can be summed up as survival. A s a social agent, people travel in transport systems designed to satisfy their need for mobility in an efficient and safe manner. These transport systems link residential clusters and social and economic activity centres (origins and destinations) together that form the physical infrastructure of society. The actual act of trip making is done by people as a transportation consumer, making decisions as to the most optimal method and route of travel in terms of cost, time, 21 • Behavioura l Leve l 1 II III IV Human Quality as a Problem Solver Road User Transportation Consumer Social Agent Pyscho-Biological Organism Problem to be Solved Vehicle Control Trip Making Activity Pattern (Communication) Satisfaction of Basic Needs Task Environment Road Road Network (Topographical Structure) Socio-Economic Structure Nature (Environment) Task Aids Vehicles, Signs, etc. Transport Mode Transport System "Culture", Technology Figure 3.1 Hierarchical Structure of Problem Solving Tasks in Traffic and Transportation (Source: Michon, 1985) 22 convenience and social acceptance. And finally, as a road user, people seek to satisfy their goals of travelling by controlling their vehicles (as in the case of driving) in a manner representative of their motivations. In light of these views, the job of the transportation professional is to plan, design and operate transportation systems that meet both the demands for use of the system and the policies set forth by the decision makers to control the demands. In response to this duty, analytical methods and tools have been developed that address the issue of transport system planning, design and operations from a behavioural approach. Land-use planning models address travelling behavior at the social agent level by determining the socio-economic structure of a region. Based on the demographic information from these land-use models, the classic four-step transportation framework models travellers in the transportation consumer level determining the number of trips generated, where they are distributed, the mode of transport taken, and finally the route selected to complete the trip. From the route plans of the trip, the road user travels in a specific environment that requires specific operational decisions and actions. These decisions and actions represent the driving task. 3.7.2 Driving Task Framework The task of driving involves many decisions and performances of varying complexity. The overall driving task can be divided in three levels of a hierarchical structure: Strategic, Tactical, and Operational (Michon, 1985). 23 Environmental Input Environmental Input Strategic (Planning) Level - - |- R o u t e - i — Speed Criteria _ ± i _ Tactical (Manoeuvering) Level Feedback Criteria Operational (Control) Level General Plans Time Constant Long Controlled Action Patterns Automatic Action Patterns sees msec F i gu r e 3.2 Dr i v ing T a s k H i e ra r chy (after Michon, 1985) As shown in Figure 3.2, the highest level of a driving task hierarchy is the strategic or planning level, which defines the behavioral goals for the driver. These goals may, for example, be based on route selection and driving time calculations. The strategic goals are achieved by activities at the tactical or manoeuvring level, which involves choosing vehicle manoeuvres within the dynamic world of traffic and traffic control devices (TCDs). The manoeuvres selected at the tactical level are carried out by the operational or control level of speed and steering control imparted from the driver to the vehicle. Operational level tasks are generally automatic (except in the case of inexperienced drivers) and performed subconsciously. Table 3.1 further details the characteristics of this driving task hierarchy. 24 Tab l e 3.1 D r i v i ng T a s k H i e r a r chy Level Characteristic Example Existing Model Strategic static, abstract planning a route, est. time for trip planning programs (Al) Tactical dynamic, physical determine ROW, passing another car human driving models Operational feedback control tracking a lane, following a car robot control systems (Source: Reece and Shafer, 1993) The driving task hierarchy can be expanded so that tactical models could be said to be a collection of models at a sub-operational level that models the driver's cognitive process (e.g. eye/head movements in visual search, memory storage, decision models). Likewise, the strategic driver model could be said to be a component of a larger human model which is the topic of some very ambitious artificial intelligence research. Figure 3.3 depicts the driving task in the context of the overall framework of travelling in a transport system at various the behavioural levels mentioned earlier. Substantial progress has been made in automating various parts of the driving task, particularly at the strategic and operational levels, but no system has yet satisfactorily implemented the tactical level to a degree that is applicable in safety and design research. Such a model could be developed based on psychological foundations with an investigation of the driver as an aggregation of components in an information-flow control structure. 25 r 1 Social Agent Residential Location Trip Generation •J I Transportation Consumer Trip Distribution Modal Choice Network Assignment (Route Se l e c t i on ) Strategic (Planning) Level Tactical (Manoeuvering) Level Road User Driving Task Hierarchy Operational (Control) Level Figure 3 . 3 Hierarchical Structure of Driving as a Social Agent, Transportation Consumer, and Road User 26 3.2 Components of a Driver Behaviour Model " A comprehensive model of driver behaviour should not only take the various levels into account, but should also provide an information flow control structure that enables control to switch from one level to the other at the appropriate points in time." (Michon 1985) A human driving model must explain exactly what and how information is processed -what features of the world must be observed, what driving knowledge is needed and how it should be encoded, and how knowledge is applied to produce actions. Constraints should be derived from a general desire to avoid collisions and a strategic driving goal of obeying traffic laws (Reece and Shafer, 1993). 3.2.1 Driver Behavior Process at an Intersection The ultimate goal of a driver is to reach a certain destination. This requires a specific route plan to be determined. This route plan then becomes the criteria or sub-goal that directs the driver on every road section travelled. When approaching an intersection along the way, the driver will select one of three option depending on the route plan: a) turn left, b) drive through, or c) turn right at the intersection. Regardless of the option selected, the driver can be said to go through five discrete chronological driving states or modes when approaching an intersection: 1) Drive Straight, 2) Approach Intersection, 3) Intersection Manoeuvre, 4) Exit Intersection, and 5) Drive Straight. Each of these modes have an objective, which when achieved the driver goes onto the next mode. This is the strategic level of a multi-level process that describes the driving task through an intersection (Figure 3.4). 27 A t any time during any of these driving modes, the driver is constantly processing decisions based on environmental (external) and mental (internal) information brought to the driver's attention. These decisions range from searching for visual cues, to deciding on specific manoeuvres to perform (tactical level processing). Such manoeuvres are then performed through the use of vehicular controls (operational level processing). But it is at the tactical level that the majority of the complex driver behavior details are processed. 3.2.2 Process Supervisor (Tactical Level Driver Behavior) The various tactical level driver behaviors can be thought of being managed by a process supervisor. In this framework, the driver utilizes facilities such as short-term and long-term memory, a visual processor (due to the scope of this research, auditory and vestibular perceptions - secondary driving senses - are ignored), and a decision processor which combines all these facilities' inputs (as well as the driver's main goal and driving mode), in order to produce operational commands. A driver motor system, which contains the driver's physiological information, then converts these operational commands into four operational actions sent to the vehicle: steering, acceleration, visual direction and signaling changes (Figure 3.5). 3.3.3 Long-Term Memory The long-term memory facility ( L T M ) primarily consists of knowledge, skills, personality characteristics and experience (Figure 3.6). Driving knowledge consists of information such as regulatory and R O W rules. Visual search skills contained in the L T M influences the visual scanning pattern of the driver. Visual differences in driver experience could be set through the use of varying sets of visual search skills. Unique to each driver are driver personality characteristics and experience. Personality attributes such as aggressiveness and desired level-of-risk affect the driver's judgment, and experience provides the driver 29 J= u> O o z cc 3 t -U l tn h - z> X cc a u cc CO < z Q DC cc U l o 1— a. Q _ o 1— O PR I c c 1— tn < o 1- U l o . U l < < cc 1- U l — i o = s U l o CO U l a < m ERGE •RATE L U 1 t - U l o u. U l u < o E <D 0 I-1 C o _l a> .c *-« o i _ o CD E CD CO V 1_ 3 a) LL with the ability to anticipate future events. The L T M can be considered as a large database containing various types of driving information and experiences. 3.3.4 Short-Term Memory The short-term memory (STM) facility is a memory stack which holds limited information regarding the driver's environment in present, past and future contexts. It can be considered as the current state of the environment as the driver perceives it. The S T M also holds dynamic variables such as the driver's stress level (influenced by the quantity and quality of information in the S T M ) , which affects the decision processor as well as the visual processor (influences the allocation of driving attention). Information passes through the S T M in a "first-in-first-out" (FIFO) order, with current information termed "fresh" and previous information termed "stale." Figure 3.7 details the components of a short-term memory model. 3.3.5 Visual Processor The Visual Processor is the facility with which the driver gathers information from the surrounding environment. This facility is dependent on the S T M and L T M as shown in Figure 3.8. As the scope of this research is on the visual system, the visual processor will be examined in more detail in the remaining chapters. 32 o E 0) E co a> « ^ o o a> E n CO <D i_ 3 iZ 3.3.6 Decision Processor and Driver Motor System The Decision Processor compares environmental information from the S T M in relation to skills and knowledge contained in the L T M to decide on operational actions that is consistent with the strategic goal (route plan). In instances where there is no suitable information in the L T M to compare to the S T M , the Decision Processor synthesizes information through estimation and prediction. If this is done repeatedly, this skill is eventually learned and stored in the L T M . The Decision Processor contains a database of " I F . . . T H E N " condition-action statements for almost every possible traffic task - similar to Anderson's production system (1983). A s shown in Figure 3.9, the Decision Calculator matches the driver environment (as portrayed in the S T M ) and the current driving goal (route plan) with a condition-action statement. Based on the matched condition statement ("IF"), the partner action program ( " T H E N " ) is executed with the required skills and knowledge acquired form the L T M . The resulting decision is a set of operational commands sent to the Driver Motor System. Decisions are not made for every perceived object but after a satisfactory acquisition of enough information in the S T M such that an understandable picture of the current environmental state is composed. The Driver Motor System (DMS) receives operational commands (in the form of four generic operations adjust: speed, travel direction, visual direction and signal) from the Decision Processor and converts these commands into physical actions necessary to act out the request. These conversions are done through the Operational Command -Physical Action Translator and in the case of driving, these physical actions are brake, accelerate, clutch/shift, steer, look ahead, look left, look right, look back, view mirrors, and signal. This structure allows for flexibility for other modes to be modelled (e.g. 35 E Q) >. CO i_ o o > TJ C (0 o V) (0 <D U O i -a. c o "w 'o Q JC H -o 1 -o (I) E (0 CO 0) 1-3 O) cyclist motor system would convert commands into pedal, hand-brake, hand signal, etc). The performance of these actions is limited to the driver's age, physical condition, and driving skill. Figure 3.9 illustrates the Driver Motor System in conjunction with the Decision Processor. 3.4 Driver Behaviour Model Characteristics 3.4.1 Processor Time Delays From the detection of a perceptual object to the execution of a response, time is required to process the information. The perception and reaction to a particular stimulus involves four distinct actions on part of the driver: perception, identification or intellection, decision or emotion, and response or volition (McShane and Roess, 1990). These four actions make up what is known as perception-reaction time (PRT). In the case of the perception and reaction to a visual cue, perception delay is due to the time required for the Perception Processor to acquire information from the environment and select from the environment a specific perceptual object. This information is transferred to the S T M and compared to knowledge stored in the L T M . If a match is found an identification is made, otherwise, the new object is learned and categorized. The time for identification and recognition is the identification delay and increases with unexpectancy, unfamiliarity and complexity of the perceptual object. Once the object is identified, an emotion and decision is made in response to the object. The time to decide on appropriate measures to take is the decision delay and is a function of the complexity of the situation, previous experience with the situation, and current stress level of the driver. And finally, time is required to physically respond to the decision. This response delay is a function of the driver's present physical condition, age, and driving skill. In 37 some instances, there may not necessarily be a response made due to the decision to do nothing. Table 3.2 summarizes the stages of perception-reaction time in relation to the components of the driver behaviour model and Figure 3.10 outlines the P R T timeline. Table 3.2 Stages of Perception-Reaction Time PRT Stages Definition Influencing Factors Model Component Association Perception detection and selection of a stimulus from a continuous series of cues - quality of stimulus in relation quality of other competing stimuli - Perception Processor Identification or Intellection recognition and identification of the stimulus - complexity of the stimulus - previous knowledge of stimulus (familiarity) - expectancy - Perception Processor - ST and LT Memory Decision or Emotion determination of an appropriate response to the stimulus based on goal, skill, and knowledge - complexity of the situation - previous experience with situation - driving stress - Decision Processor - ST and LT Memory Reaction or Volition physical response resulting from the decision - physical condition -age - skill/coordination - Driver Motor System 38 U0l}9|dLU0Q Bsuodsey u o j i n o a x g e s u o d s a y /uoiiBuiLUjejaQ u c u s p a a <D Q CD CO d O Q . CO CD DC > s CD Q o CO o CD Q CD Q co o uoiiBOjiiiuapi sninimig CD Q £ c o o re a> QC • c o a CD o ^_ a> a. « ^ o a> E CO 3 i i u o j j o a i a s s n i n u u n s u o j j i s m b o v u o i i e i u j o j u i i B j u a a i u o j i A u g CD Q . CD O CD Q_ 3.4.2 Learning Mechanism A truly intelligent driving model should be able to learn how to improve its behaviour as it gains experience driving. The steps in learning involve 1) observing an event or environment, 2) identifying or determining the cause and effect of the event and 3) developing relationships such that what was observed and determined may be used for the benefit of the observer. As this driver model incorporates an expectancy mechanism of past driving experience and knowledge as a factor in information acquisition, an appropriate learning mechanism is required in order to provide the model with the ability to dynamically develop its own expectancy database. Learned information could be merged into the driver model's long-term memory if the model were incorporated into an appropriate computer simulation. A s simulations are used for analysis of before and after scenarios, drivers in the simulation could be "trained" by having them go through a roadway section repeatedly, with abilities to store in their long-term memory items that are considered "significant" or brought to the attention of the driver's conscious repeatedly. For example, driving over a pot-hole could be considered an event that would grab the attention of a driver. After a second or third time of running over the same pot-hole, drivers in the simulation model could learn about the pot-hole's existence and location enough to have the driver avoid the roadway hazard in future iterations. Similarly, with the continuous reinforcement of certain generic stimuli, the driver could potentially "over-learn" an object such that it is expected and could eventually be "tagged" as a trivial item and ignored. Such is the case at a 2-way stop controlled intersection that is converted to a 4-way stop configuration. After frequent trips through 40 the intersection, drivers travelling in the free-flow directions would recognize that intersection as a 2-way stop controlled intersection and expect adjacent traffic to stop accordingly. Since these drivers learned that this intersection was a 2-way stop, they would not actively look for any intersection control devices after may repeated trips through the intersection. After the intersection is converted to a 4-way stop configuration, these drivers may not notice the addition of the new stop signs, resulting in potentially hazardous situations. The placement of a blinking warning light could test the driver model's ability to notice the change and re-learn accordingly. In essence, a learning or expectancy mechanism is an efficiency mechanism that allows the driver to use past information to paint the picture of a familiar environment such that re-scanning of the whole environment is not required. This reduces the time and effort required to gather the necessary environmental information every time the driver experiences a familiar environment. Only when there are significant but subtle changes to the once familiar environment is the driver in potential error. A n analogy to driver expectancy would be the cache memory in the information retrieval system of a computer (Table 3.3). 3.4.3 The Fallible Driver A s stated earlier, a driver can be regarded as a machine that reacts to information that it gathers and processes. Such a machine would require memory (long and short-term) to hold past and current information, including the specific goals or purpose of what it wants to do. But a real driver is limited in obtaining such goals and can produce contradictory actions. Therefore, the machine would need imperfections built into its design such as the ability to gather irrelevant information or be penalized due to its limitations through the use of delay mechanisms. The machine would also have a limited short-term memory 41 Table 3.3 Similarities Between the Components of a Driver Model and Digital Computer Driver Components Computer Components Relative Retrieval Speed External Environment Data from External Storage Devices/Communications ports (e.g. hard drive, Floppy drives, serial ports) minutes - seconds Visual Processor (external information acquisition) Controller Card seconds Short-Term Memory Cache milliseconds Long-Term Memory RAM nanoseconds capacity, deteriorated long-term memory, and perception and judgement errors that degrade information and in turn affect decisions. These are some of the features that are imbedded in this framework for a general driver model. Table 3.4 summarizes the characteristics of a fallible driver. 3.5 Summary A general framework of a driver behavioural model has been described that is based on a three-level driving task hierarchy derived from an overall travel behaviour premise. Based on strategic goals (route plan), operational actions (via driver motor system) are the outcome of tactical processing incorporating a visual processor, short-term and long-term memory facilities, and a decision processor. The flow of information between these components requires time which results in what is known as P R T . 42 Table 3.4 Characteristics of a Fallible Driver Model Process Limitations Information Acquisition - not omniscient (limited FOV/blockage) - subjective perception - information cone narrowed by available Information Storage and Retrieval atteritipQT(yFOV)-.Jnformation acquisition time - nmitea S T M capacity ^ increase, with .obiect ... - storedinfo degrades with time -fr^PnT^XM degrades with time but increases with repetition - LTM retrieval time relatively slow Decision Processing - decisions based on perceived info - precision reduced by stress (e.g. not enough time) Physical Abilities - motor abilities degrades with age - condition of driver may be influenced by drugs/ alcohol, lack of sleep A comprehensive driver model also require a learning mechanism to store information and synthesize skill and knowledge. Also, such a model is not infallible and must be characterized by limitations and error proneness. Based on this framework, the direction of the thesis will now focus on the development of a perception processor that models visual attention. 43 4 A DRIVER V I S U A L A T T E N T I O N M O D E L 4.1 Scope and Philosophy The philosophy of this research was to develop a model based on psychological principles. As the basic medium with which traffic engineers work is people, a good understanding of the behaviour of people in the traffic environment is crucial in understanding the interactions between the components of this environment. This generalization also allows the model to be potentially adapted to simulate any visually-based information gathering mechanism or organism. In particular to the road environment, this would imply applicability in pedestrian, cyclist, and driver models. But for the sake of the research scope, the task of driving will be the focus of study for model development. 4.1.1 Information Flows "While eye-movement research focuses on only one aspect of the total driving task, it is significant that without knowledge of the information-acquisition side of driving, no real development in driving theory such as information processing, anticipation, and learning can be developed. Any generalized predictive model of driving will have to have quantification of the informational inputs to the driver." (Rockwell, 1972) The driver can be regarded as dividing his attention to flows of information from two types of sources: external and internal. The driver is dependent on these two flows of information to make decisions. External information flow is the immense number of stimuli consisting of information from the outside world and car interior that is acquired 44 through the driver's physical senses. External information provides the driver with a picture of the driving environment. Internal information flow is the driver's mental recollection of past memories, present thoughts, and predictions of future events. Therefore, internal information helps the driver by providing meaning (from past understandings) to the present external information, as well as assisting the driver by providing predictions of possible future external information so as to prepare the driver in advanced of potential events that may occur. Both types of information flows can also be a hindrance to the driver by providing information not relevant to the driving task. For example, external information can consist of distractionary stimuli such as billboards or flashing commercial signs. Internal information can reduce the amount of driving attention by providing too much mental perusing irrelevant to the driving task at hand. Such perusing usually occur more frequently and for longer periods when the driving task is moderate and external information flow from the driving environment is not demanding (e.g. country drive). 4.1.2 Visual Attention Two important processes in this research is the selection of the stimulus or perceptual object (visual attention process) and the time of recognition and information processing. The relevance of these two processes can be summed up in two questions as identified earlier: Perceptual Object Selection: 1) What factors make a driver look at or attend to a particular object? 45 Perceptual Object Identification and Information Acquisition Duration: 2) What factors determine the length of time a driver attends to an object? If the flow of stimuli or information can be considered the demand, the supply or resource is the available attention that can handle the demand. The study of driver attention should not be limited to the selection process but also the capacity limits of attention. These questions may never be fully answered as the subject of human visual system and attention is complex and subjective. But an attempt to examine such questions may provide more clues to the driver's visual attention process, and hopefully, a better understanding of driver behaviour. 4.1.3 Perceptual Objects and their A ttributes At any given moment, the total of all parts of the environment that stimulate a person's eyes is called the visual field. Figure 4.1 shows a portrait of the visual field as seen by the left eye of physicist/psychologist Ernst Mach as he lay on a couch in his study. Reflected light from the visual field forms a retinal image, consisting of a two-dimensional representation of the perceived environment. But this two-dimensional picture carries an enormous amount of raw information and the mind provides meaning to the picture by grouping the array of "blobs" and contours into perceptual objects. These objects can be regarded as single, independent entities and are the visual stimuli or cues that are acquired through the visual system. Perceptual objects can be thought of as aggregations of smaller less significant images that form larger more significant images that have greater meaning to the driver. In the case of the driver environment, perceptual objects can be vehicles, traffic control devices, pedestrians, and various background features such as buildings and vegetation. 46 Figure 4.1 The Visual Field as Seen by the Left Eye of Ernst Mach (Source: Mach, 1959/1886) 47 In this research, each perceptual object is considered to be uniquely identified by the following attributes: • type (class hierarchy) • surface feature (color/brightness/opacity) • physical dimensions • velocity • position 4.1.4 Physical Limitations in Visual Information Acquisition In the overall process of selecting a single perceptual object of a vast number of objects within the driver's environment, many objects are removed for consideration as the driver's vision is limited by a field-of-view (FOV) defined by the physical characteristics of the eye. The driver's vision is not 360 degrees. In general, it can be described as an oval shaped domain approximately 180 degrees horizontally and 150 degrees vertically with visual acuity progressively degraded toward its boundaries (Haber and Hershenson, 1973) and these parameters vary between people and physical condition. Within the average vehicle, the driver is also provided with visual aids, such as rear and side-view mirrors, to expand his potential field-of-view (PFOV) (Figure 4.2). These mirrors literally redirect visual information normally unseen to the driver by copying such images onto themselves. But only when these mirrors are within the drivers F O V , can these images then be seen. These mirrors are most effective when the driver gazes directly at one as they are relatively small. Otherwise these mirrors just fall into the driver's peripheral vision and are infrequently attended to. But in some cases, such as a 48 flashing image reflected on the mirror, these mirrors can draw the driver's attention towards it for further investigation. The driver also has limitations regarding time of information acquisition due to the speeds of eye and head movements from one visual fixation position to another. 49 Figure 4.2 Potential Fields-of-View of the Driver in a Vehicle 4.2 Details of the VISA Model The visual processor model proposed in this thesis is a visual information acquisition model that utilizes attention and delay mechanisms in an attempt to reproduce the perception and identification phases of P R T . The proposed model is called the Visual Information Selective Acquisition driver attention model (VISA). 4.2.1 Model Components A visual model requires storage, selection and acquisition abilities. Therefore the V I S A model is comprised of four general components: • Iconic Memory • External Focusing Mechanism, • Internal Focusing Mechanism, • Delay Mechanisms These components interact with each other to perform three main tasks of the perception processor in the context of the driver behaviour framework as defined in Chapter 3: 1) select the most promising object from the driver's visual environment, 2) determine the "perceptual delay" as a result of this selection process, and 3) transfer the objects perceived visual information from the Iconic Memory to the short-term memory 50 4.2.2 Visual Focusing (Object Selection) Consider a driver that is totally field dependent. This type of driver would gather visual information in an erratic fashion, fixating on which ever stimulus is the most "pleasing" to the eye. Much of the information gathered by this driver could probably be considered irrelevant to the driving task. In fact, there is a chance that important cues may not be acquired due to this wayward method of information collection. Experiments performed by Hughes and Cole (1986) have found that an average driver pays about 30 to 50% attention to objects relevant to driving and 15 - 20% attention to traffic control devices. This is not sufficient to ensure that all or even most of these devices are noticed. Regardless, a totally field dependent driver would be considered inefficient, in terms of visual information gathering ability, and require more time to process the available visual information (Shinar et al., 1978). Similar results were made by Mourant and Rockwell (1972) in their study of novice and experienced drivers. They concluded novice drivers were more dependent on their foveal vision for gathering information than experienced drivers. On the other hand, the exact opposite kind of driver would be one that was totally set on a specific visual search pattern. This type of driver would gaze at locations irrespective of the stimuli make-up of the environment, with the search pattern already predetermined. If exactly the right visual cues were in their respective places as anticipated by the fixed search pattern, then the driver would be just fine. In fact, such a driver would then be as visually efficient as possible. But the probability of all required visual cues being located exactly where a driver expects them is almost nil. The V I S A model was designed to incorporate both of these visual information acquisition methods through the use of two mechanisms called the Internal Focusing Mechanism and 51 the External Focusing Mechanism. Experimental results from Theeuwes (1991) support the existence of such a dual-control visual selection process, which he terms exogenous and endogenous control. The Internal Focusing Mechanism can be regarded as the macro-focusing process, directing the driver's visual attention to a general proximity that is predicted to situate the most promising target based on previous knowledge stored in the short and long-term memories. The External Focusing Mechanism can be regarded as the micro-focusing process, "locking-on" to the most promising target within the driver's field-of-view. The Internal Focusing Mechanism is proactive, while the External Focusing Mechanism is reactive. 4.2.2.1 External Focusing Mechanism The External Focusing Mechanism directs the driver's visual attention towards the most promising object based on it's attributes. This mechanism relies on a total of nine visual factors or variables, classified as either Objective External Variables, or Subjective External Variables. These variables are defined independent of each other: Objective External Variables i) Opacity - relates to the reduction in visibility of an object due to a translucent object blocking it. The greater the opacity of the blocking object, the less attention the blocked object draws. Objects can be blocked by other objects, or partially visible due to blockage by translucent objects. ii) Acuity - the ability for the driver to discern details. Acuity reduces exponentially as the object is further away from the driver's foveal gaze. The higher the acuity value, the more attention the object draws. Cole and Hughes (1984) found the conspicuity of an object was strongly correlated to the object's displacement from the line of sight. 52 iii) Visual Size - the visual angle an object takes within the drivers field-of-view. The larger the relative visual size of the object, the higher the attention demand. iv) Contrast - the conspicuity or difference in color and brightness of the object with respect to it's background. The higher this difference, the more attention the object draws. v) Temporal Contrast - the change in the object's contrast over time (e.g. flashing). The higher the frequency of contrast change, the higher it's attention demand, but up to a threshold of discernment before the driver cannot detect the contrast change. vi) Longitudinal Velocity - this is the change in the object's visual size. The faster the rate of change, the higher the attention demand. vii) Latitudinal Velocity - the change in the object's tangential position. The faster the rate of change, the higher the attention demand. Subjective External Variables viii) Freshness - the uniqueness of the object based on the last time it has been focused on (i.e. selected). This variable is dependent on the driver's short-term memory. A n object that has just been attended to, will have a high freshness value within the driver's short-term memory. A n object that has never been attended to or not at least for a long while, will be less "fresh", or "stale". The lower the freshness value of an object, the higher the attention demand of that object. This variable is necessary in order to avoid the model being "locked on" to a very stimulating stimulus for long durations (i.e. large truck (visual size) ahead of driver (importance) with flashing lights (temporal contrast)). ix) Attractiveness - the subjective fondness towards an object. This variable is subject to the driver's independent affinity towards a type of object and color. The higher the attractiveness value, the higher the attention value. This variable is dependent on the driver's long-term memory which contains personal attractiveness values of all objects and colors available to the driver. 53 4.2.2.2 Internal Focusing Mechanism In concept, the Internal Focusing Mechanism directs the driver's,visual fixation to a general area of his environment directed by expectation or search patterns based on the current driving task. The definition of the pattern of such searching or probing is the focus of this mechanism. The existence of definite eye fixation sequence patterns has been studied, but with inconclusive results (Zwahlen, 1993). For the purpose of simulation, an appropriate method to model this internal focusing process is with the use of probability density maps of visual fixations rather than discrete eye fixation sequences. These maps are derived from mapping eye-fixation patterns similar to the experiments performed by Mourant and Rockwell (1970) and Burnett and Joyner (1993). A probability density map would be defined for each type of driving task, as each different task would prioritize types of information in terms of object type and location. For example, during a left lane change, vehicles would be considered the most important object type. A n d vehicles on the left side of the driver are more important than vehicles to the right, in the opposing lanes, or far at a distance from him or her, and therefore demands higher attention. The density pattern of these maps would also be dependent on the driver's level of experience (Mourant and Rockwell, 1972). 4.2.3 Determination of Attention Demand Values Once all the external and internal variables are obtained for each object, the attention demand value ( A D V ) is determined. The object with the highest A D V is the object that is selected for further investigation and stored in the short-term memory, while the rest of the objects are removed for further consideration. In cognitive terms, the object with the highest A D V is consciously processed, and the rest of the objects are stored in the subconscious. 54 The perceptual object with the highest A D V is visually fixated on in most instances, but only i f it is significant enough to warrant direct investigation via foveal vision. Otherwise, the object is examined through peripheral vision (i.e. visual angle is not changed). The threshold that determines whether an object's information is acquired through the foveal or peripheral is called the Peripheral Acquisition Threshold (PAT) , as shown in Figure 4.3. The level at which the P A T is set depends on the field dependency of the driver, with the level higher for less field dependant drivers (i.e. experienced drivers). Total Attention Capacity ADV < PAT Perceptual Object Information Acquired via Peripheral Vision (a) Peripheral Acquisition Threshold ADV (PAT) ADV r M 1 ADV > PAT Perceptual Object Information Acquired via Foveal Vision (b) F i gu r e 4.3 Fovea l / Pe r i phe ra l In format ion A c q u i s i t i o n L o g i c The A D V variable is calculated based on the relationship of the variables in the External and Internal Focusing Mechanisms. To develop such a relation would require vast amounts of empirical data of each of the variables in context with each other. This 55 relation is considered to be the core of the V I S A model but the calibration and validation of such a function could not be done as such an experiment and development of appropriate apparatus is beyond the scope of this research. Only a general relation could be established that can be used as a framework for such a calibration exercise. Chapter 5 discusses model calibration issues for validation to establish the relation and Appendix A presents an apparatus that can be used for this process. 4.2.4 Information Acquisition and Processing Delays The V I S A model portrays the perception and identification time delays of P R T based on five variables: Perception Time Delay: i) Visual Fixation Delay Identification Time Delay: ii) Recognition Delay Emotion Time Delay: iii) Expectancy Delay iv) Panic Delay v) Attraction Delay 4.2.4.1 Perception Time Delay Visual Fixation delay is defined as the search time or time it takes to physically fixate the eye (saccadic movement) and head from one object to the next. This delay mechanism 56 models the time required for stimuli detection and is assumed to be a function of the time difference between one fixation to the next. A n experiment performed by Robinson et al (1972) recorded maximum velocities for the eye of approximately 1,000 degrees/sec, and of the head about 450 degrees/sec. The head was found to usually lag the eye by about 50 msec and eye movements were observed to stop at about 40 degrees for all target angles. When the head lags the eye, the visual fixation can be considered unintentional or externally driven. The opposite can be considered intentional or internally driven. This can be argued by the fact that the driver tries to minimize the time required to acquire information from one object and fixate onto the next object (if the subsequent fixation is planned) as eye movements are quicker than head movements. 4.2.4.2 Identification Time Delay Recognition delay is defined as the time penalty due the unfamiliarity of the fixated object. In relation to identification, recognition describes a sense of familiarity whereas identification requires the production of a specific label or category name (Haber and Hershenson, 1973). Recognition delay is a function of the object's complexity (information content) and familiarity. The degree of familiarity is not a property of the stimulus, but of the perceiver. Recognition delay increases as the complexity and unfamiliarity of the object increases. This increase in time can be considered to be the time required to "learn" about the object. Less time is required to acquire information from a familiar object as once enough information is received to identify the object, the remainder of the object's attributes is "filled i n " by the previous knowledge of the object as stored in the L T M . 57 4.2.4.3 Emotion Time Delay Expectancy delay is defined as the extra delay caused by observing an object that was not expected in relation to the current driving task and level of risk assumed. Johannson and Rumar (1971) measured brake reaction times for drivers in expected and unexpected situations. They found that the reaction time averaged about 2/3 sec. in expected situations, with some taking as long as 2 sec. In unexpected situations, reaction times increased by 35 percent. Figure 4.4 shows how expectancy can influence overall reaction time. u a> J2- 3 o E c o 1 2 OC Unexpected -•* Expected 2 3 4 Information Content (Bits) Figure 4.4 Median Driver Reaction Time to Expected and Unexpected Information (Source: A A S H T O , 1990) 58 When an object is not expected, Panic delay can be initiated. Panic delay is subject to the A D V and current driver attention level (DAL) . In the V I S A model, the driver has a limited attention capacity. This attention capacity is split into two types of attention: reserve attention supply (RAS) and driving attention supply (DAS). The D A L defines the maximum amount of driving attention supply (DAS) that is allocated to the driving task with the remaining supply of attention being allocated to the R A S . But the D A L is not fixed in that it is set based on the A D V ' s of the objects the driver fixates on. Initially, the D A L is set at a level called the D A L Equilibrium, which is the "level-of-risk" that the driver perceives for a particular driving task. When an object with an A D V higher than the D A L is processed, panic is initiated and the difference between these two values determines the amount of panic delay. For the next time iteration, the D A L is then reset to the level of the processed A D V . A s well, the driver's stress level appropriately reflects this change. But after a while, if the subsequent objects selected by the driver have lower A D V ' s than this "heightened" D A L , the D A L is eventually set back to the D A L Equilibrium. Figure 4.5 demonstrates this process. Attraction delay is defined as the excess time in fixating on an object due to the driver's attraction to it. This delay is a function of the driver's subjective fondness towards the object's type and colour and is dependent on the driver's long-term memory which contains attractiveness values of all known objects and colours. 4.2.4.4 Multiple-Glancing Depending on the complexity of the driving environment, a driver will commit only a specific amount of time to visually fixate on a perceptual object. Usually, this time allotted is adequate to gather enough information about the object for identification. But in some situations, more time may be required to fully collect all the 59 j j E < < V Q Q LU ADV RAS DAS >> £TuT = Q < Q _ l E < V Q LU ADV > RAS > > DAS > o - I > • < -2 §1 o '5S o _j c o re u o c 0 c 1 c •> LO 0) 1_ 3 o C 3 .2 Q o) '-5 ^ I 5 s> J E > C S < IT Q < _J Q LU — .o el I ADV Reserve A Attention Supply (RAS) v Driving Attention Supply (DAS)  >( _i -2 »* > ro Q OL < O z necessary information. In these cases, one of two options may be taken. If the A D V exceeds the D A L , panic delay is initiated and fixation on the object is continued for a longer period. If the A D V does not exceed the D A L , the object is held on "hold" at the next fixation iteration so that another object may be viewed, after which the previous object that is on "hold" is then re-attended to in order to finish collecting the rest of its information. Whether the information is acquired through the fovea or periphery influences amount of information received for a given fixation. The rate of information transfer increases proportionally with acuity (i.e. visual bandwidth increases with increase in acuity), or to what degree the object is from the foveal line-of-sight. Information flow is also influenced by blinking rate and duration, which is a function of the driver's physiological condition (e.g. tired). Therefore to summarize, the total time required to acquire information from a selected object is dependent on the properties of the object as perceived by the driver, the current driver task, the number of fixations on the object, and method of information acquisition (i.e. via foveal or peripheral vision). 4.2.5 VISA Model Algorithm As the V I S A model is intended for use in a computer simulation model, its development required details that took this into consideration. The following is a generalized step-by-step process of a single simulation time iteration for one driver (Figure 4.6): 1) Scan all objects within the driver's environment and determine which are within the driver's potential field-of-view (includes any object reflected by the mirrors), relative to the current visual direction as determined in the previous iteration. 61 2) Determine which objects are visible (i.e. not blocked) and to what degree they are visible (based on blocking object's opacity). These objects are entered into the driver's iconic memory for temporary storage (purged after each time iteration). 3) Determine values of the external attention variables for each object. 4) Determine values of the internal attention variables for each object 5) Determine ADV for each object. 6) Select the object with the highest A D V and determine whether object information is acquired via foveal or peripheral vision. 7) If object information is acquired via foveal vision, determine new eye/head fixation direction to retina image coordinates of the selected object; transfer information to the short-term memory. 8) Determine the time delays required to acquire information from the selected object (perception, identification, and emotion time delays). 9) Update driver variables: a) adjust to for new desired speed accordingly, b) adjust to for new desired steering accordingly, c) update vehicle position, d) set next time increment for re-processing this vehicle (function of the total of all time delays). This cycle would be repeated for every driver before the next simulation time increment. 62 Iconic Memory {con t a i n s j no . of o b j e c t s ) T Identification and Emotion Delays Panic Delay Simulation Variable Updates: - new d r i ve r s p e e d - new s t ee r i ng d i r e c t i on - upda te v e h i c l e pos i t i on - new e y e / h e a d d i r e c t i o n F i g u r e 4.6 V I S A Mode l F l owcha r t 63 5 M O D E L VALIDATION P R O C E S S The overall development of the V I S A model could be defined in three parts: Part 1 of the V I S A model development project consisted of the model's conceptual design (the scope of this thesis). Part 2 is the verification phase. This process is straight forward and is a function of the quality of coding or interpretation of the model concept to computer logic. The final phase, Part 3, consists of the calibration and validation of the computer model. 5.1 Model Verification, Calibration, and Validation After coding the V I S A model into the computer, the verification, calibration, and validation of the model must be done before the model can be applied in a meaningful manner. The definition of each step is as follows: Verification To verify is to check, inspect, test, or confirm that the computer code functions as intended. This process requires a detailed testing of all logic routines and simulation processes (algorithms) of the computer code such that it conforms to the model deign as intended. This process is also called "debugging." A t this point, no attention is given to the real-life situation. Calibration Calibration is the tuning or adjusting of the simulation model with real-life or field-observed data to increase its accuracy. This requires modifying the verified computer model such that statistics gathered and reported by the model reasonably agree with empirical observations under similar conditions. In other words, the act of calibration standardizes or "customizes" a (generic) model. Only a portion of the observed data is used in this process. The remainder is used in the validation process. 64 Validation Validation is the verification, authentication, or confirmation that the model functions similar to real life over a wide range of conditions. The validation process is the comparison of the remaining observed data with the model output. If the comparisons are acceptable, the validation is complete. Otherwise, the model must be recalibrated and revalidated with more field-observed data. Only after the model is validated can it be applied to test real life situations, but only for situations that it is similarly validated for. The applicability or usefulness of the model degrades as it is used for situations dissimilar for which the validation was made. Figure 5.1 shows the three processes of model development. Real Life Flowcharts Calibration Validation Verification Computer code Figure 5.1 Verification, Calibration, and Validation Process (Source: May, 1990) 65 During model development questions can arise such as "How valid is a model?" and "How far do you go in calibration?" The answer depends on the particular task that is being modelled. For example, if visual fixation pattern results are what is desired, the model needs to have visual fixation pattern calibration done. The actual data used for calibration determines what type of calibration is done. In essence, the calibration factor obtained is really good only for the types of inputs that were used in the calibration runs. For more certainty or confidence, calibration for certain particular scenarios would be done if the model is to be used (valid) for such scenarios. Any results a model gives for a particular event are always estimates and the confidence in these estimates is correlated to the degree of validation the model is done for that event, or inversely correlated to the degree that the particular event differs from what the model is validated for. 5.2 ADV Relation Calibration The attention demand value ( A D V ) is calculated based on the relationship of the variables in the External and Internal Focusing Mechanisms. A n A D V function which combines all these variables needs to be defined and calibrated using observed data in the final phase of the V I S A research project (initially, each of the model variables would also be studied independently in order to assess the characteristics and parameters of each variable). Only once the model is calibrated and validated, may it then be used. But the calibration and validation process may prove to be the most difficult part of the V I S A model's development as detailed data would need to be collected regarding driver visual attention. To reiterate, the main goal of the model is to predict what object the driver will fixate on based on the objective and subjective characteristics of the object with respect to the driver, as well as the driver's own expectancies. The calibration of the model requires 66 collecting data on the variables that determine the A D V . The premise of this model assumes that these variables determine why, and to a degree, for how long the driver looks at a particular object within an environment of other objects. Also, the fixation onto an object is dependent on the previous object that was gazed at, as this set the initial conditions for that fixation. This requires the nine A D V variables to be gathered for each object that the driver fixates on, as well as the duration of fixation. Again the two mechanisms are the External Focusing Mechanism and the Internal Focusing Mechanism. 5.2.1 External Focusing Mechanism Objective Variables The data for the objective variables of the external focusing mechanism requires eye and head fixation patterns to be recorded. This assumes that every visual fixation is partially a function of the object's objective and subjective variables: Therefore, every time the driver fixates onto a new object data is collected for that fixation event as shown in Table 5.1. Subjective Variables The data for the subjective variables of the external focusing mechanism need to be collected in reference to predetermined subjective value tables or functions so measurements can be defined (Table 5.2). 67 Table 5.1 Objective External Variable Measurement Parameters Variable Measurement Parameter Opacity the object's visibility behind a translucent blockage Acuity the angle of the object from the previous fixation angle Visual Size the visual angle of the current object Contrast the difference in contrast of the object relative to it's background relative to the driver's point of view. Temporal Contrast the flashing frequency of the object Longitudinal Velocity the rate of change of the object's the visual size Latitudinal Velocity the rate of change of the object in the tangential direction to the driver Table 5.2 Subjective External Variable Measurement Parameters Variable Measurement Parameter Freshness duration of previous fixation to the object (a function that determines the level of interest to a previously observed stimulus) Attractiveness object type and colour (values assigned to all objects by the test driver indicating affinity to the object) 68 The difficulty in this calibration process lies in the intertwined nature of the objective and subjective variables. Data for the objective variables can be collected externally as these objective variables can be related to physical cues. On the other hand, subjective variables are based on the driver's bias and preferences. N o apparatus exists that can read the driver's mind and motives. One way subjective data can be collected is through the use of questionnaires, but these may have errors associated with differences in conscious and unconscious preferences. 5.2.2 Internal Focusing Mechanism The internal focusing mechanism is the expectancy director and is a collection of fixation probability density maps (or visual search probability maps) of the driver's visual field. A map for each driving task has to be developed which is based on the importance of an object type and its position relative to the driving task (Table 5.3). Table 5 . 3 Internal Variable Measurement Parameters Variable Measurement Parameter Importance object type and position (the driver's values of object type and their position to the driver for each driving task) The visual search probability maps of the internal focusing mechanism may be hard to isolate independently for the sake of just visual searching without being affected by the objects in the environment. But i f many experiments are done to collect enough data for various driving tasks, it is hoped that these probability maps w i l l form into distinctive patterns of "visual search probability clusters." 69 5.2.3 Driving Task Determination The variables mentioned above need be measured in relation to the driving task or manoeuvre at hand. Therefore, additional measurements of variables pertaining to driver task would be required in order to estimate the type of manoeuvre the driver performed. These variables would be related to the magnitude and amplitude of change to driving operations such as steering, braking, acceleration, visual direction, signaling, and stress level. Table 5.4 relates these variables to applicable measurement parameters. Table 5.4 Driving Task Variable Measurement Parameters Variable Measurement Parameter Lateral changes in steering angle Longitudinal acceleration, deceleration, clutch/shifting, speed Visual Direction changes in eye and head direction Signal usage of signals Stress Level heart rate, blinking rate, breathing rate, temperature 5.2.4 ADV Equation A data base of measurements for each driving task would result from experiments. A portion of this information would need to be processed in order to determine a relation of these variables to driver attention. Methods such as regression or neural networks could be employed in order to determine the "best fit" calibration equation for each driving 70 task. The remaining portion of data would be used to determine the validation of the equation. 5.3 Data Collection System A data collection system is required that can capture both the driver's physical movements (e.g. eye/head direction, steering, breaking, acceleration, signal) and stress level (e.g. heart, blinking, breathing rate; temperature). This system would need an adequate eye/head-tracking apparatus and a simulator that can realistically simulate a variety of driving conditions to human specimens. A proposed design for such a driving simulator is discussed in Appendix A . 71 6 R E C O M M E N D A T I O N S 6.1 Model Improvements As the proposed visual attention model is still conceptual and not validated, there are many improvements and further research that is needed. Some recommendations for further research are: • more details of the information flow control structure between model concepts and elements needs to be made in order for the model to be coded into a computer simulation, • further research on the nine visual variables and five delays that make up the attention and delay mechanisms of the model, and how these variables relate to each other. • development of an A D V relation that incorporates all nine visual variables • the proposed mechanism that determines information acquisition between the foveal and peripheral could be further examined and refined • the incorporation of the V I S A model into a comprehensive driver model as outlined in chapter 3 is required in order to truly verify the effectiveness and applicability of the V I S A model in a driving situation. Stang (1993) developed a detailed driver behaviour model rooted in psychological fundamentals. This model incorporated fuzzy logic to handle the subjective uncertainty of the human driver. Stang's driver behaviour model is an ideal environment in which the V I S A model could be tested and applied. 72 6.2 Validation For a model to be considered useful as a tool, it needs to be thoroughly validated in the various environments that it will be used under to verify it's applicability in those situations. In the case of the VISA model, this will require a great deal of empirical data collected regarding the driver's physiological behaviour using specialized data collection apparatus, as discussed in chapter 5. Appendix A discusses the specifications of a simulator apparatus for the collection of calibration data. 73 7 SUMMARY AND CONCLUSION 7.1 Overview The development of the Visual Information Selective Acquisition driver attention model was initiated by first reviewing general driver behaviour models in order to gain an understanding of previous conceptualizations and theories of driver behaviour. It was found that there were many different types of models developed to attempt to explain various types of driver behaviour. But most of these models were approached in a narrow view and explained only a portion driver behaviour. It was determined that a comprehensive model of driver behaviour is required that models drivers at strategic, tactical, and operational levels while providing an information flow control structure that enables control to switch from one level to another. This led to the development of a general framework of a driver behaviour model based on a hierarchical structure. Within the context of this framework, the development of a visual attention model was made. The proposed visual attention model attempts to explain the process of driver attention and information acquisition. The model facilitates visual constraints and gathers information based on a selective process that allows for events such as distractions. The visual attention in the model is influenced by two mechanisms: internal and external focusing. The internal focusing mechanism is a proactive director that moves the driver's head and eye to a general direction such that information relevant to the current task is actively searched for based on the driver's expectancy. The external focusing mechanism is a reactive focusing mechanism which causes the driver to focus and pay attention to a certain object based on it's objective attributes. This model was developed to be incorporated into a computer driver simulation model (similar to the work of Stang, 1993) to enhance the visual information selection abilities 74 of such driver models by providing the use of a broader range of variables pertaining to the driver's visual system. This model has the potential to be used in a wide range of studies requiring visual sensitivity, such in the case of in-vehicle navigation systems, and detailed enough to model a variety of visual conditions (age-related, intoxicated) within varying driving environments. The benefits of the resulting model are that it facilitates the potential to model various driver types and conditions, it permits distractions to be modelled, it permits the study of the driver's perception of detailed driving environments, and it can be used to study subjects other than drivers, such as pedestrians and cyclists. However, typical to any theoretical work, there are uncertainties and limitations to the applicability of the model that need further addressing. The model may be too complex and data hungry and especially has a high appetite for subjective data which may prove to be difficult to collect. Due to this reason the relationships between the variables within the model could not be derived. The model may also prove too simplistic to be used to investigate the complex subject of driver visual attention and further refinements may be necessary. The resulting model described in this paper is relatively complex, yet in an absolute sense, very simplistic. The human visual system is far more detailed and elaborate to which this model does not do justice. This fact reinforces the complex and convoluted nature of the human visual system and psychological behavior. But the fact remains that the nature of human behavior is quite complex. Likewise, such models are also expected to be similarly complex. Regardless, whether this type of research is worth pursuing any further is in the extent of our curiosity to learn more of how drivers behave, and the desire to apply this knowledge to improve the safety of both drivers and road users. 75 7.2 Potential Applications Through the integration of the V I S A model into a computer simulation, the model can be employed in a variety of attention studies such as: • Effects of roadway advertising on driver attention As the quality and quantity of roadside advertisements is a topic of road safety, this model could assist in measuring the degree of influence advertisements such as billboards and flashing signs could have on the driver's attention and reducing the driver's awareness of hazards. Based on findings, policies such as no advertising near sensitive areas (school zones) could be determined with a higher degree of confidence. • Effects of in-vehicle navigation systems Such devices have the potential to introduce or expand subsidiary task demands which can compete with the primary task of driving by increasing cognitive, motor and visual workload and thus degrade safety. A factor in the successful deployment of these devices will be the degree of distraction produced in relation to the driving task. • Driver physiological studies due to age, intoxication, etc. As the V I S A model incorporates variables such as cognitive process delays, limited field-of-view, and visual search patterns, drivers of varying physiological conditions could be modelled. • Reduced visibility (night time/ bad weather) studies The conspicuity of perceptual objects can be reduced in order to model reduced visibility driving conditions. 76 Effectiveness of Traffic Control Device (TCD) design and placement The design (size, colour, etc.) and physical placement of T C D s could be evaluated using the model to quantify the conspicuity of these devices. Too much or too little attention to a device would consider it a possible candidate for improved design or placement. Distractions as a means of assessing a roadway's safety As the V I S A model is an attention model, it can model distractions. One use for the modelling of distractions is to possibly use distractions as a measure of a roadway's safety. The number of incorrect fixations can be determined to define a measure of inattention or distraction. This measure can be used to evaluate a roadway (i.e. attention demand load) much like traffic conflicts are used as a predictor of accidents. In fact, inattention could be used as an indirect predictor of conflicts. And as distractions are more common than conflicts, and far more than accidents, it could be used as a more reliable measure due to its higher number of occurrences. In the context of the "safety pyramid" concept, inattention could fall within or under "potential conflicts" (Grayson and Hakkert, 1987). The V I S A model could be incorporated into a conflict simulation, such as that of the work of Sayed, et al. (1994), to be used to find correlations between conflicts and distractions. 77 B I B L I O G R A P H Y A A S H T O . (1990). A Policy on Geometric Design of Highways and Streets. American Association of State Highway and Transportation Officials, Washington, D . C . Anderson, J.R. (1983). The architecture of cognition. Cambridge, M A : Harvard University Press. Bates, J.T. (1995). Perception-reaction time. ITE Journal, Institute of Transportation Engineers, Washington, D . C , V o l . 65, N o . 7, pp. 35-36. Bhise, V . D . , and Rockwell , T . H . (1973). Toward the development of a methodology for evaluating highway signs based on driver information acquisition. Highway Research Record, National Research Council , Washington, D . C , pp. 38-56. Burnett, G . E . , and Joyner, S . M . (1993). A n investigation on the man machine interfaces to existing route guidance systems. IEEE-IEE Vehicle Navigation & Information Systems Conference, Ottawa - V N I S ' 93. Cole , B . L . , and Hughes, P . K . (1984). A field trial of attention and search conspicuity. Human Factors, V o l . 26. Connolly, P . L . (1968). Visual considerations of man, the vehicle, and the highway (Part 2). In Highway Vehicle Safety (Collected SAE Papers, 1961-1967). E d . Stonex K . A . et al . , S A E Inc., New York, N Y , pp. 352-413. Coren, S., and Ward, L . M . (1989). Sensation and Perception (3rd Ed). Harcort Brace Jovanovich, San Diego, Calif . Cremer, J.F., and Kearney, J .K. (1994). Scenario Authoring for Virtual Environments. Presented at I M A G E VII Conference, Tucson, Arizona, June 12-17 1994. 78 Dewar, R., (1992). Driver and pedestrian characteristics. Traffic Engineering Handbook (4th Ed.), Ed . J. L . Pline, Prentice-Hall, Ehglwood Cliffs, New Jersey, pp. 1-28. Drew, D . (1968). Traffic Flow Theory and Control. McGraw-Hil l , New York, N Y . Edie, L . C . , and Foote, R.S. (1960). Effect of shock waves on tunnel traffic flow. Highway Research Board, Proceedings, V o l . 39. Evans, L . (1991). Traffic Safety and the Driver. Van Nostrand Reindel, New York, N Y . Fleishman, E . A . (1975) Toward a taxonomy of human performance. American Psychologist, V o l . 30, pp. 1127-1149. Garber, N.J. , and Hoel, L . A . (1988). Traffic and Highway Engineering. West Publishing Co. , St. Paul, M N . Gordon, D . A . , (1966). Experimental isolation of driver's visual input. Highway Research Record 122, National Research Council, Washington, D . C , pp. 19-34. Grayson, G . B . , and Hakkert, A . S . (1987). Accident analysis and conflict behaviour. Eds: J .A. Rothengatter and R . A . de Bruin, Road Users and Traffic Safety, Van Gorcum, Assen/Maastricht, The Netherlands. Greenberg, H . (1959). A n analysis of traffic flow. Operations Research, V o l . 7. Greensheilds, B . D . , and Piatt, F . N . (1968). Objective measurements of driver behaviour. In Highway Vehicle Safety (Collected SAE Papers, 1961-1967). E d . Stonex K. A . et al., S A E Inc., New York, pp. 277-289. Gulian, E . , Glendon, L, Matthews, G . , Davis, R., and Debney, L . (1988). Exploration of driver stress using self-reported data. Road User Behaviour: Theory and Research, Eds: T . Rothengatter and R. de Bruin, Assen/Maastricht, The Netherlands, pp. 342-358. 79 Haber, R . N . , and Hershenson, M . (1973). The Psychology of Visual Perception. Holt, Rinehart and Winston, Inc., New York, N Y . Halsey, M . (1941). Traffic Accidents and Congestion. John Wiley and Sons, Inc., London. Herman, R., Montroll, E . , Potts, R., and Rothery, R. (1959). Traffic dynamics: Analysis of stability in car following. Operations Research, V o l . 7. Hills, B . L . (1980). Vision, visibility and perception in driving. Perception, V o l . 9, pp. 183-216. Hughes, P.K., and Cole, B . L . (1986). What Attracts Attention When Driving? Ergonomics, V o l . 29, No. 3, pp.377- 391. Hughes, P.K., and Cole, B . L . (1988). The effect of attentional demand on eye movement behaviour when driving. In Vision in Vehicles - II, Eds: A . G . Gale et al., Elsevier Science Publishers, North-Holland, pp. 221-230. Hulbert, S. (1972). Driver Information Systems. In Human Factors in Highway Traffic Safety Research, E d : T . W . Forbes, John Wiley and Sons, Inc., Toronto, pp. 110-132. I C B C . (1996). ICBC Annual Report. Insurance Corporation of British Columbia, Vancouver, B . C . Johansson, G . , and Rumar, K . (1971). Driver' Brake Reaction Times. Human Factors, V o l . 13, No. 1, pp. 23-27. Kidd, E . A . , and Laughery, K . R . (1964). A Computer Model of Driving Behavior: The Highway Intersection Situation. Buffalo, N Y : Cornell Aeronautical Laboratories, Report nr. VI-1843-V-1. 80 Klebelsberg, D . (1977). Das Modell der subjektiven und objektiven Sicherheit. Schweizerische Zeitschrift fur Psychologie und ihre Anwendungen, V o l . 36, pp. 285-294 Kramer, U . , and Rohr, G . (1982). A model of driver behaviour. Ergonomics, V o l . 25, No. 10, pp. 891-907 Levine, O . H . (1995). The Northeastern Virtual Environments Based Driver Simulator: Description and User Guide. Technical Report No. 95, Virtual Environments Laboratory, Northeastern University, Boston, M A . Luoma, J. (1988). Drivers' eye fixations and perceptions. In Vision in Vehicles - II, Eds: A . G . Gale et al., Elsevier Science Publishers, North-Holland, pp. 231-237. Lusted, H.S. , Knapp, R.B. , and Lloyd, A . M . (1993). Biocontrollers for Virtual Environments. Virtual Reality Special Report, Fall 1993, Miller Freeman Inc. Mach, E . (1959). The analysis of sensations and the relation of the physical to the psychical. Dover, New York, N Y . May, A . D . (1990). Traffic Flow Fundamentals. Prentice Hall, Englewood Cliffs, NJ . McKnight, A.J . , and Adams, B .B . (1970). Driver Education and Task Analysis Volume I: Task Descriptions Final Report. Department of Transportation, National Highway Safety Bureau, Washington, D C . McRuer, D . T . , Allen, R .W. , Weir, D . H . , and Klein, R . H . (1977). New results in driver steering control models. Human Factors, V o l . 19, 381-397. McShane, W.R. , and Roess, R.P. (1990). Traffic Engineering Prentice Hall , Englewood Cliffs, NJ . 81 Michon, J .A. (1985). A Critical View of Driver Behavior Models: What Do We Know, What Should We Do? Human Behavior and Traffic Safety, Eds: Evans L . and Schwing R., Plenum Press, New York, pp. 485-520. Michon, J .A. (1976). The mutual impacts of transportation and human behavior. Transportation planning for a better environment, Eds: P. Stringer and H . Wenzel, Plenum Press, New York, pp. 221-236, Miura, T . (1990). Active function of eye movement and useful field of view in a realistic setting. From Eye to Mind: Information Acquisition in Perception, Search and Reading. Eds: R. Groner, et al., Elsevier Science Publishers, North-Holland, pp. 119-127. Mourant, R.R. (1997) Virtual Environments Based Driving Simulator. (31 Mar. 1997). http://splinter.coe.neu. edu/~mourant/velab-drivsim-pix.html (14 Oct. 1997). Mourant, R.R., and Grimson, C . G . (1977). Predictive head-movements during automobile mirror-sampling. Perceptual and Motor Skills. V o l . 44, pp. 283-286. Mourant, R.R., Rockwell, T . H . , and Rackoff, N.J. (1969). Drivers' eye movements and visual load. Highway Research Record 292, National Research Council, Washington, D . C . Mourant, R.R., and Rockwell, T . H . , (1970). Mapping Eye-Movement Patterns to the Visual Scene in Driving: A n Exploratory Study. Human Factors, V o l . 12, No. 1, pp.81-87. Mourant, R.R., and Rockwell, T . H . (1972). Strategies of Visual Search by Novice and Experienced Drivers. Human Factors, V o l . 14., No. 4, pp. 325-335. Naatanen, R., and Summala, H . (1974). A model for the role of motivational factors in drivers' decision-making. Accident Analysis and Prevention, V o l . 6, pp. 243-261. 82 Newell, A . , and Simon, H . A . (1972). Human problem solving. Prentice Hall, Englewood Cliffs, NJ . Norman, D . A . (1976). Memory and attention. (2nd ed.). Wiley, New York, N Y . Noy, Y.I. (1990). Attention and Performance while Driving with Auxiliary In-vehicle Displays. T P 10727 (E). Transport Canada, Ottawa. Pentland, A . , and Liu, A . (1995). Toward Augmented Control Systems. la Proceedings of the Intelligent Vehicles '95 Symposium, I E E E Industrial Electronics Society. Piatt, F . N . (1968). Operations research on driver behavior. In Highway Vehicle Safety (Collected SAE Papers, 1961-1967). Ed . Stonex K . A . et al., S A E Inc., New York, pp. 261-267. Ranney, T . A . (1994) Models of Driving Behavior: A Review of Their Evolution. Accident Analysis and Prevention, V o l . 26, No. 6, pp. 733-750. Reece, D . A . , and Shafer, S .A. (1993). A computational model of driving for autonomous vehicles. Transportation Research Part A: Policy and Practice, V o l . 27A, N o . l , p p . 23-50. Reid, L . D . , and Grant, P.R. (1993). Motion Algorithm for Large-Displacement Driving Simulator. In Transportation Research Record 1403, T R B , National Research Council, Washington, D . C . , pp. 98-106. Robertson, S., and Goodwin, P. (1988). Driver stress and the process of highway design; A pilot investigation of heart rate as an aid to road system appraisal using low cost equipment. In Road User Behaviour: Theory and Research. Eds: T . Rothengatter and R. de Bruin, Assen/Maastricht, The Netherlands, pp. 359-365. 83 Robinson, G . H , Erickson, D.J. , Thurston, G . L , and Clark, R .L . (1972). Visual Search by Automobile Drivers. Human Factors, V o l . 14, No. 4, pp. 315-323. Rockwell, T . H . (1972). Skills, Judgement and Information Acquisition in Driving. In Human Factors in Highway Traffic Safety Research, E d : T . W . Forbes, John Wiley and Sons, Inc., Toronto, pp. 133-164 Rockwell, T . H . (1988). Spare visual capacity in driving-revisited: New empirical results for an old idea. In Vision in Vehicles - II, Eds: A . G . Gale et al., Elsevier Science Publishers, North-Holland, pp. 317-324. Sayed, T . , Brown, G . , and Navin, F. (1994). Simulation of Traffic Conflicts at Unsignalized Intersections With TSC-Sim. Accident Analysis and Prevention, V o l . 26, No. 5, pp. 593-607. Sayers, M . W . , and Fancher, P.S. (1993). Hierarchy of Symbolic Computer-Generated Real-Time Vehicle Dynamics Models. In Transportation Research Record 1403, T R B , National Research Council, Washington, D . C . , pp. 88-97. Schmidt, I. (1968). Visual considerations of man, the vehicle, and the highway (Part 1). In Highway Vehicle Safety (Collected SAE Papers, 1961-1967). E d . Stonex K . A . et al., S A E Inc., New York, pp. 381-351. Shinar, D . (1978). Psychology on the Road: The Human Factor in Traffic Safety. John Wiley and Sons, Inc., New York, N Y . Shinar, D . , McDowell , E . D , Rackoff, N.J., and Rockwell, T . H . (1978). Field dependence and driver visual search behavior. Human Factors, V o l . 20, No. 5, pp. 553-559. 84 Stang, N . E . (1993) Modelling the Driver for Traffic Flow Simulation. Master Theses, Department of Civi l Engineering, University of British Columbia. Stark, L . , and Ellis, S. (1981). Scanpaths revisited: Cognitive models direct active looking. In D . Fisher, R. Monty, and I. Senders (Eds.), Eye movements: Cognition and visual perception, Lawrence Erlbaum, Hillsdale, NJ, pp. 193-226. Sussman, E . D . , Bishop, H . , Madnick, B. , and Walter, R. (1985). Driver Inattention and Highway Safety. In Transportation Research Record 1047, T R B , National Research Council, Washington, D . C , pp. 40-48. Taylor, D . H . (1964). Drivers' galvanic skin response and the risk of accident. Ergonomics, V o l . 7, pp. 439-451. Tecce, J.J., Savignano-Bowman, J, and Kahle, J.B. (1978). Effects of Visual Distraction on Contingent Negative Variation and Type A and B C N V Shapes. In Multidisciplinary Perspectives in Event-Related Brain Potential Research, Ed : D . A . Otto, EPA-600/9-77-043, pp. 358-363. Theeuwes, J. (1991). Visual Selection: Exogenous and Endogenous Control. In Vision in Vehicles - III, Eds: A . G . Gale et al., Elsevier Science Publishers B . V . , North-Holland, pp. 53-61. Treat, J.R., Tumbas, N.S. , McDonald, S.T., Shinar, D . , Hume, R . D . , Mayer, R .E . , Stansifer, R .L . , and Castellan, N.J. (1979). Tri-Level Study of the Causes of Traffic Accidents: Final Report, Volumes I and U. U.S. D O T HS-805-086, (NTIS PB 80-121064). University of Iowa. (1996). Iowa Driving Simulator. (21 May 1996). http://www. ccad.uiowa.edu/media/still/index.html (14 Oct. 1997). 85 Van der Molen, H . H . , and Botticher, A . M . T . (1987). Risk models for traffic participants: A concerted effort of theoretical operationalizations. Eds: J .A. Rothengatter and R . A . de Bruin, Road Users and Traffic Safety. Van Gorcum, Assen/Maastricht, The Netherlands. Wierwille, W . W . , Antin, J.F., Dingus, T . A . , and Hulse, M . C . (1988). Visual attentional demand of an in-car naviagtion display system. In Vision in Vehicles - II, Eds: A . G . Gale et al., Elsevier Science Publishers B . V . , North-Holland, pp. 307-316. Wilde, G.J.S. (1982). The theory of risk homeostasis: implications for safety and health. Risk Analysis, V o l . 2, pp. 209-225. Wolf, J.D., and Barrett, M . F . (1978). Driver-vehicle effectiveness mode, V o l . 2. Final Report. U.S. D O T HS-804-338. Yarbus, A . L . (1967). Eye movements and vision. Plenum, New York, N Y . Zaidel, D . M . , Paarlberg, W . T . , and Shinar., D . (1978). Driver Performance and Individual Differences in Attention and Information Processing, Volume I: Driver Inattention. Institute for Research in Public Safety, DOT-HS803 793. Zwahlen, H . T . (1993). Eye Scanning Rules for Drivers: How Do They Compare with Actual Observed Eye Scanning Behaviour? Transportation Research Record 1403, T R B , National Research Council, Washington, D . C . , pp. 14-22 86 APPENDIX A - DRIVING SIMULATOR A.1 Simulators as Diver Behaviour Data Collection Apparatus In order to verify a theory of driving, experiments need to be conducted in order to collect empirical data that can support the theory. Driver behaviour experiments outside the laboratory can be performed but may prove difficult due to the uncontrolled environment of these settings. Also, data from these experiments is difficult to quantify as such experiments usually rely mostly on video recording the drivers' fields of view and visual gaze direction. This method usually requires extensive manual data coding as image recognition computer programs are either unreliable or very expensive. Therefore, computer driving simulators have been developed to simulate the driving environment and provide researchers a controlled environment for which to collect data on driver behaviour. There are a variety of types of computer driving simulators that range from basic visual simulations to full-blown motion simulators that incorporate advanced graphical displays and audio systems. The purpose of a driving simulator is to mimic the real-life environment of driving such that drivers react and behave identically to the real-life situation the simulator is simulating. The reliability and effectiveness of the simulator is measured by the degree to which the simulator can mimic real-life. Therefore it is critical that the simulator provide realistic sensory feedback to the human driver specimen. This entails realistically simulating the visual environment, auditory cues, and vestibular motion of a driving experience in an automobile. The some of the advantages and disadvantages of using a computerized driving simulator for data collection on driver behaviour are as follows: 87 A. 1.1 Advantages • controlled environment (experiments can be designed easily and repeated exactly as before) • automated data processing (information about objects and their attributes in the driving environment can be easily collected with high precision, driver performance measurements can also be collected likewise) • hazardous events can be simulated without loss of safety (e.g. high speed experiments, influences of drugs, alcohol, etc.) • various vehicle types can be simulated that otherwise would be costly to use; also vehicle prototypes can be tested before production • various road facilities can be tested before their construction resulting in designs optimized for driver safety and ergonomics. A. 1.2 Disadvantages/Reservations • simulator settings will always be inferior to real-life settings • simulators are relatively difficult and expensive to design, build and maintain • "simulator sickness" may result from poorly calibrated visual-motion simulators A.2 Current Simulators Currently, there are many computer driving simulators that are used throughout the world for various driver behavioural research. The following sections discuss examples of some of these. 88 A.2.1 IOWA Driving Simulator The University of Iowa's Driving Simulator (IDS) is one of the most impressive and comprehensive simulator in the world. The highlight of the IDS is the six-degree-of-freedom hexapod motion-base that can simulate large displacements. The motion is courtesy of six hydraulic actuators that support the base of the enclosed simulator platform at three points (Figure A . l ) . The actual cab of a real automobile is situated in an enclosed platform where the vehicle controls are connected to sensors. As well, the speedometer and tachometer display simulated speed and rpm readings. High-resolution, textured graphics are projected onto panoramic screens in front and behind the driver to display the simulated driving environment (Figure A.2). And an audio system reproduces engine, wind, and road sounds composed of processed digital samples of live audio. 89 Figure A.2 Automobile Interior (Source: University of Iowa, 1996) Experiments are monitored in a control room where operators collect information on the driver, vehicle, and simulated environment (FiguresA.3 and A4). Figure A.3 Quad Screen Driver Performance Monitor (Source: University of Iowa, 1996) 90 F igu re A.4 Iowa Dr i v i ng S imu l a t o r Con t r o l R o o m (Source: University of Iowa, 1996) The experiments performed on the IDS include influence of drugs, disease, and disabilities on driving performance; the effectiveness of computer-assisted driving aids such as collision warning devices and intelligent information systems; and the response of humans to critical events such as sudden, unexpected braking by leading vehicles. 91 A.2.2 Northeastern Virtual Environments Driving Simulator The Virtual Environments Laboratory at the Northeastern University in Boston, Massachusetts, developed a low-cost, fixed-base driving simulator using a basic 90 M H z Pentium™ P C (Levine, 1995). This simulator utilizes virtual reality technology through the use of a head-mounted display ( H M D ) attached to a head tracking arm (Figure A.5). Figure A .5 Northeastern Virtual Environments Driving Simulator HMD Setup (Source: Mourant, 1997) 9 2 Audio sounds are generated by the simulator through a pair of stereo headphones. The operator sits in front of a steering wheel column and pedal array that mimic vehicle controls (Figure A.6). The interior of a Dodge Caravan minivan was digitized into a 3-D physical model and is rendered relative to the driving environment. This model includes the windows, instrument panel, and steering wheel (Figure A.7). F igu re A.6 S t ee r i ng a n d P eda l A r r a y (Source: Mourant, 1997) The Northeastern simulator has been used in experiments to investigate drivers' spare visual capacity, driver velocity estimation, and drivers' performance in simulated nighttime lane shifts. 93 Figure A.7 Virtual Environments Screen Rendering (Source: Mourant, 1997) A.2.3 Driver Training System (DTS™) F A A C Incorporated develops and supplies a driving simulator for the purpose of training fleet operators in vehicles such as trucks, buses and emergency vehicles. The DTS™ simulator is a fixed-base simulator that incorporates a database containing a 50 square mile network of continuous roadway with topographical terrain and road side scenery. Portions of the road scenery are projected onto three high-resolution textured large-screen displays that provide a 180 field-of view. Audio sounds are also produces that mimic driver compartment, vehicle, and outside/background noises. The simulator's compartment contains duplicate vehicle controls and an instructor's seat with a remote 9 4 keyboard for real-time event initiations. The DTS™ simulator simulates a variety of autonomous vehicles on the road which interact with the driver. A.3 Simulator Requirements for VISA Model Validation The V I S A model requires the determination of an A D V function that incorporates the nine internal and external variables as discussed in chapters 4 and 5. This may prove to be the most difficult part of the V I S A model's development as detailed data would need to be collected regarding driver visual attention. Measurements need to be made that can provide data for the calibration of the A D V function based on these nine variables. A way that this can be accomplished is through the use of a driving simulator equipped with an adequate eye-tracking apparatus such that objects drawn within the simulator can be referenced to the observed scanning patterns. Such an apparatus can be relatively expensive and difficult to develop. But with the advent of improvements and declining costs in technologies such as computer graphics, virtual reality, and bio-technologies, this is becoming a more realistic possibility. A goal in the development of the V I S A model would be to develop a computer driver simulator that is relatively inexpensive, yet provides realistic visual, auditory and vestibular feedback. A.3.1 Visual Environment Simulation Since approximately 90% of driving information is visual (Rockwell, 1972), the heart of a computer driving simulator is it's visual display system. There are generally two types of display systems currently used in computer driver simulators: fixed computer screens (monitors or projection), and head-mounted displays (HMDs). Simulators that use fixed computer screens require a real or mock-up automobile cab in which the operator is 95 situated. Such simulators using standard computer monitors do not provide the operator with a realistic field of view range as the small screens only encompass a small portion of the driver's field of view. Projector-style simulators have a much larger area of display, allowing for a better peripheral coverage. But simulators with projectors are costly and prove bulky when incorporated in a motion-platform. H M D s utilize the concept of virtual reality such that the visual display is projected relative to head movements. H M D s are relatively inexpensive and provide stereo-imaging. But these usually have a limited display areas and are prone to loss of peripheral presentation much like computer monitors. More expensive H M D s provide for a much larger display area much like projector displays. For the V I S A model, a H M D would be appropriate as this is less expensive and provides the required head direction information. A H M D apparatus that also can track eye gazing direction would be ideal as the virtual environment portrayed on the display can be redrawn based on both head and eye directions. A.3.2 Auditory Simulation Drivers rely on auditory cues for shifting, driving around corners, road condition, and the presence of road users in the driving environment. T o reproduce realistic auditory sounds of the vehicle interior, engine and transmission, tire noise, and roadway environment, a stereo audio system would be required. Currently, stereo systems based on digital signal processing (DSP) can reproduce elements of real-life environments such as sound coming from a certain location in space. The utilization of such an audio system in a driving simulator would provide for life-like auditory environment. 96 A.3.3 Vestibular Simulation A n experience that is highly characteristic of driving is the sensation of motion such as lateral and longitudinal acceleration, and body roll and dipping. When simulating driving around corners, operators in simulators without motion platforms tend to overestimate the curve and run off the road. This is due to the use of the vestibular sense when navigating curves. Also, operators may feel disoriented when placed in a simulator where visual cues are not supported with vestibular cues. Motion-based platforms are designed to provide the most realistic experience of driving in simulators. In general, there are two different types of motions that need to be addressed: small-displacement and large-displacement movements. Small displacement movements simulate vehicle dynamics and vibrations. This entails motion resulting from a vehicle interaction with a road terrain through the tires, suspension and body of the vehicle. Vehicle dynamics models such as the version of Sayers and Fancher (1993) utilize sophisticated suspension kinematic models and could be used to translate forces from simulated road terrain into vehicular vibrations and motion to give simulator operators a realistic tactile feedback. There are many different ways in which large-displacement motion can be reproduced in a driving simulator. A common method is through the use of a large-amplitude hydraulic actuator-based system similar to the one use in the Iowa Driving Simulator (Reid and Grant, 1993). This type of platform reproduces motion in the x, y and z-axises, but in a relatively limited range of amplitudes. A more high-performance motion platform would be one based on a motion carriage design as shown in the conceptual illustration of the National Advanced Driving Simulator in Figure A.8. These motion platforms allow for 97 larger displacements due to their long track design and are capable of simulating long accelerations on the x-y plane, and virtually unlimited rotation on the z-axis. Track-based motion carriage simulators require a larger space than hydraulic actuator-based systems but this allows for the reproduction of complex displacements and rotations at a large amplitude, allowing for the flexibility to handle most driving situations. F i gu re A.8 9-Degrees of F r e e d o m Mo t i o n Ca r r i age (Source: University of Iowa, 1996) A.3.4 Driver Measurements Without sensors to monitor the driver, a simulator cannot collect driver behaviour information. There are two methods that can be used to monitor an operator in a driving simulator: physical monitoring, and bioelectric monitoring. 98 A.3.4.1 Physical Monitoring Body movement can be used to determine the intentions and state of a driver (Pentland and L i u , 1995). In order to estimate the driver's visual attention and driving task, the simulator must be equipped with proper monitoring devices. To determine the visual gaze of a driver, an eye-tracking apparatus can be used. One type of eye-tracking method is to record eye movements through a video camera or laser. This method is fairly accurate and fixation points can be easily mapped. In conjunction to an eye-tracker, the use of a head-tracking apparatus provides information as to whether a visual fixation was externally or internally driven. As stated in section 4.2.4.1, fixations where the head moves towards the fixation point before the eye can be stated as internally driven. The reverse can be stated as externally driven. This observation can give insight as to whether the perceived object was actively searched for or just caught the attention of the driver. The position and movements of the driver's hand and feet can give clues to the intentions or current driving task of the driver. Patterns found in these measurements can be correlated to visual measurements in relation to the state of the driving environment. Actual motion sensors placed on the driver's hand and feet could be used, or a more simpler approach would be to measure changes in steering wheel and pedal positioning. A.3.4.2 Bioelectric Monitoring A more elaborate method of tracking eye movements is through the use of biocontrollers utilizing electrooculogram (EOG) signals. Biocontroller systems such as developed by Lusted et al. (1993) have been used to control the cursor on a computer. This type of system presents itself to be simpler than the more cumbersome video camera or laser based eye-tracking systems. As these E O G systems are based on the electrical pulses that control the actual eye itself, they can also determine the focusing of the eye, allowing for 99 object tracking in x, y, and z dimensions. But the accuracy of these systems is unknown and may be more inferior to its light-based counterparts. A n interpretation of stress can be measured through the use of biosensors that can detect electroencephalogram (EEG), electromyogram (EGM), and electrocardiogram (ECG) signals. Through the use of these sensors, the rate of eye blinking, muscle flexion, heart rate, and skin temperature can be monitored to determine a measure of stress. Robertson and Goodwin (1988) developed equipment to measure heart rate as an aid to assessing the stress level of drivers. Tecce et al. (1978) associated blink frequency with stress in that negative hedonic states, such as stress, tend to increase blink frequency and positive hedonic states, such as feelings of pleasantness and contentment, tend to decrease blink rate. This stress-blink association ahs been called the "Nixon effect" due to observations of Richard M . Nixon's speech of resignation as President of the United States. A.3.4.3 Driving Environment Monitoring Every measurement of the driver requires an associated record of the driving environment so that the actions of the driver can be correlated to the visual objects and environmental situation in the driver's field-of-view. This is a critical relation for the calibration of the V I S A A D V function. As the driving simulator is computerized, synchronizing the computer-generated driving environment with driver measurements can be made into a relatively simple, if not automated task. 100 

Cite

Citation Scheme:

        

Citations by CSL (citeproc-js)

Usage Statistics

Share

Embed

Customize your widget with the following options, then copy and paste the code below into the HTML of your page to embed this item in your website.
                        
                            <div id="ubcOpenCollectionsWidgetDisplay">
                            <script id="ubcOpenCollectionsWidget"
                            src="{[{embed.src}]}"
                            data-item="{[{embed.item}]}"
                            data-collection="{[{embed.collection}]}"
                            data-metadata="{[{embed.showMetadata}]}"
                            data-width="{[{embed.width}]}"
                            async >
                            </script>
                            </div>
                        
                    
IIIF logo Our image viewer uses the IIIF 2.0 standard. To load this item in other compatible viewers, use this url:
http://iiif.library.ubc.ca/presentation/dsp.831.1-0050244/manifest

Comment

Related Items