UBC Theses and Dissertations

UBC Theses Logo

UBC Theses and Dissertations

A biology-inspired attention model with application in robot navigation Du, Yu 2018

Your browser doesn't seem to have a PDF viewer, please download the PDF to view this item.

Item Metadata

Download

Media
24-ubc_2018_september_du_yu.pdf [ 13.16MB ]
Metadata
JSON: 24-1.0365999.json
JSON-LD: 24-1.0365999-ld.json
RDF/XML (Pretty): 24-1.0365999-rdf.xml
RDF/JSON: 24-1.0365999-rdf.json
Turtle: 24-1.0365999-turtle.txt
N-Triples: 24-1.0365999-rdf-ntriples.txt
Original Record: 24-1.0365999-source.json
Full Text
24-1.0365999-fulltext.txt
Citation
24-1.0365999.ris

Full Text

A BIOLOGY-INSPIRED ATTENTION MODEL WITH APPLICATION IN ROBOT NAVIGATION  by YU DU B. Sc., Dalian University of Technology, 2004 M. Sc., Dalian University of Technology, 2007 A DISSERTATION SUBMITTED IN PARTIAL FULFILLMENT OF THE REQUIREMENTS FOR THE DEGREE OF DOCTOR OF PHILOSOPHY in THE FACULTY OF GRADUATE AND POSTDOCTORAL STUDIES (Mechanical Engineering) THE UNIVERSITY OF BRITISH COLUMBIA (Vancouver)    April 2018 © Yu Du, 2018 ii Abstract  At present, robots are applied to specific situations and needs, so special methods are adopted for determining and controlling their movements. This thesis investigates the attention and navigation control of a mobile robot for carrying out dynamically challenging tasks involving humans. Since the robot environment can be complex, dynamic, and unpredictable, the required capacities are defined as those where the robot is required to pay attention to the needed object, plan the best path to the goal location and avoid obstacles. This thesis makes significant contributions in the development of an integrated model of top-down and bottom-up visual attention with self-awareness, and a biology-inspired method of robot planning and obstacle avoidance for a mobile robot. Specifically, an integrated model of top-down and bottom-up visual attention with self-awareness for robot is developed for selecting the highest saliency area in robot view. For mimicking the human attention processes, a robot self-awareness model with a fuzzy decision making system is developed and utilized, which is an important improvement over the existing robot attention models.  Inspired by a mammal’s spatial awareness and navigation capabilities, a path planning method based on biological recognition is proposed for navigation tasks of a mobile robot in an unstructured and dynamic environment. An episodic cognitive map that encapsulates the information of scene perception, state neurons and pose perception is built to realize the environment cognition of the robot. An algorithm of the event sequence planning is presented for real-time navigation using the minimum distance between events. The method can choose the optimal planning path based on the tasks.  A visual navigation algorithm that has the scale invariant feature transform (SIFT) feature is presented. By using the SIFT feature information, the horizontal coordinate of the matched feature pairs is considered to achieve the purpose of visual navigation. An approach of obstacle avoidance for visual navigation is presented based on a risk function and feasible paths. It can choose an optimal path for obstacle avoidance and then return to path planning through the interaction with the surrounding environment. The developed methodologies are evaluated using both simulation and experimentation.   iii Lay Summary  This thesis addresses the attention and navigation control of a mobile robot for carrying out dynamically challenging tasks. The key contributions are summarized as follows: 1. A biology-based cognitive map building and navigation method based on episodic memory for behavior control of a mobile robot platform was developed. This method extended the application of biological cognition theory in the field of robotic navigation. 2. A novel robot visual attention model based on robot memory with self-awareness was presented to improve the efficiency and adaptability of the current robot visual attention models.  3. A method to realize the robot’s visual navigation and obstacle avoidance by using motion scene feature sampling, robot state and path database building and obstacle model construction was presented. It can choose an optimal path for obstacle avoidance and then return to the planned path through environmental interaction.   iv Preface  The entire work presented in this dissertation was conducted at the Industrial Automation Laboratory of the University of British Columbia (Vancouver campus) under the supervision and guidance of Prof. Clarence W. de Silva. I was responsible for such work of this research as literature survey, algorithm development, implementation and experimentation. Dr. de Silva proposed and supervised the overall research project, acquired funding and resources for the project, suggested the topic of the thesis, suggested concepts and methodologies in addressing problems in the topic, provided research facilities in his Industrial Automation Laboratory, and revised the thesis presentation. Parts of Chapter 2 have been published as: Yu Du, Clarence W. de Silva, Ming Cong, Dong Liu and Wenlong Qin. An Integrated Model of Visual Attention for Homecare Robot with Self-awareness. IEEE Conference on Robotics and Biomimetics. Zhuhai, China, 2015. Parts of Chapter 2 have been published as: Yu Du, Clarence W. de Silva, and Dong Liu. A Multi-agent Hybrid Cognitive Architecture with Self-awareness for Homecare Robot. The 9th International Conference on Computer Science & Education, Vancouver, Canada, 2014. Parts of Chapter 3, Chapter 4, Chapter 5 and Chapter 6 have been published as: Dong Liu, Ming Cong, and Yu Du. Episodic Memory-Based Robotic Planning Under Uncertainty. IEEE Transactions on Industrial Electronics, vol.64(2), pp. 1762-1772, 2017. Contribution: present a local behavioral planning method based on risk function and feasible paths using local features. The method can choose an optimal path for obstacle avoidance and then return to the planned path during visual navigation. Parts of Chapter 3 have been published as: Dong Liu, Ming Cong, Qiang Zou and Yu Du. A Biological-inspired Episodic Cognitive Map Building Framework for Mobile Robot Navigation. International Journal of Advanced Robotic Systems, vol.14(3), pp. 1-12, 2017. Contribution: propose a self-learning method by Hebbian rule. The method can autonomously and real-timely learn the environmental experience to construct episodic memory. v Parts of Chapter 3, Chapter 4, Chapter 5 and Chapter 6 have been published as: Dong Liu, Ming Cong, Yu Du, Qiang Zou and Yingxue Cui. Robotic Autonomous Behavior Selection Using Episodic Memory and Attention System. Industrial Robot—an international journal, vol.44(3), pp. 353-362, 2017. Contribution: propose a top-down attention method for visual servo control algorithm to realize target tracking and approaching. The method is used for effectively target selection under redundant environmental information.  Parts of Chapter 3 and Chapter 4 have been published as: Dong Liu, Ming Cong, Yu Du, Xiaodong Han. Robotic Cognitive Behavior Control based on Biology-inspired Episodic Memory. IEEE International Conference on Robotics and Automation, Seattle, USA, 2015. Contribution: propose a self-learning method by Hebbian rule. The method can autonomously and real-timely learn the environmental experience to construct episodic memory.   vi Table of Contents  Abstract ······························································································ ii	  Lay Summary ······················································································ iii	  Preface ······························································································· iv	  Table of Contents ·················································································· vi	  List of Tables ······················································································· ix	  List of Figures ······················································································· x	  List of Symbols ··················································································· xiii	  List of Abbreviations ············································································ xv	  Acknowledgements ·············································································· xvi	  Chapter 1: Introduction ··········································································· 1	  1.1 Motivation ·················································································· 1	  1.1.1 Homecare Robotics ······························································· 1	  1.1.2 Robot Cognitive Control ························································· 2	  1.2 Goals of the Research ····································································· 2	  1.3 Problem Definition ········································································ 3	  1.4 Related Work ·············································································· 5	  1.4.1 Robot Visual Attention ··························································· 5	  1.4.2 Robot Cognitive Map Building ················································· 8	  1.4.3 Robot Navigation ······························································· 12	  1.5 Contributions and Organization of the Thesis ······································· 16	  Chapter 2: An Integrated Model of Robot Visual Attention with Self-awareness ·· 19	  2.1 Overview ················································································· 19	  2.2 Visual Attention System ······························································· 20	  2.2.1 Robot Memory ·································································· 20	  2.2.2 Memory-based Visual Attention System with Self-awareness ··········· 21	  2.3 Self-awareness Model and Results ···················································· 27	  2.4 Experimental Results ··································································· 31	  Chapter 3: Robotic Cognitive Map Building ··············································· 35	  3.1 Overview ················································································· 35	  vii 3.2 System Software and Sensor ··························································· 36	  3.2.1 Introduction of ROS ···························································· 36	  3.2.2 Kinect Sensor ···································································· 37	  3.3 Robotic Episodic Memory Building ·················································· 39	  3.3.1 Biological Basis of Episodic Memory Modelling ·························· 39	  3.3.2. Building of Episodic Memory Model ······································· 44	  3.4 Building of Robotic Cognitive Map ·················································· 46	  3.4.1 Episodic Cognitive Map Building Process ·································· 46	  3.4.2 Overall Procedure ······························································· 49	  3.4.3 Pose Cell Network ······························································ 50	  3.4.4 Event Generation and Coordinate ············································ 52	  3.4.5 Loop Closure Detection ························································ 54	  3.5 Cognitive Map Building Experiment ················································· 55	  3.5.1 Experimental Results in Office Environment ······························· 55	  3.5.2 Experimental Results in a Dynamic Corridor Environment ··············· 59	  Chapter 4: Robotic Path Planning Based on Cognitive Map ···························· 65	  4.1 Overview ················································································· 65	  4.2 Mobile Robotic Path Planning Process ··············································· 66	  4.3 Path Planning Method Based on Cognitive Map ··································· 67	  4.3.1. State Neurons Location ························································ 67	  4.3.2. Event Location ·································································· 67	  4.3.3. Event Sequence Planning ····················································· 68	  4.3.4. Behavior Controlling ·························································· 69	  4.4 Experimental Results ··································································· 72	  Chapter 5: Visual Navigation ·································································· 78	  5.1 Overview ················································································· 78	  5.2 Image Feature Extraction and Matching Algorithm ································ 78	  5.2.1 Image Feature Introduction ···················································· 78	  5.2.2 Mathematical Definition of Image Matching ······························· 78	  5.2.3 Introduction of SIFT ···························································· 79	  5.2.4 Image Feature Extraction and Matching Based on SIFT ·················· 80	  viii 5.3 Robot Visual Navigation Method ····················································· 82	  5.3.1 Path Learning and Path Database building ·································· 83	  5.3.2 Robot State Positioning and Visual Navigation Algorithm ··············· 83	  5.3.3 Robot Visual Navigation Flow ················································ 86	  5.3.4 Navigation Error Handling ···················································· 88	  Chapter 6: Obstacle Avoidance in Visual Navigation ····································· 94	  6.1 Overview ················································································· 94	  6.2 Visual Navigation and Obstacle Avoidance System Scheme ····················· 94	  6.3 Laser-based Obstacle Detection ······················································· 95	  6.3.1 Laser Sensor ····································································· 95	  6.3.2 The Working Principle of Laser Sensor ····································· 96	  6.4 Obstacle Avoidance Algorithm in Visual Navigation ······························ 97	  6.4.1 Local Path Grid Dividing and Obstacle Model Establishing ·············· 97	  6.4.2 Risk Function Establishing and Feasible Path Analyzing. ················ 99	  6.4.3 Local Path Planning and Robot Motion Control in Obstacle Environment ·························································································· 100	  6.4.4 Return to Original Navigation Route of the Robot ························ 101	  6.5 Obstacle Avoidance Experiment in Visual Navigation ··························· 102	  6.5.1 Visual Navigation Flow under the Obstacle Condition ··················· 102	  6.5.2 Experimental Results ·························································· 102	  Chapter 7: Conclusions and Suggestions ··················································· 107	  7.1 Conclusions ············································································· 107	  7.2 Primary Contributions ································································· 108	  7.3 Limitations and Suggested Future Research ········································ 109	  References ························································································· 110	     ix List of Tables   Table 2. 1: Data Storage Location in Memory ················································· 21 Table 2. 2: Fuzzy Rules ··········································································· 28 Table 2. 3: Label Descriptions of the Input and Output Fuzzy Sets ························ 28 Table 2. 4: Parameters Used in the Experiment ··············································· 32 Table 3. 1: The Parameters Used in theExperiments ········································· 61 Table 4. 1: Robotic Event Sequence Planning Algorithm ···································· 68 Table 4. 2: Robotic Behavior Control Algorithm ············································· 71 Table 4. 3:Experimental Data ···································································· 73 Table 4. 4: Performance Analysis of Robotic Behaviour Control ··························· 74 Table 4. 5: Performance Analysis of Robotic Event Sequence Planning ·················· 76     x List of Figures  Figure 1. 1: Schematic representation of the developed system. ······························ 4 Figure 2. 1: Key sub-modules and work flow chart of robot memory. ····················· 20 Figure 2. 2: Proposed memory-based robot attention architecture. ························· 23 Figure 2. 3: Architecture of NVT and multi-scale feature maps. ···························· 24 Figure 2. 4: Self-awareness model for homecare robot. ······································ 27 Figure 2. 5: Input and output membership functions (The memberships in the two lower figures have Gaussian shape). ···································································· 30 Figure 2. 6: The results of the two fuzzy inference systems. ································ 31 Figure 2. 7: Experimental results. (In each part, the first line from left to right represents original image, depth image and bottom-up saliency map; the second line represents top-down task-related saliency map, self-aware spacial bias saliency map and self-aware hue bias saliency map.) ················································································ 34 Figure 3. 1: A comparison of computer operating systems and ROS. ······················ 36 Figure 3. 2: The robot control based on ROS adopted in this thesis. ······················· 37 Figure 3. 3: Kinect camera system. ······························································ 38 Figure 3. 4: A color image and the corresponding depth image provided by Kinect. ···· 39 Figure 3. 5: The process by which hippocampus generates episodic memory. ··········· 41 Figure 3. 6: Episodic cognitive map building process. ······································· 46 Figure 3. 7: Organization process of state neurons for episodic memory. ················· 50 Figure 3. 8: CAN model. ········································································· 51 Figure 3. 9: The connection of events. ·························································· 53 Figure 3. 10: Schematic diagram for loop closure detection. ································ 54 Figure 3. 11: Physical view of TurtleBot 2. ···················································· 55 Figure 3. 12: Snapshot of the environmental scene. ·········································· 56 Figure 3. 13: Cognitive map building results. ················································· 57 Figure 3. 14: State neurons versus events. ····················································· 58 Figure 3. 15: Transition weights between neuron states. ····································· 59 Figure 3. 16: (a) Experimental trajectory; (b) Environmental scene. ······················· 60 Figure 3. 17: Husky A200 mobile robot. ······················································· 61 xi Figure 3. 18: Episodic cognitive map building result. ········································ 62 Figure 3. 19: State versus event. ································································· 63 Figure 3. 20: Weights of state neuron transition. ·············································· 64 Figure 4. 1: Robotic event sequence planning and behavior control. ······················· 66 Figure 4. 2: Pose transform between the context events. ····································· 70 Figure 4. 3: Results of robotic event sequence planning (office environment). ··········· 74 Figure 4. 4: Analysis of robotic behavior control. ············································· 75 Figure 4. 5: Results of robotic event sequence planning (corridor environment). ········ 75 Figure 4. 6: Analysis of robotic behavior control. ············································· 77 Figure 5. 1: Image matching algorithm flow based on SIFT feature points. ··············· 81 Figure 5. 2: Reference image (left) and match image (right). ······························· 81 Figure 5. 3: Reference image (left) and match image (right). ······························· 81 Figure 5. 4: Image SIFT feature points matching sketch. ···································· 82 Figure 5. 5: The six types of local descriptor matching. ······································ 84 Figure 5. 6: The relationship between speed and angular velocity. ························· 86 Figure 5. 7: Flow of path learning and path database building. ····························· 86 Figure 5. 8: The navigation flow of robot after path learning. ······························· 87 Figure 5. 9: The Robot and its sensors. ························································· 88 Figure 5. 10: The sampling path in navigation experiment. ································· 89 Figure 5. 11: Part of images extracted in sampling. ··········································· 90 Figure 5. 12: The feature points and feature vectors of some of the images. ·············· 90 Figure 5. 13: The scene image and the matched image. ······································ 91 Figure 5. 14: Liner velocity and angular velocity in navigation. ···························· 91 Figure 5. 15: The sampling path and the navigation path. ··································· 92 Figure 6. 1: Visual navigation and avoidance schemes based on the ROS system. ······ 95 Figure 6. 2: Hokuyo URG-04LX-UG01 laser sensor. ········································ 96 Figure 6. 3: Local occupied grids under robot coordinate. ··································· 98 Figure 6. 4:The curve of the risk function. ···················································· 100 Figure 6. 5: The flow of visual navigation and obstacle avoidance. ······················· 103 Figure 6. 6: The experimental environment for obstacle avoidance in navigation. ······ 103 Figure 6. 7: v  and ω  in normal navigation without obstacle. ························· 104 xii Figure 6. 8: v  and ω  in obstacle avoidance experiment. ······························· 105 Figure 6. 9: The navigation path with and without obstacles. ······························ 106    xiii List of Symbols  b Blue  γ          Control gain Γ   Set of presynaptic neurons ( , )c di iD F F    Euclidean distance maxsfD      Maximum distance robot can reach according to the self-aware state e     A temporal sequence of events ciF      Input feature point descriptors diF        Feature point descriptors of the matching image g Green G(x, y, )σ    Gaussian kernel function θ          Threshold value Rθ         Navigation angle sfH     Most preferred hue value of robot according to self-aware state and the environment ( )I σ  Gaussian pyramid ( , )I x y     Original image 1( , )I x y    Reference image 2( , )I x y    Matched image ( )I σ  Gaussian pyramid ( )p     Pose cρ   Optimal curvature 1cρ    Barrier-free curvature sρ    Curvature of the navigation path xiv r Red iR  Risk function sR    Risk probability σ         Square error parameter ( )s     State neurons pv        Mobile robot’s linear velocity θv        Mobile robot’s angular velocity iqw     Transition weight   xv List of Abbreviations  AGV Autonomous Ground Vehicles  ART Adaptive Resonance Theory AUV Autonomous Underwater Vehicles CAN Competitive Attractor Network DOG Difference of Gaussian GS Guided Search HIS Hue, Saturation, Intensity ISAC Intelligent Soft-Arm Control NVT Neuromorphic Vision Toolkit OSRF Open Source Robotics Foundation POMDP Partially Observable Markov Decision Process RBPF Rao-Blackwellised Particle Filter ROS Robot Operating System SIFT Scale Invariant Feature Transform SLAM Simultaneous Localization and Mapping SUN Saliency Model Using Natural statistics  VOCUS Visual Object Detection with a Computational Attention System WMS Working Memory   xvi Acknowledgements  First, I wish to sincerely thank my supervisor Prof. Clarence W. de Silva for his support, advice, and guidance during my studies in UBC. His rigorous supervision and guidance have enabled me to effectively and successfully complete my Ph.D. study.  He is my spiritual mentor and I am grateful to him for all his help. I also would like to thank my Ph.D. committee for their helpful academic advice and suggestions in general. In the same time, I would like to express my appreciation to my sponsors. This work has been supported by Natural Sciences and Engineering Research Council (NSERC) of Canada, Canada Research Chairs Program, Canada Foundation for Innovation (CFI) and British Columbia Knowledge Development Fund (BCKDF). Particular acknowledgement should be made to all of my colleagues Dr. Tahir Khan, Dr. Haoxiang Lang, Dr. Dong Liu, Dr. Yunfei Zhang, Dr. Muhammad Tufail Khan, Mr. Shan Xiao, Dr. Lili Meng, Mr. Hani Balkhair, Ms. Pegah Maghsoud, Dr. Min Xia, Dr. Shujun Gao, Mr. Zhuo Chen, Mr. Teng Li, and Mr. Tongxin Shu at the Industrial Automation Laboratory (IAL) for their friendship and help. I also want to thank my friends Dr. Ming Cong and Ms. Wanling Li. Finally, I would like to express my deepest thankfulness to my father Daguang Du and my mother Zhaolan He who have given me life. 1 Chapter 1: Introduction   The capability of autonomous operation is important in many applications of mobile robots including homecare robotics. A robot must possess a high degree of machine intelligence in order for it to operate autonomously. The research presented in this thesis aims to develop biology-inspired attention and navigation methodology for mobile robots. It will facilitate a mobile robot to adapt to an unstructured and dynamic environment and improve its machine intelligence and flexibility for effectively performing the required tasks. The application focus of the present research is homecare robotics for the elderly and the disabled. It has received increasing attention in view of the associated benefits in improvement of the quality of life and economic considerations. A home environment can be complex, fuzzy, dynamic, and unpredictable. Since homecare robots take the role of human homecare workers, these robots must possess a high level of intelligence, which is particularly required for autonomous operation.   1.1 Motivation 1.1.1 Homecare Robotics The development of human symbiotic robots that can support the daily activities of humans has significant advantages in view of the shortage and high costs of skilled labor in aging societies (Guerrero et al., 2009). Homecare robots that take the place of human assistants may accomplish such tasks as housekeeping, food preparation, personal services, assistance in mobility, pet care, security, education, entertainment, and so on (Bugmann and Copleston, 2011). There are potential benefits of using homecare robots, particularly in the context of improving the quality of life of the elderly and the disabled, and others including the workforce, with added economic and social benefits. A homecare robot will allow its user to be more flexible and comfortable while allowing others in the household to pursue a relatively normal life. In a survey of 2000, 71% of the respondents answered “yes” to the question “could you imagine living daily with robots which will relieve you from certain tasks that are too difficult for you?” (Arras and Cerqui, 2005).   2 1.1.2 Robot Cognitive Control  A major challenge in modern robotics is to expand the robotic applications beyond the controlled industrial settings, and allow robots to interact with humans and dynamic environment in the real world (Khamassi et al., 2011). Current robot-automated control systems function well in environments with designed situations or predictable uncertainties. However, future robots will require autonomous control systems, which will incorporate perception, reasoning, decision making, fact generalization, and learning, to handle novel and unanticipated situations. Such cognitive control will advance “automation” to the next level (Buss et al., 2011). A cognitive system may have the capacity to receive and process enormous amounts of sensory information from the environment (attention to the external environment), focus on its current states (internal state neurons), learn, reason, and plan efficiently through understanding of the external environment. Changes in the external environment may result in changes of what decisions will be made by them (Kawamura et al., 2008).  1.2 Goals of the Research  People routinely execute many habitual motions, such as paying attention to an object, and then walking from a cupboard to a dinner table. Through repetition, they refine their actions for efficiency and smoothness, while retaining the ability to handle moving obstructions like other people that pass through, and nearby obstacles and targets (Srinivasa et al., 2012). This type of task replays a stored trajectory and continually adapts to the task and environmental uncertainty. Robot usually needs to perform several behaviors in a sequence to accomplish its task (Lee et al., 2012). It must therefore be able to select a suitable behavior to deal with the given task. The behavior-based approach (Arkin, 2008) suggests that a robot’s behavior should be generated by combining behavioral modules, each of which has access to appropriate sensory information and guides a motor function. Since the low-level sensory-motor embodiment approaches do not scale to more robust and complex behaviors, hybrid architectures were developed that combined the strengths of behavior-based and deliberative approaches (Proetzsch et al., 2010). The behavior-based approaches can be combined with the sense-plan-act approaches to deal with the coordination of behaviors at a level that is abstracted from the sensory details 3 (Richter et al., 2012). A human can better adapt to a complex environment, recall previous experience to complete a task, and produce new knowledge and skills. Such critical behaviors are often made possible through experience. Similarly, for robots, the premise is that experience does matter (Endo, 2008). It is thus natural to seek inspiration from human experience in order for a robot to autonomously learn and select a behavior for adaptivity. The present thesis argues that the learned experience can be integrated with the behavior-based approach to enable a robot to generate optimal behavioral sequences.  The main objectives of the present research are as follows: l   By mimicking a human, develop a multi-modal attention mechanism with self-awareness for robots to manage and select enormous amounts of pertinent information. Thereby improve the attention efficiency of a robot. l   Develop for a robot an episodic model and a cognitive map building method inspired by the hippocampus biological basis to represent and store the learned experiences. l   Develop a robotic behavior planning method based on a cognitive map that has the ability to autonomously guide a mobile robot to a goal location in an unstructured and dynamic environment.  l   Develop a local obstacle avoidance approach for visual navigation of a mobile robot, to achieve static obstacle avoidance and return to a planned path.  1.3 Problem Definition  The primary objective of the present research is to develop methodologies and technologies for an autonomous mobile system to have abilities of paying attention to objects, navigating in the workspace, achieving obstacle avoidance and returning to the planned path or the destination. A schematic representation of the system developed in this manner in the present work is shown in Figure 1.1. In this system, there is a laser sensor in the front of the robot, a Kinect visual sensor in the middle, and a laptop computer on the robot platform. The two sensors and the robot body are linked by the ROS supported computer. Specifically, in order to realize the separation and modular design of robot attention capability, and achieve the goal of path planning, navigation and high-accurate obstacle avoidance in navigation, this thesis uses 4 ROS to connect the robot and the sensor, which eliminates the need for the bottom development. Because of the advantages of ROS remote calls, the robot, Kinect sensors, laser sensors and the host computer for data processing are effectively separated and combined to achieve clear modular processing. The robot task is assumed to be as follows: the robot receives a task (e.g., find a specific object and transport it to a goal location) while it is navigating in the workspace. It will attempt to find the object of interest in the work environment with the help of the attention capability. Once the robot finds the object, it transports the object to its goal location based on its cognitive map. In this procedure, the robot can realize obstacle avoidance.   Biology basisEpisodic memory biology basisEpisodic memory model Cognitive mapTask objectMulti-modal attention mechanism with self-awarenessAttention modelMobile robot path planningObstacle avoidance approach for visual navigationResearch contentRobot navigationRobot platform Figure 1. 1: Schematic representation of the developed system.   The Kinect sensor can obtain the RGB-D image of a scene for use in attention and navigation procedures. Through the analysis of the acquired image, image features can be 5 extracted, and then a cognitive map and a database of robot states can be built, so as to achieve the purpose of robot planning and vision-based navigation. Laser sensor used in this thesis has a scanning angle range of -5 degree to 180 degrees. It can shoot a custom number of lasers, and then measure the distance between the obstacle and the robot for every feasible path. Through the establishment of reasonable model, it is possible to realize the function of obstacle avoidance in the vision-based navigation process of the robot.  There are three challenges in the development of the proposed system, which are all key issues in realizing a working mobile robot system for the targeted application scenarios.  The first challenge is object attention, which concerns how to build an attention system for the robot to improve the perceptual efficiency. For providing the robot the capability to manage itself effectively and adapt to changing home circumstances, self-awareness is extended to the robotic application. The second challenge is biology-based mapping and planning, which concerns how to model an episodic memory, based on which build an episodic cognitive map, and then develop the autonomous path planning and behavior control algorithms in detail. The third challenge concerns the process of static obstacle avoidance. During obstacle avoidance, the robot may deviate from the original path, again in the visual navigation. Then the robot has to perform path re-planning.   1.4 Related Work  1.4.1 Robot Visual Attention The approach of “attention selection” which determines where the robot’s attention should be directed, for immediate processing and action, can provide a fast and efficient solution to the associated problem of information overload. It will enable the robot to selectively choose a relevant piece of information for further processing while excluding the rest of the information. A humanoid robot has a complex sensing system consisting of various types of sensors, which should be able to function in a dynamic and unstructured environment rather than performing specified fixed tasks in a simple and structured environment (such as factory floor). Since the robot would be overwhelmed with sensory information, intensive computation is required for processing the perceptual information. Therefore, how to facilitate perceptual efficiency becomes an important issues in cognitive 6 robot control (Jiang and Xiao, 2007). The perceptual information may come from visual sensors, auditory sensors, tactile sensors, olfactory sensors, and taste sensors. Since visual perception plays an important role in object recognition, learning and decision making, visual attention has attracted much interest over the past several decades. Robotic environment can be dynamic, cluttered and unpredictable. Intensive computation is required particularly for processing perceptual information. A human has the ability to select interesting targets from available extensive information in a short time (Han et al., 2009). Therefore, the attention system that promotes the perceptual efficiency of a robot has become an important issue in robotics. The current visual attention models can be classified based on bottom-up processing, top-down processing, or a combination (Begum and Karray, 2011). Bottom-up attention (Borji et al., 2015) selects and gates visual information based on the saliency in the image itself by the image-based low-level cues, while top-down attention (Liu et al., 2013) is based on the prior knowledge about the scene influenced by task demands. In the daily life of a biological system, these two mechanisms work together to control which visual target would be attended upon (Borji and Itti, 2013) Among the computational models of visual attention, the models proposed in computer vision have gained widespread popularity in robotics. The early computer vision models of attention mostly dealt with bottom-up influence in attention selection. The most influential computer vision model of visual attention is the neuromorphic vision toolkit (NVT) of Itti (Itti et al., 1998, Itti and Koch, 2000), which has been extensively used in models of visual attention in computer vision and robotics. Achanta et al. (Achanta et al., 2009) presented a bottom-up attention method for salient region detection, which outputs full resolution saliency maps with well-defined boundaries of salient objects. The bottom-up cues guide the visual exploration and focus on the most salient stimuli. The effect of top-down influence is considered in recent models in accordance with the new biological findings. Wolfe (Wolfe,1994) proposed a guided search (GS) model, which hypothesized that the top-down request for a given feature would activate locations that might contain the given feature value. A later modification of NVT reported by Navalpakkam and Itti (Navalpakkam and Itti, 2006) invoked the effect of top-down selection during visual search. They built a multi-scale object representation in LTM and evaluated top–down biases based on the multi-scale feature relevance, which is determined 7 by the object representation. Frintrop (Frintrop, 2006) presented a visual attention system for object detection and goal-directed search, which is a biologically motivated computational attention system VOCUS (visual object detection with a computational attention system) that detected regions of interest in images. The top-down cues gained an active scan of the visual field in search of a pre-specified object or stimuli. However, the influence of top-down information is only limited to the case of visual search. Several computational models that combine top-down attention and bottom-up attention for robotic applications have also been proposed. Kim et al. (Kim et al., 2005) have proposed a dynamic perceptual attention model for virtual humans by combining top-down and bottom-up attention processes to select the salient object in complex virtual environments. Begum et al. (Begum et al., 2010) proposed a probabilistic model integrating top-down and bottom-up effects as a robot-centric solution for an overt visual and auditory attention problem. Yu et al. (Yu et al., 2010) presented an object-based visual attention model by extending the integrated competition hypothesis. That model selects one object of interest using low-dimensional features by combining top-down bias and bottom-up competition. Lee and Mumford (Lee and Mumford, 2010) proposed a biologically motivated computational model for object-based visual attention control in interactive environments. Bayesian models were used to combine prior knowledge and sensory information according to the Bayesian rule (Borji and Itti, 2013). Lee and Mumford (Lee and Mumford, 2010) proposed a hierarchical Bayesian inference in the visual cortex. Itti and Baldi (Itti and Baldi, 2009) proposed a formal Bayesian definition of surprise to capture subjective aspects of sensory information. Zhang et al. (Zhang et al., 2008) proposed a saliency model using natural statistics (SUN) which is a measure of saliency based on natural image statistics. Belkaid et al. (Belkaid et al., 2017) proposed a top-down modulation of sensorimotor processes based on artificial neural networks for controlling a robotic system in a visual search task. Top-down attention is learned interactively and is used to search for a desired object through biasing the bottom-up attention. Zhao et al. (2017) proposed a method of deep reinforcement learning with visual attention for vehicle classification. In recent years, the research in robot visual and auditory attention has mainly focused on overt shift of attention including change of the reference frame, dynamic IOR (inhibition 8 of return) and partial appearance, integrated space- and object-based analysis, optimal learning strategy, generality, and prior training. Some of the related issues have been solved or partially solved. For example, Jonas presented a multimodal bottom-up attention system for the humanoid robot iCub where the robot’s decisions to move its eyes and neck were based on visual and acoustic saliency maps. This work introduced a modular and distributed software architecture, which was capable of fusing visual and acoustic saliency maps into one egocentric frame of reference (Ruesch et al., 2008). Aryananda presented an integrated framework, combining an object-based perceptual system, an adaptive multimodal attention system and spatiotemporal perceptual learning, to allow a robot to interact while collecting relevant data seamlessly in an unsupervised way (Aryananda, 2006). Momotaz proposed a probabilistic Bayesian model integrating top-down and bottom-up effects as a robot-centric solution for an overt visual and auditory attention problem (Begum et al., 2009). Colombini, et al. proposed an attentional model inspired by biological systems that supports a variety of robotic sensors (Colombini et al., 2017). Although research is active in the area of robotic visual attention, there is little work on developing robots that can focus their attention not only according to its external environment but their internal state. A new attention-based approach has been proposed for self-awareness and a comparative analysis of approaches that highlights the innovation has been conducted. In that work, self-awareness is generated when attention is focused on internal states, while in the present research self-awareness is used to affect the robot attention and decision making process.  1.4.2 Robot Cognitive Map Building  Building an integrated mobile robot that can navigate freely and deliberately in an indoor environment is a challenging task (Shim et al., 2014). The robot needs to be integrated with several functional mechanisms, such as scene understanding, mapping, localization and path planning. Environmental map building and experience learning without prior knowledge are the basis of navigation and other robotic tasks. As the robotic technologies keep advancing and start interweaving into our life, it is inevitable that soon some robots will be required to perform various tasks in an unfamiliar environment. When a robot first enters an unfamiliar and complex environment, what they have is limited 9 knowledge about the places visited. They have to use this limited knowledge to find their way to a destination. Over the past decades, there has been considerable interest in solving this problem by building an internal map of environment, and navigating based on estimated localization using that map. This method is known as Simultaneous Localization and Mapping (SLAM), involving the use of grid, landmark or topological representations (Thrun and Bucken, 1999, Nieto et al., 2003, Endo and Arkin, 2003).  Spatial cognition is the basic ability of mammals to perform navigation tasks. Researchers have been investigating how animals perceive space and navigate in an environment. Cognitive map (Chatty et al., 2012, Lowry et al., 2016), which is a map-like representation that represents the spatial relationship among salient landmarks of an environment is developed to solve navigation problems. Inspired by the navigation ability of humans or animals, recent studies (Milford and Wyeth, 2010, Tian et al., 2013, Chatty et al., 2014) that are related to how mammals perform mapping, localization, and navigation, have gained significant interest of the robotics community. The cognitive map was first proposed by Tolman (Tolman, 1948) when he conducted a three-pathway maze experiment, as shown in figure 1.5. According to the results of the experiments about rodent navigation, a cognitive map is defined as a type of intelligent description of the external environment. The cognition of animals about an environment will lead to abstract expression of the external environment. Instead of collecting the path simply from the starting point to the destination point, it provides a description of the relative geometric relation with spatial environment characteristics. Liben (Liben et al., 1981) defined a cognitive map as a process composed of a series of psychological transformations by which an individual acquires, codes, stores, recalls, and decodes information about the relative locations and attributes of phenomena in their everyday spatial environment. Elvins (Elvins, 1997) defined a cognitive map as an internal description of the spatial environment based on observation, understanding and learning. A cognitive map can be applied for navigation to different locations of the environment. Some types of cognitive map building use computer-based methods. Endo et al. (Endo and Arkin, 2003) presented a method for a mobile robot to construct and localize relative to a cognitive map where the cognitive map is assumed to be a representational structure that encodes both spatial and behavioral information. The localization is performed by 10 applying a generic Bayes filter. The cognitive map was implemented within a behavior-based robotic system, providing a new behavior that allowed the robot to anticipate future events using the cognitive map. One of the prominent advantages of this approach is the elimination of the pose sensor usage, which is known for its limitations and proneness to various errors. Mallot et al. (Mallot et al., 2010) presented a view-based approach to map learning and navigation in mazes. A neural network for unsupervised learning of the view-graph from sequences of views is constructed based on graph theory, which use a modified Kohonen learning rule that transforms a temporal sequence into connectedness. Carols (Carlos, 2005) presented a hybrid representation model for robot navigation in indoor environments, which uses local reference systems as basic elements. Integration of quantitative and qualitative information is applied, obtaining a hybrid description of the world. A topological map is extracted as a natural extension from the hybrid description, which builds a cognitive map of the environment. Some types of cognitive map building utilize biology-based methods. Walker et al. (Walker et al., 1993) presented a mobile robotic control system that is modeled on the spatial memory and navigator behaviors attributed to foraging honey bees in an effort to exploit the robustness and efficiency these insects. A robot uses a self-organizing feature-mapping neural network to construct a topographically ordered map while exploring the environment. This map is then annotated with metric positional information from a dead reckoning system. The resulting cognitive map can be used by the robot to localize in the world and to plan safe and efficient routes through the environment. Wenjie et al. (Wenjie et al., 2012) presented a neural network architecture for a robot to learn a new navigation behavior by observing a human’s movement in a room. While indoor robot navigation is challenging due to the high complexity of real environments and the possible dynamic changes in a room, a human can explore a room easily without any collisions. They therefore propose a neural network that builds up a memory for spatial representations and path planning using a person’s movements as observed from a ceiling-mounted camera. Based on the human’s motion, the robot learns a map that is used for path planning and motor-action coding. Bo et al. (Bo et al., 2013) presented a cognitive map building and navigation system using an RGB-D sensor for mobile robots. A brain-inspired simultaneous localization and mapping (SLAM) system, that requires raw odometric data 11 and RGB-D information, is used to construct a spatial cognitive map of an office environment. The cognitive map contains a set of spatial coordinates that the robot has traveled. This is different from other path planning mechanisms that construct a path based on a ground-truth map, which is a collection of nodes that constitute the cognitive map. Rebai et al. (Rebai et al., 2012) presented a bio-inspired visual memory for robot cognitive map building and scene recognition by capturing some properties of primate brain and the view cells. The system enables cognition of the environment by the mobile robot using visual information extraction and the visual memory building through a Fuzzy Art architecture.  When a robot is brought into a complex unfamiliar environment to perform non-specific tasks, exploring learning through interacting with the environment is a preferred option since the robot has limited knowledge. Most existing learning methods are based on known environmental models (Wang, 2010). The used learning process is offline training, and the strategy is improved through reward, and ignores much useful information in other fields. In fact, often the environmental model is unknown in advance, and the only way to get experience through direct learning by interacting with the environment. Humans, on the other hand, accomplish tasks in such environments with ease. A human can better adapt to complex environments and tasks, accumulate knowledge, recall previous experience to complete tasks, and produce new knowledge and skills. Such critical behaviors are often made by episodic experiences, relying on their learning. Similarly, for robots, the premise here is that episodic experience does matter as well (Endo, 2008). It is thus natural to seek inspiration from human experience in order to achieve adaptivity and cognition. The present work utilizes episodic memory for which biological inspiration comes from the mammalian hippocampus (Tulving et al., 2016) to store the learned experiences. It constitutes the collection of past experiences that occurred at a particular time and place. Most research on memory has been focused on the memory structure rather than the concept of memory. The memory models were also not clear (Menegatti et al., 2004, Jockel et al., 2009, Baxter and Browne, 2011, Rebai et al., 2012, Petit et al., 2016). With regard to episodic memory modeling, working memory (WMS) and self-Agent were proposed to model an episode integrated with the internal state information (Dodd and Gutierrez, 2005). EPIROME was proposed to improve action planning based on past experiences by using 12 image appearance and behavior. The episodic memory can provide one-shot learning capabilities to a robot (Jockel et al., 2007). The episodic memories retained a sequence of experienced observation, behavior, and reward in (Endo, 2008). Simulation of navigation tasks was processed through the observation of distance information. The event modeling in this work is partly inspired by the sequence. (Stachowicz and Kruijff, 2012) adopted index data structure storing episodic memory to provide knowledge for the robotic cognitive model, and carried out lengthy simulation experiments. Kelley (Kelley, 2014) implemented a memory store to allow a robot to retain knowledge from previous experiences based on images to construct an event. An episodic memory system consisting of a cascade of two Adaptive Resonance Theory networks was presented for a robot that delivers objects (Leconte et al., 2014). Most episodic memory models are focused on episode-like memory instead of cognitive episodic memory, and they are typically unable to provide good results in the case that robot needs to adapt to non-specific tasks in a complex and uncertain environment. Episodic memory provides a mechanism for recalling past experience, but it was virtually ignored in robotic cognitive models. Recent research mostly focuses on the perspective of engineering, while paying less attention to its biological basis. A contribution of this work is the building of a robotic episode model inspired by the biological basis of hippocampus to represent the learned experience and environmental time-space characteristics. The robot has cognitive ability. The advantage of the proposed robotic episodes mapping method is that the robot can autonomously and incrementally learn the environmental experience to construct episodic memory, i.e., a cognitive map. Different from the situation sequences, which are generated by a specific grammar or rule in the learning process, the structure of event sequences in episodic memory has no inherent law, and it is organized only by the experience of the robot. The sequences are flexible and their complexity varies in accordance with the variation of the scene appearance. The proposed method can realize the accumulation, integration and updating of experience.   1.4.3 Robot Navigation  In robot navigation of a complex environment with obstacles, the robot has to use its sensors to carry out accurate autonomous movement to the target while avoiding obstacles 13 around it. As one of the key technologies in robotic research, robot navigation plays an indispensable role in robotic tasks and motion. There are many relevant research directions in this field. At present, the main methods of robot navigation include following: inertial navigation (Zhong and Huang, 2011), magnetic navigation (Kim et al. 2017), satellite navigation (such as the use of global positioning systems or GPS), and visual navigation. However, because of the increasing demand for robots in various applications involving humans, robots are required to reach human intelligence (Brooks, 1997) in complex and uncertain environments. Complex and uncertain environments, imperfect robot control, sensor noise, and unexpected environmental changes, need an efficient and reliable robotic system with cognition (Choi et al., 2011, Gu et al., 2015). Thus, this investigation concerns the visual navigation and navigation using a cognitive map. Visual navigation has become an important direction in the field of intelligent robot navigation because of the availability of rich information and multi-function environmental cognition. With the rapid development of computer hardware technologies and the emergence of various algorithms, the image processing speed has improved greatly, and the cost of visual sensors has significantly decreased. Consequently, visual systems are commonly used in robot navigation and environment perception. Therefore, using machine vision for robot navigation has become an important direction of research. In some existing applications the robot is equipped with an intelligent vision system, such as in autonomous ground vehicles (AGV) (Lategahn et al., 2011), unmanned aerial vehicle (UAV) (Cai et al., 2010), and autonomous underwater vehicles (AUV) (Brown et al., 2009).  In visual navigation of mobile robots, many effective algorithms have been proposed for visual following, planning, navigation, and road recognition. Gaspar et al. (Gaspar et al., 2000) presented a method that used a single omni-directional (catadioptric) camera for the vision-based navigation of a mobile robot in indoor environments. They proposed two main navigation modalities: Tological Navigation and Visual Path Following. Kidono et al. (Kidono et al., 2002) presented a navigation strategy that required minimum user assistance. In their method, the user first guides a mobile robot to a destination by remote control. During this movement, the robot observes the surrounding environment to create a map. Once the map is generated, the robot computes and follows the shortest path to the destination autonomously. Sim et al. (Sim et al., 2006) presented and analyzed an 14 implementation of Rao-Blackwellised particle filter (RBPF) that uses stereo vision to localize a camera and 3D landmarks as the camera moves through an unknown environment. Zhu et al. (Zhu et al., 2011) presented an autonomous navigation method based on monocular vision for the application of a mobile robot in semi-structured environment. The method includes image segmentation, information extraction and behavior decision-making. A computer simulation has verified the effectiveness of the method. Cherubini and Chaumette (Cherubini and Chaumette, 2013) proposed and validated a framework for visual navigation with collision avoidance for a wheeled mobile robot. Visual navigation consists of following a path, represented as an ordered set of key images, which have been acquired by an on-board camera. Siagian et al. (Siagian et al., 2013) presented a mobile robot navigation system guided by a novel vision-based road recognition approach. They integrated odometry and visual road recognition system into a grid-based local map that estimated the robot pose as well as its surroundings to generate a movement path. Alessandro et al. (Alessandro et al., 2015) presented a vision-based solution for decision making and a behavior-based low-level control for navigation. Charles et al. (Charles et al., 2017) demonstrated a method with a vision-guided robot that could leverage its deep neural network to navigate 50% faster than a safe baseline policy in familiar types of environments, while reverting to the prior behavior in novel environments so that it could safely collect additional training data for continuous improvement. Saurabh et al. (Saurabh et al., 2017) presented a formulation for visual navigation that unified map-based spatial reasoning and path planning, with landmark-based robust plan execution in noisy environments. This allowed the robot to efficiently navigate in novel environments given only a sparse set of registered images as input for building the representations of the navigated space. The point based algorithm, which is based on the idea of sampling, has made great progress with regard to computational efficiency and can be executed in a reasonable amount of time. Algorithms of this type have been successfully applied to a variety of robotic tasks, such as autonomous driving (Wei et al., 2012, Xu et al.,  2012), Robot motion planning (Du et al., 2010, Kurniawati et al., 2011) and human robot interaction (Gopalan and Tellex, 2015). This work realizes the function of robot visual navigation based on local image feature extraction and matching technology. A SIFT feature detection 15 algorithm is used to extract useful landmarks in the environment. These landmarks are stable in images with different frames, and can be re-detected when the robot returns to the current position. In addition, a laser can detect an obstacle with an appropriate model, effectively. In this manner, a robot is able to achieve the goal of avoiding the obstacle on the navigation path. Some researchers study mobile robot navigation based on a cognitive map. Endo and Arkin (Endo and Arkin, 2003) proposed a model of a cognitive map that allowed a robot to anticipate future path. Their simulation experiment showed that the cognitive map seemed to contribute to a robot’s ability for navigation; however, the method has not been tested on a real robot. Tian et al. (Tian et al., 2014) used the RatSLAM (Milford and Wyeth, 2008), a brain-inspired SLAM algorithm, for building a cognitive map and proposed a navigation system using an RGB-D sensor for mobile robots. Shim et al. (Shim et al., 2014) imitated a human-like behavior when travelling to a target location by following the directional guidance from someone else, and proposed a direction-driven navigation system, in which a mobile robot was instructed to follow the given directions instead of a global path. The experimental results showed that the robot could efficiently navigate to a location in a passageway except for a specific location of a place, but it could not detect a junction. Cheng et al. proposed a topological map and loop closing method to build the cognitive map and used the derived map and the Markov localization method for navigation in an indoor environment (Cheng et al., 2015). Humans can adapt to a complex environment by recalling previous experiences, and robots may use a similar approach. Naturally, in order for the robot to predict the next trajectory by reasoning the validity of the current location based on the experience on the relevant episodes in the past, which were stored in memory. Episodic memory is a form of memory that contains information associated with a particular episode of experience, and it is stored in a way such that the episode can be traced back and recalled at a later time (Tulving, 1972). Endo (Endo, 2007) presented a biologically inspired episodic memory-based approach for computing anticipatory robot behavior. He extended the episodic memroy into a framework to solve partially observable Markov decision process (POMDP) problems efficienty, i.e., to find the best action for the current situation. Kleinmann and Mertsching proposed a cognitive architecture that was substantiated by biology inspired 16 multiple memory systems, including episodic memory and other units, and demonstrated applications of it using a simulated mobile robot (Kleinmann and Mertsching, 2010). Stachowicz and Kruijff utilized index data simulation storing episodic memory to provide knowledge for the robotic cognitive model and carried out the long-time simulation experiments (Stachowicz and Kruijff, 2012). Kelley implemented a memory store to allow a robot to retain knowledge from previous experiences based on images to construct events (Kelley, 2014). Leconte et al. presented an episodic memory system consisting of a cascade of two Adaptive Resonance Theory (ART) networks, based on artificial emotion learning and episode recalling, where a robot delivered objects to people in an office area; however, different complex tasks need to be tested extensively with a higher number (Leconte et al., 2014). Liu et al. presented a robotic global planning method based on episodic memory for efficient prediction of a behavior sequence using bottom-up attention (Liu et al., 2017). Neuroscientist have found that the hippocampus plays a major role in the storage of episodic memory (Burgess et al., 2002, Fyhn et al., 2004). Considering the close relation between cognitive map and hippocampus, the present work investigates mobile robotic path planning based on episodic memory and cognitive map.  1.5 Contributions and Organization of the Thesis This Ph.D. thesis investigates and develops new techniques for effective navigation of mobile robots in unstructured and dynamic environments using attention and biology-inspired methodologies. The four main contributions of the thesis are as follows: l   An integrated model of top-down and bottom-up visual attention with self-awareness for a homecare robot is proposed for picking up the most saliency area in a robot’s view. It can improve the efficiency and adaptability of the existing robot visual attention models. l   A robotic episodic model inspired by the hippocampus biological basis is developed to represent the learned environmental experiences. The advantage of the proposed robotic episodic memory model is that by using it a robot will be able to autonomously and incrementally learn the environmental experience to construct episodic memory, i.e., cognitive map. It is able to realize the accumulation, integration and updating of experience. 17 l   An autonomous path planning and behavior controlling algorithms based on the cognitive map is proposed. Based on the episodic cognitive map, using the minimum distance between events, an algorithm of the event sequence planning is put presented for real-time navigation. Using the control algorithm, the mobile robot is able to choose the best planning path for different tasks.  l   A cognition inspired visual navigation algorithm is developed with the scale invariant feature transform (SIFT) feature. A local behavioral planning approach based on risk function and feasible paths is proposed for a robot to avoid obstacles and return to the planned path. The organization of this thesis is as follows: Chapter 1: This chapter introduces existing activities in homecare robotics and cognitive control, and highlights main research challenges of this field. Then it outlines the research objectives of the thesis and presents a literature survey to establish the related background work, mainly in the past decade.  Chapter 2: This chapter describes the proposed robot memory model and visual attention model with learning method, and then presents the proposed self-awareness model with partial results, particularly the experimental results of the attention model.  Chapter 3: This chapter introduces the biological basis of episodic memory, and simulates the organization of the episodic memory by introducing a neuron stimulation mechanism for episodic memory modelling. Then a cognitive map is built based on episodic memory for mobile robots. Several experiments are presented to validate the method. Chapter 4: This chapter chooses the next situation based on neurons transition weights and minimum event distance based on the episodic cognitive map. Then an algorithm of the event sequence planning is presented for real-time navigation. Several experiments are presented to validate the method. Chapter 5: This chapter introduces robot visual navigation including the image processing algorithm, navigation algorithm and the connection and navigation strategy between the robot system and the control system. Experimental results in the indoor scene verify the feasibility of the robot navigation method. 18 Chapter 6: This chapter introduces the obstacle avoidance scheme in robot navigation, describes the establishment of an obstacle model, the calculation of hazard function, and the study of obstacle avoidance process in robot navigation. Experimental results of robot obstacle avoidance in an indoor is analyzed, which verifies the feasibility of the present research outcomes for static obstacle avoidance in mobile robot navigation. Chapter 7 This chapter summarizes the primary contributions of the thesis, and indicates several relevant issues and possible future research directions in robotic attention and navigation. As shown in Figure 1.1, the main chapters of the thesis are interrelated as follows. During robot learning, episodic memory is self-organized using event sequences for building the environmental model based on the self-learning method (Chapter 3). Then SIFT features of scenes are extracted for building the path database (Chapter 5). During robot navigation, autonomous behavior is selected by considering the target, operating environment and possible threats. On the one hand, a visual attention algorithm with self-awareness is used for effectively selecting the target in the presence of redundant environmental information (Chapter 2). The method can be utilized to realize the tracking and approach control for a potential target. On the other hand, the autonomous behavior planning method based on episodic memory is used for robot control in an unstructured environment. By recalling the episodic memory, the robot is able to predict a behavior sequence (Chapter 4), and realize obstacle avoidance and return to the planned route while the memory is imperfect (Chapter 6).   19 Chapter 2: An Integrated Model of Robot Visual Attention with Self-awareness  2.1 Overview Robot environment can be complex, fuzzy, dynamic and unpredictable. In order to function in a dynamic and unstructured environment rather than performing specified fixed tasks in a simple and structured environment, a robot should have a sensitive perception system consisting of various kinds of sensors. Such a complex sensing system will provide ample sensory information. Then, intensive computation is required for processing the perceptual information. Humans have the ability to select interesting targets from available extensive information in a short time, but robots cannot match that ability (Han et al., 2009). Therefore, how to build an attention system for a robot to achieve perceptual efficiency is an important issue (Jiang and Xiao, 2007).  Self-awareness, which is an aspect of consciousness, is crucial in deciding and reaching a tradeoff in what you are, what you know, what you want, and what you are able to do (Morbini and Schubert, 2006). When robots take the role of human workers, they must possess a high level of intelligence. For the purpose of providing a robot the ability to manage itself effectively and adapt to changing home circumstances, self-awareness must be extended to robotic applications (Lewis et al., 2011).  In order to improve the efficiency and adaptability of the existing robot visual attention models, this section presents a novel robot visual attention model based on robot memory with self-awareness. With the ability of introspection of its internal state and the ability to reconcile oneself as an individual that will separate itself from the environment and other individuals, a robot visual attention system with self-awareness may receive and process enormous amounts of sensory information efficiently through the understanding of both external environment and internal states. Changes in any aspect may result in changes of what robots will pay attention to and what decisions will be made by them (Kawamura et al., 2008). In this chapter, bottom-up and top-down attention modes are integrated for selecting the most salient area in a robot view. A robot self-awareness model is established using sensory data and other inherent variables to generate a part of top-down bias weights 20 through a fuzzy logic method. In addition, Bayes’ rule is utilized to update the weights. The experimental results from the attention model are presented to validate the method.  2.2 Visual Attention System  2.2.1 Robot Memory A memory framework for homecare robots should be able to receive, store and renew information from visual, tactile and audio sensors, and recall past experience. What is indicated now is only the memory content related to vision. Figure 2.1 shows the corresponding key sub-modules and work flow chart of robot memory. Table 2.1 presents the list of storage locations of the data related to visual attention. RGB images and depth images are sent to the sensory memory. After the attention process, the attended data is held in the working memory. Top down task-related feature biases, and feature and spatial biases calculated by the self-awareness model are retrieved from the long-term memory and transferred to the working memory. Some of the data in working memory is then consolidated and stored in the long-term memory.     Figure 2. 1: Key sub-modules and work flow chart of robot memory.       Sensory MemoryWorking MemoryLong-term MemoryDeclarative (explicit) MemoryProcedural (implicit) MemoryProcedural MemoryEpisodic MemorySemantic Memoryattention consolidationRGB-D CameraAbandon InformationTaskForgotten Informationretrieval21 Table 2. 1: Data Storage Location in Memory. Location Data Sensory Memory RGB images, Depth images. Working Memory Top-down feature biases Self-aware feature and spatial biases Attentional selection Episodic Memory Self-awareness model. Semantic Memory Bottom-up cue influential weight Top-down cue influential weight Self-aware cue influential weight Target object’s feature  2.2.2 Memory-based Visual Attention System with Self-awareness This architecture combines the bottom-up and top-down attention mechanisms, and considers the influence of robot self-awareness at the same time. Four attention processes are applied to generate the saliency map. The overall saliency map is the weighted sum of the resulting four saliency maps. Self-awareness model is utilized to generate the self-awareness feature and the spatial biases to calculate two saliency maps separately. Bayes’ rule is used as the learning method to update the weights. As shown in Figure 2.2, an RGB-D camera generates an RGB image and a depth image. The bottom-up attention process follows the NVT model (Itti et al., 1998) to calculate the bottom-up saliency map, as shown in Fig.2.2. From the RGB input image, one can obtain an intensity map I with r (red), g (green) and b (blue) channels as( ) / 3I r g b= + + . Then I  is used to create a Gaussian pyramid ( )I σ , where [ ]0 8σ ∈ , is the scale. Four broadly-tuned color channels are created according to: ( ) / 2R r g b= − +  for red, ( ) / 2G g r b= − +  for green, ( ) / 2B b r g= − +  for blue, and ( ) / 2 / 2Y r g r g b= + − − −  for yellow. Four Gaussian pyramids ( )R σ , ( )G σ , ( )B σ  and ( )Y σ  are created from these four channels. After carrying out center-surround difference and normalization, 6 intensity maps are created as   ( , ) ( ) ( )I c s I c I s= Θ     (2.1) 22 where scale { }2,3,4c∈ , scale s c δ= + , with { }3,4δ ∈ , and Θ donates the across-scale difference between two maps. Twelve color maps are created as   ( , ) ( ( ) ( )) ( ( ) ( ))RG c s R c G c G s R s= − Θ −   (2.2)   ( , ) ( ( ) ( )) ( ( ) ( ))BY c s B c Y c Y s B s= − Θ −  (2.3) The local orientation information is obtained from I using oriented Gabor pyramids. Twenty four orientation maps are created as     ( , , ) ( , ) ( , )O c s O c O sθ θ θ= Θ  (2.4) where ( , )O σ θ  represents Gabor pyramids, and { }0 , 45 , 90 ,135θ ∈ ° ° ° °      . Orientation feature maps are encoded as a group. Local orientation contrasts between the centers and surrounding scales. In total, 42 feature maps are computed: six for intensity, 12 for color, and 24 for orientation (Itti et al., 1998).  Feature maps are combined into three “conspicuity maps.” They are obtained through across-scale addition, which involves reduction of each map to scale four and point-by-point addition. Through across-scale combination and normalization, 3 conspicuity maps are obtained as  4 22 3( ( , ))cc s cI N I c s== = += ⊕ ⊕  (2.5)   [ ]4 42 3( ( , )) ( ( , ))cc s cC N RG c s N BY c s+= = += ⊕ ⊕ +                  (2.6)   { }4 4c 2 30 ,45 ,90 ,135( ( ( , , )))cs cO N N O c sθθ+= = +∈ ° ° ° °= ⊕ ⊕∑        (2.7) The motivation for the creation of three separate channels and their individual normalization is the hypothesis that similar features compete strongly for saliency, while different modalities contribute independently to the saliency map (Itti et al., 1998). The bottom-up saliency map is the linear combinations of the 3 saliency maps as given by    1S ( ( ) ( ) ( ))3b uN I N C N O− = + +               (2.8) 23   RGB Input ImageSensory MemoryEpisodic MemorySelf-awareness ModelCenter-surround DifferenceFeature MapsEuclidean DistanceSemantic MemoryWorking MemorySemantic MemoryTask-related Feature BiasesCombination and nomalizationAttention Selection  LearningLinear CombinationsWeights UpdateConspicuity MapsAcross-scale CombinationBottom-up Saliency MapTop-down Saliency MapHue biasDifference CalculationSelf-aware Hue Saliency MapSpacial biasDepth Input ImageSelf-aware Spacial Saliency MapDifference CalculationWeight for Individual Saliency MapRGB-HSIHSI ValueFinal Saliency MapRGB-HSIHSI Value Figure 2. 2: Proposed memory-based robot attention architecture.  24    Figure 2. 3: Architecture of NVT and multi-scale feature maps.  25 In the task-related attention process, only the color features (Hue, Saturation, Intensity) (HSI) of the target object are considered. When the robot receives the command, it recalls the related features from the semantic memory, which can determine the feature biases of top-down attention at that moment. At first, the input RGB image is converted into an HSI image. Then the color difference of each pixel in the converted image and the target object is calculated as      2 2 2ij jD ( ) ( ) (i )eu i o ij o ij oh h s s i= − + − + −                   (2.9) where ijh , ijs and iji are the HSI values of each pixel in the converted image, oh , os and oiare the HSI values of the target object, and i and j indicate a pixel’s two-dimensional position in image. Top-down saliency map can be obtained using the Gaussian distribution as   2t-d( )1S ( exp( ))22euijDNλπ= −                     (2.10) Although it is difficult to build a robot’s self-awareness to completely mimic the human capability, researchers attempt to take advantage of its function for application in robots. Kawamura et al. (Kawamura et al., 2005) developed a self-agent consisting of an intention agent, a pronoun agent, a description agent, a central executive agent and an emotion agent in robot’s working memory to provide machine consciousness in intelligent soft-arm control (ISAC). Michel et al. (Michel et al., 2004) gave Robot Nico the ability to recognize its own motion in a mirror by learning through experimentation. Birlo and Tapus (Birlo and Tapus, 2011) presented a concept of self-awareness using a self-module and a self-buffer in a humanoid robot based on the existing cognitive architecture ACT-R. In 2012, Gorbenko et al. (Gorbenko et al., 2012) developed a new model using genetic algorithms to analyze a robot control system, investigate internal states and generate new internal states for the robot. The function of self-awareness model here is to generate the hue bias and the special bias to guide visual attention. The self-aware spatial saliency map can be calculated as, If maxij sfD D≤ , 26   2maxsfs( (1 ))1S ( (exp(1) exp( )))22ijsfDDNγπ−= − −            (2.11) else s 0fsS =  where ijD  is the depth value of each pixel in the input image, and maxsfD is the maximum distance the robot can reach according to the self-aware state. To simplify the fuzzy set during the calculation of hue bias due to the self-awareness model, first the hue value is converted from 325° -360°  to 36− ° - 1− °  in the fuzzy logic system, according to      sf 361H H= − °           (2.12) where sfH  is the robot’s most preferred hue value according to the self-awareness state and the environment, and { }325 ,360H ∈ ° ° . If { }0 ,324H∈ ° ° , sfH H= . If f 0sH ≥ ° ,fs sH H= , else sf 361sH H= + ° ; and if 180ij sH H− ≤ °, the self-aware hue saliency map can be calculated as  2( )1S ( exp( ))22ij ssfhH HNηπ−= −   (2.13) If 180ij sH H− > ° , then   2( (360 ))1S ( exp( ))22ij ssfhH HNηπ− −= −                 (2.14) The final saliency map is the linear combination of the four saliency maps according to    S final b u b u t d t d sfs sfs sfh sfhw S w S w S w S− − − −= + + +           (2.15) where b uw − , t dw − , spsw and sfhw are the weights for the four saliency maps. After the selection of each correct attention, Bayes’ rule is employed to update the four weights, which are stored in the semantic memory, as     1( ) ( )( )( ) ( )m mk m mm mnP A B P BP B AP A B P B==∑                    (2.16)     27 where ( )k mP B A  represents the b uw − , t dw − , spsw and sfhw in the kth  attention selection, { }1,4m∈ , ( )mP B  and ( )mP A B are the occurrence probability and the attention probability, respectively, of each saliency map. The initial values of ( )mP B  and ( )mP A Bare determined by investigating human behavior.   2.3 Self-awareness Model and Results This thesis uses the self-awareness model for homecare robot as shown in Figure 2.4. The fuzzy logic approach is used to mimic the decision making behavior of humans to generate maxsfD and fsH .  Observed VariablesSelf-awareness ModelAction Stage Activity Emergency LevelEmotionPhysical StateBattery PowerEnd Effector PositionJoint PositionInternal StateDeliberative ProcessProgram ProcessWorking TimeInherent VariablesPersonality Role in Work EnvironmentInterestFeature HobbyObject HobbyCapabilityKnowledge LevelRestrictions Figure 2. 4: Self-awareness model for homecare robot.  For the purpose of simplification, here only the robot battery power level and the robot working time are chosen as the input variables of the fuzzy decision making system to determine the maxsfD , and the robot social role, together with the air temperature are chosen as the input variables to determine sfH . Table 2.2 presents the fuzzy rules of the two fuzzy systems. Table 2.3 presents the label descriptions of the input and output fuzzy sets of the two fuzzy systems. Figure 2.5 shows the input and output membership functions. 28    Table 2. 2: Fuzzy Rules.       WT    BP WTS WTN WTL BPH EL VL LG BPM SL MM SS BPL SH VS ES         SR   T EC CO IN WC P BG R WW PB G YR WH B GY Y  Table 2. 3: Label Descriptions of the Input and Output Fuzzy Sets. Name Description Name Description WT Working time EL Extremely Long WTS Working time short VL Very Long WTN Working time normal LG Long WTL Working time long SL Somewhat Long BP Battery Power MM Medium BPH Battery Power High SS Somewhat short BPM Battery Power Medium SH Short BPL Battery Power Low VS Very Short   ES Extremely Short T Temperature P Purple WC Weather cold PB Purple-Blue WW Weather warm B Blue 29 Name Description Name Description WH Weather hot BG Blue-Green SR Social role G Green EC Elderly care GY Green-Yellow CO Cook Y Yellow IN Infant nurse YR Yellow-Red   R Red     30    Figure 2. 5: Input and output membership functions. (The memberships in the two lower figures have Gaussian shape)  The results of the two fuzzy inference systems (Mamdani) are shown in Figure 2.6.  31   Figure 2. 6: The results of the two fuzzy inference systems.  2.4 Experimental Results To evaluate the presented method, the proposed attention model is applied to 3 RGB-D images, which are obtained from a Kinect for XBOX 360 sensor system. For each image, bottom-up attention process, top-down task related attention process, self-aware spatial bias attention process, and self-aware hue bias attention process are applied separately. Table 2.4 presents the parameters used in the experiment.   32 Table 2. 4: Parameters Used in the Experiment. Image Number Top-Down Task-related HSI Dsfmax Hsf 1 H0=86 S0=0.6 I0=0.16 (green orange) 5 50 2 H0=225 S0=0.1 I0=0.83 (toilet paper) 1 216 3 H0=50 S0=0.7 I0=0.59 (yellow book) 3 0  For the first image, the green orange was chosen as the top-down task-related object. Suppose that the battery power remains at 50% and the robot has been working for 5 hours. From the self-awareness model, one can obtain max = 5msfD . Suppose that the temperature of home environment is 30℃ and the social role of the robot is that of an infant nurse. Also from the self-awareness model, one can obtain =50sfH . For the second image, the toilet paper was chosen as the top-down task-related object. Suppose that the battery power remains at 20% and the robot has been working for 7 hours. From the self-awareness model, one can obtain max = 1msfD . Suppose that the temperature of home environment is 40℃ and the social role of the robot is that of elderly care. Also from the self-awareness model, one can obtain = 216sfH . For the last image, the yellow book was chosen as the top-down task-related object. Suppose that the battery power remains at 50% and the robot has been working for 7 hours. From the self-awareness model, one can obtain max = 3msfD . Suppose that the temperature 33 of the home environment is 0℃ and the social role of the robot is that of an infant nurse. Also from the self-awareness model, one can obtain = 0sfH . It is clear that different processes lead to different saliency maps. Through the adjustment of weights, different final saliency map can be obtained. The robot attention selection is affected by both the environment and the robot itself. The results of the top-down task-related attention process and the robot self-aware hue bias attention process are good, but the result of robot self-aware depth bias attention process is not very good due to the poor quality of the input depth image from Kinect for XBOX 360.     34  Figure 2. 7: Experimental results. (In each part, the first line from left to right represents original image, depth image and bottom-up saliency map; the second line represents top-down task-related saliency map, self-aware spacial bias saliency map and self-aware hue bias saliency map.)   35 Chapter 3: Robotic Cognitive Map Building  3.1 Overview Humans can adapt to complex environments quite effectively, and they can remember what they have experienced and fulfill future tasks by recalling the past experience. Tulving and Endel proposed the concept of episodic memory, defined it as a form of memory that contains a series of information associated with a particular episode of experience, and stored in a way that the episode can be traced back and recalled at a later time (Tulving and Endel, 1972).  Some biology-related research has found that episodic memory exists in mammal hippocampus. O’Keefe and Dostrovsky adopted an electrophysiological method in some experiments (O'Keefe and Dostrovsky, 1971). The experimental results showed that there were place cells in the rodent hippocampus, and these cells could produce specific reactions to spatial position, and their action mechanism could encode the position where they were located in the environment. Eichenbaum and Dudchenko pointed out that for the hippocampus, the representation of the spatial environment information is the “memory space,” which consisted of connected episode memories rather than the absolute geometrical mapping to the actual space (Eichenbaum and Dudchenko, 1999, Milford et al., 2008). Some other research activities also have illustrated that the cognitive map is resided in hippocampus. RatSLAM is an entirely new SLAM algorithms based on neutral networks, which was first proposed by Milford et al. (Milford et al., 2010, Milford et al., 2004). It is an implementation of a hippocampus model that can perform SLAM in real time on a real robot (Milford et al., 2007). By simulating the navigation process in the rodent hippocampus, the place cells and grid cells build maps by path integration, and the RatSLAM system uses a competitive attractor network to integrate pose perception with scene perception to form a consistent representation of the environment (Nuxoll et al. 2004). Next in the present thesis, a robotic episodic model inspired by the hippocampus biological basis is developed to represent the learned experiences and the environmental time-space characteristic. In this model, the robot has cognitive ability. The advantage of the proposed robotic episodic memory model is that a robot can autonomously and 36 incrementally learn the environmental experience to construct the episodic memory, i.e., a cognitive map. Based on the RatSLAM algorithm, the proposed method can realize the accumulation, integration and updating of robotic experience.  3.2 System Software and Sensor 3.2.1 Introduction of ROS  Robot Operating System (ROS) was originally a robot project at Stanford University, which was later developed by Willow Garage Company, and is currently an open source project maintained by OSRF (Open Source Robotics Foundation, Inc.). ROS, like Windows, Linux, and IOS, is an operating system. Figure 3.1 shows the similarity between the ROS system and a traditional operating system. In the figure, the computer operating system encapsulates computer hardware and application software run on the operating system. Similarly, in ROS robot hardware, different robots, and different sensors can be expressed in the same way, and they can be called by the upper applications. ROS uses a node to express an application. Connections between different nodes are implemented by predefined format messages (Topic), services (Service), and actions (Action).   Figure 3. 1: A comparison of computer operating systems and ROS.  37 With this modular communication mechanism based on Node, developers can easily replace and update some modules in the system, and also can replace the individual modules of ROS with their own nodes, which is quite suitable for algorithm development. In addition, ROS can be used across platforms, on different computers, different operating systems, different programming languages, and different robots.   Figure 3. 2: The robot control based on ROS adopted in this thesis.  As shown in Figure 3.2, this thesis adopts robots based on ROS. The data of robot body, Kinect sensor and laser sensor can be acquisitioned by Topic, at the same time, and because of the function of the cross platform call mechanism provided by ROS, the small computer connected to the robot and the remote computer can carry out data exchange easily.  3.2.2 Kinect Sensor  Kinect for Xbox360 (Kinect) is shown in Figure 3.3. It uses real-time dynamic capture, image recognition, microphone input, voice recognition and other functions. Kinect has three lenses, the middle of which is the RGB color camera, which is used to collect color images. The left and right elements are the 3D structured light depth sensors constituting infrared transmitters and infrared CMOS cameras. They are used to collect depth data (the distance between objects in the scene and the camera). Color camera supports imaging with maximum resolution 1280*960, and the infrared camera supports 640*480 imaging. The Kinect also comes with the focus recovery technique, where the base motor moves as the 38 focus moves. Kinect has an array microphone, which uses four microphones to listen to the radio signal simultaneously. After comparison, the noise is eliminated, and the voice recognition and harmonic source localization are carried out using the collected sound.   Figure 3. 3: Kinect camera system.  There are several ways to develop a Kinect application at present. An unofficial open source development environment such as CL NUI Plaform, OpenKinect/libfreenect, and OpenNI may be used. Microsoft supports the official Kinect for Windows SDK on Windows. In the present work, because the computer that is connected to Kinect runs on Linux, and the robot uses the ROS system, Microsoft official SDK cannot be used. So, libfreenect is used as the driver software for Kinect development. Kinect can acquire color images through an intermediate RGB color camera. For depth maps, Kinect projects some random lattices through the IR, and then uses only a common CMOS sensor to capture the dot matrix. In short, when the depth of the scene changes, the dot matrix that the Kinect camera sees will change, and through this change Kinect can infer the depth information. Different from the traditional ToF or the optical structure measurement technology, Kinect uses a technique called optical encoding (Light Coding) technology, using continuous lighting (rather than pulse). Since it does not need a special sensitive chip, and uses only the common CMOS sensitive chip, it greatly reduces the cost. Figure 3.4 shows a color image and a depth image generated by Kinect.  39  Figure 3. 4: A color image and the corresponding depth image provided by Kinect.  3.3 Robotic Episodic Memory Building In this section, an episodic memory model is built for environmental cognitive learning, which can generate a cognitive map for mobile robotic navigation. 3.3.1 Biological Basis of Episodic Memory Modelling In the relevant biological studies, it has been found that hippocampus, in the brain of vertebrates, has the ability to achieve episodic memory. Episodic memory, as clarified by the psychologist Tulving (Tulving et al., 1972) reflects the episodes and scenarios that cognitive subjects had experienced in the past. In Figure 3.5 (Buzsaki and Moser, 2013), the top image shows how a woman recalls concerned scenes. The bottom image shows the process by which the hippocampus achieves episodic memory. The working mechanism of 40 episodic memory of hippocampus and the related research are the foundation of the present work.   41  Figure 3. 5: The process by which hippocampus generates episodic memory.  1) Cognitive Maps in the Brain Exist in the Hippocampus. O'Keefe and Dostrovsky (O'Keefe and Dostrovsky, 1971) conducted experiments using electrophysiological techniques and found that there exist place cells in the body of hippocampus. Place cells of hippocampus can specifically respond to space position information and then encode its location in the environment using the motility mechanism. It can be concluded that Cognitive maps exist within the hippocampus of a brain. Eichenbaum et al. (Eichenbaum et al., 1999, Shapiro et al., 1999) pointed out that the description of spatial environment information of hippocampus, instead of an absolute geometric representation of the real space, is a "memory space" consisting of interconnected episodic memories. In addition, Fortin et al. (Fortin et al., 2002) found that the hippocampus plays a unique role in the process of generating the episodic memory of a series of events, using experiments on hippocampal rodents. Due to ethical and safety factors, the research on human navigation mechanisms is much less extensive than that on rodents. However, it is found that hippocampus has a strong connection with episodic memory and is used in navigation. An experiment on taxi drivers in London (Maguire et al., 1997) has shown that drivers have larger hippocampus than others because of the requirement of remembering various places in the city and the routes. Besides, the experienced drivers have a bigger hippocampus. Researchers have used PET technology to track the process of driving through complicated routes in the city using memory (without 42 relying on maps), and found that the hippocampus plays a vital role in searching for a shortcut or a new route in a familiar environment. 2) Environmental Road Signs and Their Own Movement Information is an Important Spatial Clue of Navigation. Animals and humans form a cognitive map in the hippocampus by processing and integrating road signs, self-movement information and other spatial clues. Signposts generally refer to objects that are stable, visible, and are able to inform the location in the environment. Wang et al. (Wang et al., 2014) found that there are many types of signpost knowledge that humans form in different navigational experiences, such as the knowledge about guiding, locating and signposting sequences. Research has shown that both animals and humans can use signposts to achieve space navigation. As Smith et al. (Smith et al., 2008) pointed out, 3 to 7-year-old children have been able to use natural signposts to select the correct path. Youngstrom and Strowbridge (Youngstrom and Strowbridge, 2012) found that even in the virtual space, mice can learn with visual clues and can move to a specific location. Self-motion information, commonly referred to as ontology information, is mainly derived from sensory resources such as vision, vestibular sensation, and ontological sensation (Etienne and Jeffery, 2004). Path integration is key to information processing integration, and the above study has found that humans and animals can use road signs for navigation, and path integration can also be used to achieve spatial navigation. For example, Mittelstaedt and Mittelstaedt (Mittelstaedt and Mittelstaedt, 1980) found in experiments that female rodents were able to search in the dark and return their nests due to the strong motivation for retrieving their pups. Andel and Wehner (Andel and Wehner, 2004) found that desert ants can also return straight to the nest by path integration after exploring a certain distance, and a neural model of path integration has been proposed based on this activity. Recent research in neuroanatomy has shown that the activity of neural cells in cognitive learning for spatial environment in the brain is also affected by path integration. As pointed out by Navratilova (Navratilova, 2012), the main determinant factor of cell discharge in the hippocampus at a certain location is path integration, and the discharge of cells at their location is greatly affected when the individual moves into a new environment after their own motility information has been removed.  43 3) Hippocampus cells achieve learning by nerve cell discharging Wirth et al. (Wirth et al., 2003) studied the neural activity of monkey brain using experiments and analyzed the relationship between the neural information and the behavior of a monkey. The experimental results, to a certain extent, explain the operating mechanism of the hippocampus region and show that the hippocampus is deeply linked with the process of how long-term memory is formed by event learning. In the experiment, it was found that the curves of a monkey’s learning process and the recorded neuron activity are simultaneous, confirming that the brain achieves cognitive learning by neuron activity. Besides, neuron activity will continue for some time after cognitive learning has been accomplished, which confirms that hippocampus takes part in the construction of memory. Kobayashi et al. (Kobayashi et al., 2003) found that when rats make a round trip between two reward locations, some hippocampal neurons will correct, gradually, their discharge mode, and the discharge value will attain the maximum when a rat arrives at the reward location. When a reward does not exist but the navigation is aimed, the discharge mode will continue. This experiment confirms that when the action is aimed, the hippocampus cells of a rat can encode well-defined behaviors and place cells can be placed at specific locations.  In addition, a task-oriented model is available. From a T-labyrinth experiment it is found that there are discharge differences among hippocampus cells when the rats arrive at a T-junction of the labyrinth and decide where to go. Tests and analyses have shown that it is only the turn that decides the discharge differences, not any other factors such as speed, course or position. These results show that a special episodic memory can be encoded by hippocampus and can be applied to a navigation mission。 The episodic memory and the concerned biological basis of hippocampus as described above are the foundation on which the contextual memory model of the present work is developed. By the contextual memory model, a mobile robot is able to carry out a cognitive study of the spatial environment and form a cognitive map, which can be used for navigation to the target location. A key challenge in building this model is how to constitute the episodic memory by using information and how to describe the current environmental information of a mobile robot. 44   (1) There are many interconnected neurons in the hippocampus. During the process of spatial exploration and navigation, various scenes stimulate neurons to undergo different discharge reactions. Based on this, a neuron excitation mechanism is proposed to make the robot realize a contextual memory model and achieve cognition learning of the environment.   (2) The concept of state neurons, rooted in combination style of neurons in the hippocampus, is simplified and presented during model building. By activating and linking state neuron, episodic memory can be encoded and the model of episodic memory can gain the ability of global planning.   (3) Episodic memory is expressed by interconnected sequence of events. The concerned background knowledge, as mentioned above, about hippocampus and spatial navigation, mainly indicates the two important aspects——the signpost and the self-motion information, which facilitate the construction of the events from the episodic memory model. During the process of model building, the information, including the perception of the robotic environment and the perception of the position and posture of a robot, is stored as the knowledge learned by the robots and can be applied to generate the cognitive map of the navigating environment.  3.3.2. Building of Episodic Memory Model  Neuroscience research (Lin et al., 2005) shows that in hippocampus, there are interconnected state neurons. Episode stimulation contributes to the changing of the fire patterns of neurons, and this change is related to the characteristics of the episodic events and the environment. By imitating this mechanism, the present work adopts the stimulation mechanism of state neurons (Redish et al., 1996) to build the episodic memory model, which will realize the cognitive learning from the environment. The activation and connection of the state neurons contribute to the event connection in the episodic memory model. Episodic memory in this work comprises a temporal sequence of events ( e ) and a goal ( g ) which the mobile robot is pursuing during the episode, as represented by  ( ){ }1 2, , , , ,mE e e e g=    (3.1) 45 wherem is the number of events in the episode. The goal is defined as the environmental observation at the end of an episode. Note that the episodes depend on goals, i.e., a new episode starts from the time the robot starts pursuing a new goal and ends when the robot reaches the goal. More specifically, the event consists of a set of observation ( )o , state neurons ( )s and pose ( )p , as expressed by  ( ), , ,e o s p=  (3.2) Observation is a feature vector of the environment as captured by a vision sensor and calculated using a computer vision algorithm. Its observability plays an important role in the phases of environmental cognitive learning and relocation. It is used for cognitive learning of the environmental knowledge, and has a crucial effect on relocation. State neurons are used for estimating the current robotic state. Assuming that the mechanism of a robot’s representation of environmental cognitive learning is similar to the mechanism of hippocampal neuron sequence in memory, incremental state neuron is activated successively with the change of the environmental information. Its role is to organize the episodic memory neural network. A many-to-one mapping projection:f o s→ is defined, which indicates that there is always a unique corresponding state neuron for any observation. Pose represents the current state location ( ),x y and the corresponding orientation ( )θ . Pose perception is calculated according to the path by integrating the odometer data. Pose perception allocates the specific location information for episodic memory. Hence, the constructed cognitive map has a geometric description of the environment. The present work uses the troupe ( ), ,o s p to build the episodic memory model for robotic cognitive learning. This is done by sampling the firing patterns of hippocampal CA1 neurons to simulate the organization process of the episodic memory, abstracting the state neurons to imitate the place cells to map the high-dimensional environmental perception for reducing the computational complexity of the high-dimensional belief spaces, and improving the planning efficiency and accuracy. The proposed model can realize the robotic cognitive learning, real-time storage, incremental accumulation, and 46 integration into an unknown environment. The robot can evaluate the past event sequence, predict the current state and plan the desired behavior.  3.4 Building of Robotic Cognitive Map  3.4.1 Episodic Cognitive Map Building Process  Based on the proposed episodic memory model, state neurons have been incorporated to imitate the place cells, and it has been assumed that the episodic cognitive map is represented by a discrete finite event space e and a set of event transitions. The process of map building is shown in Figure 3.6.   The process is as follows: First template the current spatial environment input, and compare the similarity degree between the current scene perception and the existing scene  Figure 3. 6: Episodic cognitive map building process.  47 perceptions to determine whether it is a new scene. If it is a new scene, link it with a new state neuron and build the transition weights between it and the existing state neurons; if it is a familiar scene, based on the pose perception through the competitive attractive network path by integrating odometry information, update the neuron network model, state neurons transition weights and the current pose perception. According to the similarity measurement, it is possible to modulate the cumulative error occurred while path integration and solve the problem of loop closure during the map building process. Then based on the state neuron activation and pose perception, a robotic episodic cognitive learning algorithm is established, where new events and familiar events are divided for learning. For a new event, record the event; otherwise, update the events and state neuron transition weight matrix W for obtaining the event transition weight. Each pair of state neurons ( ),i js s W∈ which has a feasible transition is associated with a positive transition weight ijw . Thus, an edge-weighted episodic trajectories graph with a vertex set and edge set is obtained, as given by  { },M E W=     (3.3) The state neuron transition weight set W can be represented in the matrix form as:  ( )( )( )( )111 12221 2 21 21 2,qii qiqii iq q qi qwws t www s t wWws tw ww w w s t⎡ ⎤⎢ ⎥⎢ ⎥⎢ ⎥⎢ ⎥=⎢ ⎥⎢ ⎥⎢ ⎥⎢ ⎥⎣ ⎦ (3.4) In (3.4), the rows and columns of the matrix represent the serial numbers of the neuron states. The activation of the state neuron is  in the current time can be represented by( ) 1is t = . Then the output activity of state neuron is represented by  ( )( ) ( )( )1,0i i i nii nw s t s ts ts tθθ⎧ − >⎪= ⎨≤⎪⎩ (3.5) 48 where decay weight 1 , 10iw eτ τ−= = simulates the decay of the state neurons, and the threshold nθ determines the depth of memory for the state neurons. The transition weight can be represented by iqw as given by  ( ) ( ),iq i qw s t s t=  (3.6) It denotes the connection strength of the neuron states for constructing an episode. It remembers the context information of the state neuron qs  relevant to the current activated neuron iS . Based on the matrix (3.4), the maximum transition weight for a row vector is computed by  * max , 1,2, , ,i ijw w j q= = (3.7) If the maximum transition weight in a row is greater than 1, the corresponding neurons are defined as similar state neurons. It means that there is a variety of driving choices from the current state neuron to the target state neuron. Considering the activation status of the context neurons, the robot will choose the similar neuron state with a larger number of active context neurons as the next active state neuron. We have,  ,m nτ λ=∑  (3.8)  1 _ ( ) ( ),0nactive state k n state j notherwiseλ− = −⎧= ⎨⎩   (3.9) where ( )1,m num∈ , num  represents the value of the maximum transition weight in a row; mτ represents the number of active neurons around the thm similar state neuron; kindicates that already 1k − neurons have been activated during the path planning, and now activate the thk neuron; and { }min 1,9n k= − denotes the number of context neurons, for the decay of state neurons, whose maximum value is 9.    49 3.4.2 Overall Procedure Suppose that there are n state neurons in an episodic memory network. Among these state neurons, some are occupied and others are unoccupied. Before episode learning, all the state neurons are not occupied. Every state neuron node will integrate the related observation, behavior mode, position and head direction into an event. The detailed process of this system is given below: Step 1: Initialize the episodic memory network; Step 2: Build the Competitive Attractor Network (CAN), identify the center of the network and inject energy to generate active cell. The center is used to represent the initial pose perception of the mobile robot; Step 3: Extract and compute the initial visual image to generate the current input scene perception, and then save it into the scene perception feature set, for the work of subsequent perception similarity measurement. Step 4: For the initial frame 1, associate the initial input scene perception with the robot pose perception obtained from step 2. Then activate the first state neurons and map input perception, recording m = 1 as an occupied state neuron. The coordinates of the initial state neuron in the geometric map are (0, 0, 0). Step 5: For the frame greater than 1, calculate the similarity between the current input scene perception and each perception from the perception feature set, to determine whether it is a new scene perception. Step 6: Integrate the path using linear velocity and angular velocity information recorded by the odometer, and then update the active field of the competitive attractor network so as to update the current robot pose perception.  Step 7: Based on result of step 5, if it is a new scene, a new event need to be added, to activate one new state neuron, and create the connection weights according to    1, 1 1( ) ( )mn ci m m iiw s t s t++ +∈Γ=      (3.10) Here Γ  is the set of presynaptic neurons, cis  represents the context of current maximum activation state neurons, 1nms +  is the new added state neurons, and the weights denote the connection strengths of the events for constructing an episode. The organization process of state neurons for episodic memory is shown in Figure 3.7 (LIU et al., 2014). The yellow 50 circle represents the current activated state neuron, and the green circles represent its context state neurons. When activating a new state neuron, the connection weight ijw  between it and other context state neurons will be generated.  …ijw)(tsniis1−is1+−kiskis −iwibsco Figure 3. 7: Organization process of state neurons for episodic memory.  Step 8: If it is a familiar scene, update its connected pose perception. According to the pose perception and the scene perception, judge whether the state neuron is activated. If it is familiar event, then update its connection weights; otherwise, implement step 7. In the learning process, the system computes the robot’s current behavior mode at the same time and integrates it into the corresponding state neurons to generate events in the episodic memory. The system constantly implements the above steps, and finally builds a spatial episodic memory consisting of comprehensive representation of the environment in topology and geometry.  3.4.3 Pose Cell Network The cognitive map is built based on the RatSLAM algorithm in the present work, which is a hippocampal model for simultaneous localization and mapping (SLAM). It uses a competitive attractor network to integrate the wheel encoder information with the scene information to generate a cognitive map consisting of a consistent representation of the environment. The network can realize dynamic update by the actions: local excite, local inhibit and global inhibit (Stringer et al., 2002). 51 The CAN model was proposed by McNaughton et al. (McNaughton et al., 2006) in 2006 and shown in Figure 3.8.   Figure 3. 8: CAN model.  The competitive attractor network is a three-dimensional network and it is composed of a pose cell which is used to represent the robot’s current position ( , )x y and orientationθ. Hence it is called a pose cell network in the present thesis. Based on the attractor dynamics and the cues from the external environment, the pose cell network will generally have a single cluster of highly active units forming the activity packet. These activities compete against each other to provide an estimate of the robot’s pose. The activity for one pose cell ranges from 0 to 1, expressing the belief of the robot with regard to its own pose. Movement of the robot will modulate the dynamics of the network, causing the activity packet to change and hence update the estimated pose. Scene information becomes associated with the activity packets as well. Once the associations between the scene information and the pose estimate are learnt, the scene information will influence the position of the activity packet to update the pose estimate of the robot. For one pose cell, local excitation and inhibition is achieved through a three-dimensional Gaussian distribution of weighted connections, whose distribution is given by  2 2 2 22 2( )/ ( )// /, ,exc inhexc inhp pd da b k a b kc k c ka b c e e e eε− + − +− −= −  (3.11) 52 where pk and dk are the variance constants for place and direction, and , ,a b c represent the distances between the units in , ,x y θʹ′ ʹ′ ʹ′coordinates, respectively. The change in pose cell’s activity is given by  11 1, , , , , ,0 0 0yx nn nx y i j k a b ci j kP Pθθ ε ϕʹ′ʹ′ ʹ′−− −ʹ′ ʹ′ ʹ′= = =Δ = −∑ ∑ ∑  (3.12) where , ,x yn n nθʹ′ ʹ′ ʹ′ is the size of the pose cell network in number of cells along each of the dimensions , ,x y θʹ′ ʹ′ ʹ′, and ϕ  creates global inhibition.  3.4.4 Event Generation and Coordinate This system judges the robot’s current pose and scene perception to determine whether an event is generated. An event’s activity is dependent on the activity of the current connected scene perception and pose cell; however, their activity depends on the continuous (pose cell network) or discrete (scene perception) distance to the energy peak. The continuous injected energy is given by  ''' '0 10 12rx y rr rif rE ifr otherwiseθ θθʹ′ ʹ′ ʹ′>⎧⎪= >⎨⎪ − −⎩ (3.13) with ' '' ' 2 ' ' 2' '( ) ( ), pc ipc i pc ir ra ax x y yrrθ θθθ−− + −= = . Here ' ' ',  ,  pc pc pcx y θ  are the coordinates of the pose cell with maximum activity; ' ' ',  ,  i i ix y θ  are the coordinates of the pose cells linked to event i ; and ,  a ar θ  are the distance constants for ( , )x yʹ′ ʹ′ and orientation, respectively. When input scene perception io  is the same as the current active scene perceptioncurro , energy will be injected. The energy of event i  is given by  0 curr iix y curr iif o oEE f o oθʹ′ ʹ′ ʹ′≠⎧= ⎨=⎩  (3.14) 53 After energy injection, if no existing event has a positive activation level, a new eventje  is created and linked to the current activated pose cell and scene perception, as given by  ( , , )j j j j ije o s p p= + Δ   (3.15)     where ijpΔ is the change in the robot’s pose since the last event has been created. As shown in Figure 3.9, the disparity in position between two events is computed by    Figure 3. 9: The connection of events.   j i ijij j i j i ijj i ijx x dxdp p p y y dydθ θ θ⎛ ⎞ ⎛ ⎞⎛ ⎞⎜ ⎟ ⎜ ⎟⎜ ⎟= − = − =⎜ ⎟ ⎜ ⎟⎜ ⎟⎜ ⎟⎜ ⎟ ⎜ ⎟⎝ ⎠⎝ ⎠ ⎝ ⎠  (3.16) When the robot re-travels an explored path in the environment, no new events will be created. But, even if the events are the same and no new connections are created, information may change inside the connection. Thus, the disparity is updated according to  0.5 0 0. 0 0 0 .0 0 0new old currij ij ijdp Adp dp⎛ ⎞⎜ ⎟= + ⎜ ⎟⎜ ⎟⎝ ⎠  (3.17) 54  0.5 0 00 cos sin2 20 sin cos2 2curr old curr oldij ij ij ijold oldij ijcurr old curr oldij ij ij ijold oldij ijd d d dA d dd dd d d dd dd dθ θθ θ⎛ ⎞⎜ ⎟⎜ ⎟⎜ ⎟+ +⎜ ⎟= −⎜ ⎟⎜ ⎟⎜ ⎟+ +⎜ ⎟⎜ ⎟⎝ ⎠  (3.18)  1 11 tan tan2curr oldij ijcurr oldij ijdy dyddx dxθ − −⎡ ⎤⎛ ⎞ ⎛ ⎞= −⎢ ⎥⎜ ⎟ ⎜ ⎟⎜ ⎟ ⎜ ⎟⎢ ⎥⎝ ⎠ ⎝ ⎠⎣ ⎦ (3.19)  3.4.5 Loop Closure Detection Loop closure detection is essential for the robustness of the robotic cognitive algorithm. As shown in Figure 3.10, when exploring, the robot would estimate its pose constantly; however, when it reaches the explored position, its estimated pose may differ from the actual pose. To eliminate this error, a loop closure detection scheme is proposed.  Figure 3. 10: Schematic diagram for loop closure detection.  The present thesis uses loop closure detection to modulate the event’s pose in a cognitive map. For every event, the change in pose iPΔ  is computed as  1 1[ ( ) ( )]f tN Ni j i ij k i kij kp a p p dp p p dp= =Δ = − − + − −∑ ∑   (3.20) Due to possible changes in the orientation, the transitional information is changed as 55  1 0 00 cos sin0 sin cosnew currij i i iji idp dpθ θθ θ⎡ ⎤⎢ ⎥= Δ Δ⎢ ⎥⎢ ⎥Δ Δ⎣ ⎦  (3.21) where iθΔ are the changes in orientation of the events.   3.5 Cognitive Map Building Experiment 3.5.1 Experimental Results in Office Environment TurtleBot2 is a mobile robot, which is equipped with a Yujin Kobuki mobile base and a Kinect. A physical view of this research robot-TurtleBot2 is shown in Figure 3.11. It has a Kobuki mobile base, which consists of two driving wheels and two supporting wheels. The driving wheels are equipped with encoders, which record the travelled distance. The maximum translational velocity is 0.65m/s, and the maximum rotational velocity is 180deg/s. One Kinect is mounted on the base of the robot. It is a visual sensor whose horizontal field of view is 57deg, and the vertical field of view is 43deg. On the top plate of the robot, a laptop is mounted, which links the Kobuki mobile base with Kinect. The laptop, which uses Linux system, can launch the mobile base and control it, as well as visualize the image captured by the Kinect in real-time.   Figure 3. 11: Physical view of TurtleBot 2.  56 In the present application, the main task of TurtleBot 2 is to build a map of an office environment. Specific features in the office space are memorized through the map building process. These features serve as cues for updating the robot’s pose as well as recalling the familiar events. One robotic office is used for the purpose of the experiment. It is a typical office, which consists of office desks, chairs, clutter and some cartons. Figure 3.12 presents some environmental snapshots captured by Kinect as the mobile robot wanders in the office. The image pixel size is 640 × 480. In the present robot platform, the frequency of the topic “/odom” is 50Hz, while the topic “/camera/rgb/image_color” is 30Hz. To have the same frequency for the two topics, the frequency of “/odom” is reduced to 30Hz during the experiment. The driving translational speed of the robot is 0.2 m/s, and the rotational speed is 0.31 rad/s.   Figure 3. 12: Snapshot of the environmental scene.  By applying the proposed episodic cognitive map building method, the result of episodic cognitive map building is obtained, as shown in Figure 3.13. Here, the red line 57 indicates the raw odometry information obtained during the robot wandering in the office, as the learned path. The blue line represents the built cognitive map based on the proposed episodic-cognitive map building algorithm, as the cognitive path. The experimental result shows that the construction method of cognitive map in this thesis can achieve closed-loop detection. This means robots can correct and determine the proper path, in an uncertain environment, to compensate for the odometer cumulative error and to ensure the accuracy and stability of the constructed cognitive map. The cognitive map is constructed by a series of points, edges and circles, which respectively represent the discrete finite event sets, the connection weights of various events, and the corresponding events. The cognitive map having these geometry characteristics can be used in the scene perception, pose perception and the building of state neurons of robots in the current state. This special event connection, based on the linking mechanism of neurons, can be used for planning a robot event sequence.  	  Figure 3. 13: Cognitive map building results.  Figure 3.14 shows the activation updating process of the state neurons in the incremental learning process of event sequence forming the episodic memory. In the early 58 stage, the robot is unfamiliar with the environment, and the growing rate of state neurons will be coordinated with the occurred events. After the realization of the episodic memory, the growth of the number of state neurons does not occur when the robot come to a familiar place. However, a judgement, based on the state of neurons activated and the current robotic pose, will be executed whether a new event should be attached, which demonstrates the building theory of the episodic memory model. The cognitive map example of the present work generates 237 events and 228 state neurons, of which 120, 121, 122 and 126 were fuzzy neurons.    	  Figure 3. 14: State neurons versus events.  Based on the sequence of activated state neurons, a weighting matrix, which connects neurons, is built. The connection strength among the neurons is shown in Figure 3.15. During the growing process of neurons, the highest connection strength will arise between the current neuron and the next adjacent neuron. However, the highest connection strength can also arise the current neuron and several other neurons, when the current state is detected and recognized that robot has experienced them. This is shown in Figure 3. 14, in which the highest connection strength arises between No. 122 neurons and two other 59 neurons labelled No. 123 and No. 191.  	  Figure 3. 15: Transition weights between neuron states.  Based on the experimental results it is seen that the cognitive map is a topology and geometry map, representing the perception of a robot about its environment. For the problem of cumulative error in the exploring process, the robot can rectify and update its position by the closure detection mechanism. The results indicate that the closure detection mechanism can eliminate the error and provide good performance. State neurons are added into the episodic memory model for imitating the process of a human’s memory formation. The proposed modeling method can better simulate the organization and decay process of the human episodic memory, and avoid the redundancy of a cognitive map. The generated transition weights facilitate robotic navigation.  3.5.2 Experimental Results in a Dynamic Corridor Environment Experiments have been carried out in a dynamic environment as shown in Figure 3.16 to verify the effectiveness and robustness of the proposed cognitive map building algorithm.  60   During the experiments, the Husky A200 mobile robot (Figure 3.17) autonomously wandered along the grey solid line with arrows, as shown in Figure 3.16 (a) in the physical corridor environment, which includes pedestrians and local environmental changes. Through the Kinect2 camera sensor, the scene pictures were acquired as shown in Figure 3.16 (b). The relevant odometry information and the scene perception were recorded as events. The mobile robot used the events to build the episodic-cognitive map and then based on proposed event sequence planning and behavior controlling algorithm to realize autonomous navigation. The parameters used in the experimentation are given in Table 3.1.   (a)                       (b)  Figure 3. 16: (a) Experimental trajectory; (b) Environmental scene.  61    Figure 3. 17: Husky A200 mobile robot.  Table 3. 1: The Parameters Used in the Experiments. Description Parameter Value Linear speed pv  0.1m/s Angular speed vθ  0.05rad/s Activation threshold oθ  0.91 Neuron memory depth nθ  0.4 Neuron decay coefficient τ  10 Neuron transition weight ijw  [0,1]  The result of episodic cognitive map building, using the proposed episodic memory network, is shown in Figure 3.18. The blue dotted line represents the raw odometry information while the robot wandered in the corridor along the learning trajectory shown in Figure 3.16 (a). The red solid line represents the built cognitive map based on the proposed episodic-cognitive map building algorithm. It is the graphical topology representation of the episodic-cognitive map. The result shows that the proposed map building algorithm can solve the problem of cumulative error occurred in the phase of path integration. The system generates 561 events and 523 state neurons in total, as shown in Figure 3.19. Thus, based on the robotic cognition of the environment, the system generates an event chain, and then forms an experience map stored in the episodic memory. The experience map consists of 561 experience nodes, and each node records the corresponding observation, state neuron and pose perception. 62    Figure 3.19 shows the activation updating process of state neurons in the incremental learning process of event sequence forming the episodic memory. Before the robot wanders in the same place again, subsequent state neurons are activated in succession after the activation of the current state neuron. However, if the same observation occurs, some state neurons are activated again. For example, state neuron 50 was first activated at event 51, and it is activated again at event 461, which indicates that the robot remembers a previous experience (the same observation) in the episodic memory. Transition weights, which connect the state neurons after learning, is shown in Figure 3.20. They are constantly updated in the learning process, and have a decay process. The transition weights indicate the connection strength between each state neuron, and their structure can be adjusted during the learning process. The transition weights are used for the planning of future robotic event sequences.   Figure 3. 18: Episodic cognitive map building result.  63  In the incremental process of state neurons, the current state neuron has maximum connection strength with the next state neuron. However, when the robot encounters a familiar observation, the state neuron may have maximum connection strength with multiple state neurons. For example, state neuron 50 has maximum connection strength not only with state neuron 51 but also with state neuron 424. This kind of state neurons are termed fuzzy state neurons. Figure 3. 19: State versus event   64  By imitating human learning and cognition of environment, based on the episodic memory model and the state neuron organization mechanism, the proposed robotic episodic cognitive map building algorithm can generate an episodic cognitive map with a discrete event vertex set and an edge of event transition weights. Through the cognitive learning, the system updates the state neurons and transition weights continuously, realizes the accumulation, integration and updating of the episodic-cognitive map. Furthermore, the state neuron set can be used for planning an optimal episodic trajectory for a robot.        Figure 3. 20: Weights of state neuron transition.  65 Chapter 4: Robotic Path Planning Based on Cognitive Map  4.1 Overview  As the robotic technologies keep advancing and start to interweave into our lives, it is inevitable that some types of robots will be soon required to make highly intelligent decisions. For example, in an unfamiliar environment, the robot has to assess the current location, predict the future sequence of trajectories, and execute a series of actions to arrive at the target location through intelligent decision making. So how can we make a robot sufficiently intelligent? Here, clues can be sought from how the human brain works. Tolman suggested that for animals and humans to be able to accomplish various navigational tasks, they must have a cognitive map of their environment in their brain (Tolman, 1948). Tolman defined a cognitive map as the essential module responsible for estimating the self-position in the environment. The mammal brain hFas been studied considerably, particularly with regard to the hippocampus and its surrounding area. Passingham located the cognitive map within the mammal brain’s hippocampus (Passingham, 1979). O'Keefe and Dostrovsky first found a type of pyramidal neuron within the hippocampus, which becomes active when an animal enters a particular place in its environment. They defined this type of neurons as place cells (O'Keefe and Dostrovsky, 1971). These cells were considered the main component of the cognitive map (McNaughton et al., 1994). Grid cell is a unique spatial awareness brain cell and was discovered in 2005 by Hafting et al. They suggested that grid cell firing signaled a rat's changing position, representing a "neural odometry" for navigation (Hafting et al., 2005). The grid cells are regarded as the core of the path integration system and play a vital role in ensuring stable spatial representation during a mammal’s navigation (Navratilova et al., 2011, Moser et al., 2008). It became evident that the grid cell was a part of the cognitive map as well. The main contribution of this chapter is to present the approach of modeling an episodic memory, and based on which to build an episodic-cognitive map, and then describe the autonomous path planning and behavior controlling algorithms in detail. Typical episodic memory-based robotic navigation is focused on episode-like memory, which is a type of memory structure. The present work simulates the organization of 66 episodic memory by introducing a neuron stimulation mechanism for environmental cognitive learning. For autonomous path planning, the next situation is chosen based on the neuron transition weights and the minimum events distance. Several experiments will be presented to validate the developed methodology.  4.2 Mobile Robotic Path Planning Process Based on the episodic-cognitive map, a novel robotic path planning method is proposed here by imitating the mechanism of human memory and recall. The overall architecture of path planning and behavior controlling is shown in Figure 4.1.   The process of this system follows the steps given below: Step 1: Input the current robotic observation and the goal observation; Step 2: According to the mapping relationship from observations to state neurons, locate the current robotic state neuron and the goal state neuron; Step 3: Based on the step 2 results, locate the preferred current event and the goal event; Step 4: According to the current and goal events, the system generates a planning event sequence; Step 5: Control the robot to move to the next event along the planned path; Step 6: After step 5, the robot acquires its current observation again; Step 7: Loop execute step 2 to step 6 until the current observation is identical to the goal observation;  Figure 4. 1: Robotic event sequence planning and behavior control.  67 Step 8: The mobile robot completes the task.  4.3 Path Planning Method Based on Cognitive Map 4.3.1. State Neurons Location When a robot needs to plan a path from the current location to a target location, the robot should be provided with the target observation, and based on the camera sensor, it can capture the current observation as well. Given the current robotic observation curo , the method of sum of absolute differences is used to create the observation similarity measure set between the current observation and the observation io from the observation perception set, as given by  21 ,icur io oε =−  (4.1) where 1,2, , ,i p p= L  represents the number of observations in the observation perception set. State neuron location locates the corresponding state neurons according to the current observation. The similarity measure can be directly translated into the location’s similarity degree. Thus the robot can locate the current optical observation from the observation perception set as given by  argmax ,iopt io oo ε∈=  (4.2) Using the observation of the state neuron mapping projection :f o s→ , the robot can easily locate the current neuron state curs .  Adopting the same process, the system can locate the target neuron state tars  for the mobile robot.  4.3.2. Event Location In the proposed episodic memory model, an event comprises observation perception, state neuron and pose perception. If the current observation is not adequately similar to any observations from the experienced events, a new event will be added. As well, if the state 68 neuron or pose perception is not the same as for others from the experienced events, it will add a new event as well. So there will be one state neuron corresponding to different events. It is not possible to choose the event directly based on the state neuron, and this type of state neurons are termed fuzzy state neurons. So, the following principles are proposed for event location based on state neurons. ž  If the state neuron is the target state neuron, locate the event having the largest event number; ž  If the state neuron is not the fuzzy state neuron, locate the event that the state neuron corresponds to. ž  If the state neuron is a fuzzy state neuron, considering the event character of one-way growth, the serial number difference between the two events is defined as the event distance. The principle of minimum event distance is adopted to choose the event that is nearest to the target event as the located event. Based on these principles, the system can locate the current event cure and the target event tare  for the mobile robot.  4.3.3. Event Sequence Planning Event sequence planning plans a global path from the current event to the target event. To predict a trajectory from the current event cure to the target event tare , an event sequence planning algorithm is proposed as given in Table 4.1. Based on the one-way linear transition character of events along with time in the episodic memory, the planned event sequence is unidirectional.  Table 4. 1: Robotic Event Sequence Planning Algorithm. Input: , , , , 1cur tarM W e e i =  Output: ES (Event Sequence) 1   ( ) curES i e=  2   while cur tar<  3     ,cur cur tar tare s e s→ →  69 4     computejshaving maximum transition weight  5     if num> 1 6       compute these similar neurons’ mτ  7       find the j which maximizes mτ  8       next js s= 9     else 10      next js s= 11    endif 12    if nexts is a fuzzy state neuron 13      locate nexte based on the principle of minimum       event distance 14    else 15      locate nexte that the state neuron nexts corresponds to  16    endif 17    cur nexte e=  18    i + +  19    ( ) nextES i e=  20  endwhile 21  return ES   Based on the proposed event sequence planning algorithm, the system will generate a planning event sequence as the global path for the mobile robot to reach the target.  4.3.4. Behavior Controlling According to the predicted event sequence, the robot needs to know how to reach the next event from the current event. Figure 4.2 shows the pose transform relation between the context events.  70  The pose increment between the context events can be computed as follows:  ,j ijiij j iij ijx dxxdp p pyy dy⎛ ⎞ ⎛ ⎞⎛ ⎞= − = − =⎜ ⎟ ⎜ ⎟⎜ ⎟⎜ ⎟ ⎜ ⎟⎝ ⎠⎝ ⎠ ⎝ ⎠ (4.3)  ,j jkkjk k jk j jkx dxxdp p py y dy⎛ ⎞ ⎛ ⎞⎛ ⎞= − = − =⎜ ⎟ ⎜ ⎟⎜ ⎟ ⎜ ⎟ ⎜ ⎟⎝ ⎠ ⎝ ⎠ ⎝ ⎠ (4.4)  arctan 0,arctan2ij ijij ijiijijdy dydx dxdyotherwisedxθπ⎧ ⎛ ⎞≥⎪ ⎜ ⎟⎜ ⎟⎪⎪ ⎝ ⎠= ⎨⎛ ⎞⎪− ⎜ ⎟⎪ ⎜ ⎟⎪ ⎝ ⎠⎩ (4.5)  arctan 0,arctan2jk jkjk jkjjkjkdy dydx dxdyotherwisedxθπ⎧ ⎛ ⎞≥⎪ ⎜ ⎟⎜ ⎟⎪⎪ ⎝ ⎠= ⎨⎛ ⎞⎪− ⎜ ⎟⎪ ⎜ ⎟⎪ ⎝ ⎠⎩ (4.6)  ,ij j idθ θ θ= −   (4.7) where iθ  is the desired heading direction of event je , and jθ is the desired heading direction of event ke . Assuming that the mobile robot is controlled by velocity commands, the movement time and velocity are regulated to satisfy the behavior control need, as given by  Figure 4. 2: Pose transform between the context events.   71  2 ,ijppdpt fv= ×   (4.8)  ,ijdt fvθ θθ= ×  (4.9) where ,pv vθ  represent the mobile robot’s linear velocity and angular velocity respectively, both of which are greater than zero. If 0tθ > , the mobile robot needs to rotate for tθ  in the counterclockwise direction; otherwise, rotate for tθ  in the clockwise direction. Also, f represents the command receiving frequency of the mobile robot. To reduce the influence of the acceleration and deceleration time, the maximum linear and angular speeds set as0.1 m/s  and 0.05 rad/s , respectively. The behavior controlling algorithm is given in Table 4.2.  Table 4. 2: Robotic Behavior Control Algorithm. Input: , , , , ,cur tar pES e e v v fθ  Output: Actual path 1   while cur tar<  2     i cur=  3     compute ,ij ijdp dθ  4     compute ,pt tθ  5     go straight for pt  6     rotate for tθ  7     record actual path 8     receive the current image curimg  9     change curimg into scene perception curo  10    cur curo s→  11    cur curs e ES→ ∈  72 12    compute the angle offset between actual heading       direction and the desired direction 13    rotate for the angle offset 14  end while 15  return actual path  4.4 Experimental Results To verify the effectiveness of the proposed robotic event sequence planning algorithm, two different target planning experiments have been carried out based on the built episodic cognitive map. The driving speed of the robot is 0.1m/s and the turning speed is 0.05 rad/s. The operating frequency is about 4-5 Hz. The first experiment was carried out in the office environment shown in Figure 4. 3. The cognitive map building results are given in Chapter 3. Based on the episodic cognitive map, four experiments were performed, which have the same start event id but different target event id, and these four target event id gradually increase, as shown in Figure 4.3. The experimental data is given in Table 4.3.  73  Figure 4. 3: Results of robotic event sequence planning (office environment).  Table 4. 3: Experimental Data. Experiments (a) (b) (c) (d) Start event id 2 2 2 2 Target event id 138 170 214 220 Events distance 136 168 212 218 Number of active events 137 169 137 143  From the results it is seen that the experiments c and d, which have a larger event distance, have a smaller number of active events. This is due to the existence of the same observation during the robot wandered from the start to the end in the experiments c and d. When the mobile robot encounters a familiar observation perception, the robot can choose 74 several different trajectories to the target, based on the proposed event location principle and the event sequence planning algorithm, and the robot can locate the event that is nearer to the target event as the next active event. This cuts down the event sequence distance and improves the efficiency of task execution. A behavior control experiment, based on the mobile robot platform, is carried out. The experimental results are shown in Figure 4.2. The algorithm can maintain its robustness to guide the robot to arrive at the ideal position under the premise that there is a slight deviation between the current orientation and the ideal orientation of the robot. The result of error analysis is shown in Table 4. 4. The robot actually runs 25.492 m and the deviation to the ideal position reaches 0.182 m which is less than 1%, which confirms the high robustness of the algorithm.  	   	  Figure 4. 4: Analysis of robotic behavior control.  Table 4. 4: Performance Analysis of Robotic Behaviour Control. Ideal Position Real Position Drive Distance (m) Error (m) (-2.278,5.254) (-2.395,5.115) 25.492 0.182 75 The second set of experiments were implemented in a dynamic corridor environment (based on the cognitive map created in Chapter 3) to verify the effectiveness and robustness of the proposed path planning algorithm. The experimental results are shown in Figure 4.5.   Experiments 1 and 2 have the same initial event, but different target events, and experiment 2 has the longer event distance. Figure 4. 5 (a) and (b) show the path planning of these two experiments, (c) and (d) show the selected event id, and (e) and (f) show the active state neuron id. In experiment 1, the robot does not recall a previous experience in the episodic memory, hence the state neuron is activated successfully. . From the results it is seen that the planned path is not the successive event sequence from the initial event to  Figure 4. 3: Results of robotic event sequence planning (corridor environment).  76 the target event. On the contrary, when encountering a fuzzy state neuron, i.e., the robot can choose multiple trajectory to the target, based on the proposed principle of minimum event distance, the system will choose the event that is much closer to the target event as the next active event. The performance analysis of robotic event sequence planning is presented in Table 4.5.   Table 4. 5: Performance Analysis of Robotic Event Sequence Planning. Experiment 1 2 Initial event id 3 3 Initial state neuron id 3 3 Target event id 233 557 Target state neuron id 221 519 Events distance 231 555 Active event number 219 143 Active state neuron number 219 143  The proposed event sequence planning algorithm is capable of choosing a much better trajectory for a robot to reach the target in less time and a shorter distance, improving the efficiency of the robotic task execution. Using the mobile robot platform, a robotic behavior controlling experiment was carried out, the result of which is shown in Figure 4.6. The planned path is about 25 meters long, the target position is located at (-3.60, 22.06), but actually the robot arrives at (-3.59, 22.5). The cumulative error is less than 0.2 meter, which is about 0.8% of the total distance. So the proposed behavior control algorithm can control the mobile robot to reach the target along the planned path very well. Methods of robotic path planning are being actively explored, but most of the methods based on vision and cognitive map have poor adaption to generate environment. The teach and repeat method can realize visual positioning and path repeating with high precision. Its planned path is the cognitive map itself, and it is not a preferred plan. The path planning algorithm proposed in this thesis has the cognitive ability, compared with the teach and repeat method. When there are several choices to the target, it can avoid the unacceptable 77 trajectories and choose the shortest experienced trajectory as the global planned path. So, to some extent it can save time and improve efficiency.      Figure 4. 4: Analysis of robotic behavior control.  78 Chapter 5: Visual Navigation  5.1 Overview  In robot visual navigation, the on-board camera of a mobile robot captures images, which are processed to obtain the current robot position of the robot. If it is different from the required position according to the planned trajectory, the robot is controlled to achieve the correct trajectory of robot navigation. In the robot navigation scheme presented in this chapter, robot path planning is established by sampling the scene database, extracting the characteristics of the scene and matching with the scene in the sampling scene database. Using a navigation algorithm the motion parameters are calculated and executed so as to carry out robot navigation correctly, in the given scenario.  5.2 Image Feature Extraction and Matching Algorithm 5.2.1 Image Feature Introduction  Image features are the representative characteristics in an image. An image has its own unique features, and these characteristics of the image can provide the identity of the image, and distinguish it from other images. The features of an image may be divided into two types: natural features and artificial features. The natural features are the natural attributes of the image itself, such as size, color and shape. Point characteristics belong to one kind of natural characteristics. They include straight line, intersection point, corner point, high curvature point, center of gravity and center of a region. The feature points have strong stability, and they do not change with a changing environment and by the influence of external factors. These points can represent the picture which they are in, and the extraction of effective feature points can reduce the workload, save working time and cost with respect to the effectiveness of the associated measurement system.  5.2.2 Mathematical Definition of Image Matching Image matching is defined as the mapping of two images in gray intensity and spatial geometry. For a given reference image 1( , )I x y and the matched image 2( , )I x y , the match between the two images can be represented by the mapping:  79  2( , ) ( 1( ( , ))),I x y g I f x y=  (5.1) Here, f is the two-dimensional space geometry, and g is the one-dimensional gray intensity transformation. The purpose of image matching is to find the optimal spatial geometry transformation f and gray intensity transformation g, and make the formula (5.1) meaningful. In general, the search for optimal spatial geometry transformation f is a key problem in image matching, and the gray intensity transformation g is not required. Hence, the formula (5.1) can be simplified as  2( , ) 1( ( , )),I x y I f x y=  (5.2)  5.2.3 Introduction of SIFT The SIFT algorithm was first proposed by Lowe in 1999, and concluded in 2004 (Lowe, 2004). It selects the extreme value points of the Gaussian residual difference in the scale space, and calculates the gradient direction in the local neighborhood of the characteristic points. This algorithm brings in the image pyramid structure to the scale space to reduce the computation. The SIFT algorithm has been successfully applied in image matching (Gopalan and Tellex, 2015), panoramic splicing (WONG and CHOSY, 2006) and visual navigation (CHOSY and WONG, 2005). SIFT is an effective image local feature extraction algorithm having stability and robustness.  Specific advantages are: (1)   Stability: The local characteristics of SIFT feature are the local features of image, rotation, scale brightness transform invariance, angle change, affine transform and noise. It can maintain good stability。 (2)   Uniqueness: The information is rich and unique, and suitable for fast and accurate matching in a large database of characteristics. (3)   Extensive Information: Even a few objects can produce a large number of SIFT eigenvectors. (4)   High speed: The optimized SIFT matching algorithm can even meet real-time requirements.   80 5.2.4 Image Feature Extraction and Matching Based on SIFT A stable and effective local description is an important requirement for robot visual navigation. Mikolajczyk and Schmid (Mikolajczyk and Schmid, 2005) compared 10 local descriptions including SIFT, and the results showed that the SIFT algorithm had the best performance in similar descriptions.Therefore, this chapter extracts the key feature points of the robot moving path using SIFT in the visual navigation model. The SIFT algorithm takes the feature points of the image on different scales based on its proposed DOG (Difference of Gaussian) operator (Lowe, 2004), using the Gaussian kernel function ( , , )G x y σ  of different mean square error parameter σ to build the difference scale space, which is the difference between the adjacent layers in each group of the gauss pyramid. The feature points are obtained by the local extremum of the difference ( , , )D x y σ  of the two images on the adjacent scale:             ( , , ) ( ( , , ) ( , , )) ( , ),D x y G x y k G x y I x yσ σ σ= −                (5.3)     2 22 21( , , )= exp ,2 2x yG x y σπσ σ⎡ ⎤+−⎢ ⎥⎣ ⎦ (5.4) ( , )I x y is the original image. Through the SIFT feature calculation, one can get not only the feature point coordinates and feature point descriptor information, but also can get feature with the specified scale (sigma parameter), and the feature point principal direction. Using the dividing method of four by four sub-region, the gradient amplitude information in 8 directions is constructed for each region, and then the eigenvectors of 128 dimensions are obtained. As shown in Figure 5.1, for the reference image a and matching image b, the image matching algorithm based on SIFT feature points consists of five steps: extracting SIFT feature points, SIFT feature point matching, removing mismatches, computing the relation of the geometric transformation, geometric transformation for the matching image b. Consider two images that have some overlap, as shown in Figure 5.2, where the reference image is in the left, and the matching image is in the right. It is seen that there is a clear image rotation and a change in view between the reference image and the matched image. Next, the SIFT algorithm which maintained by Reb Hess is used to match the feature points of the two images. 81   Figure 5. 1: Image matching algorithm flow based on SIFT feature points.    Figure 5. 2: Reference image (left) and match image (right).  First, the reference image and the matching image need to be treated in grayscale, and the SIFT feature points of two images are extracted, as shown in Figure 5.3   Figure 5. 3: Reference image (left) and match image (right). 82 By connecting the reference images to the corresponding SIFT feature points in the matching image, the image feature point matching diagram is obtained, as shown in Figure 5.4.   Figure 5. 4: Image SIFT feature points matching sketch.  From the figure it is seen that the SIFT algorithm can match the feature points in two images accurately. There still are enough right matching points for estimating the projection change matrix while there is some rotation, light change, and view point change between these two images. It confirms that the characteristics of SIFT have strong stability and robustness in image distortion, such as translation, rotation, scale change, light variation, and the change of view point. Using the SIFT algorithm to match the features of the image, the implementation of the robot navigation function has been done in the present work.  5.3 Robot Visual Navigation Method The visual navigation of mobile robots consists of two modes of operation: path learning mode, and status positioning and navigation mode. Path learning mode: Robot in an unknown environment has no environmental knowledge to guide the robot behavior.  First, the robot needs to specify a path in advance. The robot continuously detects and records the environmental significant landmarks, and a corresponding state node topology map building environment. 83 Status positioning and navigation mode: Robot already exists in memory. The local characteristics of the status of the node are significant signs. Status positioning and navigation algorithm is used to predict behavior.  5.3.1 Path Learning and Path Database building The robot path learning is the process of establishing a state-based environmental topology map, each composed of several significant landmarks. When the robot moves in a pre-specified path, sampling is needed. The new state is sampled and added to the environmental topological map when the similarity threshold of the two frames of images is marked below 0.5. This method allows to select the number of states independently to accommodate different environments. The similarity of the feature points in two images is measured by the continental distance of the eigenvector. Set the feature points of the input as a benchmark, find the close and neighboring points of the input feature points in the matching image, and finally realize the target of the vector by quick match. The input feature point descriptors can be expressed as 1 2 128( , , , )ci i i iF c c c= , and the feature point descriptors of the matching image can be expressed as 1 2 128( , , , )di i i iF d d d= . Use the Euclidean distance ( , )c di iD F F  to measure the similarity of the two descriptors, to get the right pair of feature point descriptors. It should meet the condition:        ( , )( , )c di ic di pD F FD F Fθ<    (5.5) Here, ( , )c di iD F F  and ( , )c di pD F F  are the distances between ciF  and the closest point diF  and the neighboring point dpF  and θ  is the threshold value.  5.3.2 Robot State Positioning and Visual Navigation Algorithm The state positioning of the robot is done by matching the significant objects of the current scene and the feature point descriptor database which is built by sampling. After obtaining the significant objects of the current scene, the state positioning module starts to perform global positioning. If the current area of interest is positioned as the state of the robot, the robot navigation module considers this state to be the robot's current location. 84 There may be several significant signposts that may appear in the range of matches, which the system considers to be useful for robot navigation. If the robot locates a state of the given path, then it can estimate the behavior that has to be assumed according to the position and coordinate relationship between the matched local descriptor pairs. In this case, the longitudinal coordinate is ignored and the horizontal coordinate is used to apply the navigation algorithm to get the behavior of the robot. Suppose that the robot has 3 modes of motion: forward motion, right turning motion, and left turning motion. So 6 signs can be used for the local descriptor matching pair to cover all robot navigation process, as shown in Figure 5.5.   Figure 5. 5: The six types of local descriptor matching.  In the figure, the Central Line represents the center line of the input scenario and the database scenario. Define cx  as the horizontal coordinate of the SIFT local descriptor of the input scene, and dx  as the horizontal coordinate of the feature point that matched from the path database. 0x <  represents that the horizontal coordinate of the descriptor is at the left of the center line, and 0x <  represents that the descriptor is at the right of the center line. Then the behavior selection (Chang et al., 2010) of robot navigation can be expressed as follows: 85  ( 0, )( 0, )0 ( )c dc c docc dR c c docx xx x xtx xx x xtothersγγθ⎧ −− < <⎪⎪⎪ −⎪= > >⎨⎪⎪⎪⎪⎩  (5.6) Here, toc is the cycle time of the navigation algorithm, and γ  is the control gain of the navigation angle Rθ . The local descriptor pair of the current match determines the behavior of the robot. After obtaining the navigation angle (rad)Rθ  of the robot, the relationship between the angular velocity and angle is used to calculate the angular velocity of the robot:  5 ,180Rπω θ= × ×  (5.7) The linear speed of the robot's movement should be large in the straight direction and small in the direction of the robot. Therefore, define the linear speed of the robot in the situation that has no obstacle as:   min max min( ) 0.5 ( ) (1 tanh( )),wv v v v kω π ω= + × − × + − ×  (5.8) Here, minv and maxv are the minimal and maximal speed of the robot, and 12wk = . This formula has considered the operation performance and softness of the robot. The speed and angular velocity of the robot navigation is shown in Figure 5.6:  86  Figure 5. 6: The relationship between speed and angular velocity.  5.3.3 Robot Visual Navigation Flow The visual navigation of the robot may be divided into two parts: (1)  Learning of the specific path (2)  Navigation in the path that has been learned. Path learning consists of the processes shown in Figure 5.7.   Figure 5. 7: Flow of path learning and path database building.  In the process of robot path learning, first it is necessary to build a motion trajectory for the robot, including the start and end points of the path. One can place some objects that contain distinguishing features, which can help to increase the number of SIFT feature points. After planning the trajectory, it is necessary to help the robot move around the track. 87 During the moving period, ROS which is linked to the Kinect using a driver to control the Kinect and capture images of the surroundings in every time period (0.3s in the present experiment). Using the remote invocation of ROS, the images are transferred from the ROS computer to the host computer. The host computer extracts the SIFT feature of the n images, and obtains and stores the descriptors of them. In this manner the sampling learning of path planning and building of the path database are carried out. It is important to reduce the speed of the robot to increase the number of images acquired, so as to achieve the goal of enhancing the learning of the path. After the path learning process is completed, the robot will be able to navigate through that path. The main flow of the robot path scenario reconstruction (navigation) in this study is shown in Figure 5.8. When the robot is in position 𝑃#, the robot gathers an image from the scene in front of the Kinect, transfers the image1 to the host computer by using remote invocation; the host computer obtains the SIFT feature descriptors by image processing and matches them to the SIFT feature descriptors in the path database.    Figure 5. 8: The navigation flow of robot after path learning.  If image2 is successfully matched, then use the navigation algorithm introduced in Section 3.4.2 to calculate the motion parameter (linear velocity and angular velocity), 88 which are used by the robot move to the position of image2. If not, the robot stops moving and scans the environment, obtains images, processes the images, does positioning and navigation again.  5.3.4 Navigation Error Handling In robot navigation, any error in SIFT feature matching, will cause the predicted robot behavior to deviate from the actual value. For the real-time requirement of navigation, a simple adaptive threshold noise reduction method is adopted to eliminate the matching interference points. Specifically, For ( 1,2, , ),ciI diX i nσσ= =  if  i xX X σ− > ,  (5.9) Then eliminate iX and make  1n n= − . X is the mean of iX and xσ is the mean square error of iX . Experimental results show that the formula (5.9) can obtain better matching, effectively eliminating the interference and reducing the mismatched rate.  5.4 Robot Visual Navigation Experiment The robot system excluding the host control system is shown in Figure 5.9.   Figure 5. 9: The Robot and its sensors. 89     From the figure it is seen that there is a laser sensor in front of the robot, a Kinect visual sensor in the middle, and a laptop computer mounted on the top of the robot. The two sensors and the robot body are connected by the ROS supported computer, and a robot detection and control system is implemented. As shown in Figure 5.10, set the motion track of the robot from the start point A to the end point D of the path A-B-C-D。   Figure 5. 10: The sampling path in navigation experiment.  First, robot has to do path learning, by moving along the path ABCD, and extracting the image information of the surrounding environment, continuously. In the process of sampling, the host computer sends motion commands to the robot system. In the experiment, the robot moves for 0.3 s after every motion command, and does this motion 180 times. From the sampling process from A to B, t the linear velocity 200 mm / sv =  and angular velocity 0 rad / sω = , running 50 times. At the first inflection point B,	   0v =  and 0.2 rad / sω = −  (The angular velocity is positive while turning left and negative while turning right), running 25 times. From B to C, 200 mm / sv = and 0 rad / sω = , running for 35 times. At the second inflection point C, 0v =  and 0.2 rad / sω =  running 35 times. From C to D, 200 mm / sv = and 0 rad / sω = , running 45 times. As the robot moves, the host computer controls the Kinect to extract an image of the scene, and transfers to the host for storage. Overall, 180 images are extracted. The part scene images extracted in this experiment are shown in Figure 5.11.  90  Figure 5. 11: Part of images extracted in sampling.  From Figure 5.11 it is seen that there are objects that contain some salient features. These objects can make the navigation system obtain more SIFT feature points. Figure 5.12 shows the SIFT feature points and the feature vectors of some of the images.   Figure 5. 12: The feature points and feature vectors of some of the images.  The SIFT local feature descriptors obtained after sampling are stored in the host computer. For robot navigation, position the robot at the start point A, and use the navigation flow given in Section 5.4.3 to carry out scene reconstruction and navigation. Consider a point near the inflection point B, as shown in Figure 5.13. The image P1 is the scene image that Kinect extracts in the current position, and P2 is the image that is matched by SIFT matching method. It can be seen that P2 has a right turn trend compared to the image P1. The result is “Turn Right” type shown in Figure 5.5. Calculate the motion parameter by using the navigation algorithm. The motion path of the robot in navigation is shown in Figure 5.14. 91   Figure 5. 13: The scene image and the matched image.   Figure 5. 14: Liner velocity and angular velocity in navigation.  The entire process of navigation takes 35 seconds. In the period from 0  t to 1  t . The fluctuations of the linear velocity are big, and the angular velocity makes square wave like wave like fluctuations. This period corresponds to the section AB in the sampling path. From 1  t to 2  t , the linear velocity is particularly small and the angular velocity maintains on a horizontal level which is lower than 0. This period represents the first inflection point B in the sampling path. From 2  t to 3  t , the value and the fluctuation of the linear velocity 92 are both high, and the angular velocity is varying down to 0. This period corresponds to the section BC. Same as above, from 3  t to 4  t , v  is small and ω  maintains a level greater than 0. This corresponds to the left inflection point C. In period 4  t to 5  t , the changes of v  and ω  are like those in the period of the section AB. This corresponds to the section CD. The robot system completes the navigation in the planned path by using the SIFT local feature match and navigation algorithm. Due to slipping, the time taken at the two inflection points B and C is quite different. So, to make the robot pass these inflection points successfully, it is necessary to take more sampling at these points. In this navigation experiment, the motion path of sampling and navigation is shown in Figure 5.15.   Figure 5. 15: The sampling path and the navigation path.  It is seen from this experiment that, by using the SIFT local feature descriptors of the image, comparison and matching for the robot motion scene can be done, so as to determine the right direction for the robot’s target path. From the robot’s navigation angle, which is calculated by the navigation algorithm, the linear velocity and the angular velocity of the robot motion parameters v can be computed, which are used to control the robot navigation 93 correctly. Further experimentation has shown that through proper and adequate sampling, the deviation between the actual navigation path and the planned path would not exceed50mm . This satisfies the requirement of typical robotic navigation.   94 Chapter 6: Obstacle Avoidance in Visual Navigation   6.1 Overview Obstacle avoidance is a basic and important function in robot navigation. Robot obstacle avoidance depends on sensors and planning algorithms. Different sensors with different characteristics and principles of different algorithms will incorporate different time and space complexity. In visual navigation, the robot uses a database of planned paths. Then, if there are no obstacles, the robot can successfully move along a planned path and reach the destination. However, if an obstacle blocks the movement of the robot in the planned path, the robot will collide with the obstacle unless the navigation system has an obstacle avoidance strategy and algorithm. So, dealing with the obstacle avoidance in visual navigation is very important.  This chapter provides a detailed description of the robot's visual navigation program.  6.2 Visual Navigation and Obstacle Avoidance System Scheme  The overall scheme for the visual navigation and obstacle avoidance of a robot based on ROS is shown in Figure 6.1. Kinect gathers the images of the robotic environment, and determines the location of the robot with the image feature comparison methodology. Then through a series of image processing and navigation algorithms, the robot navigation is carried out. The laser sensor of the robot can obtain the distance data in a range of angles, and then calculate the distance between the obstacle and the robot, use the obstacle avoidance algorithm to control the robot movement around the obstacle, and then in combination with the navigation algorithm, make the robot continue to navigate after the avoidance. This series of navigation components are divided into two parts. The first part is based on the robot part of ROS, and contains the robot, Kinect and laser sensor, mainly to realize the function of data acquisition and motion. The second part is the upper part, which processes the received data to calculate the motion parameters for robot navigation and obstacle avoidance. The two parts are connected through the remote calling function of ROS to achieve the combination and separation of each module. 95   Figure 6. 1: Visual navigation and avoidance schemes based on the ROS system.  6.3 Laser-based Obstacle Detection 6.3.1 Laser Sensor Laser sensors use laser technology to realize distance measurement. It consists of a laser, a laser detector and a measuring circuit. Its advantages include non-contact distance measurement, high speed, high precision, high range, and immunity tolight and electrical interference. A Hokuyo URG-04LX-UG01 Scanning Laser Rangefinder is used for the obstacle detection in the present work, as shown in Figure 6.2. It is small, low-cost, and able to report ranges from 20 mm to 5600 mm (1mm resolution) in a 240° arc (0.36° angular resolution). Its power consumption, 5V 500mA, allows it to be used on battery operated platforms.   96     Figure 6. 2: Hokuyo URG-04LX-UG01 laser sensor.  6.3.2 The Working Principle of Laser Sensor In operation of the laser sensor, laser pulses are sent to the target by the laser diode. When reflected by the target, the laser scatters in all directions, and partially scattered light is returned to the sensor receiver. A photodiode is an optical sensor, which can detect very weak light signals and convert them into electrical signals. The most common is the laser range sensor, which measures the distance of the target by recording and processing the time a light pulse takes to return to the receiver. The laser sensor must accurately measure the transmission time since the speed of light is very high. For example, since the speed of light is about 83 10 /m s× , for a resolution of 1 mm, the transmission time the distance sensor electronic circuit must be able to distinguish is:   80.01 33 10 /m psm s=×, (6.1) This is a high burden for electronic technology, which may be rather costly. But now the laser ranging sensors have been able to avoid this problem, by using a simple statistical principle, or an average of law, realize the resolution of 1 mm and ensure the response speed. In robot navigation, the detection of an obstacle may be done by methods such as ultrasonic sensing, optical flow, and image recognition. The obstacle detection based on laser in this thesis has many advantages such as wide detection scope, small effect of environment, and high detection efficiency.     The method of obstacle detection by laser is mainly done through emitting a specified 97 number of lasers in a range of angles by the laser sensor (The angle range of the laser sensor in this work is from -5° to +185°), and measuring the distance from the objects of every angle to the center of the sensor by receiving the reflected laser. By establishing a specific coordinate system, the distance of each angle is calibrated to determine the distance between the robot and the obstacle.   6.4 Obstacle Avoidance Algorithm in Visual Navigation 6.4.1 Local Path Grid Dividing and Obstacle Model Establishing In the course of navigation, the robot's path and state database does not have any information about obstacles, but the obstacles can be detected by the robot's sensors. This thesis uses the advantages of a laser sensor to detect the obstacles that enter the path of robot movement. The local occupied grids in robot coordinate may be established, as shown in Figure 6.3.   98  Figure 6. 3: Local occupied grids under robot coordinate.  The local grid is semi-circular based on the laser sensor, and its vertical and horizontal expansion is limited, ignoring the impact of a further obstacle on the robot. Depending on the size of the robot, the size of the occupied grid is set at r=250mm. The data that the laser sensor detects is divided into 19 horizontal regions, and at the same time, the distance of the laser data is divided into the 1-9 area, 250mm for each area. The nearest 250mm is the stop area of the robot and the 10th area does not detect any obstacle. For the occupied grids in the detection range of the laser sensor, only the current sensor reading are considered, and for the occupied grids that are out of the range, just the previous sensor reading are used. All the occupied grids can be expressed as:  { }1 2, , , ,nC c c c=  (6.2) Denote the path of the robot that can be selected in the current coordinate as P, the corresponding curvature as iρ , and the curvature (CHERUBINI and CHAUMETTE, 2013) which is the tangent line of the robot’s straight moving direction, { }0,  2, 4  as shown in Figure 6.2. The curvature set that corresponds to the path set can be expressed as follows:                 { }1 0 1, , , , , , ,i n n Pρ ρ ρ ρ ρ ρ= − − ∈                 	  (6.3) The number of feasible paths is2 1n + , according to the self-size of the robot. Every feasible path i consists 2 areas: the collision zone iQ  and the risk zone  iD . In path 2 of Figure 6.3, the green area is the collision zone of the feasible path. The collision zone 99 decides that the robot should take steps to slow down or stop, and the risk zone is used to select the safe feasible path and determine the risk function.  The feasible path’s definition of the robot utilizes the robot’s geometry and motion characteristics. The geometrical shape of the robot defines iQ  and  iD , and describes the potential risk. The robot kinematics (curvature iρ ) defines the feasible path set P. The local plan of the robot takes corresponding measures to avoid collision, using the information of these two aspects.  6.4.2 Risk Function Establishing and Feasible Path Analyzing. Assume that the risk of the obstacles to the robot can be expressed by the risk function [0,  1]iR ∈ . The safe value ( 0R = ) indicates that the robot does not detect any obstacles in the navigation path, and it can continue visual navigation according to the methodology presented in chapter 5. The robot should reduce the speed sv  while turning to ensure that the visual sensor can track the image features rapidly. At the same time, the robot controls the output based on the status of the local grid and the navigation procedure. So, when there are no obstacles:  ssv vω ω=⎧⎨=⎩, (6.4) In the most risky situation ( 1R = ), the robot system has detected an obstacle, and the system should combine the visual information and the obstacle avoiding strategy to calculate the feasible path, for the robot move around the obstacle. In this case, the robot should reduce the linear velocity rv  and ensure that it can return to the navigation path after obstacle avoidance. So, in the presence of an obstacle, the motion parameters should be:  ,c drru u uv vvφω ρ⎧ = −⎪=⎨⎪=⎩ (6.5) Here, φρ  is the optimal curvature of the feasible path, cu  is the horizontal coordinate of 100 the input scene's SIFT local descriptor, and du  is the horizontal coordinate of the SIFT local descriptor of the next scene in the sampling database. The local planning strategy of the robot can be determined by the risk function [0,  1]iR ∈  of every feasible path. The risk function iR  can be calculated by the risk distance of the occupied unit	   ic  which is in the risky region iD  of the feasible path  i . For every occupied unit ic  in the risky region iD , the risk distance ijΔ  can be defined as the distance between the bounding frame of the robot and the obstacle along the feasible path i . In Figure 6.3, there is only an obstacle 3c  in the risky region of path 2. So the risk distance 2 3Δ = Δ , and the risk function of the robot in the feasible path can be expressed as follows:  01 11/ 2 1 tanh( ) ,1i si r i si r i si rifR ififΔ ≥ Δ⎧⎪⎡ ⎤⎪= + + Δ < Δ < Δ⎨ ⎢ ⎥Δ − Δ Δ −Δ⎣ ⎦⎪⎪ Δ ≤ Δ⎩ (6.6) Here we use 0.9mrΔ =  and 1.5msΔ = . The corresponding curve of the risk function iR  is shown in Figure 6.4.   Figure 6. 4: The curve of the risk function.  6.4.3 Local Path Planning and Robot Motion Control in Obstacle Environment There are several types of robot local path planning: 101 (1)   0cR =  represents that there are no obstacles in the navigation path, and the feasible path’s curvature can be calculated as:  = /r svρ θ  (6.7) The curvature of the feasible path is cρ ρ= . (2)   0cR ≠ indicates that there are obstacles in the navigation path, blocking the motion of the robot. Then the optimal curvature cρ  can be selected according to the path set of no obstacles ( 0iR = ), select the one that is closest to the curvature ρ  which is calculated by the navigation algorithm. If a path with 0iR =  can be found, then set the current risk function of the robot 0cR = , and if not, then select a curvature that has the minimal  iR , and set  c iR R= . Finally, the local planning strategy can be expressed as:  (1 ) ,(1 )s rr c rv R v RvR R vω θ ρ= − +⎧⎨= − +⎩ (6.8)  6.4.4 Return to Original Navigation Route of the Robot The robot deviates from the original navigation route due to the obstacle avoidance movement. So in order to be able to get back to the original path correctly, the system should combine the visual navigation, take the surrounding scene and match it with the scene in the path database so as to achieve path planning again. Using the formula 6.5, the robot can estimate the behavior according to the coordinate relationship u  of the local descriptor pairs. The navigation angle that will take the robot back to the planning path is:  00 ,0c c dr c c du if u and u utu if u and u utotherwiseλλθ⎧− < <⎪⎪⎪= > >⎨⎪⎪⎪⎩  (6.9) 102 Here t  is the running period of the navigation algorithm, and λ  is the control gain of the navigation angle rθ .  6.5 Obstacle Avoidance Experiment in Visual Navigation 6.5.1 Visual Navigation Flow under the Obstacle Condition In robotic visual navigation, when there are obstacles that block the robot’s normal navigation, the robot should avoid them, and move back to the original navigation path. The overall flow of visual navigation in the presence of obstacles is shown in Figure 6.5. First, the robot obtains the image of the scene through the Kinect visual sensor, by the scheme presented in Chapter 5, we can calculate the navigation angle Rθ  and the corresponding path curvature sρ . At the same time, the laser sensor detects the obstacles in the range from -5°to 185°in front of the robot. By building the local occupied grids and the obstacle model in robot coordinate, the risk function iR  can be calculated. Through the optimal curvature set { }1 2 21,  ,  ... ,  ρ ρ ρ  in risk condition and sρ , select the barrier-free curvature 1cρ  which is closest to the navigation path curvature sρ . Calculate the risk probability sR  of the current navigation path from iR . Calculate the feasible path’s curvature set { }11 22,  ,  ... ,  nnρ ρ ρ  which has minimal risk probability. Find the curvature that is closest to 1cρ  in the feasible path’s curvature set and set it to cρ . If 0sR = , then set the feasible path to the normal navigation path, and the curvature is sρ ; if not, then set the optimal path for obstacle avoidance to the path whose curvature is cρ .  6.5.2 Experimental Results     Figure 6.6 shows the environment of the obstacle avoidance experiment in the process of visual navigation. The driving speed of the robot is 0.1 m/s. The operating frequency is approximately 4 Hz.     The navigation path of the robot is path1, as shown by the blue line in Figure 6.6. The robot system can complete the navigation in an obstacle-free environment by using planned path sampling, motion path and state database building, feature points matching and the 103 navigation algorithm presented in Chapter 3. The navigation path is along the track A straight to D. In Figure 6.6, there is an obstacle in front of the robot, which blocks the normal navigation of the robot. For proper navigation, the robot should move around the obstacle and then return to the navigation path. The red curve of path2 is the approximate path that the robot took to navigate around obstacles.   Figure 6. 5: The flow of visual navigation and obstacle avoidance.   Figure 6. 6: The experimental environment for obstacle avoidance in navigation. 104  To complete the experiment of obstacle avoidance in the process of visual navigation, first sampling should be done to test the normal navigation without obstacles. As shown in Figure 6.7, because the navigation path1 is nearly a straight line, the variation of the linear velocity v  is quite high in the beginning. After correcting the motion direction, the linear velocity is maintained at a high value while the angular velocity ω  is small and close to 0. After the navigation experiment without obstacles, an obstacle which is in nearly of size of the robot is placed in the center of the navigation path. The obstacle avoidance control system is incorporated into the control program in the host computer so as to do the obstacle avoidance experiment in visual navigation.   Figure 6. 7: v  and ω  in normal navigation without obstacle.      Figure 6.8 shows the variation of the linear and angular velocities of navigation while there is an obstacle in the planned path. From the figure it is seen that, in the period from 0  t to 1 t , the linear velocity v  is relatively big, and the angular velocity ω  fluctuates in the neighborhood of 0. This period of time corresponds to the straight path AB, and the 105 distance between the obstacle and the robot is regarded as a safe distance, and hence the robot will move in the normal navigation mode. From 1  t to 2 t ,   ω maintains at the largest value in the positive direction (to ensure that the robot moves smoothly without slipping, the largest values of v  and   ω are set), and v  is kept at a minimal value. This period corresponds to the motion path BC. When the robot is at point B, from the data that the laser sensor collected and by using the obstacle avoidance algorithm presented in Chapter 4, the robot system calculated the risk probability of oscillation. In this manner the robot selected a correct motion path to move around the obstacle, in this period. Because the set linear velocity of risk rv  is relatively small, the speed of the robot took a minimal value. From 2t  to 3t  the robot had to avoid the obstacle. The laser detection and the obstacle avoidance algorithm showed that the robot would not oscillate with the obstacle if the robot moved forward. So the robot needed to return to the original navigation path again by finding the current position and the state of the robot and the position information of the next SIFT local descriptors through the navigation algorithm.   Figure 6. 8: v  and ω  in obstacle avoidance experiment.      In Figure 6.9 shown, the blue curve represents the navigation path of the robot in the obstacle-free scene, and the red curve represents the motion track for navigation in the 106 presence of obstacle. From the figure it is seen that the robot system avoided the obstacle and returned to the original navigation path.       Figure 6. 9: The navigation path with and without obstacles.    107 Chapter 7: Conclusions and Suggestions  This thesis developed, analyzed, and implemented a robotic attention and navigation system, which is primarily intended for robotic homecare applications, where the robot environment is dynamic and unstructured. The conclusions are summarized in the next section. The subsequent sections discuss primary research contributions made and limitations of the current work, and indicate possible further work that can be made in future research.  7.1 Conclusions This thesis proposed an integrated model of top-down and bottom-up visual attention with self-awareness for a homecare robot. For mimicking the human attention processes, a robot self-awareness model with fuzzy decision making system was developed and utilized, which is an important improvement on the existing robot attention models. Besides the task-driven object-based biasing, a robot self-awareness model can generate other parts of top-down biases in a robot visual attention model. Some results from the self-awareness model were obtained. In order to update the weights in the robot memory, the learning process was carried out through the Bayes’ rule. Three images were tested to evaluate the developed methods. Four types of saliency maps were compared to see how self-awareness can affect robot attention selection. A cognitive map based on episodic memory was built for mobile robots. Inspired by the biological findings on the mammalian hippocampus, an episodic memory model which includes a sequence of experienced observations, neutral states, behaviors and pose perceptions was proposed. Based on the memory model, a new approach was applied to the problem of Simultaneous Localization and Map (SLAM). The presented cognitive map building method can autonomously and incrementally learn the environmental experiences to construct a cognitive map and realize the accumulation, integration and updating of experiences. A new path planning and behavior control method based on episodic cognitive map is also proposed. This method can make a prediction about the future event sequence and choose a preferred trajectory from the current state to the target location, while improving the efficiency of the task execution. The method is implemented in a HUSKY mobile robot system at driving speed 0.1 m/s in an office environment. The operating 108 frequency is in the range 4-5 Hz. The cumulative error is less than 0.2 m, which is about 0.8% of the total distance. The approach based on biological cognition extended the application of biological cognitive map to the field of mobile robotic planning.  A SIFT feature extraction method was used to do sampling for each position of the robot in path planning, and build a path database. Through the deviation information of the local descriptor pairs, and the state and navigation angle of the robot were determined using a series of navigation algorithm, in order to calculate the motion parameters and instruct the robot to perform navigation. Using a laser sensor for obstacle detection in the process of visual navigation, by constructing local occupied grids in robot coordinates, building an obstacle model and setting the risk probability of each feasible path, it was possible for the robot to select an appropriate navigation path in the presence of obstacles. The method is implemented in a TurtleBot robot system at driving speed 0.1 m/s for static obstacle avoidance, and the operating frequency is approximately 4Hz. The method has good robustness for static obstacle avoidance in visual navigation.  7.2 Primary Contributions The primary contributions of this thesis were to develop a biology-based method for cognitive map building and a navigation method based on episodic memory for behavior control of a mobile robot platform. Combined with the characteristics of a cognitive map and the formation mechanism of episodic memory, the method extended the application of biological cognition theory in the field of robotic navigation. In order to improve the efficiency and adaptability of the current robot visual attention models, this thesis presented a novel robot visual attention model based on robot memory with self-awareness. With the ability of introspection of its internal state and the ability to reconcile oneself as an individual that will separate itself from the environment and other individuals, a robot visual attention system with self-awareness may receive and process enormous amounts of sensory information efficiently through the understanding of both the external environment and the internal states. A method to realize a robot’s visual navigation and obstacle avoidance by using motion scene feature sampling, robot state and path database building and obstacle model 109 construction was presented. It can choose an optimal path for obstacle avoidance and then return to the planned path through environmental interaction. Physical experimentation was developed and carried out, which showed the effectiveness of the developed methodologies in applications of mobile robot system.  7.3 Limitations and Suggested Future Research Although the mobile robot system with attention and navigation methods that was proposed in the present thesis has shown good performance in experimentation, there are potential improvements which may constitute a possible future direction of research.  An integrated model of visual attention for a homecare robot was presented. An experiment was carried out to evaluate the model and to indicate how self-awareness can affect the attention selection. Further improvement of this visual attention model will lead to a better object recognition method than the one that just uses color. An improved depth image would be desirable as well.  The biology-based method of cognitive map building and navigation was implemented and evaluated through indoor experiments. Future work may focus on outdoor experiments, and more robust vision features or more sensors to enhance the robustness for performing human-scale manipulation activities although this may increase the computational load. This investigation only considered the avoidance of static obstacles. Further study, for avoiding dynamic obstacle would be useful, for example, using the Kinect depth information to detect the dynamic obstacle and develop a scheme of dynamic obstacle avoidance. The experimentation demonstrated good performance of the developed approaches. However, the robustness of the method should be further considered concerning sensor failures. Such work may include the development of methods to detect sensor failures, incorporating more sensors, or accommodating sensor failures using a software-based approach (e.g., Kalman filter). In future, an integrated robotic system with the proposed methods would be developed followed by extensive experimentation.    110 References  Achanta, R., Hemami, S., Estrada, F. and Susstrunk, S. “Frequency-tuned Salient Region Detection,” Proc. of the IEEE Int. Conf. on Computer Vision and Pattern Recognition, pp. 1597-1604, 2009.  Andel, D. and Wehner, R., “Path Integration in Desert Ants, Cataglyphis: How to Make a Homing Ant Run Away from Home,” Proceedings of the Royal Society B Biological Sciences, vol. 271, no. 1547, pp. 1485-1489, 2004.  Arkin, R. C. “Behavior-based Robotics,” MIT Press, 1998.  Arras, K. O. and Cerqui, D., “Do we want to share our lives and bodies with robots? A 2000 people survey,” Autonomous Systems Lab, Swiss Federal Institute of Technology Lausanne: Lausanne, 2005.  Aryananda, L. “Attending to Learn and Learning to Attend”. 6th IEEE-RAS International Conference on Humanoid Robots, 2006.  Baxter, P. and Browne, W. “Memory-Based Cognitive Framework: A Low-Level Association Approach to Cognitive Architectures Advances in Artificial Life,” Lecture Notes in Computer Science, pp. 5777: 402-409, 2011.  Begum, M., Karray, F., Mann, G. K. I. and Gosine, R., “A probabilistic approach for attention-based multi-modal human-robot interaction,” The IEEE International Symposium on Robot and Human Interactive Communication, 2009.   Begum, M., Karray, F., Mann, G. K. I. and Gosine, R. G. “Re-mapping of visual saliency in overt attention: A particle filter approach for robotic systems,” IEEE International Conference on Robotics and Biomimetics, 2009.  Begum, M., Karray, F., Mann, G. K. and Gosine, R. G., “A probabilistic model of overt visual attention for cognitive robots.,” IEEE Trans. on Systems, Man, and Cybernetics, Part B: Cybernetics, vol. 40, no. 5, pp. 1305, 2010.  Belkaid, M., Cuperlier, N. and Gaussier, P., “Emotional metacontrol of attention: Top-down modulation of sensorimotor processes in a robotic visual search task,” PLOS ONE, vol. 12, no. 9, pp. 184-960, 2017. 111  Birlo, M. and Tapus, A. “The crucial role of robot self-awareness in HRI,” ACM/IEEE International Conference on Human-Robot Interaction, pp. 115-116, 2011. Borji, A., Ahmadabadi, M. N., Araabi, B. N. and Hamidi, M., “Online learning of task-driven object-based visual attention control,” Image & Vision Computing, vol. 28, no. 7, pp. 1130-1145, 2010.  Borji, A. and Itti, L., “State-of-the-Art in Visual Attention Modeling,” IEEE Transactions on Pattern Analysis & Machine Intelligence, vol. 35, no. 1, pp .185, 2013.  Brooks, R. A., “From earwigs to humans,” Robotics & Autonomous Systems, vol. 20, no. 2-4, pp. 291-304, 1997.  Brown, H. C., Kim, A. and Eustice, R. M., “An Overview of Autonomous Underwater Vehicle Research and Testbed at PeRL,” Marine Technology Society Journal, vol. 43, no. 2, pp. 33-47, 2009.  Bugmann, G. and Copleston, S. N. “What can a personal robot do for you?” Conference on Towards Autonomous Robotic Systems, 2011.  Burgess, N., Maguire, E. A. and O'Keefe, J., “The human Hippocampus and spatial and episodic memory,” Neuron, vol. 35, no. 4, pp. 625, 2002.  Buss, M., Hirche, S. and Samad T., “Cognitive Control,” The Impact of Control Technology, 2011. Cai, G., Chen, B. M. and Tong, H. L., “An overview on development of miniature unmanned rotorcraft systems,” Frontiers of Electrical & Electronic Engineering in China, vol. 5, no. 1, pp. 1-14, 2010.  Buzsaki G , Moser E I. “Memory, navigation and theta rhythm in the hippocampal-entorhinal system,” Nature Neuroscience, 2013, vol. 16, no.2, pp. 130-138.  Carlos J, Broch P, Teresa M, et al., “Cognitive Maps for Mobile Robot Navigation: A Hybrid Representation Using Reference Systems,” Proceedings of the 19th International Workshop on Qualitative Reasoning, 2005.  Chang, C. K., Siagian, C. and Itti, L., “Mobile robot vision navigation & localization using Gist and Saliency,” IEEE/RSJ International Conference on Intelligent Robots and Systems, pp. 4147-4154, 2010.  112 Chatty, A., Gaussier, P., Hasnain, S. K., Kallel, I. and Alimi, A. M., “The effect of learning by imitation on a multi-robot system based on the coupling of low-level imitation strategy and online learning for cognitive map building,” Advanced  Robotics, vol. 28, no. 11, pp. 731-743, 2014.  Chatty, A., Gaussier, P., Kallel, I., Laroque, P. and Alimi, A. M., “Adaptation capability of cognitive map improves behaviors of social robots,” IEEE International Conference on Development and Learning and Epigenetic Robotics, pp. 1-6, 2012.    Cheng, H., Chen, H. and Liu, Y., “Topological Indoor Localization and Navigation for Autonomous Mobile Robot,” IEEE Transactions on Automation Science & Engineering, vol.12, no. 2, pp. 729-738, 2015.  Cherubini, A. and Chaumette, F., “Visual navigation of a mobile robot with laser-based collision avoidance,” The International Journal of Robotics Research, vol. 32, no. 2, pp. 189-205, 2013.  Cho, S. Y. and Wong, J. J. “Probabilistic Based Recursive Model for Face Recognition,” Lecture Notes in Computer Science, 2005.  Choi, B. S., Lee, J. W., Lee, J. J. and Park, K. T., “A Hierarchical Algorithm for Indoor Mobile Robot Localization Using RFID Sensor Fusion,” IEEE Trans. Ind. Electron, vol. 58, no. 6, pp. 2226-2235, 2011.  Colombini, E. L., Simões, A. D. S. and Ribeiro, C. H. C., “An Attentional Model for Autonomous Mobile Robots,” IEEE SYSTEMS JOURNAL, vol. 11, no. 3, pp. 1308-1319, 2017.  Dodd, W. and Gutierrez, R. “The role of episodic memory and emotion in a cognitive robot,” IEEE International Workshop on Robot and Human Interactive Communication, pp. 692-697, 2005.  Du, Y. A POMDP., “Approach to Robot Motion Planning under Uncertainty,” The Workshop on International Conference on Automated Planning & Scheduling, 2010.  Eichenbaum, H., Dudchenko, P., Wood, E., Shapiro, M. and Tanila, H., “The Hippocampus, memory, and place cells: is it spatial memory or a memory space?” Neuron, vol. 23, no. 2, pp. 209, 1999.  113 Elvinsand Todd, T., “VisFiles: virtually lost in virtual worlds-wayfinding without a cognitive map,” Acm Siggraph Computer Graphics, vol. 31, no. 3, pp. 15-17, 1997.  Endo, Y., “Anticipatory robot control for a partially observable environment using episodic memories,” IEEE International Conference on Robotics and Automation, pp. 2852–285, 2007.  Endo, Y. and Arkin, R. C., “Anticipatory robot navigation by simultaneously localizing and building a cognitive map.,” IEEE/RSJ International Conference on Intelligent Robots and Systems, vol. 1, pp. 460-466, 2003.  Etienne, A. S. and Jeffery, K. J., “Path integration in mammals,” Hippocampus, vol. 14, no. 2, pp. 180, 2004.  Fava, A. D., Satler, M. and Tripicchio, P., “Visual navigation of mobile robots for autonomous patrolling of indoor and outdoor areas,” Control and Automation, 2015.  Feng, G., He, Y. and Han, J., “Active Persistent Localization of a Three-Dimensional Moving Target Under Set-Membership Uncertainty Description Through Cooperation of Multiple Mobile Robots,” IEEE Trans. Ind. Electron., vol. 62, no. 8, pp. 4958-4971, 2015.  Fortin, N. J., Agster, K. L. and Eichenbaum, H. B., “Critical role of the Hippocampus in memory for sequences of events,” Nature NeuroScience, vol. 5, no. 5, pp. 458, 2002.  Frintrop, S., “VOCUS: A Visual Attention System for Object Detection and Goal-directed Search,” Lecture Notes in Artificial Intelligence, vol. 3899, pp. 1-5, 2006.  Fyhn, M., Molden, S., Witter, M. P., Moser, E. I. and Moser, M. B., “Spatial Representation in the Entorhinal Cortex,” Science, vol. 305, no. 5688, pp. 1258-1264, 2004.  Gaspar, J., Winters, N. and Santosvictor, J. “Vision-based Navigation and Environmental Representations with an Omni-d-d Camera,” IEEE Transactions on Robotics and Automation. pp. 890-898, 2000.  Gopalan, N. and Tellex, S., “Modeling and solving human-robot collaborative tasks using POMDPs,” Proc. Robot., Sci. Syst., Rome, Italy, 2015.  114 Gorbenko, A., Popov, V. and Sheka, A., “Robot self-awareness: Exploration of internal states,” Applied Mathematical Sciences, no. 13-16, pp. 675-688, 2012.  Guerrero, P., Solar, J. R. D., Romero, M. and Herrera, L., “An integrated multi-agent decision making framework for robot soccer,” Robotics Symposium (LARS), Latin, USA, pp.1-6, 2009.  Gupta, S., Fouhey, D., Levine, S. and Malik, J., “Unifying Map and Landmark Based Representations for Visual Navigation,”  arXiv Paper Daily, UC Berkeley, 2017.  Hafting, T., Fyhn, M., Molden, S., Moser, M. B. and Moser, E. I., “Microstructure of a spatial map in the entorhinal cortex.,” Nature, vol. 436, no. 7052, pp. 801, 2005.  Han, B. Tcheang, L., Walsh, V. and Gao, X., “A novel feature combination method for saliency-based visual attention,” International Conference on Natural Computation, pp. 18-22, 2009. Itti, L., Koch, C. and Niebur, E., “A Model of Saliency-Based Visual Attention for Rapid Scene Analysis,” IEEE Transactions on Pattern Analysis & Machine Intelligence, vol. 20, no. 11, pp. 1254-1259, 1998. Itti, L. and Baldi, P., “Bayesian surprise attracts human attention,” Vision research, vol. 49, pp. 1295-1306, 2009. Itti, L. and Koch, C., “A saliency-based search mechanism for overt and covert shifts of visual attention,” Vision Research, vol. 40, pp. 1489-1506, 2000. Jockel, S., Lindner, F. and Zhang, J. “Sparse distributed memory for experience-based robot manipulation,” IEEE International Conference on Robotics and Biomimetics, pp. 1298-1303, 2009. Jockel, S., Westhoff, D. and Zhang, J. “EPIROME-A novel framework to investigate high-level episodic robot memory,” IEEE International Conference on Robotics and Biomimetics, pp. 1075-1080, 2007 Kawamura, K. and Ratanaswasd, P., “Implementation of cognitive control in a humanoid robot,” International Journal of Humanoid Robotics, vol. 5, no. 4, 2008. Kawamura, K., Dodd, W., Ratanaswasd, P. and Gutierrez, R. A., “Development of a robot with a sense of self,” IEEE International Symposium on Computational Intelligence in Robotics and Automation, Cira 2005, pp. 211-217, 2005. 115 Kawamura, K., Gordon, S. M., Ratanaswasd, P., Erdemir, E. and Hall, J. F., “Implementation of cognitive control for a humanoid robot,” International Journal of Humanoid Robotics, vol. 5, no. 4, pp. 547-586, 2008. Kelley, T. D., “Robotic Dreams: A Computational Justification for the Post-Hoc Processing of Episodic Memories,” International Journal of Machine Consciousness, vol. 06, no. 2, pp. 109-123, 2014. Khamassi, M., Lallée, S., Enel, P., Procyk, E. and Dominey, P. F., “Robot cognitive control with an eurephysiologically inspired reinforcement learning model,” Frontiers in neurorobotics, vol. 5, no. 1, pp. 1, 2011. Kidono, K., Miura, J. and Shirai, Y., “Autonomous visual navigation of a mobile robot using a human-guided experience,” Robotics & Autonomous Systems, vol. 40, no. 2, pp. 121-130, 2002. Kim, H. S., Seo, W. and Baek, K. R., “Indoor Positioning System Using Magnetic Field Map Navigation and an Encoder System,” Sensors, vol. 17, no. 3, pp. 651, 2017. Kim, Y., Velsen, M. V. and Jr, R. W. H., “Modeling Dynamic Perceptual Attention in Complex Virtual Environments,” International Workshop on Intelligent Virtual Agents, pp. 266-277, 2005. Kleinmann, L. and Mertsching, B., “Learning to adapt: Cognitive architecture based on biologically inspired memory,” Industrial Electronics and Applications, pp. 936-941, 2011 Kobayashi T, Tran A H, Nishijo H, et al., “Contribution of hippocampal place cell activity to learning and formation of goal-directed navigation in rats,” Neuroscience, 2003, vol. 117, no. 4, pp. 1025-1035. Kurniawati, H., Du, Y., Hsu, D. and Lee, W. S., “Motion Planning under Uncertainty for Robotic Tasks with Long Time Horizons,” Int. J. Robot, vol. 30, no. 3, pp. 308-323, 2011. Lategahn, H., Geiger, A. and Kitt, B., “Visual SLAM for autonomous ground vehicles,” IEEE International Conference on Robotics and Automation, pp. 1732-1737, 2011. Leconte, F., Ferland, F. and Michaud, F. “Fusion Adaptive Resonance Theory Networks Used as Episodic Memory for an Autonomous Robot,” International Conference on Artificial General Intelligence, pp. 63-72, 2014. Lee, T. S. and Mumford, D., “Hierarchical Bayesian inference in the visual cortex,” JOSA A, vol. 20, no. 7, pp. 1434-1448, 2003. 116 Lewis, P. R., Chandra, A. and Parsons, S. et al, “A Survey of Self-Awareness and Its Application in Computing Systems,” Fifth IEEE Conference on Self-Adaptive and Self-Organizing Systems Workshops, pp. 102-107, 2011 Liben, L. S., Patterson, A. H. and Newcombe, N., “Spatial representation and behavior across the life span,” Womens Health Issues, vol. 11, no. 3, pp. 148-159, 1981. Lin, L. N., Osan, R. Shoham, S. et al, “Discovery and identification for the functional units of real-time encoding on episodes’ experience of mouse hippocampal neural networks,” Journal of East China Normal University: Natural Science, no. 5, pp. 208-216, 2005. LinWang, WeiminMou and XianghongSun, “Development of Landmark Knowledge at Decision Points,” Spatial Cognition & Computation, vol. 14, no. 1, pp. 1-17, 2014. Liu, D., Cong, M., Du, Y., Zou, Q. and Cui, Y., “Robotic autonomous behavior selection using episodic memory and attention system,” Industrial Robot, vol. 44, no. 3, pp. 353-362, 2017. Liu, D., Cong, M., Gao, S., Han, X. and Du, Y., “Robotic episodic learning and behavior control integrated with Neuron stimulation mechanism,” Jiqiren/robot, vol. 36, no. 5, pp. 576-583, 2014. Lowe, D. G., “Lowe, D.G.: Distinctive Image Features from Scale-Invariant Key-points. Int. J. Comput. Vision 60(2), 91-110,” International Journal of Computer Vision, vol. 60, no. 2, 2004. Lowry, S., Sünderhauf, N. and Newman, P. et al., “Visual Place Recognition: A Survey,” IEEE  Transactions  on  Robotics, vol. 32, no. 1, pp. 1-19, 2016. Maguire, E. A., Frackowiak, R. S. and Frith, C. D., “Recalling routes around london: activation of the right Hippocampus in taxi drivers,” Journal of NeuroScience the Official Journal of the Society for NeuroScience, vol. 17, no. 18, pp. 7103, 1997. Mallot, H. A., Bulthoff, H. H., Scholkopf, B. and Yasuhara, K., “View--Based Cognitive Map Learning by an Autonomous Robot,” ICANN, Paris, France, vol. 2, pp. 381-386, 1995. Mcnaughton, B. L., Battaglia, F. P., Jensen, O., Moser, E. I. and Moser, M. B., “Path integration and the neural basis of the ‘cognitive map’,” Nature Reviews Neuroscience, vol. 7, no. 8, pp. 663-678, 2006. Mcnaughton, B. L., Knierim, J. J. and Wilson, M. A., “Vector encoding and the vestibular foundations of spatial cognition: Neurophysiological and computational mechanisms,” 1993. 117 Menegatti, E., Maeda, T. and Ishiguro, H., “Image-based memory for robot navigation using properties of omnidirectional images,” Robotics & Autonomous Systems, vol. 47, no. 4, pp. 251-267, 2004. Michel, P., Gold, K. and Scassellati, B., “Motion-based robotic self-recognition,” IEEE/RSJ International Conference on Intelligent Robots and Systems, pp. 2763-2768, 2004. Milford, M. and Wyeth, G., “Persistent Navigation and Mapping using a Biologically Inspired SLAM System,” The International Journal of Robotics Research, vol. 29, no. 9, pp. 1131-1153, 2010. Milford, M. J. and Wyeth, G. F., “Mapping a Suburb with a Single Camera Using a Biologically Inspired SLAM System,” IEEE Transactions on Robotics, vol. 24, no. 5, pp. 1038-1053, 2008. Milford, M. J., Wyeth, G. F. and Prasser, D., “RatSLAM: a hippocampal model for simultaneous localization and mapping,” IEEE International Conference on Robotics and Automation, Proceedings, ICRA, pp. 403-408, 2004. Milford, M., Schulz, R., Prasser, D., Wyeth, G. and Wiles, J., "Learning spatial concepts from RatSLAM representations," Robotics & Autonomous Systems, vol. 55, no. 5, pp. 403-410, 2007. Mittelstaedt, M. L. and Mittelstaedt, H., “Homing by path integration in a mammal,” Naturwissenschaften, vol. 67, no. 11, pp. 566-567, 1980. Morbini, F., “A Self-Aware Agent,” Dissertations & Theses - Gradworks, 2009. Navalpakkam, V. and Itti, L., “Top-down attention selection is fine grained.,” Journal of vision, vol. 6, no. 11, pp. 1180-1193, 2006. Navratilova, Z., “The role of path integration on neural activity in Hippocampus and medial entorhinal cortex,” Dissertations & Theses - Gradworks, 2012. Nieto, J., Guivant, J., Nebot, E. and Thrun, S. “Real time data association for FastSLAM,” Robotics and Automation, 2003. Proceedings. ICRA '03. IEEE International Conference on, pp. 412-418, 2003. Nuxoll, A. and Laird, J. E., “A Cognitive Model of Episodic Memory Integrated with a General Cognitive Architecture,” International Conference on Cognitive Modelling, pp. 220-225, 2004. O'Keefe, J. and Dostrovsky, J., “The Hippocampus as a spatial map: preliminary evidence from unit activity in the freely-moving rat,” Brain Research, vol. 34, no. 1, pp. 171-175, 1971. Passingham, R. E., “The Hippocampus as a cognitive map,” Neuroscience, vol. 4, no. 6, pp. 863, 1979. 118 Petit, M, Fischer, T, and Demiris Y, “lifelong augmentation of multimodal streaming autobiographical memories,” IEEE Transactions on Cognitive and Developmental Systems, vol. 8, no. 3, pp. 201-213, 2016. Proetzsch, M., Luksch, T. and Berns, K., “Development of complex robotic systems using the behavior-based control architecture iB2C,” Robotics & Autonomous Systems, vol. 58, no. 1, pp.46-67, 2010. Rebai, K., Azouaoui, O. and Achour, N. “Bio-inspired visual memory for robot cognitive map building and scene recognition,” IEEE/RSJ International Conference on Intelligent Robots and Systems, pp. 2985-2990, 2012. Redish, A. D., Elga, A. N. and Touretzky, D. S., “A coupled attractor model of the rodent head direction system,” Network Computation in Neural Systems, vol. 7, no. 4, pp. 671-685, 1996. Richter, C. and Roy, N. “Safe Visual Navigation via Deep Learning and Novelty Detection,” Robotics: Science and Systems, 2017. Richter, M., Sandamirskaya, Y., and Schoner, G. “A robotic architecture for action selection and behavioral organization inspired by human cognition,” IEEE/RSJ International Conference on Intelligent Robots and Systems, 2012. Ruesch, J., Lopes, M., Bernardino, A.and Hornstein, J. “Multimodal saliency-based bottom-up attention a framework for the humanoid robot iCub,” IEEE International Conference on Robotics and Automation, 2008. Sang Hyoung Lee and Il Hong Suh, “Motivation-Based Dependable Behavior Selection Using Probabilistic Affordance,” Advanced Robotics, vol. 26, no. 8-9, pp. 897-921, 2012. Shapiro, M. L. and Eichenbaum, H., “Hippocampus as a memory map: Synaptic plasticity and memory encoding by hippocampal Neurons,” Hippocampus, vol. 9, no. 4, pp. 365, 1999. Shim, V. A., Tian, B., Yuan, M., Tang, H. and Li, H. “Direction-driven navigation using cognitive map for mobile robots,” IEEE/RSJ International Conference on Intelligent Robots and Systems, pp. 2639-2646, 2014. Siagian, C., Chang, C. K. and Itti, L. “Mobile robot navigation system in outdoor pedestrian environment using vision-based road recognition,” IEEE International Conference on Robotics and Automation, pp. 564-571, 2013. 119 Sim, R., Elinas, P., Griffin, M., Shyr, A. and Little, J. J. “Design and analysis of a framework for real-time vision-based SLAM using Rao-Blackwellised particle filters,” Proc. of the 3rd Canadian Conference on Computer and Robotic Vision, 2006. Smith, A. D., Gilchrist, I. D. and Cater, K. et al., “Reorientation in the real world: the development of landmark use and integration in a natural environment,” Cognition, vol. 107, no. 3, pp. 1102, 2008. Srinivasa, S. S., Berenson, D. and Cakmak, M. et al., “Herb 2.0: Lessons Learned from Developing a Mobile Manipulator for the Home,” Proceedings of the IEEE, vol. 100, no. 8, pp. 2410-2428, 2012. Stachowicz, D. and Kruijff, G. J. M., “Episodic-Like Memory for Cognitive Robots,” IEEE Transactions on Autonomous Mental Development, vol. 4, no. 1, pp. 1-16, 2012. Stringer, S. M., Rolls, E. T., Trappenberg, T. P. and de Araujo, I. E., “Self-organizing continuous attractor networks and path integration: two-dimensional models of place cells,” Network Computation in Neural Systems, vol. 13, no. 4, pp. 429-446, 2002. Thrun, S. and Bucken, A., “Learning Maps for Indoor Mobile Robot Navigation,” Artificial Intelligence, vol. 99, pp. 21-71, 1996. Tian, B., Shim, V. A. and Yuan, M. et al. “RGB-D based cognitive map building and navigation,” IEEE/RSJ International Conference on Intelligent Robots and Systems, pp. 1562-1567, 2014. Tolman, E. C., “Cognitive maps in rats and men.,” Psychological Review, vol. 55, no. 4, pp. 189, 1948. Tulving, E., “Episodic and Semantic Memory,” Organization of Memory, pp. 381-403, 1972. Tulving, E., Kapur, S., Craik, F. I., Moscovitch, M. and Houle, S., “Hemispheric encoding/retrieval asymmetry in episodic memory: positron emission tomography findings,” Proceedings of the National Academy of Sciences of the United Stated of America, vol. 91, no. 6, pp. 2016-2020, 1994. Walker, A., Hallam, J. and Willshaw, D., “Bee-havior in a mobile robot: the construction of a self-organized cognitive map and its use in robot navigation within a complex, natural environment,” IEEE International Conference on Neural Networks, pp. 1451-1456, 1993. Wei, J., Dolan, J. M., Snider, J. M. and Litkouhi, B., “A point-based MDP for robust single-lane autonomous driving behavior under uncertainties,” IEEE International Conference on Robotics and Automation, pp. 2586-2592, 2012. 120 Wirth, S., Yanike, M. and Frank, L. M. et al., “Single Neurons in the Monkey Hippocampus and Learning of New Associations,” Science, vol. 300, no. 5625, pp. 1578-1581, 2003. Wolfe, J. M., “Guided Search 2.0 A revised model of visual search,” Psychonomic Bulletin & Review, vol. 1, no. 2, pp. 202-238, 1994. Wong, J. J. and Cho, S. Y. “Facial emotion recognition by adaptive processing of tree structures,” ACM Symposium on Applied Computing, pp. 23-30, 2006. Xu, W., Wei, J., Dolan, J. M., Zhao, H. and Zha, H., “A real-time motion planner with trajectory optimization for autonomous vehicles,” IEEE International Conference on Robotics and Automation, pp. 2061-2067, 2012. Yan, W., Weber, C. and Wermter, S. “A neural approach for robot navigation based on cognitive map learning,” International Joint Conference on Neural Networks, pp. 8, 2012. Yanrong, J., Nanfeng and X., "A selective attention-based contextual perception approach for a humanoid robot," Journal of Control Theory & Applications, vol. 5, no. 3, pp. 244-252, 2007. Youngstrom, I. A. and Strowbridge, B. W., “Visual landmarks facilitate rodent spatial navigation in virtual reality environments,” Learning & Memory, vol. 19, no. 3, pp. 84, 2012. Yu, Y., Mann, G. K. I.and Gosine, R. G., “An Object-Based Visual Attention Model for Robotic Applications,” IEEE Trans. on Systems, Man, and Cybernetics, Part B: Cybernetics, vol. 40, no. 5, pp. 1398-1411, 2010. Z W Wang, “Research on behavior learning for intelligent robot with cognitive ability,” Ph.D. dissertation, School of computer Science and technology, Harbin Engineering University, 2010. Zhang, L., Tong, M. H., Marks, T. K., Shan, H. and Cottrell, G. W., “SUN: A Bayesian framework for saliency using natural statistics,” Journal of vision, vol. 8, pp. 32, 2008. Zhao, D B, Chen, Y R, Lv L, “Deep reinforcement learning with visual attention for vehicle classification”, IEEE Transactions on Cognitive and Developmental Systems, vol. 9, no. 4, pp. 356-367, 2017. Zhong, M. and Huang, B., “A MEMS-Based Inertial Navigation System for Mobile Miniature Robots,” Advanced Materials Research, vol. 383-390, pp. 7189-7197, 2011. 121 Zhu, Y., Sun, C., Han, Z. and Yu, C., “A visual navigation algorithm for mobile robot in semi-structured environment.,” IEEE International Conference on Computer Science and Automation Engineering, pp. 716-721, 2011.  

Cite

Citation Scheme:

        

Citations by CSL (citeproc-js)

Usage Statistics

Share

Embed

Customize your widget with the following options, then copy and paste the code below into the HTML of your page to embed this item in your website.
                        
                            <div id="ubcOpenCollectionsWidgetDisplay">
                            <script id="ubcOpenCollectionsWidget"
                            src="{[{embed.src}]}"
                            data-item="{[{embed.item}]}"
                            data-collection="{[{embed.collection}]}"
                            data-metadata="{[{embed.showMetadata}]}"
                            data-width="{[{embed.width}]}"
                            async >
                            </script>
                            </div>
                        
                    
IIIF logo Our image viewer uses the IIIF 2.0 standard. To load this item in other compatible viewers, use this url:
https://iiif.library.ubc.ca/presentation/dsp.24.1-0365999/manifest

Comment

Related Items