International Construction Specialty Conference of the Canadian Society for Civil Engineering (ICSC) (5th : 2015)

Maintaining vertical gardens using quadrotor aerial inspection Bulgakov, Alexey; Emelianov, Sergey; Schach, Rainer; Sayfeddine, Daher; Erofeev, Vladimir Jun 30, 2015

Your browser doesn't seem to have a PDF viewer, please download the PDF to view this item.

Item Metadata

Download

Media
52660-Bulgakov_A_et_al_ICSC15_067_Maintaining_Vertical_Gardens.pdf [ 591.13kB ]
52660-Bulgakov_A_et_al_ICSC15_067_Maintaining_Vertical_Gardens_slides.pdf [ 2.85MB ]
Metadata
JSON: 52660-1.0076460.json
JSON-LD: 52660-1.0076460-ld.json
RDF/XML (Pretty): 52660-1.0076460-rdf.xml
RDF/JSON: 52660-1.0076460-rdf.json
Turtle: 52660-1.0076460-turtle.txt
N-Triples: 52660-1.0076460-rdf-ntriples.txt
Original Record: 52660-1.0076460-source.json
Full Text
52660-1.0076460-fulltext.txt
Citation
52660-1.0076460.ris

Full Text

5th International/11th Construction Specialty Conference 5e International/11e Conférence spécialisée sur la construction    Vancouver, British Columbia June 8 to June 10, 2015 / 8 juin au 10 juin 2015   MAINTAINING VERTICAL GARDENS USING QUADROTOR AERIAL INSPECTION Alexey Bulgakov1,5, Sergey Emelianov1, Rainer Schach2, Daher Sayfeddine3, Vladimir Erofeev4  1 South West State University, Russian Federation 2 Technical University Dresden, Germany 3 South Russian State Polytechnic University, Russian Federation 4 State University Mordovia, Russian Federation 5 a.bulgakow@gmx.de Abstract: The upkeep of the building requires continuous monitoring to maintain the safety aspects as well as aesthetic elements of the building including facades, marble constructions and gardens.  With the decrease of the amount of green areas, human race will face climatic catastrophes, which will affect the quality, type and life longitude of the Earth habitants. To tackle this issue, vertical gardens were designed to fit in the overwhelming world of concrete and to adapt in order to do photosynthesis. The growth of the plants is supported by spraying insecticide periodically. In this paper, we offer to use an unmanned aerial vehicle, to recognize the targeted green areas using HSV algorithm, track the generated trajectory and perform the spraying process all along the recognized area. We believe this will automate the maintenance of the gardens, improve the effectiveness of the treatment and assist to the ecologic factor, by applying the necessary amount of chemicals on the targeted place only.       1 INRODUCTION Recently unmanned aerial vehicles are used to perform several civil tasks. Particularly, in construction fields, the idea of aerial robot is directly related to photogrammetry, site scanning and buildings status. In this regard, many literatures supported the cost efficiency, safety and quality control for such technological implementation (Bulgakov A. et al. 2014;  Emelianov S. et al. 2014; Rango,  et al. 2009.  Chao, H et al. Shang et al. 2010) Similarly, aerial robots can be used to scan vertical gardens and improve the irrigation system by controlling the quantity of water supplied to a certain site. This can be of great importance when talking about large agricultural areas, mega-polis with limited greenery and quantifying forest fire impact.  The technology being used to sense is based on satellite inspection. The limitation of such technology consists of the low resolution and the update ratio of the images provided. On the other hand, UAVs are more cost effective, can reach difficult scanning corners and provide more accurate visual information, as they can be equipped with high definition cameras. In this paper, we offer to use a vertical take off and landing aerial robot to perform visual inspection for vertical gardens.  067-1 2 RELATED WORKS UAV’s successful results in improving military tasks have pushed the developers of such technology to try it in the civil market. The verdict: we got a miniature class of rotorcraft assisting in fire fighting, cinematography, television broadcasting and so on. The technology is being used to improve irrigation systems (Chao H, et al.), to address the climatic changes in the north and south poles (Lucieer A. et al. 2012), inspection of bridges and facades  (Bulgakov A. et al. 2014;  Emelianov S. et al. 2014). As a sensing technology, band -reconfigurable fixed wing UAV were used more frequently. This is due to the optimized flight range, lifting capability and improvement in Micro (MEMS) and Nano electromechanical systems (NEMS). The sensors used are based on thermal infrared and machine vision. But these sensing technologies have limitations, especially the thermal infrared: the cost involved and the reliability of the acquired data especially in a humid atmosphere (Lucieer A. et al. 2012). The other limitation is the UAV itself: stability problem leads to false reading. Having analyzed the pros and cons of aerial inspection, the optimization problem can be divided into two subtasks: 1. Coverage control problem consisting of path planning and control of the UAV; 2. Geo-reference problem consists of registration of each acquired pixel from the aerial images including temporal and spatial information (time and location). Both of the subtasks can be addressed in either of  two ways: global path planning consisting of trajectory planning before the flight and local planning that generates the trajectory during flight. 3 AIM OF THE PAPER The aim of the paper is to address the aforementioned subtasks in order to inspect vertical gardens in the cities using quadrotors. As it is noticeable in most of the mega polis in the world, where green areas are lacking. The uncontrollable population growth is destroying the remaining of “mother nature”. Engineers and social workers have invented the concept of green wall or vertical gardens. The idea, as shown is figure 1, consists of planting wall climbing trees and plants on vertical walls. Due to the ergonomics involved the idea is widely spread.  Figure 1: Vertical garden concept The role of the quadrotor is to acquire visual information such as images and pictures and spray the insecticide on the target area.  3.1 Coverage Control Problem This subtask is divided into two points: path planning and control. Since, for the geo-reference problem, we will be using GPS data to acquire the temporal and spatial information, we will limit ourselves with the control part of it. As it is known, a quadrotor is a take off and landing rotorcraft, having six degrees of 067-2 freedom. But this miniature aircraft can perform only four flight regimes: roll, pitch, yaw and hover. This mechanical limitation makes the quadrotor nonlinear system. The dynamics of the quadrotor can be described as follows: [1] ; [2]  [3] ;  [4]   [5]   [6] .  Where,  ,  and  are the projection of the linear acceleration of the quadrotor in the Earth fixed axis,  ,  and  are projection rotational acceleration of the quadrotor in the body fixed axis,  the gravitational acceleration,   the torque generated from the rotors,  the mass of the quadrotor, ,  and   are the projection of the Inertia of the quadrotor,  is the rotational speed of the propellers, ,    and  are the roll, pitch and yaw angle consequently, , ,  and  are the aforementioned flight regimes.  3.2 Fuzzy Logic Position Controllers  As it is important to know the value of the error, it is similarly critical to understand how it is changing over time. The error and its derivative in time are one of the possibilities to configure a Fuzzy logic controller. It is an artificial intelligence approach that computes mathematical operations based on degree of truth rather than the conventional True-False Boolean logic. The Fuzzy logic allows having more adaptable controller specially when dealing with nonlinearities (i.e. nonlinear aerodynamic model of the quadrotor as described in equations 1-6) and uncertainties (i.e. permanent changes in the flight circumstances, wind, temperature, obstacles positions etc.). Although in many cases, heuristic algorithms were ruled out, due to the time consumption, we saw a need to include this algorithm in our survey to cover most of the techniques used. We would like to clarify that the term heuristic is used to describe the estimation ability in artificial intelligence.   Moreover, the linguistic power of the fuzzy logic may be extremely useful while navigating in totally unknown areas.  The trick in such controller is how to set up the linguistic rules for input and output.  We fine tuned the rules by using the resulting graph (figure 2) for the function de(t)=f(e(t)), where e(t) is the deviation in position of the quadrotor with reference to the desired trajectory and de(t) is the variation of the deviation.   Figure 2: Resulting Graph de(t)= f(e(t)) 067-3 Using the graph in figure 2, we assign the following labels for the numeric values: Negative big (NB) =-1, Negative Small (NS) = -0.5, Zero (Z)=0, Positive Small (PS)= 0.5 and Positive Big (PB) =1. As a result we obtain the linguistic rules listed in table (1). A linguistic rule defines the output of the fuzzy controller based on discrete logic (i.e. if e (t)= NB and d(e(t)= PB, then the output is Z). the graphical representation of the linguistic rules is illustrated in figure 3.   Table 1: Fuzzy rules         de   e NB NS Z PS PB NB NB NS NB NB Z NS NS  NS NS Z  NS Z NB NS Z PS PB PS NS Z PS PS PS PB Z PB PB PB PB   Figure 3: Graphical Representation of the linguistic rules The inputs of the fuzzy controller de(t) and e(t) are represented using triangular membership functions as shown in figure 4 a and 4b. The output of the fuzzy control is also 5 triangular membership functions (figure 4c).  Figure 4 a: membership function for input “e(t)” 067-4  Figure 4 b: membership function for input “de(t)”  Figure 4 c : membership function for fuzzy regulator output The results of controlling the quadrotor using fuzzy logic regulators are shown in figure 5a and b for position and rotation control consequently.  Figure 5a: Quadrotor position control with reference to GPS waypoints. Horizontal axis – time [s], vertical axis- position [m]  067-5  Figure 5b: Quadrotor rotation control with reference to obtained position coordinates in 3a. Horizontal axis – time [s], vertical axis- orientation [degree/s] The results shown in figure 5a and 5b give a fair idea about the reliability of the fuzzy controller. For instance, figure 5a illustrates the fact of stabilizing the linear movement of the quadrotor (described in equations 1-3, X (red curve), Y (blue curve), Z (green curve)) without overshooting. This allows the quadrotor to move along Earth-axis OXYZ smoothly. The other advantage of avoiding overshooting is contributing to better energy-efficient control systems (i.e. overshooting in altitude control means flying higher and consuming more power).  Taking into consideration the nonlinearity of the aerodynamic model of the quadrotor, linear movement cannot be achieved unless stabilizing Euler angles (roll (red curve), pitch (blue curve) and yaw angles (green curve)) as shown is figure 5b. Control signals are generated in order to increase or decrease Euler angles (equation 4-6) with reference to linear position control task (equations 1-3).    3.3 Geo-reference Identification Problem  Identification of the vertical garden can be done using the green pattern. Many machine vision algorithms are dedicated to recognize colors. We will be using the HSV algorithm due to its mathematical model, which is more reliable than the other methods. The HSV consists of giving a certain color 3 coordinates: Hue, saturation and value as shown in picture 4. HSV is used is many literatures in order to detect static and dynamic agents/objects based on histogram criteria, such as recognition of traffic lights (Yixin Chen et Al. 2013), gesture identification (Mokhtar M. Mohsen et al. 2010), image and structure retrieval (Durgesh N. et al. 2013) and face detection (Naresh Kumar R. et al. 2014,Sayfeddine D. 2013).     Figure 6: HSV shade representation 067-6 From figure 6, we can see clearly that the green color is varying between 60 and 180 degrees. Hence, the role of the algorithm is to identify the green patches on the wall and apply the insecticide on the recognized area. This can be done by dividing the frame of the camera mounted on the quadrotor into 9 identical cells. Hereafter the registered pixel coordinates will be modified to metric using the following equations: [7]    [8]  [9]  [10]  [11]  [12]  Where,  and  the projection of the altitude on the pixel coordinates,  is the flying altitude,  and  are the horizontal and vertical pixel coordinates of the recognized area,  and  are the scale factors,  and  are the rotation of the quadrotor in the pixel axis,  and  are the rotation of the diaphragm of the camera from the body axis. After identification of the desired pixels based on color criteria, Cartesian subtraction is computed to identify the distance between the center of the fixed camera and the centroid of the detected colored surface. The subtraction result represents the distance that the quadrotor has to fly in pixels. The following approach is known as visual odometry. Depending on the position of the centroid, metric control task is generated for flight altitude stabilization (equation 7 and 8), linear movement (equations 9 and 10) and rotational movement (equation 11 and 12).   Using the concept of reactive control and based on the results obtained in equations (8,9 and 11,12), control signals are generated to initialize one of the four available flight regimes described in equations (3-6). 4 REAL-TIME RESULTS   HSV and visual odometry algorithms were tested in real-time in order to trace the possibility of their implementation on high-rise structures covered with vertical gardens. As figure 7 shows, the identification process was successful and the generation of the pixel coordinates was obtained. Figure 7 can be interpreted as follows; the green surface is detected by algorithm HSV (i.e. position Middle-Middle in the 9-cells frame). The green pixels are generated (i.e.  = 69, ). The Cartesian subtraction is obtained and metric control task is formed. The graph is figure 7 shows how the quadrotor changes its position (X (green curve), Y (purple curve)) with the changes of the pixel coordinates  (blue curve) and  (red curve).  A comparison between real time two dimensional flight results (figure 7) and simulation results (figures 5a/5b) shows that the deviation in sensors readings is as follows: roll reading (0.045°/s); yaw reading (0.125°/s); pitch reading  (0.15°/s). The reasons of such deviations can be caused by the Euler angles MEMS gyroscopes, asymmetric structure of the quadrotor, influence of shifted center of gravity on the control systems, and propellers non-uniformity. The obtained deviation values lay in the tolerance interval and allowed Metrologic norms (deviation of real-time results ±5% from the simulation results).     067-7  Figure 7: Trajectory tracking 5 CONCLUSIONS AND FUTURE WORK The paper focused on how to use the quadrotor to maintain vertical gardens on high-rise structures. The adopted concept was to divide the autonomous flight problem into control of the quadrotor over an identified area using HSV algorithm. The control was performed using fuzzy logic position regulators based on reactive control concept interconnecting the rotation speed of the propellers with the positioning of the quadrotor. No overshooting was registered in the simulation results. Regarding the identification of the green area, HSV algorithm was used. The pixel coordinates were modified to metric in order to create flight mission. The identification and control algorithms were tested in real time using a commercial quadrotor (AR DRONE) linked to a stationary computer using datagram protocol UDP through a Wi-Fi connection. Although the results were outstanding and the computational power was enough to drive the quadrotor to the end of the experience, we have noticed the influence of the light nature and temperature on identifying the green color. Having just the color as tracking criteria, the stationary computer could run into local minimum problem (between green and blue). This issue was addressed earlier as extrapolation of motion function. A solution was suggested based on nonlinear neural network model with exogenous input (NARX). The results can be accessed in the following literatures (Sayfeddine D. 2014, Sayfeddine D. et al. 2014). References Bulgakov, A., Emelianov, S., Bock, T. and Sayfeddine, D. 2014. Control of hovering altitude of a      quadrotor with shifted center of gravity for inspection of high-rise structures. ISARC, Sydney, Australia:      762-768. Chao, H. and al.  Band reconfigurable multi- UAV based cooperative remote sensing for real time water      management and disturbed irrigation control. 9 pages. Durgesh Nirapure, Udaypal Reddy. 2013. Fast retrieval of images using filtered HSV color level detection.      International journal of Emerging Technologies and Advanced Engineering. Volume 3, issue 8: 414-     419. Emelianov, S., Bulgakov A. and Sayfeddine, D. 2014. Aerial laser inspection of building facades using      quadrotor. Proceedia of Creative construction. CC2014. Prague, Czech Republic: 24-30. Lucieer, A. and al. 2012. Using a micro UAV for ultra high resolution multi sensor observations of      Antarctic moss beds. International Archives of the photogrammetry, remote sensing and spatial      information science. Volume XXXIX-B1, 2012: 429-433. Mokhtar M. Mohsen, Pramod K. Misra. 2011. HSV brightness factor matching for gesture recognition      system. International Journal of Image Processing (IJIP), Volume (4): issue 5: 456-467. Naresh Kumar R., P. Varum Varma, P. Surya Abhilash. 2014. Face detection using HSV color space and      minimal distance maximum variation approach. International Journal of Electrical, Electronics and Data      Communication, Volume 2, issue 4: 8-12. 067-8 Rango, A. and al. 2009. Unmanned aerial vehicle based remote sensing for rangeland assessment,       monitoring and management. Journal of Applied Remote sensing, 4: 033452. Sayfeddine D. 2013. Face identification and tracking using quadrotor. Translated from Russian: Control       problems, processing and data transmission (ATM 2013) : 178-180. Sayfeddine, D. 2014. Nonlinear autoregressive neural network with exogenous inputs based solution for       local minimum problem of agent tracking using quadrotor. Electronic scientific journal. Engineering       Journal of Don. 2: http://www.ivdon.ru/en/magazine/archive/n2y2014/2355 Sayfeddine, D. and al. 2014. Position prediction of a dynamic agent tracked by quadrotor using neural       networks. Electronic scientific journal. Engineering Journal of Don. 1:       http://www.ivdon.ru/en/magazine/archive/n1y2014/2293   Shang, H. and al. 2010. Low cost UAV based thermal infrared remote sensing: platform, calibration and       application. IEEE: 38-43. Yixin Chen, Yi Xie and Yulin Wang. 2013. Detection and recognition of traffic signs based on HSV vision       model and shape features. Journal of computers, Vol.8., No.5: 1366-1370.  067-9  5th International/11th Construction Specialty Conference 5e International/11e Conférence spécialisée sur la construction    Vancouver, British Columbia June 8 to June 10, 2015 / 8 juin au 10 juin 2015   MAINTAINING VERTICAL GARDENS USING QUADROTOR AERIAL INSPECTION Alexey Bulgakov1,5, Sergey Emelianov1, Rainer Schach2, Daher Sayfeddine3, Vladimir Erofeev4  1 South West State University, Russian Federation 2 Technical University Dresden, Germany 3 South Russian State Polytechnic University, Russian Federation 4 State University Mordovia, Russian Federation 5 a.bulgakow@gmx.de Abstract: The upkeep of the building requires continuous monitoring to maintain the safety aspects as well as aesthetic elements of the building including facades, marble constructions and gardens.  With the decrease of the amount of green areas, human race will face climatic catastrophes, which will affect the quality, type and life longitude of the Earth habitants. To tackle this issue, vertical gardens were designed to fit in the overwhelming world of concrete and to adapt in order to do photosynthesis. The growth of the plants is supported by spraying insecticide periodically. In this paper, we offer to use an unmanned aerial vehicle, to recognize the targeted green areas using HSV algorithm, track the generated trajectory and perform the spraying process all along the recognized area. We believe this will automate the maintenance of the gardens, improve the effectiveness of the treatment and assist to the ecologic factor, by applying the necessary amount of chemicals on the targeted place only.       1 INRODUCTION Recently unmanned aerial vehicles are used to perform several civil tasks. Particularly, in construction fields, the idea of aerial robot is directly related to photogrammetry, site scanning and buildings status. In this regard, many literatures supported the cost efficiency, safety and quality control for such technological implementation (Bulgakov A. et al. 2014;  Emelianov S. et al. 2014; Rango,  et al. 2009.  Chao, H et al. Shang et al. 2010) Similarly, aerial robots can be used to scan vertical gardens and improve the irrigation system by controlling the quantity of water supplied to a certain site. This can be of great importance when talking about large agricultural areas, mega-polis with limited greenery and quantifying forest fire impact.  The technology being used to sense is based on satellite inspection. The limitation of such technology consists of the low resolution and the update ratio of the images provided. On the other hand, UAVs are more cost effective, can reach difficult scanning corners and provide more accurate visual information, as they can be equipped with high definition cameras. In this paper, we offer to use a vertical take off and landing aerial robot to perform visual inspection for vertical gardens.  067-1 2 RELATED WORKS UAV’s successful results in improving military tasks have pushed the developers of such technology to try it in the civil market. The verdict: we got a miniature class of rotorcraft assisting in fire fighting, cinematography, television broadcasting and so on. The technology is being used to improve irrigation systems (Chao H, et al.), to address the climatic changes in the north and south poles (Lucieer A. et al. 2012), inspection of bridges and facades  (Bulgakov A. et al. 2014;  Emelianov S. et al. 2014). As a sensing technology, band -reconfigurable fixed wing UAV were used more frequently. This is due to the optimized flight range, lifting capability and improvement in Micro (MEMS) and Nano electromechanical systems (NEMS). The sensors used are based on thermal infrared and machine vision. But these sensing technologies have limitations, especially the thermal infrared: the cost involved and the reliability of the acquired data especially in a humid atmosphere (Lucieer A. et al. 2012). The other limitation is the UAV itself: stability problem leads to false reading. Having analyzed the pros and cons of aerial inspection, the optimization problem can be divided into two subtasks: 1. Coverage control problem consisting of path planning and control of the UAV; 2. Geo-reference problem consists of registration of each acquired pixel from the aerial images including temporal and spatial information (time and location). Both of the subtasks can be addressed in either of  two ways: global path planning consisting of trajectory planning before the flight and local planning that generates the trajectory during flight. 3 AIM OF THE PAPER The aim of the paper is to address the aforementioned subtasks in order to inspect vertical gardens in the cities using quadrotors. As it is noticeable in most of the mega polis in the world, where green areas are lacking. The uncontrollable population growth is destroying the remaining of “mother nature”. Engineers and social workers have invented the concept of green wall or vertical gardens. The idea, as shown is figure 1, consists of planting wall climbing trees and plants on vertical walls. Due to the ergonomics involved the idea is widely spread.  Figure 1: Vertical garden concept The role of the quadrotor is to acquire visual information such as images and pictures and spray the insecticide on the target area.  3.1 Coverage Control Problem This subtask is divided into two points: path planning and control. Since, for the geo-reference problem, we will be using GPS data to acquire the temporal and spatial information, we will limit ourselves with the control part of it. As it is known, a quadrotor is a take off and landing rotorcraft, having six degrees of 067-2 freedom. But this miniature aircraft can perform only four flight regimes: roll, pitch, yaw and hover. This mechanical limitation makes the quadrotor nonlinear system. The dynamics of the quadrotor can be described as follows: [1] ; [2]  [3] ;  [4]   [5]   [6] .  Where,  ,  and  are the projection of the linear acceleration of the quadrotor in the Earth fixed axis,  ,  and  are projection rotational acceleration of the quadrotor in the body fixed axis,  the gravitational acceleration,   the torque generated from the rotors,  the mass of the quadrotor, ,  and   are the projection of the Inertia of the quadrotor,  is the rotational speed of the propellers, ,    and  are the roll, pitch and yaw angle consequently, , ,  and  are the aforementioned flight regimes.  3.2 Fuzzy Logic Position Controllers  As it is important to know the value of the error, it is similarly critical to understand how it is changing over time. The error and its derivative in time are one of the possibilities to configure a Fuzzy logic controller. It is an artificial intelligence approach that computes mathematical operations based on degree of truth rather than the conventional True-False Boolean logic. The Fuzzy logic allows having more adaptable controller specially when dealing with nonlinearities (i.e. nonlinear aerodynamic model of the quadrotor as described in equations 1-6) and uncertainties (i.e. permanent changes in the flight circumstances, wind, temperature, obstacles positions etc.). Although in many cases, heuristic algorithms were ruled out, due to the time consumption, we saw a need to include this algorithm in our survey to cover most of the techniques used. We would like to clarify that the term heuristic is used to describe the estimation ability in artificial intelligence.   Moreover, the linguistic power of the fuzzy logic may be extremely useful while navigating in totally unknown areas.  The trick in such controller is how to set up the linguistic rules for input and output.  We fine tuned the rules by using the resulting graph (figure 2) for the function de(t)=f(e(t)), where e(t) is the deviation in position of the quadrotor with reference to the desired trajectory and de(t) is the variation of the deviation.   Figure 2: Resulting Graph de(t)= f(e(t)) 067-3 Using the graph in figure 2, we assign the following labels for the numeric values: Negative big (NB) =-1, Negative Small (NS) = -0.5, Zero (Z)=0, Positive Small (PS)= 0.5 and Positive Big (PB) =1. As a result we obtain the linguistic rules listed in table (1). A linguistic rule defines the output of the fuzzy controller based on discrete logic (i.e. if e (t)= NB and d(e(t)= PB, then the output is Z). the graphical representation of the linguistic rules is illustrated in figure 3.   Table 1: Fuzzy rules         de   e NB NS Z PS PB NB NB NS NB NB Z NS NS  NS NS Z  NS Z NB NS Z PS PB PS NS Z PS PS PS PB Z PB PB PB PB   Figure 3: Graphical Representation of the linguistic rules The inputs of the fuzzy controller de(t) and e(t) are represented using triangular membership functions as shown in figure 4 a and 4b. The output of the fuzzy control is also 5 triangular membership functions (figure 4c).  Figure 4 a: membership function for input “e(t)” 067-4  Figure 4 b: membership function for input “de(t)”  Figure 4 c : membership function for fuzzy regulator output The results of controlling the quadrotor using fuzzy logic regulators are shown in figure 5a and b for position and rotation control consequently.  Figure 5a: Quadrotor position control with reference to GPS waypoints. Horizontal axis – time [s], vertical axis- position [m]  067-5  Figure 5b: Quadrotor rotation control with reference to obtained position coordinates in 3a. Horizontal axis – time [s], vertical axis- orientation [degree/s] The results shown in figure 5a and 5b give a fair idea about the reliability of the fuzzy controller. For instance, figure 5a illustrates the fact of stabilizing the linear movement of the quadrotor (described in equations 1-3, X (red curve), Y (blue curve), Z (green curve)) without overshooting. This allows the quadrotor to move along Earth-axis OXYZ smoothly. The other advantage of avoiding overshooting is contributing to better energy-efficient control systems (i.e. overshooting in altitude control means flying higher and consuming more power).  Taking into consideration the nonlinearity of the aerodynamic model of the quadrotor, linear movement cannot be achieved unless stabilizing Euler angles (roll (red curve), pitch (blue curve) and yaw angles (green curve)) as shown is figure 5b. Control signals are generated in order to increase or decrease Euler angles (equation 4-6) with reference to linear position control task (equations 1-3).    3.3 Geo-reference Identification Problem  Identification of the vertical garden can be done using the green pattern. Many machine vision algorithms are dedicated to recognize colors. We will be using the HSV algorithm due to its mathematical model, which is more reliable than the other methods. The HSV consists of giving a certain color 3 coordinates: Hue, saturation and value as shown in picture 4. HSV is used is many literatures in order to detect static and dynamic agents/objects based on histogram criteria, such as recognition of traffic lights (Yixin Chen et Al. 2013), gesture identification (Mokhtar M. Mohsen et al. 2010), image and structure retrieval (Durgesh N. et al. 2013) and face detection (Naresh Kumar R. et al. 2014,Sayfeddine D. 2013).     Figure 6: HSV shade representation 067-6 From figure 6, we can see clearly that the green color is varying between 60 and 180 degrees. Hence, the role of the algorithm is to identify the green patches on the wall and apply the insecticide on the recognized area. This can be done by dividing the frame of the camera mounted on the quadrotor into 9 identical cells. Hereafter the registered pixel coordinates will be modified to metric using the following equations: [7]    [8]  [9]  [10]  [11]  [12]  Where,  and  the projection of the altitude on the pixel coordinates,  is the flying altitude,  and  are the horizontal and vertical pixel coordinates of the recognized area,  and  are the scale factors,  and  are the rotation of the quadrotor in the pixel axis,  and  are the rotation of the diaphragm of the camera from the body axis. After identification of the desired pixels based on color criteria, Cartesian subtraction is computed to identify the distance between the center of the fixed camera and the centroid of the detected colored surface. The subtraction result represents the distance that the quadrotor has to fly in pixels. The following approach is known as visual odometry. Depending on the position of the centroid, metric control task is generated for flight altitude stabilization (equation 7 and 8), linear movement (equations 9 and 10) and rotational movement (equation 11 and 12).   Using the concept of reactive control and based on the results obtained in equations (8,9 and 11,12), control signals are generated to initialize one of the four available flight regimes described in equations (3-6). 4 REAL-TIME RESULTS   HSV and visual odometry algorithms were tested in real-time in order to trace the possibility of their implementation on high-rise structures covered with vertical gardens. As figure 7 shows, the identification process was successful and the generation of the pixel coordinates was obtained. Figure 7 can be interpreted as follows; the green surface is detected by algorithm HSV (i.e. position Middle-Middle in the 9-cells frame). The green pixels are generated (i.e.  = 69, ). The Cartesian subtraction is obtained and metric control task is formed. The graph is figure 7 shows how the quadrotor changes its position (X (green curve), Y (purple curve)) with the changes of the pixel coordinates  (blue curve) and  (red curve).  A comparison between real time two dimensional flight results (figure 7) and simulation results (figures 5a/5b) shows that the deviation in sensors readings is as follows: roll reading (0.045°/s); yaw reading (0.125°/s); pitch reading  (0.15°/s). The reasons of such deviations can be caused by the Euler angles MEMS gyroscopes, asymmetric structure of the quadrotor, influence of shifted center of gravity on the control systems, and propellers non-uniformity. The obtained deviation values lay in the tolerance interval and allowed Metrologic norms (deviation of real-time results ±5% from the simulation results).     067-7  Figure 7: Trajectory tracking 5 CONCLUSIONS AND FUTURE WORK The paper focused on how to use the quadrotor to maintain vertical gardens on high-rise structures. The adopted concept was to divide the autonomous flight problem into control of the quadrotor over an identified area using HSV algorithm. The control was performed using fuzzy logic position regulators based on reactive control concept interconnecting the rotation speed of the propellers with the positioning of the quadrotor. No overshooting was registered in the simulation results. Regarding the identification of the green area, HSV algorithm was used. The pixel coordinates were modified to metric in order to create flight mission. The identification and control algorithms were tested in real time using a commercial quadrotor (AR DRONE) linked to a stationary computer using datagram protocol UDP through a Wi-Fi connection. Although the results were outstanding and the computational power was enough to drive the quadrotor to the end of the experience, we have noticed the influence of the light nature and temperature on identifying the green color. Having just the color as tracking criteria, the stationary computer could run into local minimum problem (between green and blue). This issue was addressed earlier as extrapolation of motion function. A solution was suggested based on nonlinear neural network model with exogenous input (NARX). The results can be accessed in the following literatures (Sayfeddine D. 2014, Sayfeddine D. et al. 2014). References Bulgakov, A., Emelianov, S., Bock, T. and Sayfeddine, D. 2014. Control of hovering altitude of a      quadrotor with shifted center of gravity for inspection of high-rise structures. ISARC, Sydney, Australia:      762-768. Chao, H. and al.  Band reconfigurable multi- UAV based cooperative remote sensing for real time water      management and disturbed irrigation control. 9 pages. Durgesh Nirapure, Udaypal Reddy. 2013. Fast retrieval of images using filtered HSV color level detection.      International journal of Emerging Technologies and Advanced Engineering. Volume 3, issue 8: 414-     419. Emelianov, S., Bulgakov A. and Sayfeddine, D. 2014. Aerial laser inspection of building facades using      quadrotor. Proceedia of Creative construction. CC2014. Prague, Czech Republic: 24-30. Lucieer, A. and al. 2012. Using a micro UAV for ultra high resolution multi sensor observations of      Antarctic moss beds. International Archives of the photogrammetry, remote sensing and spatial      information science. Volume XXXIX-B1, 2012: 429-433. Mokhtar M. Mohsen, Pramod K. Misra. 2011. HSV brightness factor matching for gesture recognition      system. International Journal of Image Processing (IJIP), Volume (4): issue 5: 456-467. Naresh Kumar R., P. Varum Varma, P. Surya Abhilash. 2014. Face detection using HSV color space and      minimal distance maximum variation approach. International Journal of Electrical, Electronics and Data      Communication, Volume 2, issue 4: 8-12. 067-8 Rango, A. and al. 2009. Unmanned aerial vehicle based remote sensing for rangeland assessment,       monitoring and management. Journal of Applied Remote sensing, 4: 033452. Sayfeddine D. 2013. Face identification and tracking using quadrotor. Translated from Russian: Control       problems, processing and data transmission (ATM 2013) : 178-180. Sayfeddine, D. 2014. Nonlinear autoregressive neural network with exogenous inputs based solution for       local minimum problem of agent tracking using quadrotor. Electronic scientific journal. Engineering       Journal of Don. 2: http://www.ivdon.ru/en/magazine/archive/n2y2014/2355 Sayfeddine, D. and al. 2014. Position prediction of a dynamic agent tracked by quadrotor using neural       networks. Electronic scientific journal. Engineering Journal of Don. 1:       http://www.ivdon.ru/en/magazine/archive/n1y2014/2293   Shang, H. and al. 2010. Low cost UAV based thermal infrared remote sensing: platform, calibration and       application. IEEE: 38-43. Yixin Chen, Yi Xie and Yulin Wang. 2013. Detection and recognition of traffic signs based on HSV vision       model and shape features. Journal of computers, Vol.8., No.5: 1366-1370.  067-9  Maintaining vertical gardens using quadrotor aerial inspection The speaker‘s attendance at this conference was sponsored by  the Alexander von Humboldt Foundation.   Prof. Dr. Sc. Alexey Bulgakov  http://www.humboldt-foundation.de Vertical Gardens concept •  Big cities, limited green surfaces •  Ergonomic design  •  Difficult to maintain, specialization required  Conventional ways limitations  •  Vertical planting layout •  High-rise structures •  High risk involved •  Insecticide  Why quadrotor? •  Cheap, maintenance-friendly •  Practically no flight setup required •  Ability to transport loads •  Can be equipped with sensors •  High maneuverability •  Ability to reach difficult corners   Why automating maintenance process? •  Resource efficiency •  Avoid risks/ severe injuries •  Multitasking  Experimental Setup Quadrotor dynamics Coordinates Transformation Fuzzy Logic Controller Simulation Results Altitude effects on flight stability F- the load of the wind on the quadrotor S- the exposed area P- the wind pressure C is the drag coefficient Kz - the exposure coefficient  G - the gust factor H- flight altitude  Visual Odometry real-time results Possible applications  •  Visual odometry + proximity sensor = 3D maintenance   •  Single or swarm robotics  •  Vertical/ horizontal plantation zones, roof plantation •  Visual Odometry + infrared thermal image acquisition =   Moisture detection (rice, tea plantation)        Thank you very much for your attention! 

Cite

Citation Scheme:

        

Citations by CSL (citeproc-js)

Usage Statistics

Share

Embed

Customize your widget with the following options, then copy and paste the code below into the HTML of your page to embed this item in your website.
                        
                            <div id="ubcOpenCollectionsWidgetDisplay">
                            <script id="ubcOpenCollectionsWidget"
                            src="{[{embed.src}]}"
                            data-item="{[{embed.item}]}"
                            data-collection="{[{embed.collection}]}"
                            data-metadata="{[{embed.showMetadata}]}"
                            data-width="{[{embed.width}]}"
                            async >
                            </script>
                            </div>
                        
                    
IIIF logo Our image viewer uses the IIIF 2.0 standard. To load this item in other compatible viewers, use this url:
http://iiif.library.ubc.ca/presentation/dsp.52660.1-0076460/manifest

Comment

Related Items