UBC Theses and Dissertations

UBC Theses Logo

UBC Theses and Dissertations

A machine vision and handling system for measurement of containerized tree seedlings 1996

You don't seem to have a PDF reader installed, try download the pdf

Item Metadata


ubc_1998-0128.pdf [ 14.09MB ]
JSON: 1.0080883.json
JSON-LD: 1.0080883+ld.json
RDF/XML (Pretty): 1.0080883.xml
RDF/JSON: 1.0080883+rdf.json
Turtle: 1.0080883+rdf-turtle.txt
N-Triples: 1.0080883+rdf-ntriples.txt

Full Text

A MACHINE VISION AND HANDLING SYSTEM FOR MEASUREMENT OF CONTAINERIZED TREE SEEDLINGS by Wing Hang Poon B.Eng. (Mechanical Engineering), Technical University of Nova Scotia Nova Scotia, Canada, 1992 A THESIS S U B M I T T E D IN P A R T I A L F U L F I L L M E N T OF T H E R E Q U I R E M E N T S F O R T H E D E G R E E OF M A S T E R OF A P P L I E D S C I E N C E in T H E F A C U L T Y OF G R A D U A T E STUDIES Department of Mechanical Engineering We accept this thesis as conforming to the required standard T H E U N I V E R S I T Y OF BRITISH C O L U M B I A October, 1996 ©Wing Hang Poon, 1996 In presenting this thesis in partial fulfilment of the requirements for an advanced degree at the University of British Columbia, I agree that the Library shall make it freely available for reference and study. I further agree that permission for extensive copying of this thesis for scholarly purposes may be granted by the head of my department or by his or her representatives. It is understood that copying or publication of this thesis for financial gain shall not be allowed without my written permission. Department of M~&Cf)A1\J I Q l F N & I N"5Fg(N Q The University of British Columbia Vancouver, Canada Date OCT. >5, 1 ^ 6 DE-6 (2/88) 11 A B S T R A C T The commercial forest industry in British Columbia, along with the Ministry of Forests and private nurseries, produce over two hundred mill ion containerized tree seedlings annually. A t harvest, these seedlings are lifted out of their growing container and graded to remove inferior stock based on their dimensions and appearance. Manual grading is a labor-intensive and costly operation. The work is subjective and susceptible to human error. Advances in the capability and a parallel decline in cost of machine vision hardware have spurred interest in automated grading as an alternative. A PC-based machine vision and handling system providing rapid measurement of containerized seedling morphological features has been developed in this research. The system, which employs a novel design of line-scan imaging using fiber optics in conjunction with structured backlight, addresses the production needs of commercial forest nurseries. The multiple processing approach of simultaneously scanning an entire row of 14 seedlings satisfies the time constraint of 8 seconds to measure the stem diameter and shoot height of all seedlings based on their high contrast images projected onto the optical fiber sensors. The system has a high transverse resolution equal to the 0.25 mm diameter of the optical fibers for precise measurement of stem diameter. Designed for on-line inspection, the system also provides a multi-functional, menu-driven graphical user interface. The interface offers a convenient and interactive means to control the scanning operation and adapt to various grading criteria. In addition, it supports different Ill scanning and display capabilities, and produces simple statistics for each measured feature which would be valuable for quality control and quality improvement purposes. Tests were conducted to evaluate the performance of the system using different seedling samples and a test object of known size representing a seedling. Average classification error of tree seedlings was found to be 9%. The accuracy of diameter measurement had a low standard deviation of 0.12 mm. The speed of scanning depended upon the desired accuracy of the length measurement. A t a belt speed of 4 cm/s, the accuracy of length measurement was found to be 0.6 cm with a corresponding seedling throughput of 1.75 seedlings per second. iv T A B L E O F C O N T E N T S A B S T R A C T : ii T A B L E O F C O N T E N T S iv L I S T O F T A B L E S vi L I S T O F F I G U R E S :....... vii A C K N O W L E D G M E N T '. ix 1. I N T R O D U C T I O N 1 1.1 MOTIVATION. 1 1.2 THESIS OBJECTIVES 3 1.3 THESIS O V E R V I E W ; 4 2. T H E N U R S E R Y B A C K G R O U N D 6 2.1 C H A P T E R O V E R V I E W 6 2.2 E A R L Y LIFTING OPERATION ; 6 2.3 PREVIOUS A T T E M P T S ON A U T O M A T I O N -9 3. R E V I E W O F P R E V I O U S R E S E A R C H 13 3.1 C H A P T E R O V E R V I E W 13 3.2 SEEDLING H A N D L I N G A N D SORTING • 14 3.3 M A C H I N E VISION APPLICATION 18 4. D E S I G N O F A M A C H I N E V I S I O N P R O C E S S 22 4.1 C H A P T E R O V E R V I E W : 22 4.2 ILLUMINATION : • 22 4.3 I M A G E ACQUISITION 26 4.4 I M A G E SEGMENTATION 29 4.5 F E A T U R E E X T R A C T I O N • 31 4.6 I M A G E INTERPRETATION 35 5. D E S I G N A N D D E V E L O P M E N T O F T H E P R O T O T Y P E 36 5.1 C H A P T E R O V E R V I E W 36 5.2 ASSUMPTIONS 36 V 5.3 SEEDLING G R A D I N G CRITERIA 38 5.4 S Y S T E M H A R D W A R E 38 5.4.1 Conveyor System 41 5.4.2 Fiber Optics 43 5.4.3 Illumination 46 5.4.4 Camera System 52 5.4.5 Computer • 58 5.5 S Y S T E M SOFTWARE 58 5.5.1 Description of the Program Modules 59 5.5.2 Camera Initialization 60 5.5.3 Scanning Operation 63 5.5.4 Scanning Modes 68 5.5.5 Grading Parameters 73 5.5.6 Result Statistics •„...:.•.=.•..:.-..:;.....-...•.• 75 6. CALIBRATION, TESTING AND EVALUATION 77 6.1 C H A P T E R OVERVIEW 77 6.2 CALIBRATION : 77 6.3 A C C U R A C Y OF O B J E C T CLASSIFICATION 83 6.4 A C C U R A C Y OF S T E M D I A M E T E R M E A S U R E M E N T 85 6.5 A C C U R A C Y OF L E N G T H M E A S U R E M E N T •• 86 6.6 S P E E D OF SCANNING 89 6.7 REPEATABILITY IN M U L T I P L E SCANNING 92 7. CONCLUSION AND RECOMMENDATIONS 93 7.1 CONCLUSIONS A N D CONTRIBUTIONS OF T H E RESEARCH 93 7.2 R E C O M M E N D A T I O N S FOR FURTHER WORK 95 REFERENCES • 9 7 APPENDIX A : On-OFF Motor Control Circuitry • 100 APPENDED B : Assembly Drawings for the Camera Mount and Fiber Optics Unit 105 APPENDED C : Dimensional Drawings for the Protype Frame 113 v i L I S T O F T A B L E S Table Page TABLE 5-1 : FEATURES OF DIFFERENT SCANNING MODES 68 TABLE 6-1: CLASSIFICATION RESULT FROM A SET OF OBJECTS CONSISTING OF 8 SEEDLINGS AND 2 PLUGS 84 TABLE 6-2 : GRAPHICS PERFORMANCE TEST ON VARIOUS COMPUTER CONFIGURATIONS 91 L I S T O F F I G U R E S Figure Page FIGURE 1-1 : STYROFOAM BLOCKS • 2 FIGURE 1-2 : CONTAINERIZED SEEDLINGS 2 FIGURE 2-1: EARLY LIFTING SETUP 7 FIGURE 2-2 : MANUAL WRAPPING 8 FIGURE 2-3 : EXTRACTION MACHINE 9 FIGURE 2-4 : SEEDLING PROCESSING SYSTEM DEVELOPED BY THE M O F 10 FIGURE 2-5 : SINGULATION BELT • 11 FIGURE 2-6 : LINE-SCAN MODULE 1 2 FIGURE 3-1 : SKETCH OF THE DESIGN PROPOSED BY M A W [17] 15 FIGURE 3-2 : LINEAR DISPLACEMENT DEVICE FOR MEASURING SEEDLING DIAMETER..... 16 FIGURE 3-3 : THE ROTATING SEEDLING HOPPER WITH THE VACUUM WEDGE NOZZLE 17 FIGURE 4-1 : DIFFERENT ILLUMINATION TECHNIQUES 2 4 FIGURE 4-2: SPECTRAL EMISSION OF SOME COMMON ILLUMINATORS USED IN MACHINE VISION (SOURCE: E G & G RETICON 95/96 IMAGE SENSING MANUAL) • 2 6 FIGURE 4-3 :256 GRAY-SCALE IMAGE OF A SEEDLING 30 FIGURE 4-4: A BIMODAL GRAY LEVEL HISTOGRAM • 31 FIGURE 4-5 : ELEMENTS OF EDGE DETECTION : DARK OBJECT ON A LIGHT BACKGROUND 33 FIGURE 5-1: T H E PROTOTYPE OF THE SCANNING SYSTEM 39 FIGURE 5-2 : SCANNING SYSTEM USING FIBER OPTICS .: • 4 0 FIGURE 5-3 : MOTOR CONTROL CIRCUITRY FOR THE SORTING CONVEYOR 43 FIGURE 5-4 : STRUCTURE OF OPTICAL FIBER AND DIAGRAM OF LIGHT TRANSMISSION 44 FIGURE 5-5 : SPECTRUM ATTENUATION OF THE S K SERIES OPTICAL FIBERS (SOURCE : MITSUBISHI RAYON CO., LTD) 4 5 FIGURE 5-6: FIRST ILLUMINATION DESIGN WITH CYLINDRICAL LENSES • 4 7 FIGURE 5-7: SECOND ILLUMINATION DESIGN WITH STRUCTURED LIGHT 49 FIGURE 5-8 : OPTICAL SETUP FOR THE PROTOTYPE WITH THE CUSTOM MADE OPTICAL BENCH 50 FIGURE 5-9: TRANSFORMING OF A POINT IMAGE PNTO A LINE IMAGE USWG A GLASS ROD 51 FIGURE 5-10 : CAMERA SETUP 5 4 FIGURE 5-11: RAW IMAGE AS VIEWED BY THE CAMERA SYSTEM (NO OBJECT DETECTED) , 54 FIGURE 5-12 : SENSORS MAPPING 5 6 v i i i FIGURE 5-13 : MOTION AND RESOLUTION .. ••••• 57 FIGURE 5-14: USER INTERFACE OF THE PROTOTYPE 59 FIGURE 5-15: USER INTERFACE - CAMERA INITIALIZATION 62 FIGURE 5-16: BINARY SEEDLING IMAGE AROUND THE ROOT COLLAR ZONE 64 FIGURE 5-17: PLOT OF DARK PIXEL COUNT VS LENGTH 65 FIGURE 5-18: PLOT OF NUMBER OF TRANSITIONS VS LENGTH 65 FIGURE 5-19: FLOWCHART OF THE GRADING ALGORITHM................ 67 FIGURE 5-20: USER INTERFACE - SELECTING DIFFERENT MODES 69 FIGURE 5-21 : LAYOUT OF DIFFERENT SCANNING MODES 70 FIGURE 5-22 : FLOWCHART OF THE i_scan_1 () ROUTINE 71 FIGURE 5-23 : USER INTERFACE - PARAMETER SETTING • • 73 FIGURE 5-24 : DEFAULT VALUE OF THE PARAMETERS 7 4 FIGURE 5-25 : USER INTERFACE - STATISTICS OPTION 75 FIGURE 5-26 : LAYOUT OF THE STATISTICS ROUTINE 75 FIGURE 6-1: THRESHOLD SETTING 78 FIGURE 6-2 : WIDTH FACTOR OF ALL 14 SECTIONS .-. 79 FIGURE 6-3 : RELATIONSHIP BETWEEN INPUT VOLTAGE AND BELT SPEED 80 FIGURE 6-4: HEIGHT COEFFICIENT OF IMAGE MODE 81 FIGURE 6-5 : HEIGHT COEFFICIENT OF GRAPH MODE • 82 FIGURE 6-6: HEIGHT COEFFICIENT OF FOUR MODE 82 FIGURE 6-7: HEIGHT COEFFICIENT OF A L L MODE 83 FIGURE 6-8 : OBJECT USED FOR THE ACCURACY OF STEM DIAMETER AND HEIGHT MEASUREMENT 85 FIGURE 6-9: RESULT OF VARIATION IN LENGTH MEASUREMENT 87 FIGURE 6-10 : RESULT OF STANDARD DEVIATION OF LENGTH MEASUREMENT IN ROWS 87 FIGURE 6-11: COMPARISON OF THE HEIGHT COEFFICIENT OF ALL 4 MODES 88 FIGURE 7-1: FUTURE WORK • 96 FIGURE 7-2 : MOTOR CONTROL CIRCUIT 1 0 1 FIGURE 7-3 : CIRCUIT RESPONSE CURVES : 1 0 3 ix A C K N O W L E D G M E N T I would like to thank my supervisors, Dr. Peter D . Lawrence from the Electrical Engineering Department and Dr. Farrokh Sassani from the Mechanical Engineering Department for their gracious guidance and immense patience throughout the course of my study. Without their invaluable insight, this thesis would not have been possible. I would like to acknowledge N S E R C and the Advanced Systems Institute for the financial contribution to this research. I would also like to express my gratitude to M r . Steve Pelton of Pelton Reforestation for giving me the opportunity to familiarize myself with the basic forest nursery operation. In addition, many individuals in the Mechanical Engineering Department assisted me in various aspects of my works. Many thanks to Tony Besic, Anton Schreinders, and Dave Camp from the machine shop for their professional technical assistance and valuable suggestions during the construction of the prototype. I would like to thank Don Bysouth in the instrumentation shop for his technical help in designing the motor control circuitry and of course, Alan Steeves for his constant availability and help with computer related problems. I am also grateful to all my colleagues and friends for their continuous support and encouragement. Finally, my deepest gratitude is extended to my fiancee, Chihiro Otsuka, and my family who are always behind me and with me at all times. 1 Chapter One 1. INTRODUCTION 1.1 Motivation Reforestation is the key to the ecosystem and promises renewable supplies of lumber and paper products. Compared to a decade ago, more than double the number of seedlings are planted in the province of British Columbia to replace trees that have been harvested, destroyed by fire or damaged by pests as the demand for forest renewal continuously rises. Last year more than 200 million seedlings were planted in British Columbia and container- grown seedlings constitute over 90% of the figure [2]. With a higher survival rate than that of bare root seedlings, the trends of reforestation in the foreseeable future wi l l go towards the production of containerized seedlings [20]. Containerized seedlings are grown in modular styrofoam blocks which contain a number of cells arranged in a regular pattern. Depending on the species and age stock types, the number of cells in the block varies. A typical size is 14 X 8 horizontal and vertical cells'respectively, and each cell is planted with a single seedling (Fig. 1-1 & 1-2). Once the seedlings are large enough to be planted, they are extracted from the blocks, graded to remove inferior stocks and bundled for shipping. Manual grading is a labor intensive process and is susceptible to human errors induced by long operating hours. Cost of labor could range to a substantial level of the total production costs. A s well, it is not practical for the workers to grade more than two classes and record Chapter I: Introduction 2 statistical throughput because of so many different grading standards for different species and different stocks. With all of these shortfalls and the advances in the capability of machine vision hardware, it has opened the way to make automation possible and attractive. Figure 1-1 : Styrofoam blocks Figure 1-2 : Containerized seedlings A study was conducted in a private local nursery to examine the feasibility of developing a fully integrated automated system for seedling lifting against the manual operation at production-line rates. The lifting process consists of three major phases, namely, extraction, grading and bundling. It was found that in the past few years, there have been several Chapter 1: Introduction 3 previous automation efforts made on each phase that have yielded reasonable results in a few local nurseries. However, when it comes to solving the problem as a whole, so far the result has been unsatisfactory. This is mainly due to the lack of coordination between mechanical components of different phases and/of the results are not cost effective for the expected quality standards. The process of each component should be synchronized with each other and the methodology of how the seedlings are handled, transported, and graded through the system play a vital role in solving the entire problem. 1.2 Thesis Objectives The primarily goal of the research described in this thesis has been to develop an automated grading and sorting system for containerized seedlings with the use of machine vision in a commercial nursery environment. In the design of the system, accuracy of the measurement and system throughput are the main concerns. The adaptation and coordination with all packaging tasks, like the automated row extraction machine that is widely used in many nurseries in the province, should also be investigated. The following specific objectives were pursued to accomplish the goal: (1) To study the general lifting operation in a nursery. (2) To conduct a review of technology that has been used in the past in relation to seedling handling and grading, and the capabilities and limitations of the technology. Chapter I: Introduction 4 (3) To design and construct a prototype which could simplify the seedling morphological measurements while achieving a tight accuracy of 0.1 mm in the transverse direction and 1 cm in the longitudinal direction. (4) To develop and implement a machine vision system and algorithm for seedling grading based on the on-line measurement, and a user interface for the system prototype to control the process and be able to adjust for different grading standards. (5) To evaluate the performance of the implementation in terms of measurement accuracy, reliability and speed. 1.3 Thesis Overview The thesis has seven chapters. Chapter 2 begins with a general background of the nursery under study. This includes an overview of the manual lifting operation and a discussion of the early attempts on automation. Chapter 3 consists of a literature review of seedling handling and sorting techniques, and the use of machine vision to grade bare root seedlings, both for quality control sampling and for on-line inspection. In Chapter 4, the fundamental concepts of machine vision are reviewed, including the five basic processes which are applicable to the design of the vision system used in the prototype. The main part of this thesis, which is the prototype design and development, is presented in Chapter 5. In this chapter, the discussion of the equipment used for constructing the prototype and the description of various functions of the scanning program are given. Chapter 6 is devoted to Chapter I : Introduction 5 the calibration and evaluation of the machine vision system. The performance of the system was evaluated in terms of object classification, measurement accuracy of the stem diameter and shoot height, speed of scanning, as well as the measurement repeatability. Lastly, Chapter 7 serves as a concluding chapter of the entire thesis in which the contribution of the research and the recommendations for future work are addressed. 6 Ch apter Two 2. T H E NURSERY BACKGROUND 2.1 Chapter Overview Here in British Columbia, different containerized nurseries either perform the common procedures of manual lifting or exploit mechanization in various levels in conjunction with manual work. None o f them has yet achieved total automation in the entire production line. This chapter presents an overview of the basic manual lifting operation and previous automation efforts made in a private local nursery, Pelton Reforestation. It is one of the largest containerized seedling producers among all other players in the province. This study is essential because it serves as a cornerstone of problem formulation as well as a reference to the machine vision system proposed and described in this thesis. 2.2 Early Lifting Operation Each year Pelton Reforestation produces 60 million seedlings. Interior spruce, Lodgepole pine and Douglas-fir are the three major species. In the nursery, 70% of the total production is one-year-old stock (1+0) and 30% is two-year-old stock (2+0) (the 0 means "old"). A t harvesting, blocks of seedlings are delivered from the field to the graders in the packing shed which basically consists of several conveyor belts (Fig. 2-1). Graders standing along either side of the conveyor load one block at a time in front of them and extract and grade Chapter 2 : The Nursery Background 7 the seedlings at the same time. The way graders extract the seedlings is to grasp the root collar o f the seedlings and forcibly pull them out of the container. Since the seedlings are often not easily removed without damaging the roots, it usually requires using a "block shaker" to loosen the plugs prior to the delivery to graders at the belt. Grader Grader Grader I CD | Wrapper Grader Grader Grader Packer Figure 2-1 : Early lifting setup The graders sort the seedlings into classes of acceptable and cull (unacceptable) according to several morphological characteristics: root collar diameter, shoot height (distance between the seedling top and the root collar) and the plug integrity. The measurement is done to the nearest 1 mm for the root collar diameter and 1 cm for the shoot height by using a meter stick. The graders reject the cull and place the acceptable seedlings in groups o f 15 Chapter 2 : The Nursery Background 8 to 20 onto the conveyor passing to the bundling (or wrapping) phase. A n experienced grader can normally grade as many as 25,000 seedlings per day of 7 to 7 Vi hours (close to one seedling per second), while a beginner starts at 8,000 seedlings per day for the same task. However, the job is very monotonous and prone to human error. A commercial polystyrene film wrapper is placed at the end of every conveyor belt. It is used to wrap the grouped seedlings into a bundle to facilitate shipping planting, retain plug moisture and assist planters (Fig. 2-2). Usually only one worker is required in this phase and another worker helps in packing the bundles into boxes which wi l l be shipped to the customers. A typical bundling station consists of a roller to dispense the film; a platform to wrap the seedling; and a hot wire to cut the plastic wrap. Figure 2-2 : Manual wrapping Chapter 2 : The Nursery Background 9 2.3 Previous Attempts on Automation Due to the uniformity and inherently separated nature of containerized seedlings, it is possible to exploit automation as an alternative to manual operation. A machine which is used for extracting seedlings from a block was designed by a company in Washington State (Danzco) and is currently adapted for use in the nursery (Fig. 2-3). (a) Front view (b) Side view Figure 2-3 : Extraction machine The extractor requires an operator feeding one block at a time vertically from the side. Two pneumatic linear actuators controlled by a programmable logic controller (PLC) are used to Chapter 2 : The Nursery Background 10 synchronize the extractor motion. The horizontal actuator, which attaches to a changeable row of pins, pushes the bottom row of seedlings from the back of the block. Once the entire row is out of the block, the horizontal actuator retracts and the vertical actuator pushes the block downward to access the next row of seedlings. The process then iterates until the top row of seedlings is removed. So far the performance of the extractor has been quite satisfactory without causing too much damage to the plugs. The time between extraction of rows is about 8 seconds. In conjunction with the extractor, there was an attempt to automate the entire lifting operation by using a seedling processing system developed by B . C . Ministry of Forests. The seedling processing system comprised a singulation belt, a vision grader, a bundler and a mechanical wrapper (Fig. 2-4). Vision system Flipping plate Bundler Extractor Semi-automatic wrapping system Figure 2-4 : Seedling processing system developed by the M O F Chapter 2 : The Nursery Background 11 Figure 2-5 : Singulation belt As shown in Fig. 2-5, a row of seedlings dropped onto a black grading conveyor after being extracted from the block container. A vertical singulation belt and an alignment chain attached to a series of trays were used to separate the seedlings about 2 to 3 inches apart before passing underneath a black-and-white camera which captured the entire image of each individual seedling. The images were digitized and processed by a PC-based frame grabber board. The board offered a resolution array of 510 X 480 horizontal and vertical lines, respectively. A solid-state monitor was also used to display images. However, the singulation mechanism often failed to work properly causing seedlings in the trays to cross one another and lose their orientation in the alignment chain. Due to this mechanical problem, the vision system could not grade the seedling correctly and achieve the full inspection rate of 3 to 4 seedlings per second. A nursery worker had to stand beside the machine to correct the orientation and singulation. It was also found that the worker had to frequently clean the buildup of soil on the alignment chain to avoid jamming. The other problem with the system was that the seedlings could not assemble nicely after dropping Chapter 2 : The Nursery Background 12 down to the bundler. The count of the seedlings in the bundle was triggered by two sensors mounted on the sides of the end of the conveyor. A s soon as the required number of seedlings were attained, a flipping plate was dropped to block the seedlings from falling down to the same bundler while a new bundler was indexed for the next group of seedlings. At the time when a new bundler was ready, the flipping plate was lifted up again. Apart from the aforementioned equipment, a line-scan based vision system had also been tested in the nursery (Fig. 2-6). The line scanner, which graded seedlings lengthwise as they passed between a line of photo-detectors and a backlight, was developed by a B . C . company (Byronix). High contrast silhouette images were obtained in this case. The grading rate of the scanner depended upon the average length of the seedlings. A n actuator located at the lower end of the belt rejected seedlings which did not meet the grading specifications. Generally the system was quite reliable, but the shortcoming was that the grading rate for this configuration is too limited and required the design of a new parallel transporting module in order to implement it in a production line. Figure 2-6 : Line-scan module 13 Chapter Three 3 . REVIEW OF PREVIOUS RESEARCH 3.1 Chapter Overview While the majority o f nurseries in British Columbia grow containerized seedlings, planting bare root seedlings is most common in nurseries in the United States, particularly in the southern regions where most of the active research on nursery mechanization takes place. Reflected in the number of studies found in the past decade, there has been a growing interest in the mechanization of the nursery lifting operation, especially in the field of bare root pine seedlings which are grown directly in soil. Seedling morphological properties such as root collar diameter, shoot height, projected root area and root/shoot ratio have been shown to affect seedling survival and growth [14]. These characteristics have a direct impact on the nursery production practices and hence the ease of the forest regeneration effort. Precise handling and accurate measurement o f morphological properties o f tree seedlings have always been tightly linked to each other and are also very time-consuming and delicate processes i f performed by hand. In order to gain a better understanding of how to devise an efficient scheme for automatic grading, it is helpful to first study the seedling handling and sorting methods reported via previous research. The term "handling" in the current context implies proper separation of seedlings, particularly prior to the sorting or grading process. Chapter 3 : Review of Previous Research 14 The term "sorting" implies the ability of partitioning into classes of acceptable and cull, whereas grading means the ability of partitioning into multiple classes. In the first part of this chapter, several handling and sorting technologies related to bare root tree seedlings are discussed. These applications are selected because the idea could be extended to containerized seedlings where no work has been previously done. The second part of the chapter is devoted entirely to machine vision for grading bare root seedlings. In general, grading of bare root seedlings is a more challenging task than that of containerized seedlings due to the nature of root formation. A machine vision system designed to grade bare root seedlings could easily be modified to grade containerized seedlings as well. Additionally, through the review of different techniques used in machine vision, we hope to gain more insight into designing a robust machine vision system targeted at container grown seedlings. 3.2 Seedling Handling and Sorting One of the pioneering developments regarding handling and sorting applied to bare root seedlings was an automatic plant selector by M a w and Suggs [17] as shown in Fig. 3-1. It employed a precision seeding and taping system [11] to load uniformly spaced seedlings on Mylar or plastic tape and roll the tape and seedlings all together into a bundle prior to introduction of a sorting machine. Seedlings were classified as good or cull on the basis of a length measurement made by a line of photo-transistors when a seedling intersected light directed onto it. A digital comparator circuit was designed to handle plants being presented Chapter 3 : Review of Previous Research 15 to the detector on a continuous flow basis. Cul l seedlings were destroyed by a guillotine knife. The success of the detector depended largely upon the shadow crispness projected over a certain number of photo-transistors. A seedling with a close canopy was more consistently detected with accuracy than one with spreading leaves. The results indicated the difficulty involved in obtaining a correct length measurement in a continuous production process. Ardalan and Hassan [3] compared a similar optical detection system with a mechanical linear displacement device to measure singulated bare root pine seedlings loaded onto plastic tape. The plant selection in the study was on the basis of stem diameter rather than height. Again, both designs were incorporated into the continuous flow operation. The mechanical displacement detection system made use of a linear vertical potentiometer attached to a roller (Fig. 3-2). A s the tape loaded with seedlings was forced up the roller, a '•EXECUTION PHOTOTRANSISTOR COMPUTATION PHOTOTRANSISTOR DESTROYED P L A N T S Figure 3-1 : Sketch o f the design proposed by Maw [17] Chapter 3 : Review of Previous Research 16 voltage proportional to the seedling diameter was generated regardless of the feeding speeds. However, the linear displacement system requires direct contact with the object being measured and could be a hazard to the seedling, particularly when running at low speed where seedling bending would occur. For the optical system, which was sensitive to speed variations, infrared modulated light was used so as to eliminate the interference of daylight. They found that detection was somewhat sensitive to the speed of feeding seedlings to the detector. 55 % of the seedlings were found to be measured within 0.1 mm of a specified diameter when fed to the detector at speeds between 15 and 26.3 cm per second. The sorting rate was approximately 3 seedlings per second, which was too slow for the replacement of manual sorting. Direction of Seedling Motion • * Figure 3-2 : Linear displacement device for measuring seedling diameter Chapter 3 : Review of Previous Research 17 Graham and Rohrbach [10] developed and tested a mechanical seedling singulator to remove single bare root pine seedlings from a bundle. Providing about 65% singulation success rate, the system made use of a wedge shaped vacuum and a rotating triangular seedling hopper (Fig. 3-3). The vacuum nozzle was designed to catch only one seedling by sucking it into the wedge where it would block the nozzle orifice. The seedling hopper rotated one third revolution for each seedling selection to prevent root entanglement. The developers determined that with two singulators working independently, a single seedling would be available for planting 95% of the time. Such an apparatus could be useful to singulate seedlings before the automatic grading process. ROTATING TRIANGULAR SEEDLING HOPPER SEEDLING S T E M (CROSS - S f r T l O M l VACUUM WEDGE NOZZLE - HOPPER \ WALLS » CROSS- SECTIONAL END VIEW UP AND DOWN VACUUM WEDGE NOZZLE TRAVEL SEEDLINGS PLATE Figure 3-3 : The rotating seedling hopper with the vacuum wedge nozzle Chapter 3 : Review of Previous Research 18 3.3 Machine Vision Application The need for faster recording and accurate measurement of seedling attributes has long been recognized as an important aspect in automating the grading process. Considerable research has been conducted on the application of computer vision for inspection of fruit and vegetables, but to a much lesser extent for the coniferous industry. One early example applied to coniferous plants was a digital device for measuring and recording of stem diameter, shoot height, and root mass area index of tree seedlings developed by Buckley [5]. The device used potentiometric transducers to measure diameter and height, and a moving linear 1024 element photodetector to measure root mass area index or silhouette area. The device required an operator to open the area sensor cover, place the root collar of a seedling in the stem transducer, close the area sensor cover, position the height transducer, and press a button to initiate the area measurement. The mechanism of the device was similar to that of the present day photocopiers except that it used backlighting. This apparatus was an improvement over manual measurement techniques and achieved high measurement accuracy. However, it is not suitable for grading large quantities of seedlings. Suh and Miles [31] used a machine vision system installed on an I B M - A T microcomputer networked to a Sun workstation to measure morphological properties of loblolly pine seedlings. Backlighting, which provides a sharp contrast image and is ideal for dimensional measurements, was used to simplify the image analysis. Images were first taken using an R C A CMR-300 camcorder with a zoom lens, stored on tapes, and then digitized by a Data Translation (DT) 2851 frame grabber (480 X 512 resolution) with an auxiliary frame Chapter 3 : Review of Previous Research 19 processor (Data Translation DT-2858). They used markers and template techniques to locate seedling root collars and terminal buds. Approximately 40 seconds were required for measuring and recording the data of each seedling with the system. However, the study demonstrated that machine vision and image processing can be used in a commercial production situation to grade seedlings into classes with high measurement accuracy. A n average relative error of 0.4 mm for measurement of root collar diameter, and 4.5 mm for measurement of shoot height were reported in the study. Similar in concept, Wilhoit and Kutz [32, 33] developed a machine vision system to measure bare root pine seedlings for quality control sampling. The system utilized two Watec 80 IN C C D cameras attached to a DT-2861 image multiplexer board and DT-2853 480 X 512 digitizer board to focus on different seedling parts to obtain maximum resolution. Again, it used fluorescent backlighting to create seedling silhouette for image analysis. To correct differences in lighting conditions, they used an adaptive thresholding technique [27]. Because Wilhoit 's study emphasized quality control analysis other than on- line grading, the processing rate was expected to be slower. In fact, it took about 14 seconds to process a seedling in the study. Rigney and Kranzler [22] assembled an inspection system for grading pine seedlings on- line. The seedling classification as acceptable or cull was based on minimum criteria for stem diameter, shoot height, and projected root area. A n International Robomation Intelligence (IRI) D256 machine vision development system was used with an image resolution of 256 X 240 pixels. Two Hitachi KP-120U black and white television cameras Chapter 3 : Review of Previous Research 20 were mounted above a conveyor belt painted in matte black. One camera measured the entire seedling with 2.0 mm resolution and the other focused on the root collar area with 0.5 mm resolution. A synchronized strobe xenon flash under fluorescent room lighting was used to freeze a singulated seedling as it traveled underneath the two cameras. To obtain an accurate measure of the stem diameter, they investigated several techniques such as binary thresholding, gradient edge detection, Laplacian edge detection, and a moments technique. A method employing a modified 3 X 3 Laplacian vertical edge detector, with a fast measurement time of 0.035 second for convolving with the root collar zone, was used in the study. The algorithm, written in C, was reported to be capable of grading between two [23] to four [21, 22] seedlings per second, and correcting for variations in seedling orientation and lateral position of up to + 30 degrees and ± 6 cm, respectively. A nursery worker was required to manually separate the seedlings and place them on the conveyor belt before the grading process. The overall inspection rate would depend upon the rate at which the worker could singulate the seedlings (in which case 4 seedlings per second is difficult to achieve). Nevertheless, in view of mechanizing the complete inspection process, the task of automated singulation is not easy which has been shown in [10]. To increase measurement precision and overall inspection rate, Rigney and Kranzler [24] considered the design of a line-scan based machine vision system. They realized the concept in their later works [25, 26] by building two prototypes, each using a single line-scan camera in combination with backlighting. Their first prototype, which made used of a linear 1024- Chapter 3 : Review of Previous Research 21 pixel camera ( E G & G Reticon, 1902), was installed on a 3 3 M H z V M E bus-based computer system equipped with a DataCube (Max-Scan) multifunction digitizer board and a (Max^ Graph) graphics board. The digitizing board provided camera timing, image digitization, thresholding and run-length encoding, and the graphics board supported display on an image monitor. The prototype could produce a grading rate of as high as 10 seedlings per second for 1+0 seedlings with a spatial resolution of 1 mm longitudinally and 0.1 mm laterally. Results of their study also showed that machine vision measurements had a standard deviation of approximately one-third those of manual measurement. This, again, proved that machine vision could comfortably outperform manual measurement, not only in terms of accuracy, but also of speed. Based on a similar idea, their second prototype only differed from the first one in terms of the platform and camera resolution. With the use of a 2048-pixel line-scan camera ( E G & G Reticon, LCI912) and a DT-2856 digitizer card housed in a 50MHz 486DX P C , they achieved a higher transverse resolution of 0.05 mm. Another improvement of the system was an interactive user interface through which an operator could modify the setpoints of the measurement as well as observe the graphical results in real time. However, because of the overhead of graphics display, the inspection rate dropped down to about 3 seedlings per second. One limitation of Rigney and Kranzler's work is that their systems (both 2-camera configuration and line-scan configuration) could only perform grading in a one-by-one seedling basis, i.e. only one seedling appearing within the camera field of view at a time. 22 Chapter Four 4. DESIGN OF A MACHINE VISION PROCESS 4.1 Chapter Overview Machine vision, also referred to as computer vision, has been described as "the integration of image sensors with digital computers to obtain useful information" [24]. If properly applied, machine vision can dramatically reduce production costs and improve product quality by enabling accurate and inexpensive 100% inspection of the products. In order to better understand how machine vision works, it is necessary to understand the processes or the elements of which a machine vision system is comprised. Therefore, the objective of this chapter is to present an overview of the five basic processes of machine vision in relation to the system developed in this thesis. The five processes are (1) illumination, (2) image acquisition, (3) image segmentation, (4) feature extraction, and (5) image interpretation. 4.2 Illumination The formation of an image begins with the generation of light reflected from the object being observed. The quality of the image can be greatly affected by certain fundamental characteristics of the illumination source, such as the wavelength of the source and the orientation of the source with respect to the object. Suitable illumination not only can Chapter 4 : Design of a Machine Vision Process 23 increase image quality but also facilitate inspection, avoiding the need for complex image processing algorithms. Research has shown the importance of illumination techniques associated with different conveying methods in produce inspection [1]. Different scene illumination techniques are useful in the acquisition of different object features (Fig. 4-1). When certain key surface features must be studied, such as texture and label on a container, diffuse front lighting is desirable. It floods the area of interest with light such that the surface characteristics wi l l act as the defining features in the image and is particularly useful for working in gray scale. In contrast, backlighting provides high contrast silhouette images, useful in locating image boundaries for dimensional measurement or recognition of object presence. To detect surface defects, both dark and light fields of specular illumination can be used. In the dark field specular illumination, as light striking a specular or mirror surface reflects off at an angle that is equal and opposite to the incident ray, the only illumination going to the sensor is the redirected light rays caused by the defects or scratches. As a result, defects show up as bright spots in a dark field in the image. The light field illumination uses the same principle except that the image sensor is positioned in line with the reflected ray. The only time illumination is not transmitted is when deformity of the specular surface exists which shows up as dark spots in a light field in the image. Chapter 4: Design of a Machine Vision Process 24 Camera Light source (A) Front Lighting - Diffuse (D) Front Lighting - Directed Bright Field Transparent T Table o o o (B) Back Lighting (E) Strobe Lighting Screen with pattern (C) Front Lighting - Directed Dark Field (F) Structured Light Figure 4-1 : Different illumination techniques Chapter 4: Design of a Machine Vision Process 25 Another comrnonly employed illumination technique used in machine vision is strobe lighting. It is useful when it is desired to freeze the apparent motion of moving objects. Strobe lighting is provided by a short pulse of a high-intensity light source, like a xenon flash; but close attention must be paid to both the duration of the flash and the synchronization with the image sensor during the presence of the object. In a broad sense, structured light is the use of illumination of the object with a special pattern or grid. Depending on the shape of the surface feature, the intersection of the object and the projected illumination results in a unique pattern which can be used to determine the three-dimensional shape and orientation of the object. One example of structured light application is the use of a laser line projected on an object to measure its depth through triangulation. Another consideration in proper image illumination is the choice of the light source. The source of light should be chosen in a way that its output/wavelength matches the sensitivity/wavelength of the image sensor [4]. For example, flash tubes are considered to be inefficient when used with solid state sensors because most of the output has a short wavelength, whereas the sensors are sensitive in the higher wavelength range (400 - 1100 nm). Fig. 4-2 shows the spectral distribution of some common illuminators used in machine vision. Moreover, i f the images are to be processed in color, the colors of interest can be detected only when the appropriate wavelengths exist in the illuminators. Chapter 4 : Design of a Machine Vision Process 26 1.4 1.2 + 1.0 Relative o.8 -j~ response 0.6 + He-Ne Laser Neon Lamp Fluorescent Lamp Flash 0.4 + Tubes 0.2 f N s — LED N \ S N Tungsten / \ \ Lamp / \ \ Quartz Halogen x \ Type Lamp Laser Diode 4 1 h H 1 1 —H 200 400 600 800 1000 1200 1400 1600 1800 2000 Wavelength (u.m) Figure 4-2 : Spectral emission o f some common illuminators used in machine vision (Source: E G & G Reticon 95/96 Image sensing manual) Note that the above mentioned illumination techniques and light sources are not meant to be inclusive. They are just a few of the common types used in machine vision systems. 4.3 Image Acquisition Once the scene has been properly arranged, an imaging device is needed to create the two- dimensional representation for further processing and interpretation. A n imaging device is a sensor which collects light from a scene (typically through a lens) and then converts the light into an electrical signal through the use of some type of photosensitive target. There are two basic types of imaging device for use in the machine vision industry: tube sensors Chapter 4: Design of a Machine Vision Process 27 and solid state sensors. Tube type (vidicon) cameras were commonly used in the early vision industry but were later replaced by solid state cameras because of several important advantages. In addition to being smaller than vidicon cameras, solid state cameras are also more rugged and tolerant of intense illumination and magnetic fields. Their performance is not degraded by geometric distortion and lag. Image lag appears as a ghost of a bright object after it has moved, and results from electric charge remaining on the sensor after the scan. The most common type of sensors used in solid state cameras is the charge-coupled device (CCD). These sensors can be fabricated in different geometric configurations, such as linear, matrix and rectangular. When light passing through the camera lens strikes the array, each photosensitive element converts the portion of light falling onto it into an analog electric signal. The entire image is thus broken into arrays of individual picture elements (pixels). The magnitude of the analog voltage for each pixel is directly proportional to the average of the light intensity variation over the area of the pixel. C C D arrays differ from others sensor types, like CID (charge-injected device), primarily in how voltage is extracted from the sensors. Typical matrix array solid state cameras have 1024 X 1024 photosensitive elements, although a number of other configurations, such as 256 X 256, 512 X 512, 2048 X 2048 are also readily available. The output from matrix cameras may or may not be compatible with commercial T V standards (RS-170) to accommodate a more flexible frame rate. Linear array or line scan solid state cameras may have from 256 to 4096 or more (8000 under development) elements. A complete two-dimensional scene can be acquired by a matrix Chapter 4 : Design of a Machine Vision Process 28 camera with a single exposure. For a line scan camera to acquire the same image, it must be positioned in relation to the scene and then the camera must be moved past the scene, or the scene moved past the camera. However, the use of a line scan camera facilitates many types of applications, such as inspection of a part traveling on a conveyor or mechanical scanning device using a rotating mirror which requires the object to be analyzed on a line-by-line basis. In addition, there are two main reasons to consider using line scan cameras : resolution and speed. Generally, resolution means either greater field-of-view or greater accuracy in measurement. Resolution is basically the number of pixels in a line. Matrix cameras nowadays offer high resolution (2048 X 2048 elements); but their use may not be justified due to the higher cost (large total number of sensing elements and the up-to-date technology). On the other hand, line scan cameras having 4096 pixels are not uncommon. One line scan camera can be used instead of two matrix cameras for the required horizontal resolution. When the exposure time (or integration time in electronic equivalent term) of the line scan camera is well adjusted with the velocity of the inspection belt, reasonably high vertical resolution can also be acquired. In terms of speed, line-scan cameras generally have a higher pixel output rate than a matrix camera. Image data can be processed line-by-line, as opposed to acquisition of an entire frame rate before processing on conventional systems. In other words, the image data in the line-scan camera is always ready for processing. In contrast to a matrix camera, extra care should be taken on the proper triggering of the image capture. Chapter 4 : Design of a Machine Vision Process 29 The output of the camera usually results in a series of voltage levels representing light intensities over the area of the image. This preliminary analog video must be converted into digital format before it can be stored or processed in the computer. This is usually accomplished through some sort of plug-in interface board. The range of interface board could vary from a simple A / D converter to a full scale image processing system depending upon the complexity of the image operation. A typical low-end commercial frame grabber contains features like : signal digitizer, look up table, fast storage buffer, display support, external trigger and some basic image manipulation routines such as image subtraction and file type conversion. Although there exists such a variety of functionality, all interface boards perform the basic transformation of pixel intensity into discrete gray level of 8 to 24 bits long. 4.4 Image Segmentat ion The third general step in the vision sensing process is to distinguish the subject from the background. For the simplest case, an object may be adequately segmented from its background with proper illumination. With the use of backlighting, an object appears as a silhouette on a white background. Typically the imaging device might produce a gray scale image like the one as shown in Fig. 4-3 with its corresponding gray scale histogram which, basically, is a plot of the frequency distribution of the gray levels in the image (Fig. 4-4). In the case here, the histogram of the backlit object contains a group of pixels predominately in the dark area (the object) and a group of pixels in the predominately light area (the Chapter 4 : Design of a Machine Vision Process 30 background). Such a distribution is bimodal. A n optimal binary segmented image can be obtained by thresholding the image at a gray level between the two peaks. A l l pixels darker than or equal to the threshold are mapped to black and all pixels lighter than the threshold are mapped to white. A n appropriate choice of thresholding value could have a significant effect on the image segmentation and thus the shape and size of the object in the binary image. Figure 4-3 : 256 gray-scale image of a seedling Chapter 4: Design of a Machine Vision Process 31 6,000 Gray scale Figure 4-4 : A bimodal gray level histogram 4.5 Feature Extraction The reason that machine vision systems are being increasingly employed in automation processes is that typical parts from the object being examined tend to have distinct shapes or properties that can be recognized on the basis of very elementary features. Very often these distinctive features are simple enough to allow identification of the parts or the objects, or the corresponding position, orientation, or size. The process of determining these elementary features of the image is referred to as feature extraction. In most cases, the features selected are highly dependent upon the specific application and the analysis technique employed in a later stage. To measure geometric properties of an object, feature like "edges" are usually found to be useful and the operation for extracting Chapter 4: Design of a Machine Vision Process 32 these is called "edge enhancement", which is typically followed by a thresholding/segmentation process [16]. A n edge is a boundary within an image where there is a large change in light intensity between adjacent pixels. These boundaries usually correspond to real edges in the object being examined and thus are very important for dimensional measurement. Edges are usually determined by using one of a number of different "edge masks" which mathematically calculate the presence of an edge point by a weighted summation process (convolution). To illustrate the idea, consider an image of a simple dark object on a light background having an intensity profile along a horizontal scan line of the image (Fig.4-5). It is noted from the profile that an edge is modeled as a ramp, rather than as an abrupt change of intensity. The first derivative (the gradient) of the profile is zero in all regions of constant intensity, and constant during the intensity transition. The second derivative (the gradient change), on the other hand, is zero except at the start and end of an intensity transition. Based on these observations, it is evident that the gradient magnitude can be used to detect the presence of an edge, while the sign of the gradient change can be used to determine whether an edge pixel lies on the light (background) or dark (object) side of an edge. Chapter 4 : Design of a Machine Vision Process 33 Figure 4-5 : Elements o f edge detection : Dark object on a light background In a digital image, the magnitude of the gradient at any pixel is computed by convolution of an edge mask with that pixel and its neighbors. To save computation time, the size of the mask or "convolution kernel" is usually 3 by 3 and it is by far the most common size used in Chapter 4 : Design of a Machine Vision Process 34 industrial computer vision due to the low computation requirement compared to large convolution kernels. There also exists a large number of edge masks and among the most widely used are the Sobel, Gradient Directional and Laplacian edge enhancement operators [16]. Sobel is computationally very efficient; Gradient Directional favors edges in a selected direction; and the Laplacian is insensitive to edge orientation and generates sharper edge definition due to its second derivative nature (i.e. it detects magnitude of gradient changes). Depending on the edge mask used, the transformed image wi l l represent either the gradient magnitudes or the magnitude of gradient changes. The gradient or edge-extracted image may be used for dimensional measurement. Another operation commonly used in the feature extraction process is called runlength encoding which is used to compress the amount of data required to be stored. This operation is similar to edge detection in binary images. In runlength encoding, each line of the image is scanned and transition points from black to white or white to black are noted, along with the number of pixels between transitions. This runlength data is then stored in memory instead of the original image and serves as the starting point for later processing. Both feature extraction operations (edge enhancement and runlength encoding) stores only the features of the image instead of the entire image in memory, thereby substantially reducing the amount of memory required. However, runlength coding is much simpler to implement for a simple image in an on-line inspection process where speed is of concern. Convolution is always computationally expensive and its use is only justified when it is supported in special purpose hardware. Chapter 4 : Design of a Machine Vision Process 35 4.6 Image interpretation When a system has completed the process of analyzing image features, some conclusion must be made about the findings, such as the verification of the presence of the object, the identification of the specific part in the object or the geometric properties of the object. Based upon these conclusions, certain decisions can then be made about the object or the production process. These conclusions are formed by comparing the results of the analysis with a certain set of standard criteria. In an inspection operation, the whole image interpretation process is usually incorporated into the system through software means. The decision made by the system may then be used as the input to other processes in the whole production line. 36 Chapter Five 5. DESIGN AND DEVELOPMENT OF T H E PROTOTYPE 5.1 Chapter Overview From the study of previous automation efforts, it can be seen that the handling problem of how well the seedlings can be separated and orientated prior to grading is the limiting factor in automating the whole production line. In addition, the inspection configuration plays an essential role in efficient grading. In this chapter, a novel design of a machine vision and handing system is introduced and the development of the prototype is discussed. This design, which is based on line-scan imaging of fiber optics in conjunction with the use of structured backlighting, aims at eliminating the complicated process of separating the seedling as well as providing reasonably high measurement accuracy and inspection rate. The chapter first starts with the assumptions of the design, followed by the grading criteria used in the system. Then the equipment used for the construction of the prototype is described. Lastly, the grading algorithm and the design of the user interface are presented. 5.2 Assumptions A s was mentioned earlier, building a proof-of-concept prototype was one of the primary objectives o f this research. It could be used to prove the concepts as well as to confirm that mechanical and electrical parts could be integrated with the computer and vision system in Chapter 5 : Design and Development of the Prototype 37 order to simulate the operation at a reasonable rate. There were, however, several assumptions made regarding this prototype. • A s the automated process of extracting seedlings out from the block had already existed and had high performance in operation, we would merely adopt the technology, while requiring minimum changes to be made to the existing system configuration. A s a result, a seedling processing rate fast enough to synchronize with the extraction rate (8 - 10 seconds per row) would benefit the overall production rate. • Containerized seedlings are grown in styroblocks with various configurations of cavities. It could range from 8 to 16 cavities per row in a styroblock. A s being one of the most popular sizes for 1+0 seedlings, the 14 seedlings per row block was used in this study. • Interior spruce is one of the major and popular tree species used by the commercial nursery industry in the province of British Columbia. Because of the morphological differences between species of containerized seedlings, this study was limited to the grading of containerized interior spruce. However, it is anticipated that the methodology and grading algorithm developed here could be adapted to the grading of other species of containerized seedlings. Chapter 5 : Design and Development of the Prototype 38 5.3 Seedling Grading Criteria Morphological characteristics are used for grading most nursery stock. These characteristics include stem diameter at the root collar, shoot height, root growth and plug integrity. Stem diameter and shoot height are generally given priority and were adopted as the grading criteria for this project. Stem diameter is typically considered more important. The measurement accuracy of the stem diameter and shoot height is to be 0.1 mm and 1 cm respectively for machine vision grading. 5.4 System Hardware This section describes the equipment used for constructing the prototype. The main components were a conveyor system, fiber optic strips, the camera system, a structured backlight and a PC. Fig. 5-1 shows a picture of the prototype and Fig. 5-2 shows the corresponding schematic (however, only 4 seedlings are shown to keep the diagram simple). Figure 5-1 : The prototype o f the scanning system Figure 5-2 : Scanning system using fiber optics Chapter 5 : Design and Development of the Prototype 41 5.4.1 C o n v e y o r System To simulate production inspection and sorting operations, two sets of variable speed conveyors were constructed. The first conveyor, the inspection conveyor, consisted of two sections of conveyor belt aligned end-to-end in the same direction of travel. A row of 14 seedlings was placed manually on the first section of the inspection conveyor to which the extractor was expected to connect. Driven by a 12V D C gear motor with a 2 A power supply, the inspection conveyor was capable of operating at a speed between 2 to 6 cm/s. The width of both sections were 61 cm (2 feet) and the length of the first and second section were approximately 34 cm and 17 cm respectively. A set of 2.5 cm tall guide-rails was used in the first section to separate the seedlings and to constrain their lateral movement before entering the inspection stage. With tops pointed toward the direction of travel, the seedlings were individually scanned as they crossed a 1 cm gap between the two sections. Strips of plastic fiber optic ribbons, each corresponding to the transverse field-of-view of a seedling, were used as the scanning mechanism and were sandwiched between two aluminum plates mounted at approximately 5.5 cm above the gap. A s the seedlings passed the gap, they intersected the structured backlight projected onto the fiber optics proportionally to their size and shape, forming high-contrast images. These signals or images were then used for the measurement of the seedling morphological features. A more detailed description of the scanning mechanism is provided in the coming sections. However, to provide more stability, the inspection conveyor was made of non-slip foam. Chapter 5 : Design and Development of the Prototype 42 The second conveyor system was the sorting conveyor, running perpendicularly to the inspection conveyor. The sorting conveyor was made up of two synchronous belt drives with 4 timing pulleys of equal size and was powered by a 90V gear motor with a controller to vary the speed. Synchronous belt drives eliminated the problems of noise, slippage and lubrication because power was transmitted by positive engagement of belt teeth rather than by friction as in conventional belt drives. Two sets of 14 seedling holders were bolted across the belt drives directly opposite to the other and were used for picking up the seedlings from the end of the inspection conveyor. The motion of the sorting conveyor was controlled in such a way that the sorting conveyor remained motionless upon the arrival of seedlings for a short duration; then it moved to transport the row to the next stage and stop again when the other set of holders settled in position awaiting for the next row of seedlings. Whenever the sorting conveyor stopped, the seedling holders were aligned with the seedlings coming out from the inspection conveyor. For this simple "stop and run" motion, a circuitry was designed to add to the motor controller to create a duration of 2- seconds stop and 6-seconds run. Fig. 5-3 shows the circuitry and the details of design is documented in Appendix A . Chapter 5 : Design and Development of the Prototype 43 Figure 5-3 : Motor control circuitry for the sorting conveyor 5.4.2 F i b e r Opt ics Fiber optics is best known for its communication applications. Nevertheless, it was used for the application of image transmission / light guiding as part of the image acquisition process in this project. A s briefly mentioned in the previous section, 14 strips of optical fibers were used as the linear sensing device to scan seedlings individually as they crossed the gap between the two sections of the inspection conveyor. Each strip was made up of three 13 mm wide plastic fiber ribbons (Mitsubishi Rayon, SK-10R) taped side-by-side, thereby forming an effective transverse field-of-view (FOV) of approximately 4 cm for each seedling. A s for the ribbon itself, it was made up of bare plastic fibers with 250 urn in diameter aligned and bonded parallel to each other. Each fiber has a concentric double-layer structure consisting of a core of transparent polymethyl methacrylate ( P M M A ) covered with a thin layer of fluorine polymer cladding. The refractive index of the core is higher than that of the cladding, so light in the core that strikes the boundary with the cladding at a glancing angle is confined in the core by total internal reflection, as shown in Fig. 5-4. Chapter 5: Design and Development of the Prototype 44 Clad ( n 2 ) Core (rij) lower refractive index Total Internal Reflection Figure 5-4 : Structure o f optical fiber and diagram of light transmission Numerical aperture (NA), which relates to the brightness of the optical fiber, i.e. quantity of input light, is expressed by the formula as follows : NA = y] n\ -n\ where «, and n2 represent the refractive index of the core and cladding respectively. For our S K series fibers with nx = 1.492 and n2 = 1.417, A M = 0.467. Therefore, the acceptance angle (a ) the angle over which light rays entering the fiber would be guided along its core, is: a = 2 s i n - 1 n\ - n\ = 56° In the design of the fabrication of fiber optics, the acceptance angle or the numerical aperture is an important parameter because fiber optics usually require a lens or some form of light structuring to be useful. In the next section of illumination, two different designs of Chapter 5 : Design and Development of the Prototype 45 illumination system wi l l be discussed in order to project light onto the fibers so that the proper silhouette image could be formed as the seedling moved past these "line-scan" fiber optic sensors. Other considerations in choosing this particular series of optical fibers were its flexibility, ease of handling (i.e., it can readily be cut without special tools) and low attenuation in the visible spectrum (400 - 800 rim) as shown in Fig. 5-5. -Collimated Light 600 700 Wave Length inm) soo Figure 5-5 : Spectrum attenuation o f the S K series optical fibers (Source : Mitsubishi Rayon C o . , Ltd) Attenuation measures the amount of signal lost in transmission by comparing output power with input power as defined as [9]: Chapter 5 : Design and Development of the Prototype 46 LOSS in decibels (db) = - 10 log r Power ^ 10 Power ,„ J A t a wavelength of 633 nm, which corresponds to that of Helium-Neon laser, the signal only experienced an approximately 250 db/km loss. This translated to an efficient 94% net light transmittance per meter. 5.4.3 I l l u m i n a t i o n Selection of the illumination technique and the setup o f the corresponding optics are critical to the system performance. This section describes two illumination systems designed for the prototype based on the technique of backlighting. The second illumination system was designed to overcome problems encountered by the first system. The objective of the both designs, however, was to couple light properly to the fiber's F O V . Fig. 5-6 shows the first design of the illumination system in which two long plano-convex cylindrical lenses were arranged in axial alignment with each other such that a light ray was transferred from one focal point (the source) to the other (the fibers). The image detected by the fibers and thus transmitted to the camera system was o f silhouette form. It was produced by a series of halogen lamps shining from behind while the fibers pointed straight down to the object (seedling) as shown. Note that the use of backlight, which provided high contrast, eliminated the need for computationally expensive edge enhancement process. Chapter 5 : Design and Development of the Prototype 47 Fiber Figure 5-6 : First illumination design with cylindrical lenses Similar in appearance to spherical plano-convex lenses but rectangular in shape, these two cylindrical lenses are chosen for two reasons. The first was the long focal length (91 cm) because it provided a reasonable clearance for the seedling passing between the fibers and the lens. The second is the light gathering capability, quantified as follows: fl# = F + D NA = 05 + f/# where / / # = f-number, F = focal length, D = diameter of the lens and N A = numerical aperture. Chapter 5 : Design and Development of the Prototype 48 In Fig. 5-6, theta (f?) illustrates how the numerical aperture of the lens controls the angle of light converged by the lens. A s the f l# increases, theta, the acceptance angle, decreases and the lens becomes more capable of converging light. To be sensitive in detecting the object, it is important that the acceptance angle of the lens (f?) is as small as possible within the range covered by the fiber {a). In our case, the chosen cylindrical lens yielded the best combination of physical length, focal length and light gathering capability, giving F = 91 mmand a = 41.8 ° < 0 = 5 6 ° . However, test results with this optical setup were unsatisfactory. Due to the light-emitting nature of the source, light rays coming at an angle to the length of the lenses interfered with the projection of the shadow of the object being measured. The lens combination was only capable of focusing light in one direction. A 2 dimensional focusing lens combination would have been used to adjust the problem or some sort of light control "louver" had to be used to restrict the direction of light prior to the lens system. Nevertheless, the ideal condition of light coupling would simply be a plane of collimated light projecting onto the field-of-view of the optical fibers, which was approximately 61 cm (2 feet) wide. It is difficult to optically generate such a wide span of collimated light. However, a second optical illumination system was designed in an attempt to simulate such a condition. Chapter 5 : Design and Development of the Prototype 49 Fiber Strips Laser beam 30/70 Splitter 50/50 Splitter Right angle prism Figure 5-7 : Second illumination design with structured light The concept of combining structured light and backlighting effect was used to project a single line pattern on the fiber optics as shown in Fig. 5-7. A n 8 m W helium-neon laser with a wavelength of 633 nm and a beam diameter of 3 mm was used as the illumination source. B y using two mirror-type beam splitters (30/70 and 50/50, %reflectivity/%transmission).and a right angle prism, the laser beam was broken into 3 separate beams with approximately one-third of the original input power. A section of glass rod was placed in front of each laser beam to project a plane of light with the thickness equal to the original beam thickness Chapter 5 : Design and Development of the Prototype 50 onto the fiber optic sensors. Two dividers were used to separate each section as shown and they were each wrapped with a piece of black cotton to avoid specular reflection from the stray light, which could lead to false readouts from the sensors. With this method of light projection, light rays entering the fiber sensors closely approximated that of collimated light. The discrepancy in this case was adjusted through calibration. Fig. 5-8 shows the corresponding optical setup. Figure 5-8 : Optical setup for the prototype with the custom made optical bench The effect of transforming a point of light into a line image by using a glass rod is basically the same as that by using a cylindrical lens. Fig. 5-9 illustrates the idea. Chapter 5: Design and Development of the Prototype 51 I s Beam radius =1.5 mm Rod diameter = 3 mm F (5/ 2, Half fan angle Figure 5-9 : Transforming o f a point image into a line image using a glass rod A s the beam passing through the glass rod, the dimension of the beam in the y-direction is stretched or magnified while its dimension in the x-direction remains unchanged. Using a simple geometric calculation, the fan angle (j3) can be determined based on the radius of the beam and the focal length of the rod. The advantage of using a glass rod over cylindrical lens is that the focal length of a glass rod is much smaller given the same curvature. A s a result, a wider fan angle and a longer line can be obtained. To be compatible with the set distance of 42 cm between the rod and the fibers, a 3 mm diameter glass rod was used for each of the three sections dividing up the entire 61 cm span. Each section of this setup achieved a fan angle of approximately 120° and generated a line of 145 cm long. Since the intensity of the laser beam is Gaussian's distributed, the Chapter 5 : Design and Development of the Prototype 52 corresponding line generated is "bright" around the center region and fades out towards the ends. Therefore, it was desirable that a much wider line be used to cover each section of the span so that the portion projected onto the optical fibers would be as uniform as possible as the "bright" region was stretched to a longer range while the remains of the line portion were blocked or cut out by the divider which separated the sections. 5.4.4 C a m e r a System The use of fiber optics in this project has realized grading of more than one seedling at a time in parallel without sacrificing resolution by using only a single area camera. The fiber strips were arranged such that they were all perpendicular to the direction of motion. The diameter of the individual fiber dictated not only the transverse resolution but also the longitudinal resolution as viewed by the C C D camera. This section describes the camera system used in the prototype and how the output signal from the fibers was mapped to the C C D sensors. A digitally controlled/digital output television-like monochrome camera (Electrim E D C - 1000HR) with a half-size (16-bit) plug-in card was used as the camera system. The camera has a resolution of 753 (H) X 244 (V) pixels operating in non-interlaced mode and 753 (H) X 488 (V) in interlaced mode. The EDC-1000 H R camera system is an inexpensive alternative to a standard T V camera attached to a frame-grabber board. It has the feature of fixed pixels (direct mapping between pixels and computer memory bytes) and a spectral range from 0.4 urn to 1.1 um. The output from the camera system is an 8 bit digital signal Chapter 5: Design and Development of the Prototype 53 corresponding to the quantized value of brightness at serially sampled spatial data points (256 gray levels). The computer-controlled exposure time can be varied between 1 millisecond and 10 seconds and the pixel transfer rate is up to 1.6 Mb/sec. Frame rate can vary depending on the exposure time, display resolution, and the computer clock speed. Image data existing in computer R A M after each scan is processed by the computer C P U . The camera head unit was equipped with a 12.5 - 75 mm focal length zoom lens and an additional lens extension ring was used to insert between the lens and the camera in order to allow a shorter focal distance (or working distance). A s shown in Fig. 5-2, the ends of each fiber strip were assembled in such a way that one stacked on top of the other with a small clearance in-between so that the linear images conducted by the optical fiber strips would map to certain rows of the C C D sensor arrays. To achieve this high-precision light coupling, a fiber holding unit was constructed as shown in the assembly drawing in Appendix B . Pieces of approximately 1 mm thick plastic plate were used to separate the fiber strips and the side facing the camera was painted in matte black to distinguish the output signal of the fibers. The fiber unit, which had a size of 3 X 3 X 1 lA inch, were housed in one side of the camera mount made of an aluminum square tube, while the camera was secured in the other side of the mount. The camera mount also accommodated a slot for the fiber holding unit to slide in either direction along the camera's optical axis to adjust the desirable F O V . Fig. 5-10 shows a picture of the camera mount supported by a frame above the conveyor system. Detailed dimensions of the frame can be Chapter 5: Design and Development of the Prototype 54 found in Appendix C. Fig. 5-11 shows an instance of an "unprocessed live" image as viewed by the camera. Figure 5-10 : Camera setup Figure 5-11: Raw image as viewed by the camera system (no object detected) The camera was orientated in a fashion that the fiber strips lay horizontal in the F O V , and was operated in non-interlaced mode (i.e. 244 rows of pixels). Interlaced mode would otherwise only enhanced the image for human viewing; and it was not desirable for machine Chapter 5 : Design and Development of the Prototype 55 vision processing because the double-vertical-resolution effect was generated by shifting down alternative rows by one-half pixel. A n aperture (f-number) of f / 4 was found to be the optimal value used for contrast setting. With the fiber unit positioned at 13 cm in front of the camera, the zoom lens was set at a focal length of 25 mm to obtain a transverse spatial resolution and a longitudinal spatial resolution both at approximately 0.1 mm. Higher spatial resolution in both directions could be achieved by further decreasing the F G V (i.e. reducing the focal length in the zoom lens). However, the choice of this spatial settings was justified as follows. Basically, the accuracy of the seedling's diameter measurement is governed by the resolution (or diameter) of the Optical fibers which is 0.25 mm. Nyquist's sampling theorem states that in order to recover the information in an undistorted form after a sampling process, the original information must be bandwidth limited to half the sampling frequency. In digital image processing, sampling frequency is equivalent to spatial resolution because of the finite size pixels. Since in our case the image as viewed by the fiber optics was already limited to a resolution of 0.25 mm, in order to retain the accuracy of the stem diameter measurement or be able to measure the diameter of the fiber with subpixel accuracy, the sampling has to be performed at a resolution higher than 0.125 mm (i.e. < (0.25 +2) numerically). A resolution lower than 0.125 mm (under-sampling) would miss the features of the image and introduce artifacts. Therefore, a transverse (or the scan direction as shown in Fig. 5-12) resolution of 0.1 mm was an appropriate choice in terms of speed and the conservation of the stem diameter Chapter 5 : Design and Development of the Prototype 56 accuracy imposed by the fibers. A higher value could be chosen at the expense of lower processing speed due to higher number of pixels representation. Optical fiber Pixel average value as seen by the sensors Figure 5-12 : Sensors mapping With the longitudinal spatial resolution set at 0.1 mm, two rows of the C C D array were used to represent the resolution of a fiber strip in the direction perpendicular to the scan. In a static situation, the resolution of the fiber or the two-rows length in the C C D sensors determines the longitudinal resolution capacity of the system (assuming that the numerical difference between the resolution of the fibers and that of the C C D sensors are insignificant, i.e. 0.25 - 0.10 x 2 = 0.05 mm). In a dynamic situation where motion is present, the longitudinal resolution is also affected by the exposure time and the speed of the seedling's Chapter 5: Design and Development of the Prototype 57 motion (i.e. the speed of the inspection conveyor). Fig. 5-13 shows an illustrative example of the effect of motion upon the longitudinal resolution of two coupled rows of pixels. Distance travelled during exposure time Motion u so / End of intergration \ Start of integration Figure 5-13 : Motion and resolution The darkly shaded area is a projection of the coupled array pixels on the conveyor surface and the lightly shaded area shows the actual portion of the belt seen during the exposure time. In such case, the effective longitudinal resolution consists of a static resolution component plus a dynamic component equal to the distance the object moves in the direction of motion during a given exposure time. The same argument was used for the calculation of the longitudinal resolution for the height measurement in the system, which can be expressed as : Longitudinal Resolution = Static Longitudinal Resolution + [ {Belt Velocity) x {Exposure Time) ] Chapter 5 : Design and Development of the Prototype 58 where static longitudinal resolution = 0.10 x 2 = 0.20 mm. One obvious advantage of this setup was that the longitudinal/height resolution could be individually adjusted by either the belt velocity or the exposure time. In addition, two dimensional images could be constructed line-by-line "endlessly" for each section as the seedling moved past the fiber optic strip. 5.4.5 C o m p u t e r The image processing operation was implemented in software by the computer C P U . The host computer was a 486 DX4-100Mz P C equipped with 16 M b of R A M . A n A T I mach 32 chip set V E S A graphics card with 1 M b of memory was used as the graphics display card for both image display and command interface. One 16-bit (ISA) slot in the computer housed the digitization board of the camera system which provides software-controllable camera exposure and image digitization at a rate up to 1.6 x 10 6 pixels per second. With 753 pixels in a row, 2 rows representing a single fiber strip, and 14 strips in total, the maximum image acquisition rate was approximately 76 Hz , resulting in a lowest attainable exposure time of 13.2 milliseconds. 5.5 System Software Acquiring the line scan data into the computer is just the beginning phase of the entire measurement process. A software program, written in the C programming language, was developed for the on-line morphological inspection of the seedlings. The software also Chapter 5 : Design and Development of the Prototype 59 provides a convenient, multi-functional, menu-driven graphical user interface (Fig. 5-14) which supports different scanning and display options, camera controls, grading criteria selection, statistical reports, and options such as saving runlength data to file. This section wi l l begin with the overall description of the program modules. The stages involved in the initialization process wi l l next be discussed. Then the scanning operation and the measurement (grading) algorithm wi l l be presented, followed by the description of the different functions supported in the software. * a Fiber Scan 3 a l \m\ ml Effjg A [ Display Sett ings S t a t i s t ics Quit \ ! / Welcome to fl.S.G.S. 1 m By Wing Poon A u t o m a t i c S e e d l i n g G r a d i n g System Figure 5-14 : User interface o f the prototype 5.5.1 Desc r ip t ion of the P r o g r a m M o d u l e s The interactive grading software is the predominate part of the entire system. It is composed of four program modules: control.c, menux, graphics.c and chartx. A l l of these programs used standard C functions under the Microsoft C/C++ development environment (version 1.51). Module control.c contains the main() program function to control flow among Chapter 5 : Design and Development of the Prototype 60 different menu options (run(), display(), setting(), stat()) as well as the runlength encoding (run_length()) and measurement (grade()) functions. In addition, it contains a camera initialization routine (init_camera()), four scanning routines (i_scan_1(), g_scan_1(), scan_4() and scan_all()), which correspond to different display or scanning modes, and other supporting routines like memory allocation, for image and display buffer, status bar animation for object detection, and results display in both graph plotting and table format. Module menux consists o f routines which build up the menu-driven interface and handle error messages. Graphics.c has routines which are responsible for general screen layout and color setting for each display mode. The last module, chart.c, contains.routines which are used when the "show result" option under statistics menu is selected. For example, pie_chart() routine, which is used for displaying the percentage distribution of detected objects in pie chart format, and plot_result(), which is used to plot the height and diameter distribution, are two of the statistics routines defined in the chartx module. A short description of all the 47 (3 camera's linkable and 44 user-defined) functions/routines listed in an alphabetical order can be found in the function prototype section in control.c. 5.5.2 Camera Initialization Initialization (init_camera()) has to be performed prior to any scanning operation (Fig. 5- 15). The process involved consists of four stages. Chapter 5 : Design and Development of the Prototype 61 Stage 1 : Memory space is first allocated (build_section()) for each section variable (of structure section) enough to hold a 500-row-long continuous runlength image and other parameters. Stage 2 : The pointers pointing respectively to the beginning of two lines corresponding to the location of a fiber strip in the image buffer (characterized by an array of 244 pointers) are assigned to the two line parameters (linel and Iine2) of a section variable. This is done for all 14 fiber strips. The locations (or line numbers) of each of the fiber strips in the image buffer array were determined in advance through a simple program designed to display only the rows selected; and the corresponding lines were examined visually to see i f they covered the entire strip. Stage 3 : After the assignment of all the line parameters, the function hsubcam1() of the camera is called and an image is read into the image buffer which has an allocated size of 753 x 244 bytes. Function hsubcam1() is a linkable routine provided with the camera system and is used for the sub-array scanning operation. It is passed the parameters like exposure time, clear count, antiblooming flag, interlace flag, the address of the pointer to the start of the first line in the image buffer array, and the address of the line length array which specifies the number of pixels to be read from the beginning of each individual line selected. The antiblooming function is turned on, but the interlace option is turned off during the operation. The default value of the exposure time and clear count is set to 100 msec and 4 respectively and it can be changed under the "Setting" menu. The camera is Chapter 5: Design and Development of the Prototype 62 instructed through the line length array to read all 28 rows (28 x 753 bytes) associated with the fiber strips under the set exposure time. Stage 4 : A s last stage of initialization, routine init_section() is called and is used to find the effective F O V (or width) of each fiber strip in terms of pixel location characterized by the parameters offset and span of the section variable. Since the F O V of a fiber strip takes approximately 400 pixels in width under the current lens setting (40 cm 0.1 mm per pixel), it proportionately reduces the processing time by processing only the corresponding pixel span as opposed to the full horizontal row consisting of 753 pixels. If init_section() detects any section having a span less than 200 pixels, the user wi l l be prompted to perform initialization again. Such a situation occurs when the user forgets to turn on the laser before initialization, or misalignment of the lines mapped to the fiber strips occurs. Figure 5-15 : User interface - Camera initialization Chapter 5: Design and Development of the Prototype 63 5.5.3 Scanning Operation The scanning operation of the prototype involves a loop in which 3 sequential processes take place repetitively. The first process involves the loading of the images of the pixel rows corresponding to a number of the fiber strips into the buffer under a set exposure time. A s soon as the images are ready, image segmentation and extraction of features like stem diameter and shoot height are then performed, followed by the display of results based on the measured features. In a detailed view of how seedlings are measured, the procedure for measuring the seedlings is described as follows. On any image row returned from the camera, the high contrast, gray scale pixels representing the F O V of the fiber strips are first thresholded to produce a binary image which segments the object (seedling) and the background light. Thresholding here refers to the process of examining each pixel's gray value, which is between 0 (black) to 255 (pure white) inclusively, so that a value less than or equal to the threshold is considered to be black (object) and a value greater than the threshold is considered to be white (background light). Next, the binary image is runlength encoded. The number of transitions between the object and the background light, and the maximum consecutive dark pixel count (henceforth refers to as dark pixel count) of a row are recorded as the runlength parameters. B y defining these two parameters, the beginning of the plug may intuitively be expected in the location where there is an abrupt, positive change of the dark pixel count. The detection of a seedling is triggered by the change of the number of transitions from zero to non-zero and a row counter is used along the length of the seedling. The shoot height could then be Chapter 5 : Design and Development of the Prototype 64 determined based on the distance in row below which marks the beginning of the plug. The root collar zone is defined as the region from the 9/10th to the full length of the shoot height. Needles and branches contribute many transitions to the image rows (as shown in Fig. 5-16). A s a result, the row in the root collar zone in which only two transitions are recorded (corresponding to the edges of the stem) and the associated dark pixel count is less than the expected stem width wi l l be identified as a candidate row for diameter measurement. In such a case, the root collar diameter is taken as the average of the dark pixel count of the candidate rows. Figs. 5-17 and 5-18 show typical plots of the two runlength parameters of a seedling against its length in rows. Figure 5-16 : Binary seedling image around the root collar zone Chapter 5 : Design and Development of the Prototype 65 300 100 150 200 Length in rows Figure 5-17 : Plot o f dark pixel count vs length -r 50 100 150 200 Length in rows 250 300 Figure 5-18 : Plot o f number o f transitions vs length 350 In the program code, the functions which carry out the runlength encoding and the measuring scheme are run_length() and grade() respectively. Function run_length() is Chapter 5 : Design and Development of the Prototype 66 passed a threshold, the number and the address of a section; function grade() is passed only the number and the address of the same section. Grade() is called whenever the end of the seedling is detected as indicated by the change of the number of transitions from non-zero to zero. However, to prevent false detection, the grade() function wi l l not be activated i f the object's height is less than 15 rows in length In the grade() function, any one of the three attributes, tree, unknown or plug, is assigned to the object after the measurement process. If the beginning of the plug is located and at least one candidate row of the stem diameter in the root collar zone is found, the object wi l l be classified as a "tree". The shoot height in millimeters is calculated from the corresponding length in rows multiplied by a height coefficient which is a function of the inspection belt speed. In a similar manner, the stem diameter is converted to millimeters by multiplying the corresponding average pixel count by a width factor determined from calibration. Due to the imperfect light collimation, each section has its own width factor which is different from the other. The calibration of both height coefficient and width factor wi l l be further discussed in the next chapter. If the start of the plug is located but no candidate row of stem diameter is found, the object w i l l be classified as "unknown". Its entire length from the tip to the end is recorded instead, and the diameter is assigned as zero. On the condition that the start of the plug cannot be found, the object wi l l be attributed as "plug". In such case, the diameter measured can be seen as the average width of the plug as it is calculated based on the average of the dark Chapter 5 : Design and Development of the Prototype 67 pixel count of the rows which satisfy with only two transitions, and the entire length wi l l be determined. Fig. 5-19 summarizes the heuristics of the grading process. O b j e c t r e p r e s e n t e d by t h e r u n l e n g t h p a r a m e t e r s object = U n k n o w n dia. = 0 height = entire height < / ' Any row that has only 2 transitions 11111111 Yes Calculate the average width based on the average dark pixel count from these rows object = P l u g dia. = average width height = entire height No £ object = U n k n o w n dia. = 0 height = entire height Figure 5-19 : Flowchart of the grading algorithm Chapter 5 : Design and Development of the Prototype 68 Note that with this setup of grading process, even the seedling rolls during scanning (as is usually the case when being transported in belt), the algorithm wi l l still be able to correctly measure the seedling so long as it travels under the F O V of its corresponding fiber strip. 5.5.4 Scanning Modes There are four different modes for the interactive scanning operation, which vary in terms of display capability and number of sections (or fiber optic strips) that are supported." The features of the four different scanning modes in the prototype are summarized in Table 5-1 and the layout is shown in Fig. 5-21. Mode Corresponding function in the program Description Image mode i_scan_1 () • no. o f sections supported = 1 • on-line binary image display • result in table form (height, dia. & object) Graph mode g_scan_1 () • no. of sections supported = 1 • on-line plotting display • result in table form (height, dia., object & plug location) Four mode scan_4() • no. of sections supported = 4 • object detection with animated bar • result in table form (height, dia. & object) All mode scan_all() • no. of sections supported = 14 • object detection with animated bar • result in table form (height, dia. & object) Table 5-1 : Features o f different scanning modes Chapter 5 : Design and Development of the Prototype 69 The scanning mode is selected through the "Display" menu (Fig. 5-20) prior to the scanning operation activated by the "Start scanning" option under the "Run" menu. The default mode set in the program is Image mode, which is applied to section 8. Other sections can be chosen by entering the desired section number after selecting "Single section..." option. To switch to Graph mode, the "Graph" option should be chosen under the "Format..." option as shown. Selection of the other two modes is trivial as shown in the user interface. Figure 5-20 : User interface - Selecting different modes Chapter 5 : Design and Development of the Prototype Image mode Graph mode Animated detection Four mode AH mode Figure 5-21 : Layout of different scanning modes Chapter 5: Design and Development of the Prototype 71 In Image mode, the mechanism of the scanning operation can be understood via the flowchart shown in Fig. 5-22. G r a p h i c s I n i t i a l i z a t i o n I _KJ L o a d i m a g e i n t o b u f f e r w i t h t h e s e t e x p o s u r e t i m e P e r f o r m r u n l e n g t h e n c o d i n g Yes No D i s p l a y i m a g e o n s c r e e n S h o w r e s u l t i n t a b l e No I s V ' k e y b o a r d p r e s s e d P e r f o r m G r a d i n g Yes R e t u r n t o t h e U s e r I n t e r f a c e Figure 5-22 : Flowchart of the i_scan_1 () routine Chapter 5 : Design and Development of the Prototype 72 The mechanism of the Image mode routine is similar to that of the Graph mode routine, except that result data is plotted rather than displayed on screen. The Graphics initialization process involved, first, setting the monitor display into a certain screen resolution. Then a gray scale display driver, supplied with the camera system, is used to map the color palette into gray scale form. Some screen decorations, like drawing buttons and overlaying text, are added as the last step of the Graphics initialization process. The same entire process is used for the other two modes of scanning. The only difference is that, instead o f analyzing one section after refreshing the image buffer each time, four and fourteen sections are analyzed sequentially for the Four and All mode respectively. To indicate that an object is triggered and being processed, an animated bar, which is right next to the result table of the corresponding section, is used. It is worth noting that the Four mode option is most useful in experimenting with and setting-up of the system since fewer seedlings need to be handled. The setup of the number of sections currently in the prototype was 14, which corresponded to the number of seedlings in a row of the styroblock in study. However, the program has the ability to adapt for more sections for future expansion i f desired. Theoretically, i f 2 rows are used for each section which is the case in this project, up to 144 sections could be supported in a similar fashion. In such case, a more powerful system should be used to compensate for the longer processing time. Chapter 5 : Design and Development of the Prototype 73 5.5.5 Grading Parameters There are 7 parameters associated with the scanning operation. The parameters are threshold, exposure time, clear count, inspection belt speed, data storage, plug & stem difference, and the maximum stem diameter. Each parameter can be set interactively under the "Setting" menu as shown in Fig. 5-23. Fiber Scan ! * J l Q EfSM: Figure 5-23 : User interface - Parameter setting A l l parameters, except the data storage ("Output file..." option), affect the result of the scanning operation. The optimum value of the 2 camera control parameters, exposure time and clear count, was found to be 100 msec and 4 respectively. Clear count is the number of times charge is transferred down the C C D to the clearing drain to clear it. The camera manufacturer suggested 7 times to clear all charge. However, as taking exposures in succession, 4 clearing cycles was found to be satisfactory, at a saving of time. Chapter 5: Design and Development of the Prototype 14 Threshold, belt speed, plug & stem difference, and maximum stem diameter are parameters related to the grading process. The proper value of threshold was found to be 70. The belt speed, which is expressed in terms of the input voltage to the D C motor driving the inspection belt and is later converted into the unit of cm/s, is used to determine the longitudinal resolution of the object measured. The plug & stem difference is basically the change in the number of dark pixel counts expressed in millimeters, which is used to identify the beginning of the plug. The maximum stem diameter is the parameter which is used for locating the diameter rows in the root collar zone. The two parameters, plug & stem difference and maximum stem diameter, allow flexibility in the grading scheme which are essential to the variation of seedling species and size. The option "Output f i le . . . " is used to save the runlength parameters into a file for further analysis i f required. Initially, i f the user does not change any setting of the parameters, a set of default values is used as shown in Fig. 5-24. [SI Fiber Scan l Au t o Zj c i i i i i iel m m\B A | ~~ISSfli Diculav S e t t i n g . S t a t i s t i c s Quit I n i t i a l i z a t i o n j Start scanning Automatic Seedlin Display section > Section 6 Threshold > ?8 Exposure time > 188 ms Clear count ~—> 4 Belt speed > 3.8 cm/s Output f i l e > NIL Plugr & stem diff --> 5.0 mm tlax* ster» dia > 4.8 mm Figure 5-24 : Default value o f the parameters Chapter 5 : Design and Development of the Prototype 75 5.5.6 Resul t Statistics A s a powerful feature, the program allows a user to examine the result statistics through the "Show results" option under the "Statistics" menu, Figs. 5-25 and 5-26. Thus, 100% inspection of objects is permitted and this is another benefit of machine vision. I Hun Di sp la Quit Automatic Seedling Scanning System Figure 5-25 : User interface - Statistics option Percentage distribution o f the detected objects expressed in pie chart form Plot of measured diameter and height of the tree (seedling) Table shows the total number of objects and the number o f trees detected Figure 5-26 : Layout o f the statistics routine Chapter 5 : Design and Development of the Prototype 7t> A t this point, the program supports only simple statistics such as plotting the stem diameter and shoot height of the object detected as a tree, and showing the distribution of all the objects being measured. A s for the commercial interest, a specification of grade could be implemented in the program such that the number of seedlings associated with different grades (such as acceptable and cull) could be shown. 77 Chapter Six 6. CALIBRATION, TESTING AND EVALUATION 6.1 Chapter overview Proper calibration of threshold value, width factor and height coefficient is essential to optimum algorithm or scanning performance. This chapter begins with the discussion of the calibration of the vision system, followed by result and discussion of different tests performed on the system. The performance of the prototype was evaluated in terms of 5 categories: (1) accuracy of the object classification, (2) accuracy of diameter measurement, (3) accuracy of length measurement, (4) speed of scanning under different hardware configurations and (5) repeatability in multiple scanning. 6.2 Calibration Different calibration steps were performed with the use of a seedling sample and a slender rod of known size. The seedling simple was used to determine the proper threshold value, whereas the calibration rod was used to determine the horizontal scale factor for each section, and the height coefficient as a function of conveyor speed which were incorporated into the grade() routine. To determine the proper threshold value, image of a seedling sample (Fig. 4-3) as viewed by a fiber section was retrieved and analyzed. The brightness distribution of the image (Fig. 4- Chapter 6: Calibration, Testing and Evaluation 78 4) showed that a threshold value between 45 to 165 was an appropriate choice to separate the seedling and backlight represented by two peaks. A threshold value of 70, which is in the lower side of the range and is therefore less prone to noise, was selected and set as the value for the default setting (Fig. 6-1) as well as the testing of the measurement algorithm. Figure 6-1 : Threshold setting The width factor, expressed in millimeters per pixel, represents the transverse resolution of a section. The procedure involved to obtain the width factor is described as follows. Scanning is performed on the calibration rod to measure the corresponding width in pixels, while the width factor was initially set to 1. Extra care was taken to make sure that the calibration rod pointed in the direction of travel at all times while being scanned. Due to the shape resemblance to a plug, the resulting diameter measured by the algorithm showed the average width of the rod in terms of number of pixels. The width factor was then recalculated by dividing the known width (or diameter) of the calibration rod by the measured diameter in pixels, averaged from 10 measurements. This calibration process was Chapter 6: Calibration, Testing and Evaluation 79 done for all 14 sections and the value for each section is shown in Fig. 6-2. Note that the resulting values were in close agreement with the measured transverse resolution of approximately 0.1 mm per pixel when the camera was first set up. 0.05 7 8 Section 10 11 12 13 14 Figure 6-2 : Width factor o f all 14 sections In a similar manner, the height coefficient, expressed in centimeters per row, was obtained by dividing the known length by the number of rows the calibration rod represented in the image. This was done for different conveyor speeds and for different scanning modes. However, since the belt speed is a function of the motor input voltage, it was necessary to first obtain a relationship as shown in Fig. 6-3. Chapter 6: Calibration, Testing and Evaluation 80 o 3 F(x) = 0.7755x-1.4846 - ' A Experiment Fitted Curve K 2 3 4 5 6 7 8 9 10 11 Input DC Voltage (V) Figure 6-3 : Relationship between input voltage and belt speed Once the above linear function was established, the number of rows representing the length of a calibration rod was recorded for different voltages ranging from 2 to 5 cm/s correspondingly. This was done for all 4 modes. Because of the variation of the number of sections being processed and the complexity of the graphics display used in each mode, the number of rows representing the length of the same calibration object varied; so did the height coefficient which is summarized in the Figs. 6-4 to 6-7 shown below for different modes. The height coefficient shown in the figures is in the "per-2-rows" basis, which is the unit length representing the diameter of the optical fiber (see Fig. 5-12), so that a comparison could be made with the ideal longitudinal resolution (as discussed in the camera system section of the previous chapter) which depends on a static portion, 0.020 cm, and a Chapter 6: Calibration, Testing and Evaluation 81 dynamic component equal to the product of exposure time and belt velocity which, in this case, is 0.1 x cm. Each data point of a height coefficient was calculated by dividing the known length by the number of representing rows averaged from 4 test runs, and then multiplied by 2 to obtain the unit of centimeters per 2 rows. A l l of the height coefficient functions shown and obtained from linear regression were for reference purposes. The actual functions implemented in the grade() routine were in the per-row basis and took on motor voltage as input. 1.4 ^ Experiment Fitted Curve, F(x) Ideal, I (x) 0.4 0.2 l(x) = 0.1x + 0.020 0 2 3 4 5 6 Belt speed (cm/s) Figure 6-4 : Height coefficient o f Image mode Chapter 6: Calibration, Testing and Evaluation 82 Figure 6-5 : Height coefficient o f Graph mode Figure 6-6 : Height coefficient o f Four mode Chapter 6: Calibration, Testing and Evaluation 83 • Experiment Fitted Curve, F(x) Ideal, I (x) 1 2 3 4 5 6 Belt Speed (cm/s) Figure 6-7 : Height coefficient o f A l l mode 6.3 Accuracy of Object Classification The test conducted to evaluate the performance of the measurement capability of the prototype in terms of object classification entailed measuring a set of 10 seedlings (1+0 spruce) obtained from the nursery of the Forestry Department at U B C . Among the set, 2 seedlings were cut flat right above the top of the plug so that they represented the plug. The rest of the set had the stem diameter measured manually at the stem caliper ranging from 2.3 to 3.2 mm, and the shoot height measured from the base of the plug to the tip of the seedling ranging from 11.4 to 16.2 cm. With the use of one fiber section (section 11), each object (seedling or plug) was manually placed on the inspection conveyor running at approximately 3.5 cm/sec and scanned repetitively for 10 times. Setting of the two parameters, plug & stem diff. and max. stem dia., were fixed at 5 mm and 3.8 mm Chapter 6: Calibration, Testing and Evaluation 84 respectively and the scanning mode was set at Image mode so that image could be visually examined as for conformation. Results of the measurements were recorded as shown in Table 6-1. Since the objective of the test was to see i f the algorithm was able to identify the correct object as either "tree" or "plug", attention was not paid on the measured value of diameter and shoot height so long as the deviation of the measured value of the diameter and height from the true value was below 0.5 mm and 2 cm respectively (in fact, this was the case). A n informal investigation on the distribution of measured stem diameter and shoot height through the statistical display in the program revealed the compliance of the distributions of the seedling set. No. of Objects Percentage Expected Result Total 100 100% 100% Detected as 'Tree - ' 71 71 % 80 % Detected as "Plug" 15 15% 20% Detected as "Unknown" 14 14 % 0% Table 6-1 : Classification result from a set o f objects consisting o f 8 seedlings and 2 plugs A s indicated from the result, there was a 9%.classification error for tree and a 5% classification error for plug, both of which added up to the 14% misclassification as unknown. The cause of the first case was due to one of the seedlings in the set which had branches protruded into the collar zone. This resulted in more than two transitions counted in each row along the root collar zone which caused the algorithm fail to locate the stem Chapter 6: Calibration, Testing and Evaluation 85 diameter row and consequently identify it as unknown. The latter case could be explained by the portion of the plug leaning out of the fiber's F O V during scanning, which was caused by the lateral movement of the cylindrical plug due to rotation. It was observed on the on- line image displayed when this phenomenon happened. Since there was only one transition along the plug's length, it was subsequently classified as unknown. 6.4 Accuracy of Stem Diameter Measurement The stem diameter measurement of a seedling varies subjective to the rotation about its longitudinal axis because different views of a seedling have different sizes projecting onto the optical fibers. A s a result, it was decided that a thin, rectangular plate attached to a rod having an uniform diameter of 3.12 mm (see Fig. 6-8) was used instead to determine the accuracy of the stem diameter measurement in the program. 10.0 cm 12.5 cm M W H M Dia. = 3.10 mm 1 1 Figure 6-8 : Object used for the accuracy of stem diameter and height measurement This object, which was seen as a "tree" in the grading algorithm, was scanned repetitively for 50 times by using Graph mode. The section used for scanning was section 7. Conveyor speed was set at 3 cm/s (though it would not affect the measurement result). The diameter of each measurement was recorded. The mean value, standard deviation and the coefficient Chapter 6: Calibration, Testing and Evaluation 86 of variation (standard deviation divided by mean) was found to be 3.12 mm, 0.12 mm and 4.0% respectively. In general, measurement accuracy specifies the maximum total deviation of the true dimension from the measured dimension. For the diameter measurement under high contrast image condition, the best accuracy was expected to be equal to the resolution of the optical fibers (even though the fiber was sampled by the camera sensors at approximately 0.09 mm per pixel). However, the measurement results showed high measurement accuracy obtained from this vision setup. 6.5 Accuracy of Length Measurement To determine the accuracy of the length measurement in the system, the same object for diameter measurement was used. Testing was conducted by scanning the object for 20 times for 4 different speeds (2.5, 3.5, 4.5 and 5.5 cm/s) under different scanning modes. A n effort was made to randomly select different sections to scan the object when the scanning mode was at Four or All mode which supports more than one section scanning simultaneously. Unlike the stem diameter measurement, the number of rows representing the object's shoot length (12.5 cm) was recorded. Results of the measurement are presented in terms of coefficient of variation and the standard deviation for the number of rows measured as a function of belt speed (Figs. 6-9 and 6-10). Chapter 6: Calibration, Testing and Evaluation 87 Figure 6-9 : Result o f variation in length measurement Image mode —•—Graph mode — 4 - Four mode —X—All mode 2 .2 0 • - * v ; — , x 1 1 L • 1 2 2.5 3 3.5 4 4,5 5 5.5 6 Belt Speed (cm/s) Figure 6-10 : Result o f standard deviation o f length measurement in rows The low coefficient of variation, which range from 1.37 to 3.99%, suggested the high consistency of the length measurement regardless of speed. The low standard deviation, Chapter 6: Calibration, Testing and Evaluation 88 which ranged from 0.63 to 1.65 rows, measured a low spread about the average value (or the mean) of the sample data. During the testing, it was observed that the difference of row length was always ± 2. This in fact was in close agreement with the unit of accuracy in rows representing the resolution of the optical fiber. The accuracy in terms of the actual length in centimeters was closely related to the overall system resolution in the longitudinal direction, which is the height coefficient. Being a step further from the calculation of ideal resolution, the height coefficient accounts also for the time to process the image as well as the time it takes to display the graphics. This could be understood through the comparison of the height coefficient as shown in Fig. 6-11. 1.40 Belt Speed (cm/s) Figure 6-11: Comparison o f the height coefficient o f all 4 modes Chapter 6: Calibration, Testing and Evaluation 89 At a speed of 4 cm/s, the expected accuracy of the system is 0.60 cm by scanning all 14 seedlings simultaneously (All mode), which is within the required accuracy of 1 cm for the length measurement. This speed would also provide a maximum allowable length of 32 cm for a seedling that could be measured accordingly (4 cm/s x 8 sec. of extraction time between rows). Assuming all seedlings are below the maximum length limit, this speed would yield a seedling throughput of 1.75 seedlings per second. 6.6 Speed of Scann ing For a given belt speed as shown in Fig. 6-11, the difference in the height coefficient of any of the modes from the ideal longitudinal resolution is due to sacrificing of time on the image processing and displaying of graphics. Because a certain length of time was needed to be allocated for these two tasks, the time between camera exposures was elongated. Consequently, a fewer number of rows would represent the same length of the object or lower resolution could be expected given the same belt speed. The large offset from the Image mode was sorely the result of on-line image display. The speed of scanning depends upon the desired, accuracy or the resolution of the length measurement which relates to the exposure time of the camera, the speed of the host computer, and the speed of the video card. For a specific object length, a shorter exposure time translates into a higher sampling frequency in the longitudinal direction. However, too short of an exposure may result in insufficient illumination to the sensing elements; but too long an exposure time may also introduce image blur given the same moving speed of the Chapter 6: Calibration, Testing and Evaluation 90 object. In our prototype system, it was found that an exposure time between 80 to 100 milliseconds was the optimum value for the best performance of image algorithm. It is definitely not surprising that the speed of the computer plays a role in having better accuracy in length measurement. In the case of grading all 14 seedlings simultaneously, the sooner the runlength encoding and grading routines finish executing all 28 lines (2 lines for each section), the sooner the next data sample is acquired, which means more lines or rows wi l l represent an object of the same length given the same exposure time and conveyor speed. A faster computer wi l l be capable of executing the image processing algorithm in a shorter time as all of the processing power relies on the host C P U . Similar to the effect of a faster computer, a faster speed of video transfer to the monitor w i l l also help improve the speed of the overall system and the precision of length measurement. Since the user interface is graphically very demanding (like plotting results on-line and showing a moving bar indicating that an object being scanned in the section), a lot of processing time is dedicated to the display of graphics. A faster and better video card (for example, having a V R A M instead of D R A M ) wi l l certainly be helpful in shortening the stall. A s proof to this argument, a demo program was created which reads image data from files and has the same mechanism of data acquisition as the program reading directly from the camera. To simulate the exposure time of the camera, a simple time delay function corresponding to the length of exposure time was incorporated into the demo program. The demo program was then run on computers with different configurations with the results summarized in Table 6-2. The basis of comparison was made by reading the same image, Chapter 6: Calibration, Testing and Evaluation 91 processing and displaying it line-by-line (Image mode) until a field of 244 lines had been acquired and processed. A s 2 lines were acquired during each 100 millisecond exposure, a simple calculation, ignoring the expense of computation and image display operation, gave a total processing time of 12.2 seconds. Computer Conf igurat ion T ime in s e c . taken for 1 cyc le* ( 244 l ines / 1 2 2 readout ) C P U V ideo Card V ideo R A M 486-100 MACH 32 1 DRAM 26.58 P5-75 MACH 64 2 DRAM 13.12 P5-100 MACH 64 1 DRAM 13.02 P5-166 MACH 64 2 VRAM 12.96 P6 - 200 MACH 64 2 DRAM 13.04 •Note: Comparison was based on Image mode, exposure time = 100 msec. Table 6-2 : Graphics performance test on various computer configurations A n interesting observation is that the speed of the computer does not significantly affect the performance so long as it is faster than that of a 486 D X 4 computer. The video card configuration, on the other hand, was an important factor for performance improvement, which also could translate into higher measurement accuracy in length. It is also important to note that each of the four different graphics modes (Image mode, Graph mode, Four mode and All mode) supported in the program has a different level of complexity in terms of graphics display operation. The Image mode is the most time consuming mode among the Chapter 6: Calibration, Testing and Evaluation 92 four and it was chosen for overall performance comparison based on various hardware configurations. 6.7 Repeatability in Multiple Scanning A s the last test of the prototype system, an attempt was made to investigate the repeatability of the seedling measurements by scanning the same seedling which was stored previously in file (Fig. 4-3). The All scanning mode was used in this test and all 14 sections read the same image file and processed it simultaneously. This was done for 20 repetitions. The setting of all grading parameters was kept unchanged during the simulation. Results showed that the exact values of stem diameter and shoot height were obtained each time after scanning. This, therefore, showed high reliability of the measurement program and algorithm. 93 Chapter Seven 7. CONCLUSION AND RECOMMENDATIONS 7.1 Conclusions and Contributions of the research A research project investigating the use of machine vision technology for measuring morphological properties of containerized seedlings has been initiated. A prototype P C - based machine vision and handling system has been developed. The prototype system, which employed a novel design of line-scan imaging of fiber optics in conjunction with the use of structured backlight, simplifies the whole on-line measurement process and addresses the production needs of commercial forest nurseries. The multiple processing approach of simultaneously scanning an entire row of 14 seedlings satisfies the time constraint of 8 seconds to measure the diameter and shoot height of all seedlings based on their high contrast images projected onto the optical fiber sensors. The system has a high transverse resolution equal to the resolution of the optical fiber. The longitudinal resolution depends on the exposure time, belt speed as well as the speed of the host computer and graphic display board. A multi-functional, menu-driven graphical user interface was developed for the system prototype. It offered a convenient means to control the camera settings (like exposure time) and various grading criteria (like threshold and maximum stem diameter) interactively. Additionally, it supports a different number of scanning sections (1 ,4 and 14) and has Chapter 7: Conclusion and Recommendations 94 various display capabilities. The interface also supports simple statistics for each measured feature which could be valuable for quality control. The performance in terms of the object classification in the measurement routine, grade(), was good. Most of the time the algorithm could correctly identify a seedling, with the only exception being when branches protruded into the root collar zone, resulting in more than 2 transitions counted in every diameter row. However, this algorithmic flaw could be eliminated by adding several more heuristic criteria in the algorithm which makes use of the prior knowledge of the stem diameter. The measurement accuracy for stem diameter was excellent. A test object having a "tree" shape and a uniform stem diameter of 3.10 mm was measured through 50 repetitive scans. The standard deviation was found to be 0.12 mm. The accuracy of height measurement depended on the longitudinal resolution of the system (or height coefficient). A t a belt speed of 4 cm/s under the All scanning mode, the accuracy of length measurement was found to be 0.6 cm and a seedling throughput of 1.75 seedlings per second could be expected. Given the same speed, the accuracy could be increased by using a better video graphics board which could minimize the overhead due to graphics display. Chapter 7 ; Conclusion and Recommendations 96 Computer bus Memory A / D board Graphics board Data acquisition board Camera Display Driver Figure 7-1 : Future work Chapter 7: Conclusion and Recommendations 96 CPU (Decision Making) 1 Computer bus ^ ^ ^ ^ Memory Data acquisition board Camera Driver Figure 7-1 : Future work 9 7 R E F E R E N C E S [I] Affeldt, H.A.; Heck, R.D. 1992. Illumination methods for automated produce inspection: Advantages and disadvantages. Paper No. 92-3028, A S A E , St. Joseph, MI. 36 pp. [2] Anonymous. 1995. Annual report of the Ministry of Forests for the fiscal year ended March 31,1995. Ministry of Forests, British Columbia. [3] Ardalan, S.H.; Hassan, A.E. 1982. Automatic feeding and sorting of bare root seedlings. Trans, of the A S A E , 25(2):266-270. [4] Browne, A. , Norton-Wayne, L. 1986. Vision and information processing for automation. Plenum Press, 233 Spring St., New York, N Y 10013. [5] Buckley, D.J.; Reid, W.S.; Armson, K . A . 1978. A digital recording system for measuring root area and dimensions of tree seedlings. Trans, of the A S A E , 21(2):222-226. [6] Chin, R.T.; Harlow, C.A. 1982. Automated visual inspection: A survey. IEEE Trans, on Pattern Analysis and Machine Intelligence, PAMI-4(6)557-573. [7] ELECTRIM corporation. 1992. EDC-1000HR computer camera technical manual. ELECTRonic IMaging, 356 Wall St., Princeton, NJ 08540. [8] E G & G Reticon. 1995/96. Image sensing and solid state camera products. 345 Potrero Avenue, Sunnyvale, C A 94086. [9] GALILEO electro-optics corporation. Technical memorandum: Fiber optics - theory and application. Galileo Park, P. O. Box 550, Sturbridge, M A 01566. [10] Graham, L.F.; Rohrbach, R.P. 1983. Mechanical singulation of bare root pine seedlings. Paper No. 83-1093, A S A E , St. Joseph, MI. 15 pp. [II] Hassan, A.E . 1989. Singulation of bare root seedlings - feasibility study. Paper No. 89- 1084, A S A E , St. Joseph, MI. 28 pp. [12] Hassan, A.E. ; Tohmaz, A S . 1990. Use of vision system for recognizing plantable seedlings. Paper No. 90-7518, A S A E , St. Joseph, MI. 13 pp. [13] Hines, R.L.; Sistler, F.E.; Wright, M.E. 1986. A vision system for grading container- grown plants. Paper No. 86-3043, ASAE, St. Joseph, MI. 13 pp. 98 [14] Larsen, H.S., South, D.B., Boyer, J.N. 1986. Root growth potential, seedling morphology and bud dormancy correlate with the survival of loblolly pine seedlings. Tree Physiology 1:252-263. [15] Lawyer, J.N. 1981. Mechanization of nursery production of bare root deciduous planting stock. Forest Regeneration. A S A E Publication 10-81. ASAE, St. Joseph, MI. pp. 30-37. [16] Lindley, C.A. 1991. Practical image processing in C. John Wiley & Sons, Inc., New York, N Y 10158. [17] Maw, B.W.; Suggs, C.W. 1980. Progress in the mechanical singulation and sorting of seedlings. Paper No. 80-1055, A S A E , St. Joseph, MI. 16 pp. [18] Maw, B.W.; Brewer, H.L. 1983. Transducers for an automatic seedling sorter. Paper No. 83-1091, A S A E , St. Joseph, MI. 13 pp. [19] National Semiconductor Corporation. Linear databook 3. 1988. pp. (5)42-43. [20] Province of B.C. Ministry of Forests, Nursery & Seed Operations Branch. Forest seeding contract. Internet www page, at URL: < http: // www.for.gov.bc.ca / nursery / heatqtrs / contract.htm >. [21] Rigney, M.P.; Kranzler, G.A. 1988. Machine vision for grading southern pine seedlings. Trans, of the A S A E , 31(2):642-646. [22] Rigney, M.P.; Kranzler, G.A. 1988. Computer vision for grading tree seedlings. Proceedings, combined meeting of the Western Forest Nursery Associations. August 8-11: Vernon, British Columbia. General Technical Report RM-167, USDA Forest Service, Fort Collins, CO. pp. 182-187. [23] Rigney, M.P.; Kranzler, G.A. 1989. Performance of machine vision based tree seedling grader. Paper No. 89-3007, ASAE, St. Joseph, MI. 12 pp. [24] Rigney, M.P.; Kranzler, G.A. 1990. Computer vision: A nursery management tool. Target Seedling Symposium: Proceedings, combined meeting of the Western Forest Nursery Associations. August 13-17: Roseburg, Oregon. General Technical Report RM-200, USDA Forest Service, Fort Collins, CO. pp. 189-194. [25] Rigney, M.P.; Kranzler, G.A. 1993. Line-scan inspection of conifer seedlings. Proceedings of SPIE - The International Society for Optical Engineering. Vol . 1863. Publ by Int Soc for Optical Engineering, Bellingham, WA. pp. 166-174. 99 [26] Rigney, M.P.; Kranzler, G.A. 1995. Machine vision system for measuring conifer seedling morphology. Proceedings of SPIE - The International Society for Optical Engineering. Vol . 2345. Society of Photo-Optical Instrumentation Engineers, Bellingham, WA. pp. 26-35. [27] Ruzhitsky, V .N . , Ling, P.P, 1992. Machine vision seedling measurement using multiple cameras. Proceedings of SPIE - The International Society for Optical Engineering. Vol . 1863. Publ by Int Soc for Optical Engineering, Bellingham, WA. pp. 175-184. [28] Simonton, W; 1989. Geranium stock processing using robotic system. Paper No. 89-7054, A S A E , St. Joseph, MI. 19 pp. [29] Simonton, W; Pease, J. 1990. Automatic plant feature identification of geranium cuttings using machine vision. Trans, of the A S A E , 33(6):2067-2073. [30] Steinmetz, V. , Delwiche, M.J., Giles, D.K., Evans, R. 1994 Sorting cut roses with machine vision. Trans, of the A S A E , 37(4):1347-1353. [31] Suh, S.R.; Miles, G.E. 1988. Measurement of morphological properties of tree seedlings using machine vision and image processing. Paper No. 88-1542, ASAE, St. Joseph, MI. 16 pp. [32] Wilhoit, J.H.; Fly, D.E.; Kutz, L.J.; South, D.B. 1992. A low cost machine vision system for seedling morphological measurements. Paper No. 92-3030, A S A E , St. Joseph, MI. 17 PP- [33] Wilhoit, J.H., Kutz, L.J., Vandiver, W.A. 1995. Machine vision system for quality control assessment of bareroot pine seedlings. Proceedings of SPIE - The International Society for Optical Engineering. Vol . 2345. Society of Photo-Optical Instrumentation Engineers, Bellingham, WA. pp. 36-49. APPENDIX A ON-OFF Motor Control Circuitry 101 In order to provide a "stop-and-run" motion for the sorting conveyor, the controller of the motor is modified to add a timer and relay circuitry. A schematic is shown below. 9VDC •4- Initial Delay Adjustment r — Timer Circuit 2a Contact switch Main switch C =68 pF 4 ~j 3 555 Timer 1N914 _L. -s 6V DC Relay (DPDT) 4, If 1N914 1N914 6V DC Relay (DPDT) Motor controller Power supply Figure 7-2 : Motor control circuit The function of the circuitry is to enable the sorting conveyor to come to rest for a short duration so that seedlings can settle down on it as they arrive from the inspection conveyor. 102 After the "resting" time, the graded seedlings can be transported to the next rejection and bundling process (future work). The circuitry has an 8 second period (6-seconds " O N " and 2-seconds "OFF") which is referenced to the extraction time between rows performed by the extractor. The circuitry is divided into two main parts: a basic astable timer circuit and an initial delay adjustment relay. A s pins 2 and 6 of the 555 timer chip are connected together, the lower circuit triggers itself each timing pulse and functions as an oscillator. Cx charges through (Rw + RlF) and (R2V + R2F), but discharges through (R2V + R2F). The charges on C, ranges from 1/3 to 2/3 of the input voltage which in this case is 9V and therefore, the value of the input voltage has no effect on the timing cycle. To avoid false triggering, pin 5 is tied to ground via a 0.1 pF capacitor. D2 absorbs voltage generated by the relay coil #1 when it is switched off. The value chosen for C, (68 uF) is based on the selected oscillation frequency (0.125 Hz) according to the specification chart given in [19]. The value of resistors is calculated according to the equations: t,= 0.632 ((RiV+R:F) + (R2V+R2F)) t2= 0.632 (R2V+R2F) With t] = 6 seconds and t2 = 2 seconds, ( R w + RlF) and ( R2V + R2F) are obtained as 47 k Q and 94 k f i respectively. However, i f it is desired to vary these times, it is only necessary 103 to adjust the Rw and R2V . The upper graph in Fig. 7-3 shows the output pulse from the timer circuit. Output pulse with the the first pulse timing error Charge on C, Output pulse after the release of contact switch 1 1 + ' delay time (s) time (s) time (s) Figure 7-3 : Circuit response curves A s shown in the upper portion of Fig. 7-2, the second relay (coil #2) is used to adjust the pulse error introduced by the first " O N " pulse delay in the timing circuit. While the contact switch is depressed, the relay 2a contact are opened whenever the timer energizes the relay coil #1 via the contact l a . This prevents the power supply from reaching the motor controller or the motor. When the contact switch is released after the timer has stabilized, relay coil #2 is de-energized permanently and power to motor is controlled by contact l b 104 only. The bottom graph in Fig. 7-3 shows the timing pulse to the motor with the elimination of the first pulse timing error. The contact switch was released during the 2 seconds " O F F " interval. This adjusted output pulse is important for the correct placement of the holders used to transport the seedlings in the sorting conveyor. 105 APPENDIX B Assembly Drawings for the Camera Mount and the Fiber Optics Unit 106 >- \ _ i u u 00 X <L CJ 107 108 u PQ ZD CD J5 •o cf & CZ O o Po  <c -P g CD o a 0 ~> 0" > +» 3 P ro  >-pa <t P ro  oi +* ea <z ID > <: PI <r ZD a <: u o X * ' ID OJ ro X * in OJ co 109 110 Ill 112 APPENDIX C Dimensional Drawings for the Prototype Frame 114 115 —> OJ o "> tz o +» >- 1=1 in OS OJ £ C5 i_ in L _ c d O Vi OJ c E: OJ c5 £ u n T ~ T 116 i- a Q_ - H 1=1 in ON ON Hi -15)1- _0J _Q c: O Ul a OJ E OJ Ul d c5 "iL OJ +» c5 o3 75 O O > E W E OJ Z5 -Q 2 5 OJ L. c5 CJ in 117 u h- <r _ i > U I 1 > (Y U m -r- 00 oo 00 U Z (_> <c O J + > O O J >- X X •st- % a> S L. O J U S_ 1/1 u o # N C  =) fo  O J 01 I o Hi t h o i o r O J <4- nc  Q. cs C5 L +» C5 O J ~ C c U d , —' —' L. T5 a a ~i. L +> +> 00 00 O J -p o z +> g SI


Citation Scheme:


Usage Statistics

Country Views Downloads
China 22 0
United States 3 0
Japan 1 36
City Views Downloads
Beijing 22 0
Redmond 1 0
Sunnyvale 1 0
Mountain View 1 0
Tokyo 1 0

{[{ mDataHeader[type] }]} {[{ month[type] }]} {[{ tData[type] }]}


Share to:


Related Items