Open Collections

UBC Theses and Dissertations

UBC Theses Logo

UBC Theses and Dissertations

An analysis of machine shape defects in British Columbia sawmills and their classification using neural.. 2003

You don't seem to have a PDF reader installed, try download the pdf

Item Metadata

Download

Media
ubc_2003-0290.pdf [ 9.81MB ]
Metadata
JSON: 1.0074991.json
JSON-LD: 1.0074991+ld.json
RDF/XML (Pretty): 1.0074991.xml
RDF/JSON: 1.0074991+rdf.json
Turtle: 1.0074991+rdf-turtle.txt
N-Triples: 1.0074991+rdf-ntriples.txt
Citation
1.0074991.ris

Full Text

AN ANALYSIS OF MACHINE SHAPE DEFECTS IN BRITISH COLUMBIA SAWMILLS AND THEIR CLASSIFICATION USING NEURAL NETWORKS by H E L E N KATR INA E. R A S M U S S E N B.A.Sc., The University of British Co lumbia , 1994 A T H E S I S S U B M I T T E D IN PART IAL F U L F I L M E N T O F T H E R E Q U I R E M E N T S F O R T H E D E G R E E O F M A S T E R O F A P P L I E D S C I E N C E in T H E F A C U L T Y O F G R A D U A T E S T U D I E S ' I (Faculty of Forestry) W e accept this thesis as conforming to the required standard T H E UN IVERS ITY O F BRITISH C O L U M B I A February, 2003 © Helen Katrina E. Rasmussen , 2003 U B C Rare Books and Special Collections - Thesis Authorisation Form Page 1 of 1 In presenting t h i s thesis i n p a r t i a l fulfilment of the requirements for an advanced degree at the University of B r i t i s h Columbia, I agree that the Library s h a l l make i t f r e e l y available for reference and study. I further agree that permission for extensive copying of th i s thesis for scholarly purposes may be granted by the head of my department or by his or her representatives. I t i s understood that copying or publication of th i s thesis for f i n a n c i a l gain s h a l l not be allowed without my written permission. The University of B r i t i s h Columbia Vancouver, Canada http://www.library.ubc.ca/spcoll/thesauth.html 4/9/03 ABSTRACT Ideally, the edges of lumber are parallel to each other and its ends are rectangular and in line with each other. However, sub-optimal occurrences in the sawing processes cause deviations from this ideal shape. In the sawmill, these deviations are often detected as off-size variations in thickness, and one particular defect shape is not necessarily distinguished from another in the downgrading process. These defects, different from machine defects like torn grain or skip, are referred to as machine shape defects in this thesis. The first part of this thesis implements a survey to analyse machine shape defects in British Columbia sawmills, while the second part employs neural networks as an experimental approach in the classification of these defects. A survey was designed and implemented to determine the industrial significance of machine shape defects in British Columbia sawmills. Completed in 2000, the survey focussed on six machine shape defects commonly caused by the sawing process: snipe, flare, wedge, taper, thin snake and fat snake. Responses came from mills located across BC and from both large and small forest companies responsible for 33% of BC softwood lumber production in 2000. Characterising BC sawmills according to machine shape defects and annual production shows that for each category of mill, with one exception, there is over a 20% probability of producing at least five types of machine shape defects. The most common grade cited for all machine shape defects was No. 2 Structural. By ranking the machine shape defects in terms of occurrence and by determining which ones are most serious in terms of final quality, it was established that thin snake, snipe and taper have the most serious impact on the industry. Neural networks were trained to detect and classify snipe in rough green lumber, using more than one hundred trim ends sampled randomly from a mill experiencing difficulty processing frozen wood. A self-contained measuring apparatus was constructed to support measuring equipment and to convey the sample boards through the measuring range of six lasers at a steady rate, using the automatic feedrollers of a shaper table. A statistical model was developed to interpret the physical characteristics of the board's surface, focussing on its shape. This model was used to preprocess the laser data into a set of variables, simplifying the data set for input into the neural networks. It was demonstrated that neural networks can be applied with limited success to detect machine shape defects, in particular snipe, in random samples of rough green lumber. However, it was established that more training data is required to train the neural networks to classify the sample cases with combination snipe. ii TABLE OF CONTENTS ABSTRACT II TABLE OF CONTENTS Ill LIST OF TABLES ...VI LIST OF FIGURES VII ACKNOWLEDGEMENT VIII BACKGROUND 1 TROUBLESHOOTING METHODS 2 REAL-TIME LUMBER SIZE CONTROL SYSTEMS 3 THESIS OBJECTIVES 4 PART I - LUMBER SHAPE DEFECT SURVEY 5 INTRODUCTION 5 Objective of the Survey of BC Sawmills 7 METHODOLOGY 7 RESULTS 8 Response 8 Probability of Shape Defect 10 Shape Defects Affecting Quality 15 Grade 16 Causes of Shape Defect 16 Nomenclature 23 DISCUSSION 25 Probability of Shape Defect 25 Shape Defects Affecting Quality 26 Grade 28 Causes of Shape Defect 31 Nomenclature 33 CONCLUSIONS 34 PART II - CLASSIFICATION OF MACHINE SHAPE DEFECTS 36 INTRODUCTION 36 Objective of Neural Network Classification 37 iii METHODOLOGY 3 9 Sample Measurement 39 Measurement Equipment 39 Sample Boards and Measurement Process 43 Data Validation 45 Sources of Error & Tolerance 4 5 Calibration Block 4 5 Validation of the data 4 6 Data Preprocessing 47 Laser Data 4 ^ Modelling the 3-D Shape of the Board 50 Input & Output Variables 53 Neural Network Trials 53 Software 53 Neural Network Training and Testing 54 Neural Network Trial Procedure 55 Training Process: Snipe Classification Approaches & Training Strategies 55 RESULTS 5 9 Data Validation 59 No Change in Apparatus 59 Laser Measurement Repeatability 59 Data Preprocessing 60 Neural Network Trials 62 Interpretation of Results 62 Description of Neural Networks 63 Evaluation of Neural Networks 63 Two Class Network Problem: No Snipe & Snipe 64 Base Condition - Two Class Networks 64 Shuffled Condition - Two Class Networks 65 Pruned Condition - Two Class Networks 66 Extended Duration Condition - Two Class Networks 67 Three Class Network Problem: No Snipe, Snipe & Combination Snipe 68 Base Condition - Three Class Networks • 68 Shuffled Condition - Three Class Networks 68 Pruned Condition - Three Class Networks 69 Extended Duration Condition- Three Class Networks 69 iv DISCUSSION 71 Data Validation 71 Neural Network Trials 71 Description of Neural Networks 72 Type of Network 72 Complexity 74 Evaluation of Neural Networks 75 Error: Training, Verification & Testing 75 Performance 77 Classification Analysis of Sample Cases 77 CONCLUSIONS 81 FUTURE APPLICATIONS 82 SUMMARY 83 BIBLIOGRAPHY 84 APPENDIX A - LUMBER SHAPE DEFECT SURVEY 87 APPENDIX B - TOLERANCE CALCULATION 90 APPENDIX C - PREPROCESSED DATA 92 C l PREPROCESSEDTRAINING DATA SET 92 C. 2 R 2 AND F VALUES 92 APPENDIX D - DATA VALIDATION SUMMARY OF RESULTS 97 D. 1 No CHANGE IN APPARATUS 97 D.2 LASER MEASUREMENT REPEATABILITY 97 APPENDIX E - EXAMPLE CLASSIFICATION ANALYSIS OF SAMPLE CASE 100 v LIST OF TABLES Table 1. Comparison of grades with respect to skip and wane 30 Table 2. Top three most common causes of each machine shape defect 31 Table 3. List of Equipment 40 Table 4. Defects found in sample boards 43 Table 5. Equipment Tolerances 45 Table 6. Selection of control parameters for the Base Condition neural networks 57 Table 7. Summary of results for detecting a change in the measuring apparatus 59 Table 8. Summary of results for testing repeatability 60 Table 9. Base Condition - Two Class Networks 65 Table 10. Shuffled Condition - Two Class Networks 66 Table 11. Shuffled Condition Second Round- Two Class Networks 66 Table 12. Pruned Condition - Two Class Networks 67 Table 13. Extended Duration Condition - Two Class Networks 67 Table 14. Base Condition - Three Class Networks 68 Table 15. Shuffled Condition - Three Class Networks 68 Table 16. Pruned Condition - Three Class Networks 69 Table 17. Extended Duration Condition - Three Class Networks 70 Table 18. Sample excerpt of the output values for network N10EC3C 75 Table 19. Classification data for top ten neural networks 78 Table 20. Types of Misclassification 79 vi LIST OF FIGURES Figure 1. Graphical representation of each machine shape defect 6 Figure 2. Annual production of respondents by class 9 Figure 3. Machine shape defect proportion of respondents by class 9 Figure 4. Probability of shape defect for all mills 11 Figure 5. Graphical representation of mismatch 11 Figure 6. Probability of shape defect by defect proportion 12 Figure 7. Probability of shape defect: high vs. low defect proportion mills 13 Figure 8. Probability of shape defect by annual production 14 Figure 9. Probability of shape defect: large vs. small production mills 15 Figure 10. Machine shape defects most serious in terms of the quality of the final product 16 Figure 11. Machine shape defect causes: benchmark results 18 Figure 12. Machine shape defect causes: thin snake 20 Figure 13. Machine shape defect causes: snipe 20 Figure 14. Machine shape defect causes: taper 21 Figure 15. Machine shape defect causes: fat snake 21 Figure 16. Machine shape defect causes: wedge 22 Figure 17. Machine shape defect causes: flare 22 Figure 18. Machine shape defect causes: mismatch 23 Figure 19. Combined ranking of machine shape defects 28 Figure 20. Photos of measurement equipment 40 Figure 21. Diagram of data acquisition hardware 41 Figure 22. Configuration of lasers measuring a sample board 42 Figure 23. Sketch of pushboard in the apparatus 44 Figure 24. Drawing of calibration block 46 Figure 25. Graphs of logged laser data from a No Snipe board sample 49 Figure 26. Graphs of logged laser data from a Snipe board sample 50 Figure 27. Model of regression lines on a sample board 52 Figure 28. Graphical representation of the snipe classification problem 56 Figure 29. Top Surface: F test values by class 61 Figure 30. Bottom surface: F test values by class 61 Figure 31. Top surface: coefficients of multiple determination by class 62 Figure 32. Bottom surface: coefficients of multiple determination by class 62 Figure 33. Illustration of N9SC2C, linear network with 6 input variables and 0 hidden units... 72 Figure 34. Illustration of N10BC2C, MLP network with 12 input variables and 7 hidden units 73 Figure 35. Illustration of N10BC3C, RBF network with 4 input variables and 1 hidden unit.... 73 vn ACKNOWLEDGEMENT The execution of this project and completion of this thesis would not be possible without the support and assistance from several people. I have categorised them below in an attempt to avoid overlooking anyone. I thank each individual heartily for their time, consideration, patience and in some cases, their sense of humour. Academic Support Thomas Maness Robert Kozak Shawn Mansfield Dallas Foley Darrell Wong Tom Wray Robert Furst Margaret Graham Gary Schajer Tony Besik Christine Lilley Industrial Support Bruce Lehmann, Thin Kerf Technologies Ken Del Puppo, Slocan Group Todd Buchanan, CAM Technologies Nick Barrett, CAM Technologies Dennis Stein, Interfor Phil Warnery, Interfor Terry Maloff, Kalesnikoff Personal Support Marc Sauze Jocelyn Sauze Marian Marinescu Ross Farrell Hauke Chrestin Leah Palmer Kristen Lane viii BACKGROUND In a 1996 survey of B.C. sawmills, increasing fibre recovery was the primary reason for upgrading technology, while improving product quality was the second most important reason (Lee et al 1999). The results of this survey reflect a trend in the wood products industry that is the result of high raw material costs and a highly competitive commodity market. There is a recognition of the importance that technology plays in reducing production and material costs, controlling processes to meet customer demands, and increasing profits. The majority of sawmills in B.C. have adopted quality control methods to increase sawmill recovery and to maximise the value of output lumber by monitoring the process for inconsistencies in lumber sizes (Maness et al 1994). Once a quality control problem develops, it is essential to detect and rectify the problem as soon as possible in order to reduce waste and maximise production. A good sawing process produces lumber with a smooth flat surface and a uniform width and thickness down the length of each board. Problems with the sawing process can result in changes to the board's profile through the width and thickness. Different types of sawing problems cause various shape defects on a board's surface, resulting in different shape profiles. For instance, snake occurs when the blade teeth move laterally in the cut due to a lack of side stability (Thunell 1988). This instability can be the result of either the wood not being held firmly in place or the blade not being tensioned enough. As the resulting profile of the lumber is not uniform, the board requires further processing to remove the sawing defect, reducing recovery and increasing production costs. If the shape defect cannot be rectified or eliminated, the board is downgraded or it becomes waste. Quality control problems in the sawing process are usually detected by inspecting the board's sawn surface, and the cause of the problem is deduced by examining the board's surface characteristics, often manually. Depending on the quality control supervisor's experience and knowledge, troubleshooting a sawing problem can take a considerable amount of time, materials, effort and money. Fortunately, laser-based non-contact sensing systems can be developed to determine the profile or shape of a board in real-time. These systems would detect when a sawing problem is occurring, reducing the manual labour required in monitoring the sawmill process. 1 This project develops a proof-of-concept method to detect sawing defects by classifying board profiles into shape categories, using neural networks. Ultimately the system would incorporate a troubleshooting function to aid in determining the cause of the detected sawing defect. Troubleshooting Methods Troubleshooting systems in the sawmill industry are capable of detecting process problems in real-time but currently are not capable of diagnosing them. Process troubleshooting is primarily accomplished by the quality control (QC) supervisor, using manual techniques. The QC supervisor often relies on his/her sawmill experience to recognise the problem and consequently, deduce its cause. If it were possible to set forth the cause and effect relationship to locate operating difficulties, troubleshooting a mill would be an easy matter. Such is not the case. The effects or symptoms of trouble often arise from any one of a number of possible causes, but more often from a combination of causes. Therefore, troubleshooting a sawmill often becomes a difficult task (Williston 1988). Below are several troubleshooting methods that have evolved to mitigate the difficulties with finding the causes of quality problems in the sawmill process. Unfortunately, these existing methods are time-consuming and cumbersome. Furthermore, the valuable information gained using these techniques risks being forgotten or lost, since sawmills rarely keep a database of quality problems and their causes. One of the most common troubleshooting methods in sawmills is measuring lumber with calipers in order to infer the location of a quality problem within the sawmill process. The QC supervisor collects a sample of boards, measures them and then investigates the running equipment to identify the delinquent part relating to the position of the defect on the boards (Lehmann Interview). Although this procedure takes a considerable amount of time and energy to complete, it continues to be used for lack of a more successful and efficient method. As defect problems tend to reoccur, the QC supervisor often recognises lumber defects and corrects the process problem after quickly verifying their cause. This knowledge is accumulated through experience and through discussions with maintenance and production personnel (Brown 1982). Further insight is developed by reviewing maintenance records, since the source of a particular cyclical problem can be deduced by relating the negative change in quality with some maintenance event during the same period. Consequently, the amount of time, material, effort and money spent troubleshooting a sawing problem depends largely on the QC supervisor's experience, knowledge, memory and availability. 2 Real-Time Lumber Size Control Systems Lumber size control is defined as: A systematic procedure that, properly carried out, identifies and locates problems occurring in sawing-machine centers, sawing systems, or setworks systems (Brown 1986). As a result, lumber size control is often used to troubleshoot quality problems caused by equipment malfunction, though "frequently, several remedies may be applied to one problem area" (Brown 1982). A detailed review by Maness et al (2002) discusses lumber size control systems and their statistical implementation in the sawmill industry; hence, only a brief description follows. Digital calipers are used to make several measurements per board for a sample of boards. The variation in the width and/or thickness of the lumber is determined from the data points. This variation is measured in terms of between-board and within-board standard deviation. The former is literally the variation in size from board to board in the process. It is often related to the performance of the setworks (Brown 1986). The latter is the variation between the measurements taken on the board itself for a sample of boards. It is often used to indicate how accurately a saw is cutting (Brown 1986). The difficulty is that the reliability of using within and between standard deviation alone to pinpoint specific manufacturing problems is low because the sawing process is highly complex and is affected by many factors (Lehmann Interview). For example, a saw cutting through a knot has been shown to deflect through the cant for several feet before recovering, due to the effect of the clearance gap. Analysing the within-board standard deviation indicates a problem with the sawing (depending on the number of samples taken), but in effect, the problem is more complex, involving the saw's interaction with natural defects like knots as well as the saw's parameters like thickness and tensioning (Lehmann 1993). Thus, the problem could also be the type or quality of the raw material. Furthermore, the specific problem with the saw is not known; it could be the saw setup or the choice of saw itself for the particular application. Essentially, there is not enough detail in the standard deviation statistic to establish the actual cause of the quality problem. Additional evidence from the lumber and its defects is essential to making this method reliable. While many sawmills rely on manual methods to control the size of lumber, such as caliper measurement, increasingly they are turning to real-time technology to monitor their processes. However, to date, these real-time systems are limited to identifying the fact that there is a problem in the process and, furthermore, do not take the shape of the lumber itself into account. These systems continuously measure lumber with optical scanners and create variability control 3 charts with the data to determine whether the process is operating in control. When the product output begins to show consistent defects, i.e. the variation becomes too great, an alarm sounds to indicate the manufacturing process is out of control (Maness 1992). In effect, these automatic systems are extensions of the manual caliper method and use generalisations based on the variation in the lumber dimensions rather than analysing the shape of the board in a logical manner. While there are several products on the market, which perform measurement and control charting, there is no evidence that any of them consider the lumber shape or include process troubleshooting functions. Therefore, shape defect detection and analysis is a new approach to controlling the sawmill process, as it models the surface of the lumber to logically determine whether a problem in the process has occurred and where it likely originated. Thesis Objectives The two main objectives of this thesis are to study machine shape defects produced in British Columbia mills and to introduce a method for automatically distinguishing between the different defects. The different classes of machine shape defects in rough green lumber will be determined and defined in terms of their characteristics. In addition, these machine shape defects will be ranked with respect to their industrial significance in British Columbia. Furthermore, an experimental method to detect and to classify these machine shape defects, using neural networks, will be developed as a proof of concept 4 PART I - LUMBER SHAPE DEFECT SURVEY Introduction Several different sawing processes can be employed by a sawmill. The primary breakdown process, which mainly converts the logs into cants, can be accomplished by a headrig bandsaw, a quad bandsaw, a chip'n'saw or an optimising canter, depending on the sophistication of the mill equipment, the size of the logs, the production goals of the mill, and the desired product mix (Williston, 1988). The secondary breakdown, which converts cants into flitches, is normally done by horizontal arbour or vertical arbour saws, either double or single arboured (one or two sets of saws process the cant). The flitches are then processed into lumber by an optimising edger or a reman edger, using chipper heads and/or saws to square the sides and cut multiple boards from an optimised pattern. This pattern is determined automatically at the optimiser edger and manually at the reman edger. Ideally, after edging, all of the lumber edges are parallel to each other and the ends of the boards are rectangular and in line with each other. However, sub-optimal occurrences in the sawing processes described above cause deviations from this ideal shape. Often these deviations are detected as off-size variations in thickness, and one particular defect shape is not necessarily distinguished from another in the downgrading process. Nevertheless, there are six general lumber shapes caused by sawing problems in the mill process. These defects, different from machine defects like torn grain or skip, are referred to as machine shape defects in this paper. Described below, they include snipe, flare, wedge, taper, thin snake and fat snake (Figure 1). Each machine shape defect is described in terms of a possible reason for its occurrence in order to emphasize not only its physical differences, but also its causal differences. The reason for this approach is that some process problems are suspected to cause certain machine shape defects more often than others. For instance, snipe is often associated with mis-timing of the hold-down rolls. When a feedroll lands on a cant too soon, the cant is forced into the saws at an angle, removing a triangular-shaped section from the end. Flare, the complement of snipe, is likely formed by the same phenomenon, but a triangular-shaped section is added onto the end of the board. When some type of misalignment prevents the cant from passing through the saws on a straight course, taper, a gradual thinning (or thickening) down the length of the piece is often visible on the lumber. Conversely, wedge, a gradual thinning (or thickening) across the width or through the thickness of the piece is often linked to problems with the saws, which prevent them from cutting the piece evenly. Snake is frequently attributed to the saws' movement during the cut; fat snake refers to a thicker board, while thin snake is its complement. 5 a. Thin snake b. Fat snake e. Taper f. Wedge Figure 1. Graphical representation of each machine shape defect Although there are several ways in which machine shape defects can reduce the profitability of a sawmill, the extent to which machine shape defects have an economic impact on the entire industry is unknown. Their deviation from the ideal shape of lumber implies that an increase in machine shape defects will reduce mill recovery because of the additional steps in the process required to compensate for them. These steps include increasing target sizes, trimming, and remanufacturing. Thin snake and taper tend to be undersized boards. Typically, sawmills increase their lumber target sizes to compensate for undersizing problems in the process. Therefore, producing snake and taper in significant quantities is likely to cause increases in lumber target sizes. Sawmill simulations which estimate the value and volume recoveries from changes in lumber target sizes have demonstrated that reductions in target sizes significantly increase mill revenues. For instance, an interior B.C sawmill producing dimension lumber would increase its net revenue by $27,160/month for every 0.010-inch reduction in lumber target size (Maness and Lin 1995). Monitoring and controlling these machine shape defects will reduce problems in achieving target sizes and, therefore, result in economic benefits to the sawmilling industry. 6 Snipe and flare are likely to increase trimloss, since they tend to occur at either end of the boards and are trimmed off to avoid downgrading the lumber or to prevent problems in drying and planing. Although the contribution of these machine shape defects to trimloss has not been quantified, a mill experiencing a trimloss of 42 MMBF/year would increase its revenue by $134,400 with a 2% reduction (Thomlinson 1992). On the other hand, wedge and fat snake are remanufactured in order to remove or reduce the effects of the machine shape defects. In some instances, reprocessing the defective lumber at the trimmers or the remanufacturing edger will not be sufficient to rectify the machine shape defects, resulting in downgraded or rejected lumber. Therefore, the mill not only incurs additional reprocessing costs, but must also suffer a reduction in recovery value. Object ive of the S u r v e y of B C Sawmi l l s The objective of this research was to rank the aforementioned machine shape defects found in rough green lumber with respect to their industrial significance. A survey was designed and implemented to determine the industrial significance of each of these machine shape defects in British Columbia sawmills. By ranking these machine shape defects in terms of occurrence and by determining which ones are the most serious in terms of final quality, it can be established which of the machine shape defects have the greatest impact on the industry. This information is important for focussing subsequent research on the classification of machine shape defects. A further benefit of this survey lies in the identification of common causes of each machine shape defect within the sawmill process. This information can be used to develop a troubleshooting guide for analysing various process problems using machine shape defects. M e t h o d o l o g y To determine which of the machine shape defects most affected British Columbia sawmills, a facsimile survey was implemented in the fall of 2000 (Appendix A). The survey focussed on six machine shape defects commonly caused by the sawing process: snipe, flare, wedge, taper, thin snake and fat snake. To avoid a naming bias, the machine shape defects were only graphically represented on the survey. The questionnaire was designed to find out, not only which machine shape defects occur in the sawmill process, but also which ones occur the most frequently, what the defects are likely to be called in the mills, what grades they are typically assigned and what the most common causes are. Furthermore, the questionnaire was designed to identify the most serious machine shape defects with respect to the quality of the final product and to determine which machine shape defects, if any, the survey may have missed. 2 Assuming S200/MBF less the value of S40/MBF in chips. 7 In order to expedite the research process and to reduce costs, the surveys were distributed to British Columbia sawmills by facsimile. The distribution list was compiled from two different forest industry directories. An attempt was made to include every company listing sawmills in British Columbia. The graphical representation of the machine shape defects enabled the survey to be faxed out on one page plus a covering letter. In addition, it was expected that a short and simple questionnaire would encourage a good response rate. The initial survey was followed up three months later by a reminder letter with an identical questionnaire. In the first round, the survey was addressed to the Head Sawfiler &/or Planerman, while in the second round it was addressed to the Quality Control Supervisor &/or Head Sawfiler. The rationale for this decision was that many Planermen may have not responded in the first round because the questionnaire dealt only with the sawmill process. Results Response The survey was distributed to 110 mills throughout British Columbia, with 33 mills returning completed questionnaires for a response rate of 30%. These 33 mills were responsible for 33% BC softwood lumber production in 2000 (4.5 billion Board Feet out of a provincial total of 13.6 billion Bdft)3. Responses came from mills located across BC and from both large and small forest companies. Figure 2 shows the annual production of responding sawmills, while Figure 3 shows the cross section of the responding mills in terms of proportion of machine shape defects. This proportion is the approximate percentage of machine shape defects produced annually in a respondent's sawmill process. 3 Council of Forest Industries (2000), 12/2000 year-to-date total B C softwood lumber production. 8 p 23 78.4 133.8 189.2 244.6 More M M B F Figure 2. Annual production of respondents by class Defect % Figure 3. Machine shape defect proportion of respondents by class Nonresponse bias was statistically tested by comparing the average production of machine shape defects between late and early respondents. Specifically, the mills were grouped according to whether they had responded to the first or the second faxing. The total production of machine shape defects for each mill was determined. These values were then used to calculate the average production of machine shape defects for each respective group of mills. From a t-test of the equality of two means (independent samples and equal variances), no 9 significant difference was observed at an alpha level of 0.05. This finding indicates that nonresponse bias was likely not present in this analysis. Probability of Shape Defect In order to identify the machine shape defects that most affect the industry, the importance of each defect must be established in terms of its probability of occurrence during the sawmill process. Each responding sawmill was given a graphic representation of six machine shape defects and asked to indicate which ones occurred or did not occur in its mill (Figure 1). The mill was then asked to rank the machine shape defects in order of frequency. The responses indicated that snipe and taper each had a 94% chance of occurring in the sawing process, the highest probability by far (Figure 4). Thin and fat snake followed with probabilities of 61% and 58%, respectively. Flare, with a 52% chance of occurring, occurred more frequently than wedge4 at 33%, while mismatch had the lowest probability of occurrence with 12%. Mismatch was not originally included in the survey questionnaire. However, several respondents indicated that mismatch occurred in their mills by sketching it in the available space, so it was included in the analysis (Figure 5). 4 Although the survey cover letter explained that the defects of interest were machine shape defects, and not natural defects like wane, there was some confusion with wedge in three responses. These specific misinterpretations were not used in the analysis. 10 100% "° S 80% | | 60% 3 3 40% o o £ O 20% 0% S t ® ^ J- ^ ^ ^ ' Shape Defect Figure 4. Probability of shape defect for all mills Figure 5. Graphical representation of mismatch Each mill was also asked to indicate their annual production and to approximate the proportion of their total production with machine shape defects. The size of the mills varied between 0.5 and 300 MMBF annual production, while the estimated proportions of machine shape defects varied between 0% and 10%. This information was used to categorise the types of mills experiencing machine shape defects based on annual production and machine shape defect proportion. To analyse the effect of machine shape defect proportion, the mills were separated into four defect proportion categories of 2.5% gradations, starting from 0%. The probability of each machine shape defect occurring in each category was calculated (Figure 6). Al l but one category of the mills surveyed have a 100% likelihood of producing snipe and taper. However, 1 1 in the remaining category, mills with defect proportions of 2.5% or less, snipe and taper still have over an 80% chance of occurring. Either type of snake is most likely produced in mills with defect proportions between 7.5% to 10.0%, at 100% probability, while flare is most likely to occur in mills with defect proportions between 5.0% and 10.0% (two categories), also at 100% probability. At over 50% probability, wedge has the highest chance of occurring in mills with defect proportions between 5% and 7.5%. Mismatch was only reported to occur in mills with defect proportions between 2.5% and 5%. 100% •= 5 60% 5 t O o °- 20% 0% 2.5% 5.0% 7.5% Defect % 10.0% • T h i n Snake • S n i p e • Fat Snake • F l a r e • T a p e r • Wedge • Mismatch Figure 6. Probability of shape defect by defect proportion To simplify the relationship between machine shape defect types and machine shape defect proportions, the mills were divided into high defect and low defect proportion mills. The demarcation between high and low defect proportions was set at 3.75% by observing the results of the histogram combined with the data's measures of central tendency (Figure 7). As expected in this generalisation, all the machine shape defects have a higher probability of occurring in high defect proportion mills than in low defect proportion mills. Snipe and taper are the most likely to occur in each type of mill, followed by thin snake, flare and fat snake which have similar probabilities of occurring. Wedge, followed by mismatch, has the lowest probability of occurring. It is interesting to observe the large difference in probabilities between the two categories of mills for some of the defects. The probability of wedge occurring in a high defect proportion mill is almost four times its probability of occurring in a low defect proportion mill, while for mismatch, this probability is more than doubled. Snipe and taper each have over a 10% higher chance of occurrence in a high defect proportion mill than in a low 12 proportion mill. For thin snake, flare and fat snake, this difference in probabilities is less Q High • L o w 0% 20% 40% 60% 80% 100% Probabil ity of Occurrence Figure 7. Probability of shape defect: high vs. low defect proportion mills To analyse the effect of mill size, the mills were separated into six annual production categories of 50 MMBF gradations, and the probability of each machine shape defect occurring for each category was calculated (Figure 8). Mills with less than 50 MMBF have the highest chance of producing flare at 100% probability. Mills, in all cases except those in the 50 to 100 MMBF category, have a 100% chance of producing snipe and taper. Both thin and fat snake are most likely to occur in mills between 150 and 200 MMBF at over 70% probability, while wedge is most likely to occur in mills between 200 to 250 MMBF at just under 35% probability. At 50% probability, mismatch was reported the most frequently by mills with annual productions between 250 to 300 MMBF. 13 MMBF Figure 8. Probability of shape defect by annual production An interesting trend to note is that the probability of producing either type of snake increases as the annual production increases, until 200 MMBF when it begins to decrease. Flare, on the other hand, shows a downward trend as the annual production increases to 150 MMBF. After this point, its probability increases and then drops off sharply only to increase again at 250 MMBF. Wedge and mismatch both have loose upward trends as the annual production increases, while snipe and taper are consistently at 100% probability except for a dip at 50 MMBF. To simplify the relationship between machine shape defect and annual production, the mills were divided into large and small production mills. The demarcation between large and small production was set at 145 MMBF by combining the results of the histogram with the data's measures of central tendency (Figure 9). In this generalisation, all the machine shape defects have a higher probability of occurring in large production mills than in small production mills, except mismatch. Note that mismatch was not included in the original survey but was reported more often by the small production mills. Snipe and taper are the most likely defects to occur in each type of mill, followed by thin snake, flare and fat snake, which have similar probabilities of occurrence, and finally, by wedge and mismatch. 14 E Large • Small 0% 20% 40% 60% 80% 100% Probabil ity of Occurrence Figure 9. Probability of shape defect: large vs. small production mills To determine whether mills with certain defects are more likely to experience other defects, the occurrence of each machine shape defect in mills was analysed with respect to each other. It was found that all of the mills with taper also produce snipe and vice versa. Less than 55% of the mills with snipe (and taper) have flare. 95% of mills with thin snake also have fat snake. Wedge has no discernible pattern with regard to the probability of other defects. Over 58% of the mills with snipe (and taper) experience fat snake, while over 61% of those mills with snipe (and taper) experience thin snake. 65% of mills with thin snake have flare, while over 68% of mills with fat snake have flare. Shape Defects Affecting Quality In order to ascertain which machine shape defects most affect the industry, the machine shapes that cause the most problems with regard to the quality of the final product must be identified. To that end, each mill was asked to indicate which machine shape defects were the most serious in terms of the quality impacts on the final product (Figure 10). The responses clearly show that thin snake is the most serious machine shape defect in terms of quality for sawmills with 32.8% of the responses, followed by snipe with 24.1%. Taper is the third most serious for mills with 22.4% of the responses, followed by fat snake with 12.2%. Wedge and flare are considered the least serious problems in terms of final product quality, comprising 5.2% and 3.4% of the responses, respectively. No information was obtained on mismatch in this section of the survey. 15 Figure 10. Machine shape defects most serious in terms of the quality of the final product Grade To determine the grades most associated with machine shape defects, mills were asked to cite the most common grade for each of the machine shape defects depicted in the questionnaire. Based on the responses, the most common grade for all machine shape defects was No. 2 Structural (No. 2). Nevertheless, in the majority of mills, thin snake and snipe also required resawing and trimming, respectively. Should thin snake not make No. 2 or be resawn, the lumber was downgraded to No. 3 Structural (No. 3), Economy or rejected as cull in some mills. The lowest grade assignment for snipe was No. 3 or Utility. Fat snake was planed out or resawn and occasionally rejected in mills where it did not make No. 2, while flare was typically sent to the trimmers or the remanufacturing edger. Taper was planed out, resawn or downgraded to No. 3 or Utility, but wedge was typically resawn or downgraded to Economy. There were no comments on the grade assignments for mismatch. Causes of Shape Defect To pinpoint critical process problems which produce machine shape defects, the respondents were asked to list the most common causes for the machine shape defects which occur in their sawmills. To facilitate analyses, these reported causes were organised by machine shape defect in a master list and then grouped into eight general categories. These categories are described below: 16 1. Piece Stability - causes associated with the movement and position of the wood piece as it is processed, including mechanisms to hold the piece in place such as linebars and pressrolls. 2. Saw Condition - causes related to the physical condition of the saws themselves, like worn or hot saws. 3. Alignment - causes associated with the machine's alignment, including misaligned rolls, saws and fences. 4. Saw Stability - causes associated with the movement of the saws in the cut, such as worn guides. 5. Feeding - causes related to the wood pieces' entering the saws improperly, like feedroll timing and overfeeding. 6. Piece Condition - causes associated with the physical condition of the wood piece itself, like frozen or bowed cants. 7. Setup - causes associated with the setup of the machine, including clearances. 8. Operator - causes associated with operator intervention or error. These categories of reported causes were analysed to determine the most common causes of machine shape defects. These percentages were calculated by dividing the total tally for each cause category from all the machine shape defects by the total number of causes reported. The results from this general analysis set a benchmark against which to judge the individual machine shape defects. These individual percentages were calculated by dividing the tally from the machine shape defect of interest for each cause category by the total number of causes reported for that machine shape defect. To illustrate, the formulae for Saw Condition and Thin Snake follows: , „ Total tally of Saw Condition reported % Reported Causes = Total number of causes reported for all machine shape defects , _ „ , . , Tally of Saw Condition reported for Thin Snake % Reported Causes of Thin Snake = Total number of causes reported for Thin Snake The benchmark results show the most common process problems causing machine shape defects (Figure 11). Piece Stability and Saw Condition tie for the most prevalent cause of machine shape defects at 19.5% of all reported causes, followed closely by Alignment with 18.1%. Saw Stability and Feeding follow with 14% and 13.5%, respectively, outranking Piece Condition at 17 10.2%. With just over 5% of the reported causes combined, Setup and Operator are the least common causes of machine shape defects, at 3.7% and 1.4%, respectively. Piece Stability Saw Condition Alignment <D to Saw Stability 3 ( j Feeding Piece Condition Setup Operator 0% 10% 20% 30% 40% 50% % R e p o r t e d C a u s e s Figure 11. Machine shape defect causes: benchmark results Thin Snake: According to the survey results, Saw Condition is by far the most common cause of thin snake with just under 35% of the reported causes, followed by Saw Stability with just over 20% (Figure 12). Both easily exceed their Benchmark scores. Feeding and Piece Condition are fairly common causes with over 15% of the responses each, while Alignment is the least common cause of thin snake with just over 5% of the responses. Setup and Operator are not reported as causes of thin snake. Snipe: In keeping with the general results, Piece Stability is the most common cause of snipe, far exceeding the general score with 42% of the reported causes (Figure 13). In contrast, Saw Condition is the least common cause, since Operator is not reported to cause snipe. Alignment, second after Piece Stability, is less than half as common with 20% of the reported causes. Feeding, at 12% of reported causes, is the third most common cause of snipe. Taper: Alignment is the most common cause of taper at 31.6% of the reported causes (Figure 14). Piece Stability, Saw Condition and Feeding tie for second at 15.8% each, half as common as Alignment. Saw Stability, third, is half as common at 7.9% of the reported causes. Like the benchmark results, Operator is the least common cause. 18 Fat Snake: As with thin snake, Saw Condition is by far the most common cause of fat snake at 34.5% (Figure 15). However, Piece Condition follows with just under 20% of the reported causes. Saw Stability is the third most common cause with 17.1%, while Piece Stability is the least common cause with 4.9%. Setup and Operator are not reported to cause fat snake. Aside from these last two, Piece Stability and Alignment are the only causes to lag behind their respective Benchmark scores. Wedge: Saw Condition is also the leading cause of wedge with 35.7% of the reported causes, followed by Alignment with 28.6% and Saw Stability with 14.3% (Figure 16). Here, the sequence somewhat mimics the Benchmark results. However, Piece Stability, Feeding and Piece Condition are the least common with 7.1% each, given that Setup and Operator do not rate as causes. Flare: Flare, like snipe, has Piece Stability as the most common cause, followed by Alignment. (Figure 17). Saw Stability and Feeding are the third most common causes of flare at 13.8%, while Piece Condition and Operator are the least common at 6.9%. Flare is the only machine shape defect not to have Saw Condition reported as a cause. The causes exceeding their respective Benchmarks significantly are Piece Condition, Setup and Operator. Mismatch: The reported causes for mismatch include Saw Condition, Alignment and Saw Stability (Figure 18). Saw Condition and Alignment are the leading causes with 40% each, while Saw Stability is half as common at 20%. This sequence is somewhat consistent with the Benchmark results, though no other causes are reported and the scores are much higher. 19 Figure 13. Machine shape defect causes: snipe 20 Figure 14. Machine shape defect causes: taper 0% 10% 20% 30% 40% 50% % R e p o r t e d C a u s e s Figure 15. Machine shape defect causes: fat snake 21 Figure 17. Machine shape defect causes: flare 22 0% 10% 20% 30% 40% 50% % Reported Causes Figure 18. Machine shape defect causes: mismatch Some of the reported common causes grouped into the eight general categories highlight problems with a specific part of the equipment, in addition to a process problem. These equipment-specific causes show up in different forms across the eight general categories. Three machine shape defects show this pattern of equipment-specific cross-category causes. They are snipe, flare and taper. In the case of snipe, the predominant equipment-specific cause is problems with the rolls, including feedroll timing, outfeed roll misalignment or worn bed rolls, for example. These roll problems account for 24.0% of the total reported causes for snipe. Flare is also caused by problems related to the rolls, like roll pressure, feedroll timing and outfeed roll misalignment. These roll equipment problems arise in several of the general categories and account for 24.1% of the total reported causes of flare. For taper, the predominant cross-category cause relates to problems with the linebar. These linebar equipment problems account for 15.8% of the total reported causes of taper and include, but are not limited to, linebar misalignment, pieces not tight to the linebar, and internal linebar failure. Nomenclature To ensure that the nomenclature of the machine shape defects in the survey was consistent with industry terminology and to determine whether it is consistent throughout the industry, the respondents were asked to label the graphical representation of each defect with its common mill name (Figure 1 on p.6). Thin snake and fat snake were both primarily called 'snake' or a variation of snake such as 'snaking' or 'snakey sawn' (Figure 1. a & b). One mill referred to thin snake as 'negative 23 snake' and fat snake as 'positive snake', while two mills differentiated between the two snakes by qualifying them as 'thin' or 'thick' boards. Otherwise, no distinction was made by the mills. Interestingly, two mills named the snake defect with respect to its process cause. The first mill employed the term, 'resaw dip', which relates to problems with the handsaws on the resaw. The second mill used the term, 'linebar deviation', referring to cants not being properly held against the gang saw linebar. Snipe was primarily called 'snipe' or a variation qualified by the term, 'end', such as 'sniped end' or 'end snipe' (Figure l.c). One mill described the end of the board in question by using the terms 'lead end snipe' and 'tail end snipe' to locate the snipe. The two naming exceptions were 'tapered end' and 'bull head'. Flare was mainly called 'flare', though three mills did not differentiate it from snipe (Figure 1. d). The two exceptions were 'bump' and 'club foot end'. Taper was mainly called 'taper'. However, slightly less than a third of the mills referred to it as 'wedge', while three mills used the two terms interchangeably (Figure 1. e). One mill called it 'bevel'. Yet another mill pointed to the cause of the taper defect by naming it 'thin line bar board', referring to problems with the cant at the cant optimizer. Wedge was mainly called 'wedge', although several mills used the term 'bevel', and one mill interchanged these two terms (Figure 1. f). One mill incorrectly referred to this defect as 'snipe', while another referred to it as 'taper'. Three responses demonstrated confusion with the graphical representation of wedge by naming natural defects. Two of these mills called it 'wane', while the third called it 'sidecut', which can result in waned edges. As mentioned previously, mismatch was not originally included in the survey; however, several respondents sketched in the defect and labelled it 'mismatch' or 'saw mismatch' (Figure 5). One mill labelled it 'step'. 24 Discussion Probability of Shape Defect Respondents ranked machine shape defects with regard to frequency of occurrence as follows: 1. Snipe and Taper 2. Thin Snake 3. Fat Snake 4. Flare 5. Wedge 6. Mismatch From the results of the response analysis, it is clear that snipe and taper are the most frequently occurring machine shape defects in British Columbia sawmills. They are distantly followed by thin and fat snake, which lag behind by over 30% of the responses. However, this evidence alone does not ensure that snipe and taper should be the primary focus of subsequent research because their impact on the quality of the final product must also be taken into account. A machine shape defect that occurs the most frequently, but that can be easily corrected in subsequent processes such as planing, is less threatening to the profitability of a mill than one that occurs frequently, but must be downgraded or scrapped. Nonetheless, this information brings to light the fact that a problem with these types of defects most assuredly exists, highlighting the need to improve certain aspects of the sawing process in BC mills. Characterising BC sawmills according to machine shape defects and annual production shows that, for each category of mill, but one, there is over a 20% probability of producing at least five types of machine shape defects. The mills making up the 5 to 7.5% defect proportion category and those making up the 150 to 200 MMBF annual production category generally have the highest probabilities of shape defect, suggesting that they have the most work ahead of them with respect to controlling their sawing processes and reducing the instances of machine shape defects. Further research is required to pinpoint the reasons why mills in these two particular categories experience more problems with machine shape defects than the others. Simplifying the sawmill categories into high and low defect proportions produces results in keeping with expectations. Lower defect proportion mills have a lower probability of producing machine shape defects, since it is presumed that these mills have better control over the quality of their sawing process. The fact that they still experience problems with some of these defects suggests that the causes are either difficult to control or are not known to the mills. Separating the mills into large and small production mills is also consistent with expectations. Large production mills likely experience more problems with machine shape defects because faster, 25 high volume processes create difficulties in detecting and correcting problems, often before significant volumes of defective lumber are produced. Very few mills, if any, have real-time quality control systems, which would address this difficulty in controlling the process. It was found that most mills experiencing problems with one type of machine shape defect are likely to encounter problems with other types of machine shape defects as well. These co- occurrences are linked to the fact that machine shape defects share common causes with each other (p.18- 19). Shape Defects Affecting Quality The processes normally following primary and secondary breakdown in a sawmill are edging, trimming, drying and planing. It is in trimming and planing that a machine shape defect is often corrected, although certain defects may only effectively be corrected via resawing or remanufacturing. It is often less desirable to trim rather than to remanufacture for two reasons. First, in trimming a piece, it becomes shorter. This process improves its grade but reduces final product options. On the other hand, remanufacturing removes the undesirable characteristic of the piece, allowing it to remain full length in most instances. Second, trim ends result in trimloss and reduce the recovery of a mill (as discussed above). In contrast, remanufacturing yields higher recovery rates, although an extra step is applied to the piece. The trade-off is the costs associated with the additional steps in remanufacturing. Respondents ranked machine shape defects with regard to quality as follows: 1. Thin Snake 2. Snipe 3. Taper 4. Fat Snake 5. Wedge 6. Flare In light of the above discussion, it is logical that thin snake is the most serious machine defect in terms of the quality of the final product, since it is the most difficult machine shape defect to correct in subsequent processes. If not remanufactured into 1" boards, thin snake results in skip at the planer and is downgraded according to the degree and depth of scantness (NLGA 2000). Snipe, the second most serious defect, also results in skip at the planer, unless the more scant areas of the board are trimmed off. In the case of taper, the third most serious machine shape defect, a scant end is removed at the trimmers, but an over-thick end is either planed out or resawn. Also over-thick in areas, fat snake is corrected through planing or remanufacturing to remove the excess ripples of wood from the piece, while flare is corrected by remanufacturing 26 or by trimming to remove the additional section of wood. The most problematic type of wedge is a gradual thinning (or thickening) across the width of the piece, as opposed to a gradual thinning through the thickness which is easily corrected at the edger. However, wedge across the width can normally be corrected by resawing, unless the thin side is too scant, a problem which results in significant downgrading as wane. Combining the quality rankings with the frequency of occurrence rankings identifies the machine shape defects with the greatest impact on the sawmill process (Figure 19). The machine shape defects in the upper right hand quadrant of the graph appear to have the most impact on the industry in BC. However, a complete assessment must also incorporate the influence of lumber downgrading. The combined results show that thin snake is the most serious in terms of the quality of the final product, but the third most frequently occurring defect. Thus, BC mills have recognised the problems with thin snake and are addressing them. However, snipe also requires attention. It is one of the most frequently occurring defects and the second most serious in terms of the quality of the final product. In addressing the problems causing snipe, mills will increase their recovery by decreasing the trimloss incurred from correcting snipe downstream. Taper, the other most frequently occurring machine shape defect, is the third most serious in terms of the quality of the final product. Likely its reprocessing appears insignificant because a scant end is removed at the trimmers or an over-thick end is planed out or resawn. 27 Most Common Fat Snake Least • - Serious Flan are Wedge Tamper Snipe Thin Snake 1 ̂ Most Serious Least Common Figure 19. Combined ranking o f machine shape defects Grade Grading rules set minimum guidelines for the final dimensions and characteristics o f each board. The grading rules employed in this research are set by the National Lumber Grades Authority (2000). Machine shape defects are not specifically characterised by these grading rules, but are limited by certain characteristics specified in the rules. The main characteristics applied in grading machine shape defects are skip and wane. Skip accounts for scantness resulting in areas on the face or edge o f a piece that fail to surface clean at the planer. Wane accounts for a "lack o f wood from any cause on the edge or corner o f a piece o f lumber" ( N L G A 2000). Note that wane itself is a natural defect, not a machine shape defect. However, the two are treated the same way in accordance with the grading rules. Downgrading o f machine shape defects takes place when the defect characteristics exceed the minimum guidelines for N o . 2 Structural, the most commonly cited grade. A s some machine shape defects are downgraded to N o . 3 or Ut i l i ty , and others to Economy, a short comparison o f the guidelines for these grades is in order (Table 1). It is immediately apparent that the guidelines for N o . 3 and Uti l i ty are identical wi th respect to skip and wane. A s expected, the tolerances for more skip and larger wane increase as the grade decreases from N o . 2 to N o . 3 (or 28 Utility) to Economy. However, Economy is the only grade with an allowable depth of skip exceeding 1/8". It is also the only grade allowing through portions, which are essentially gaps in the lumber. An example from the survey response highlights the cost of downgrading lumber associated with machine shape defects. Based on the survey data, the average BC sawmill produces approximately 5.65 MMBF of machine shaped defective lumber. If only 3% of the machine shape defects result in downgrading the lumber from No. 2 to No. 3 Structural, the average sawmill loses just over $12,700 every year5. That snipe and taper are usually only downgraded to No. 3 or Utility suggests that the mills either correct the machine shape defects in downstream processes or misjudge the pieces' final thicknesses. For the most part, snipe and taper result in hit or miss skip, and in less than 10% of the cases, heavy skip at the planer. Some mills may have the same problem misjudging the final thicknesses of lumber with thin snake, since it is downgraded to No. 3. Other mills may not consider it worthwhile to resaw the snaked lumber into 1" and downgrade it to Economy or even reject it. Wedge, on the other hand, is likely easier to judge since it is graded as a wane characteristic. The piece is either successfully remanufactured or downgraded to Economy, which has a fairly generous wane allowance. Fat snake also appears to be easier to manage as it is generally corrected and infrequently rejected. Neither fat snake nor flare appears to be downgraded, likely because they are simple machine shape defects to correct. 5(0.03 * 5.65MMBF/year*Sdi f f ce/MBF= and No. 3 random lengths 170MBF*75=12,712.5$/yr) Based on a $75 difference between No. 2 29 No. 2 Structural No. 3 Structural Utility Economy Skip hit and miss*, hit or miss with a hit or miss with a 1/4" scant in with a maximum maximum of maximum of thickness and/or of 5% of the 10% of the 10% of the width. Not pieces containing pieces containing pieces containing limited in length. hit or miss** or heavy skips. heavy skips. heavy*** skip 2' or less in length. Wane 1/3 thickness and 1/2 thickness and 1/2 thickness and 3/4 width, full 1/3 width full 1/2 width full 1/2 width full length. If length, or length, or length, or through the edge, equivalent on equivalent on equivalent on equivalent to each face. Wane each face. Wane each face. Wane area of 75% of not to exceed 2/3 not to exceed 7/8 not to exceed 7/8 cross-section. thickness or 1/2 or 3/4 width for or 3/4 width for Through portion width up to 1/4 up to 1/4 length. up to 1/4 length. not to exceed 2' length. in length. If across the face, 1/2 width must not exceed 1/4" scant in thickness for 1/3 length or, as equivalent longer. * Hit and miss skip is a series of skips not over 1/16" deep with surfaced areas in between * * Hit or miss skip means completely or partly surfaced or entirely rough. May be 1/16" scant. * * * Heavy skip is not over 1/8" in depth Table 1. Comparison of grades with respect to skip and wane 30 Causes of Shape Defect The fact that Piece Stability and Saw Condition, followed closely by Alignment, are the most common causes of machine shape defect draws attention to two issues (Figure 11). First, these process problems are the most difficult to control and, second, they generally require the most attention to reduce or eliminate machine shape defects. However, it is also important to consider which specific shape defects are produced by these problems, since a more frequently occurring defect indicates that its cause is not being addressed or recognised by the mill. Results of the survey indicated that Piece Stability and Saw Condition rate as the most common causes of individual machine shape defects, with the exception of taper (Table 2). Piece Stability is the primary cause of snipe and flare, while Saw Condition is the primary cause of thin snake, fat snake and wedge. Alignment is the primary cause of taper, but both Alignment and Saw Condition are responsible for causing mismatch. Machine Shape Defects Top Three Causes Thin Snake Snipe Taper Fat Snake Wedge Flare Mismatc h 1 Saw Condition Piece Stability Alignment Saw Condition Saw Condition Piece Stability Saw Condition & Alignment 2 Saw Stability Alignment Saw Condition, Piece Stability & Feeding Piece Stability Alignment Alignment Saw Stability 3 Piece Condition & Feeding Feeding Saw Stability Saw Stability Saw Stability Saw Stability & Feeding Table 2. Top three most common causes of each machine shape defect That snipe and flare are most commonly caused by problems associated with the stability of the piece follows expectations in two respects. First, they have the same types of causes because they are essentially each other's respective shape complements. Second, they both are the result of problems like a mis-timed roll forcing the piece to move during the sawing process. However, the fact that both types of snake and wedge are most commonly caused by problems relating to the condition of the saw is unexpected, since it was thought that they were created by 31 the saws' moving during cutting. In retrospect, this finding is reasonable because a hot or warped saw does not make a straight cut and results in a wavy surface. It was expected that Alignment problems most commonly cause taper because this defect results from sawing the piece at a steady angle. It is not surprising that mismatch has both Saw Condition and Alignment as its most common causes, since it is often attributed to either improperly filed saws on double-arbour edgers or problems with offset in aligning the two sets of saws (Williston 1988). Snipe and taper are the most frequently occurring machine shape defects, indicating that Piece Stability and Alignment are difficult problems and require attention from the mills. Nonetheless, Saw Condition still needs to be improved in order to reduce the frequency of thin snake. Piece Stability is very difficult to control in the sawing process because of equipment vibration and jarring movements from the hold-down mechanisms and from the logs or cants themselves. Several different methods of controlling the piece through the sawing process have been developed. Generally, most systems rely on some type of hold-down mechanism to steady the piece as well as some type of mechanism to keep it centered as it is processed. On a headrig, the knees or dogs hold the log in place as it is sawn, often against a fence, while a canter usually has a chain or slat bed feed with hold-down rolls interspersed strategically down its length to keep the cant in its optimal position (Williston 1988). One centering device, which is still used on some older models of Chip 'n' Saws, chips a 2x4 key in the bottom of the log to hold the piece centered through the canter. This key acts like a wheel on a rail and is later sawn off to make a low grade 2x4. With the increased demand and value of fibre, this concept was refined to use splines to center the log through the canter. For arbour saws, the cant is transported on a slat bed or narrow chain, using a rollcase and linebar or a rollcase and centre-feed system to position the cant in the saws. A sawmill's best strategy may be to maintain a rigorous preventative maintenance schedule to keep the hold-down mechanisms well maintained, since wear reduces their control over the piece and precision in positioning it. Monitoring the lumber on the outfeed for defect shapes would help to indicate when the stability problem gets out of control. Alignment is also very difficult to control due to machine vibration and jarring from the piece being processed. The taut-string method of aligning the equipment by ensuring that the rolls, saws and/or chipping heads are lined up and centred in the vertical and horizontal planes has long been replaced by optical methods using lasers (Williston 1988). However, the alignment performed on the equipment is still static, whereas a sawing operation is dynamic, and therefore, 32 1 requires some type of dynamic test to ensure that the sawing and chipping actions take place in the manner prescribed by the optimisation system or anticipated by the operator. Because saws are constantly running and their performance is affected by many factors, such as feedspeed and wood species, Saw Condition is difficult to control as well. Most sawmills mitigate problems with their saws by changing them regularly throughout the shift. They are heavily reliant on their Sawfiler's ability to recognise the fundamental cause of sawing quality problems and on his/her expertise to resolve them effectively and economically. Not only do Sawfilers consider the size, shape and metallurgy of the saw required for a particular situation, but they also design the saw tips and gullets to meet specific needs (Lehmann 1993). Problems with rolls and linebars are noteworthy in terms of the number of responses as a cause of snipe, flare and taper. Preventative maintenance measures should, therefore, focus on these particular components of the sawing process in order to reduce incidents of these machine shape defects. From the data, it appears that monitoring the timing and alignment of the rolls, as well as the condition and alignment of the linebar, would prove beneficial. Nomenclature The nomenclature applied to machine shape defects is fairly consistent throughout BC sawmills. This result was surprising given that there are no set definitions of these defects in the Standard Grading Rules For Canadian Lumber and no other definition sources have been located to date. However, grading competitions and regional lumber grade inspectors may contribute to the uniformity in naming defects. Furthermore, the sawmilling community in British Columbia is fairly close-knit and information is often shared between mills, especially as employees change jobs. Finally, the quality control programs in British Columbia's educational institutions may influence the nomenclature in mills as the number of graduates increases in the sawmill workforce. Despite this consistency, difficulties did arise in labelling the graphical representation of taper. It was frequently confused with wedge. This confusion is likely due to the influence of ordinary expressions rather than a misinterpretation of the sketch. For instance, a tapered piece of lumber resembles a wedge used to block a door open. When this survey was designed, there was some discussion with regard to using the term wedge rather than bevel to describe this machine shape defect. Thus, it is not surprising that several mills used the latter term to describe this shape. However, it is difficult to understand how it was confused with snipe. 33 Conclusions Based on this survey, the majority of BC sawmills produce machine shape defects and recognise that these defects have the potential to affect the qualify of their final products. However, the mills surveyed take different approaches with respect to the machine shape defects they encounter. Often the machine shape defect is corrected by downstream processes, like trimming, or downgraded in the planermill. Unfortunately, these both incur negative consequences to the profitability of a mill through reduced recoveries and decreased product values. An example from the survey response highlights the cost of downgrading lumber. Based on the survey data, the average BC sawmill produces approximately 5.65 MMBF of machine shaped defective lumber annually. If only 3% of the machine shape defects result in downgrading the lumber from No. 2 to No. 3 Structural, the average sawmill loses just over $12,700 every year. While many mills recognise the causes of these machine shape defects, it is not clear whether the difficulty of addressing the processing problem impedes them from preventing the defect or whether preventing the defect is not a priority because it can be corrected downstream. However, the fact that thin snake is considered the most serious in terms of the quality of the final product, but is actually the third most frequently occurring defect, indicates that BC mills have recognised the problems with thin snake and are addressing them. Nevertheless, snipe also requires attention. Not only is it one of the most frequently occurring defects and the second most serious in terms of the quality of the final product, but in correcting snipe downstream, a mill's recovery is decreased through increased trimloss. Taper, the other most frequently occurring machine shape defect, is considered the third most serious in terms of the quality of the final product. Likely, it has not been addressed because, like snipe, it is only downgraded to No. 3 or Utility. In addition, its reprocessing may appear insignificant, since a scant end is removed at the trimmers or an over-thick end is planed out or resawn. Characterising the mills according to their proportions of machine shape defects encountered and annual productions shows that each category of mill, with one exception, has over a 20% probability of producing at least five types of machine shape defects. The mills making up the 5 to 7.5% defect proportion category and those making up the 150 to 200 MMBF annual production category generally have the highest probabilities of machine shape defects, suggesting that they have the most work ahead of them with respect to controlling their sawing processes and reducing the instances of machine shape defects. Further research is required to pinpoint the reasons why sawmills in these two particular categories experience more problems with machine shape defects than others. 34 This survey has identified three priority areas of focus for mills desiring to reduce their production of machine shape defects. These critical areas are Piece Stability, Saw Condition and Alignment. Stabilising the piece will significantly reduce snipe and flare, while improving and monitoring the condition of the saw will significantly reduce thin snake, fat snake and wedge. Concentrating on machine alignment will decrease taper significantly, as well as reduce mismatch, which is also affected by the stability of the piece. Choosing to concentrate on any of these improvements will have ripple effects on reducing the production of machine shape defects in general because they share common causes. 35 PART II - CLASSIFICATION OF MACHINE SHAPE DEFECTS Introduction Several different sawing processes are employed by a sawmill. The primary breakdown process, which converts the logs into cants, can be accomplished by a headrig bandsaw, a quad bandsaw, a chip'n'saw or an optimising canter, depending on the sophistication of the mill equipment, the size of the logs, the production goals of the mill, and the desired product mix (Williston 1988). The secondary breakdown, which converts cants into flitches, is normally done by horizontal arbour or vertical arbour saws, either double or single arboured (one or two sets of saws process the cant). The flitches are then processed into lumber by an optimising edger or a reman edger, using chipper heads and/or saws to square the sides and cut multiple boards from an optimised pattern. This pattern is determined automatically at the optimiser edger and manually at the reman edger. Ideally, after edging, all of the lumber edges are parallel to each other and after trimming, the ends of the boards are rectangular and in line with each other. However, sub-optimal occurrences in the sawing processes described above cause deviations from this ideal shape. Often these deviations are detected as off-size variations in thickness, and one particular defect shape is not necessarily distinguished from another in the downgrading process. Part I of this thesis identified six general lumber shapes caused by sawing problems in the mill process. These defects, different from machine defects like torn grain or skip, are referred to as machine shape defects. They are snipe, flare, wedge, taper, thin snake and fat snake (Figure 1). Classifying these machine shape defects is advantageous for several reasons. First, as each type of machine shape defect is primarily caused by a particular combination of process problems, their classification facilitates process troubleshooting. For instance, producing large quantities of snipe indicates that there is a piece stability or alignment problem, from a linebar failure or misaligned roll for example (Table 2). Second, the type of machine shape defect prevalent in the mill process shows how the mill is affected by the process problem. With snipe for example, downstream processes like trimming are impacted by increased flow, or the amount of lumber downgraded to No. 3 Structural or Utility is increased. Third, the frequency of occurrence of these machine shape defects in the sawmill process indicates the magnitude of the problem in the mill. For example, if snipe accounts for approximately 23%6 of the 5.65 MMBF machine shape defective lumber produced by an average BC sawmill every year, the sawmill can estimate the cost of the snipe defect problem. If the mill frequently downgrades sniped 6 Based on percent chance of occurring in the sawing process (Figure 4) 36 lumber to No. 3 or Utility, the cost of the problem is based on the difference between selling prices. However, if the mill normally trims off the sniped end, then that cost is calculated based on decreased recovery and increased downstream processing. A neural network is a generalisable model built from a set of training data, using experience rather than explicit rules. The model is actually a set of functions, called units, which are linked together by weights that describe the effect each unit will have on the overall model. For a complete understanding of these functions, please refer to Bishop's Neural Networks for Pattern Recognition (1995). Using neural networks is a desirable way to classify machine shape defects because their pattern recognition capability enables them to classify systems that are too complex to model with rules (Swingler 1996). Neural networks also have the potential to perform these classification tasks in real-time, while maintaining their fault tolerance performance (Skrzypek 1991). They provide better fail safety or fault tolerance than classic sequential computing systems. For instance, rather than stalling the whole system, problems occurring in one part of the neural network can be overcome because the information is distributed throughout the network with mainly locally connected nodes (Tulunay 1991). Experimentation has been done on diagnostic systems similar to grading where there are large sets of rules used to recognise problems. For example, on an experimental basis, neural classifiers performed the recognition task to diagnose system faults in an automotive control system (Williams 1993; Marko 1989). Furthermore, neural networks appear to be relatively easy and inexpensive to implement and to maintain. In this application, additional hardware and software requirements are minimal. However, programming required beyond the off-the-shelf software depends on the level of automation desired by the user or sawmill. Once trained and installed, neural networks demonstrate a certain flexibility for changing conditions and ability to accommodate defective hardware, useful attributes in the sawmill environment (Tulunay 1991). This pilot experiment uses neural networks to classify sample boards by the machine shape defect(s) they contain. The study focusses on the snipe defect because it was determined to have the greatest impact of these six machine shape defects on the Wood Products industry in the results of the survey in Part I of this thesis. Objective of Neural Network Classification This research focusses on developing a method to detect machine shape defects in the sawmill. The objective is to establish whether neural network classification of the aforementioned machine shape defects found in rough green lumber is feasible. In particular, can neural 37 networks be trained to distinguish between those boards which contain snipe (or a combination of snipe and another defect) and those which do not? This question will be addressed by a proof of concept study, not by a full development. For this reason, this experiment is restricted to one machine shape defect: snipe. As mentioned previously, snipe is the machine shape defect of choice because it was found to be the most important in British Columbia sawmills in Part I of this thesis. The main benefit of employing neural networks to classify machine shape defects is the potential for automated real-time identification of these defects in the sawmill process. 38 Methodology Sample Measurement Measurement Equ ipment A self-contained measuring apparatus was constructed to support the measuring equipment and convey the sample boards, driven by an automatic feeder (Figure 20). Three laser displacement sensors (lasers) were positioned above the measuring bed and three were positioned below. A table nearby held the data acquisition hardware, including the data acquisition card and PC (Table 3). Cables ran from the lasers to the data acquisition hardware to the PC. 39 b. Overview of measuring apparatus (centre) and data acquisition hardware (right) Figure 20. Photos of measurement equipment Table 3. List of Equipment Equipment Type Equipment Description Qty Laser LDS Displacement sensor model LDS 80/10 6 Data Acquisition National Instruments PCI-MIO-16E-4 1 Card Desktop PC Pentium 100MHz 1 Automatic Feeder Pertio power feeder 1 Measuring Apparatus Custom design 1 . Power Converter draws 110 V 1 Software DAQ Sim 1 NI DAQ 6.5 Driver 1 Windows NT platform 1 Microsoft Excel 1 The lasers used were Dynavision LDS Displacement sensors, model LDS 80/10. These lasers were capable of rates up to 500kHz. They automatically compensated for differences in the board's surface and colour by varying the power to the laser diode (LMI). The laser beam emitted from a laser diode struck the board surface and reflected onto a position sensitive detector. The current output signals from this detector were translated into distance by signal processing electronics, in this case, a data acquisition card (Figure 21). The data acquisition card was a National Instruments PCI-MIO-16E-4 board with a NI-DAQ 6.5 driver, using a Windows NT platform. It used custom designed Visual C++ software. The software program, called DAQsim, translated the current output signal from each laser into a distance measurement by dividing the signal into intervals called divisions. In this case, the division size was 2.4 micrometers for the laser measurement range of 10 mm and the data acquisition card resolution of 1 in 4096. The resolution was the smallest change in the signal from the lasers that can be detected by the acquisition card. Therefore, the conversion from divisions to metric used the following formula: measurement range in m * (data acquisition card resolution)"1 = micrometers/division So, 10"3m * (4096)"1 =2.4 micrometers/division 40 In this application of DAQSim, six channels logged the data from the six lasers. Each channel was set to a channel size of 10, a rate of 600 points per second and a queue size of 10. The laser data was logged into a comma separated file (*.csv) for each sample board and can be imported into Microsoft Access or Microsoft Excel. I N P U T L A S E R 0 - ± 1 5 V D A T A A C Q U I S I T I O N -O GRD C H A N N E L P O W E R C O N V E R T E R G- G" + 5 V G R D -G O U T P U T P C B O A R D Figure 21. Diagram of data acquisition hardware Six lasers measured a sample board continuously. Three lasers were lined up across the width of the top of the board and three were lined up across the width of the bottom (Figure 22). The configuration of the lasers enabled both the top and the bottom surfaces of the sample board to be measured in three places: two sides and the center. The lasers were staggered for two reasons. Their size was relatively large compared to the width of the board and the lasers had to be oriented perpendicular to the feed direction of the sample board. 41 FEED SECTION LOOKING EAST Figure 22. Configuration of lasers measuring a sample board The lasers were fixed in the measurement apparatus at an operating distance of 80 mm from the surface of the rough green lumber sample. A rise or dip in the surface of the sample was detected by a corresponding change in the data readings. Using the lasers in pairs, this profile measurement was made at the same rate along the board. The thickness of the lumber sample can also be determined from each pair of lasers; however, this research was focussed on analysing the shapes of the lumber surfaces, not the absolute thickness changes in the lumber. 42 S a m p l e B o a r d s a n d M e a s u r e m e n t P r o c e s s One hundred and three rough green trim ends were sampled randomly from a mill experiencing difficulty processing frozen wood. The resulting sample boards ranged from 7 5/8" to 24 5/16" in length. The board defects were primarily machine shape defects, although some boards also had natural, seasoning and/or manufacturing defects (Table 4). The sample boards were manually assessed for machine shape defects by the author. The number and types of machine shape defects were recorded with the sample board number for the categorization required in preprocessing. Snipe was the most prevalent defect, found in over 40% of the samples, while very few samples had no defects at all. Table 4. Defects found in sample boards Defect Characteristic* Classification* # Samples"1" snipe machine shape 43 machine gouge manufacturing 31 mismatch manufacturing 30 wane natural 17 wedge machine shape 16 skip or roughness manufacturing 14 sawcut manufacturing 8 taper machine shape 4 knot tearout manufacturing 4 no defect N/A 3 thin manufacturing 2 pickaroon hole manufacturing 2 split seasoning 2 * (NLGA 1998) except machine shape defects + Number exceeds total of 103 due to defect combinations in samples The automatic feeder conveyed the sample boards through the measuring range of the six lasers at a steady rate. From experimentation, it was calculated that the measurement rate was 46 points per inch (18.11 points per cm). Because the automatic feed rollers could not drive the sample boards completely through the lasers, a pushboard followed each sample board through the rollers to ensure each sample board was completely scanned by the lasers (Figure 23). The pushboard was also designed to signal the end of each sample board. The 5 mm notches on the top and bottom surfaces of the pushboard signalled the end of the sample board and the beginning of the pushboard by spiking the data to the out-of-range value, 4095 divisions. The 43 pushboard also doubled as a spare calibration block throughout the data acquisition, having been set in the same orientation following every sample piece. Though the lasers were not wired to signal out of range, the leading edge of the sample piece can be detected by a sudden change in the data from the steady reading of 4095 divisions, the maximum value set by the DAQsim program. S E C T I O N L O O K I N G N O R T H Figure 23. Sketch of pushboard in the apparatus The data for each sample board consisted of six sets of laser readings paired with a sample number. Each laser measured the board continuously, while the software logged the data from all the lasers into one file for each sample board. Physically, the measurement data was split into top and bottom sets because the lasers were positioned above and below the sample board. The top set of lasers was T l , T2 and T3, while the bottom set was BI, B2 and B3. As the lasers were staggered in the measuring apparatus, the data in the files was recorded in the sequence the lasers detected and measured the sample board. The top lasers detected the sample board in numerical order: first T l , second T2, and third T3, as did the bottom lasers: first BI, second B2, and third B3. This sequence of board detection was critical for the analysis of the top surface and of the bottom surface, since the laser data start points were verified using the top and bottom lasers in pairs. 44 Data Validation S o u r c e s o f E r r o r & T o l e r a n c e Sources of error were associated with the construction of the measuring apparatus, the fabrication and design of the calibration block, the capabilities of the lasers and other hardware, as well as the human intervention factor. Tolerances and margins of error were evaluated in order to outline the limitations of this pilot experiment (Table 5). The intention was to enable future endeavours to be built upon this project by setting the results in the context of their limitations. The Total Tolerance was estimated by adding up the tolerance of the individual components of the measuring apparatus, from the jig holding the lasers to the acquisition card (Appendix B). The Within Laser Tolerance did not include the component tolerance from the laser jig because data testing for each laser assumed the laser's position remained constant relative to itself. Results outside the Within Laser Tolerance would show the laser may have moved. Tolerances for the separate components were obtained from several sources including equipment manuals, websites, design drawings and physical measurement. The tolerance was calculated in divisions to facilitate the data verification and testing process. The Total Tolerance was the Within Laser Tolerance plus the component tolerance from the laser jig. Table 5. Equipment Tolerances Equipment Component Estimated Tolerance Tolerance in divisions Laser jig height 0.1 mm 40.950 Laser accuracy 0.05 mm 20.474 Calibration Block height +/- 0.0254 mm 20.803 Acquisition Card precision 2.44 mV 0.999 accuracy 4 microseconds to settle Within Laser Tolerance 0.103 mm 42.3 divisions Total Tolerance 0.203 mm 83.2 divisions C a l i b r a t i o n B l o c k A calibration block was designed and fabricated for use in testing the validity of the laser measurement data (Figure 24). The thickness of the calibration block was based on the rough green thickness of typical sawmill target sizes. Steps were machined in the block for two major reasons. First, the steps were for testing the lasers' ability to detect the changes anticipated in 45 the machine shape defect lumber samples. From the specifications, the average resolution of the lasers was less than 0.1 micrometers. As this figure was an average, it was important to ensure that the lasers can detect the dimensional changes which distinguish machine shape defects. Second, the steps were for testing the effective measurement range of the lasers. It was important to establish that the measurement data remained accurate throughout the whole range, not just at operating distance, since this information was not available from the manufacturer. The smallest step was 5 thousandths of an inch, while the largest step was 391 thousandths of an inch (9.9 mm). — r- " fc - 2" - - 2" - - 2" - - 2" - j 4" I TOP VIEW 1.72 0.005 r 0.0 1 0"| L o.u 0.391 SIDE VIEW NTS Figure 24. Drawing of calibration block The material chosen for the calibration block was Acetron GP Acetal for its dimensional stability and machinability. The comments in the product profile for Acetron GP also highlighted such desirable features as low moisture absorption, high strength, stiffness, easy to machine and no centerline porosity (DSM 14). Validation of the Data The purpose of periodic calibration was to verify that the data was measured consistently by the lasers and was valid based on the calibration block and test measurements taken throughout the experiment. The validation of the data had three testing aspects. 1. Step size detection. 46 2. No change in apparatus before and after measurement. 3. Laser measurement repeatability. The laser data used in these tests was from static measurements of the calibration block designed for this purpose. The first set of tests checked the step size detected by the lasers to ensure their accuracy in measuring changes in the surface relief of the sample boards. Comparing the step measurement averages to the caliper measurements, it appeared that the lasers were offset by 0.5 mm. Likely, the lasers were not aligned perfectly perpendicular to the surface of the measuring bed, resulting in the offset. Unfortunately, the nature of the measurement apparatus made the positioning of the lasers unadjustable. The second set of tests checked that the measuring apparatus did not change during the board measurement process by testing the calibration results from before and after board sampling. The residual means of each matched before-and-after pair were tested against the expected mean of the differences of the matched pairs (Bluman 1997). Two-tailed t-tests were used to test for differences between before and after board sampling, whereby the expected mean of the differences equalled zero. One-tailed t-tests were used to test whether the difference was less than the Within Laser Tolerance of the measurement apparatus. In this instance, the residual means should be less than the expected mean of the differences which was the Within Laser Tolerance. The third set of tests checked the repeatability of each laser's measurements to ensure the lasers were measuring the data consistently. Five sets of data were collected for each of the six lasers by measuring the calibration block five consecutive times. The repeatability of the lasers was tested using paired t-tests for each laser. The difference between the sets of data for each laser should be less than the Within Laser Tolerance of the measurement apparatus. Data Preprocessing In order to train the Neural Networks to differentiate between shape defects, the data was preprocessed for a set of variables. Preprocessing is a fixed transformation of the variables and often greatly improves the performance of a pattern recognition system (Bishop 1995). It simplifies the classification task by reducing the dimensionality of the input vector, and by minimising the amount of data required in each data set (Williams 1993). The main advantage that preprocessing offers is to decrease the time to train the network, while maintaining the level of information in the training set. In this pilot project, the input variables enabled the neural network to classify board sample data into shape defect categories. These output categories were Snipe, No Snipe and Snipe 47 Combination. The raw measurement data was preprocessed using multiple linear regression. The top and bottom surfaces of the sample boards were modelled to obtain complex slope and intercept characteristics using data from all six lasers. These input values, together with the machine shape defect assessment of the sample board, for the data set were used to train, verify and test the neural networks. L a s e r Da ta In effect, each laser draws an imaginary line down the length of the sample board being measured. These lines are called laser lines in this paper. Laser lines T l , T2 and T3 describe the top surface of a sample board, while laser lines BI, B2 and B3 describe the bottom surface. There was a lag in the laser's detection of the sample board for two reasons. First, the board was conveyed down to the lasers, which were staggered in the feed direction and relative to the fence. Second, the user manually initiated the data recording function because the data acquisition software distinguished between reading and recording the laser data. The start of the board data was determined by reviewing the data manually. The sample board's leading edge was detected by finding the out-of-range values for each of the lasers in the data set. The DAQSim program displayed a reading of 4095 divisions when nothing was detected by lasers Tl , T2, T3, B2 and B3. In the case of laser BI, a reading of 0 divisions indicated that the object was out of range, as this laser was setup differently. A rapid change in the out-of-range values flagged the sample board's leading edge in the laser data. An equal number of data points, beginning from the established start of the readings, was used in the regression analysis for each of the lasers in order to simplify the physical interpretation of the analysed surface. Although the top and bottom surfaces of the sample boards were analysed separately, the laser data start points were matched up in top and bottom pairs to ensure consistency in the interpretation of the results (Figure 22). These laser pairs were Tl and BI, T2 and B2, and T3 and B3 (Figure 25 & 26). Data from samples with no snipe was generally fairly flat with an occasional jag. In contrast, data from samples with snipe was irregular and a slope was easily detected from the graphs. Note that the data from lasers T2 and B2 cross over as a function of how it was graphed. It was important to recognise that the logged laser data may not represent the entire sample board and push board due to the lag between reading the data points and logging the data points. It was for this reason that an equal number of data points, matched up between laser pairs, was analysed, instead of the entire raw data set of each laser. 48 N o S n i p e 4000 c 3000 | 2000 •| 1000 •laser T1 -laser B1 T - CM OO T— CM CO •sr IO CO 00 CO ^ IO ID N CO O) Sample Number N o S n i p e 5000 » 4000 .2 3000 ~ 2000 Q 1000 0 -laser 12 -laser B2 o o i o r ^ - C D v - c o m r ^ c n i - ( M C O i f ( O S O ! l < I ) 0 T - ( \ f ) * l O C D S O O O Sample Number N o S n i p e 4000 n 3000 (A C ; § 2 0 0 0 ~ 1 0 0 0 -laser T3 -laser B3 n i o s o i i - t o m s o i T - C M C O T j - C D r ^ C O a J O i - C N C O ' 4 - U O C O r ^ O O O Sample Number Figure 25. Graphs of logged laser data from a No Snipe board sample 49 S n i p e 5000 g 4000 .2 3000 •5 2000 5 1000 0 • laser T2 •laser B2 C M C O ^ - t O t O r ^ O O O O O O C D ^ f C M O O O C O ^ - C O Sample Number S n i p e 5000 m 4000 .2 3000 •5 2000 O 1000 0 •laser T3 •laser B3 c o m r ^ o ) T - c o m r ^ a > o o i O ' t N T - o i s i o n • > - o \ i r o - ^ - ' < t m c o r ^ Sample Number Figure 26. Graphs of logged laser data from a Snipe board sample M o d e l l i n g the 3-D S h a p e o f the B o a r d A statistical model was developed to interpret physical characteristics of a board's surface, primarily the shape, using data from the lasers. The advantage of this model was that it 50 described the three dimensional shape of the board. Three laser lines were compared on one regression surface with their slopes and intercepts. Equation 1-1 models the top surface, and Equation 1-2 models the bottom surface. Combining the results from the top and bottom surfaces defines the shape of the board, by forming a shape from the three connected laser lines of each surface. For example, connecting lines T l , T2 and T3 from the No Snipe graphs in Figure 25 results in a fairly flat top surface. However, connecting lines T l , T2 and T3 from the Snipe graphs in Figure 26 results in an indented and sloped top surface. This statistical model was also particularly important for the simplification of the data set required for input into the neural networks. Rather than using every laser data point, the regression coefficients will represent the changes in a board's surface. Yi =TU + TIS*Z + T2I*D2 +T2S*D2 * Z + T3I * D3+T3S * Z)3 *Z + e, (1-1) Where Yj = i t h observation of the laser data set T i l = intercept for laser line Tl T1S = coefficient of slope for laser line Tl T21 = intercept for laser line T2 T2S = coefficient of slope for laser line T2 T3I = intercept for laser line T3 T3S = coefficient of slope for laser line T3 Z = sample number (reset to 1 for each laser) D2 = dummy variable for laser line T2 D3 = dummy variable for laser line T3 £j = error component Yi =BU + BIS*Z + B2I*D2 +B2S*D2 *Z + B31*D} +B3S*D3 *Z + e,. (1-2) Where Yj = i t h observation of the laser data set B l l = coefficient of intercept for laser line Bl B1S = coefficient of slope for laser line B1 B2I = coefficient of intercept for laser line B2 B2S = coefficient of slope for laser line B2 B31 = coefficient of intercept for laser line B3 B3S = coefficient of slope for laser line B3 Z = sample number (reset to 1 for each laser) D2 = dummy variable for laser line B2 D3 = dummy variable for laser line B3 51 E j = error component Using the dummy variables, DI and D2, gave the possibility of describing three different laser lines in the model for the surface. The dummy variables were valued 0 or 1 depending on which laser line the data came from. Each laser line was represented by an intercept and a slope. For example, the laser line for the top surface, made by laser T l , was represented by the intercept T i l and the slope T1S. A configuration of three laser lines allowed testing for shape across the board width as well as along the length (Figure 27). This option became important for modelling wedge as well as the other defects. Figure 27. Model of regression lines on a sample board An F-test was used to test the significance of the model for a given sample surface, and t-tests were used to test the significance of the coefficients. However, insignificant variables were not eliminated from the regression model, since this research focusses on analysing the shape of the surface, rather than finding the best model to fit that shape. Therefore, for every sample board, the multiple linear regression produced six coefficients for the top surface and six for the bottom surface. These coefficients revealed the physical characteristics of the sample board by showing what the regression surface looked like. The slopes and intercepts indicated what one part of the surface was doing relative to the other. For example, if the slope coefficient B3S equalled zero, then laser line B3 was flat or horizontal. A negative value for the slope coefficient predicted a downward slope, while a positive value reflected an upward slope. 52 Therefore, one expected a negative slope coefficient for a sniped sample and a near-zero slope coefficient for a flat board sample with no snipe. Microsoft Excel?, Data Analysis Regression function was used to perform the multiple linear regression tests for the top and bottom surfaces of each sample board. The coefficient values were recorded under each respective input variable in a separate workbook that was formatted for importing into the Statistica Neural Networks software. Input & O u t p u t V a r i a b l e s In order to build a data set for training the neural networks from the laser data, the number and type of input variables required to model the 3-D shape of the board must be determined and examples of the output categories must be collected and defined. In the workbook formatted for importing into the Statistica Neural Networks software, each row contained a set of regression coefficients for each sample board (Appendix C.1). There were twelve coefficients in each set. These coefficients were, in effect, the values of the input variables for the data set. The six variables for the top surface were T i l , T1S, T2I, T2S, T3I and T3S and the six variables for the bottom surface were B1I, BIS, B2I, B2S, B3I and B3S. One half of the coefficients represented slopes and the other half represented intercepts. Input variables were named for the surface and laser line represented and for the associated intercept or slope. For example, T i l indicated the top surface by T , the laser line by '1', while T represented the intercept. These variables were numeric. The last variable in the row was SNIPE, the output variable, and represented the machine shape defect indicator value for the sample board. The three values for SNIPE were 'y', 'n' or 'c', and indicated Snipe, No Snipe or Combination Snipe respectively. Consequently, the output variable was nominal. In order to train the neural network, the value of the output variable was determined manually by visually examining the board samples and categorising them for snipe. Otherwise, the neural network performed this classification task by assigning the value. Neural Network Trials S o f t w a r e The software choice was based on the classification requirement for pattern recognition algorithms. The decision was to use Statistica Neural Networks because of the software's flexibility and its classification features. The five different types of networks that can be applied to classification problems, using this software, were multilayer perceptrons (MLP), radial basis function (RBF), Kohonen Self-Organising Feature Map networks (Kohonen), linear 53 networks (linear) and (Bayesian) probabilistic neural networks (PNN). Each of these network types represented a different pattern recognition algorithm. The Intelligent Problem Solver (IPS) feature enabled training decisions to be controlled on several levels from automatic to advanced fine-tuning. The basic version of the IPS is primarily automated; however, the user can set control parameters which determine how classification is performed, such as the number and types of networks saved in the solution set and the duration of training. In the advanced version of the IPS, the user can specify design parameters such as the classification confidence threshold, the number of hidden units and the network type. Using this version required more experience and familiarity with neural networks and the classification problem itself. Regardless of the version chosen, the software automatically interpreted nominal output variables for classification and generated statistics on overall classification performance (Statsoft 1999). In addition, it had clear and useful graphics, as well as intelligible manuals. N e u r a l N e t w o r k T r a i n i n g a n d T e s t i n g Before a neural network can be used to classify machine shape defects, it must be trained using a data set comprised of input data paired with the correct output categories (Eggers 1991). Each pair is called a sample case. Training a neural network is essentially the process of tuning a set of parameters to describe a statistical model of its data set (Swingler 1996). The outcome of this training is actually a series of different types of networks which have a variety of different characteristics and parameters. The user chooses the network most appropriate for the application based on its attributes, striking a balance between classification accuracy, training time, classification time, memory usage and fault tolerance (Cornforth 1993). Low error and high performance demonstrate good classification accuracy in a model. The main goal in pattern recognition was to develop a neural network which generalises well, so that it can successfully predict the correct output category from new data. (Bishop 1995). The simplest way to ensure that a neural network has good generalisation ability was to reserve part of the data set for verification and another part for testing. Verification is an independent check on the performance of the network during training, indicating that overlearning has occurred by an increase in the verification error. Overlearning can occur when the neural network model is overfitted to the training data, resulting in a loss of generalisation ability (Swingler 1996) Testing is a final check for bias in the network's performance results. Statistica Neural Networks splits the data set in three and randomly assigned the sample cases in a 2:1:1 ratio for training, verification and testing respectively (Statsoft 1993). If overlearning or a bias in the performance results was reported, the network can be improved by enlarging the data set, 54 by changing the type of network, or by modifying the training process. These options must be weighed with consideration to factors like cost and availability of data, time constraints for training, as well as the levels of noise, required generalisation ability and network simplicity for the particular application (Swingler 1996). N e u r a l N e t w o r k Tr ia l P r o c e d u r e In this pilot project, the major constraints governing the collection of sample pieces and the construction of the measurement apparatus were time and money. Within these limits, a significant effort was made to minimise noise in the data by selecting appropriate lumber samples and by careful measurement of these samples. The lumber samples included a random variety of machine shape defects, though snipe is the primary defect of interest, comprising 40% of the samples. Training Process: Snipe Classification Approaches & Training Strategies Two major approaches in the training process were used for the snipe classification problem: as a two-class problem or as a three-class problem (Figure 28). The input and output variables were pre-processed the same way for both approaches, but some board samples, namely the snipe combination samples, were classed differently between approaches as explained in the following paragraphs. Four training strategies were used in the process of training the neural networks for both approaches to the snipe classification problem: Base Condition, Shuffled Condition, Pruned Condition, and Extended Duration Condition. Each training strategy resulted in a series often different networks. The control parameters for the Base Condition were described subsequently, followed by summaries of the other training strategies. S n i p e C l a s s i f i c a t i o n P r o b l e m Two Class Networks Three Class Networks Base Condition Base Condition -10 networks -10 networks Shuffled Condition Shuffled Condition -10 networks -10 networks Pruned Condition Pruned Condition -10 networks -10 networks Extended Duration Condition Extended Duration Condition -10 networks -10 networks 55 Figure 28. Graphical representation of the snipe classification problem The first approach, called Two Class Networks, treated the snipe classification as a two-class problem in which the network was trained to distinguish between Snipe and No Snipe categories. The Snipe category consisted of sample boards having an edge or center piece cut out from a sudden movement in the saw blade or the wood. In the data, this cutout, called snipe, was observed as a deviation in the board's profile where the lumber thickness had been reduced. In some instances, snipe resembled wane without the bark because of the nature of the sawing process. Note that the Snipe category in the two-class problem did not include the snipe/wedge combination samples. These snipe/wedge samples were classed in the No Snipe category, which primarily consisted of sample boards which did not contain snipe. The second approach, called Three Class Networks, treated the snipe classification as a three- class problem, by adding a third category, Combination Snipe. Again, the Snipe category was comprised of all the samples which contained snipe, except the snipe/wedge samples. In the three-class problem, these snipe/wedge samples were classed in the Combination Snipe category, which generally included samples having snipe and at least one other machine shape defect. That said, in this proof of concept, only the snipe/wedge samples were included in the Combination Snipe category, due to the small numbers of the other defect combinations. There were not enough examples for each type of snipe combination, so they were classed in the Snipe category. In the three-class problem, the No Snipe category consisted of sample boards which did not contain any snipe. Therefore, these samples contained other machine shape defects or no defects at all. Identical control parameters were selected for the Two and Three Class Networks, using Statistica Neural Networks' Intelligent Problem Solver (IPS) feature to set up the training for the Base Condition networks (Table 6). The Base Condition networks resulted from selecting the basic version of the IPS, which automated as many of the training decisions as possible. The other training strategies differed by whether the data was shuffled once or twice, the input variables were pruned or the training time extended. They are described in more detail below. The Standard problem treated the sample cases in the data set as independent (Statsoft 1999). The dependent or output variable was SNIPE, while the twelve independent variables, the input variables, were Ti l , T1S, T2I, T2S, T3I, T3S, B1I, BIS, B2I, B2S, B3I and B3S. The option to search for an effective subset of specified variables was selected to allow the IPS to discard those variables deemed irrelevant for a particular network solution. 56 The default setting in the Basic version automatically determined the single threshold to minimise the misclassification rate. Because the classes overlapped, a threshold was set between the doubt and acceptance regions for each class (Swingler 1996). A value below the threshold meant the sample case was rejected for that class. Positioning the threshold was a balance between minimising the classification error and discarding good data (Swingler 1996). For example, if the threshold was split evenly between two classes, no sample cases were rejected, but the classification error was high. The amount of time for the IPS to spend designing an effective neural network for the application was specified in broad terms as 'medium'. However, the actual duration was relative to the available amount of data in the set and ranged from minutes to hours. Any number of neural networks may be saved in the set, but saving ten networks appeared to give a reasonable variety of types and complexities of networks in the trials. Thus, ten networks were saved each time training took place. To maintain diversity, the selection of networks saved balanced the performance against type and complexity. Basically, 'complexity' refers to the number of hidden units connecting the input units to the output units through transformations and 'performance' gauges the predictive accuracy of the network, while 'type' indicates the network model used for the pattern recognition, i.e. RBF versus MLP (Statsoft 1999). Table 6. Selection of control parameters for the Base Condition neural networks Design Control Parameter Choice Version of Intelligent Problem Solver Basic Problem type Standard Output variable selection SNIPE Input variables selection Ti l , T1S, T2I, T2S, T3I, T3S, B1I, BIS, B2I, B2S, B3IandB3S Duration of design process Medium Number of networks to save 10 Selection of networks to be saved Balance performance against type & complexity Action if network set too full Increase network set size Normally, newer networks replaced existing networks in the set to minimise the quantity of redundant networks. In choosing to increase the network size when the set becomes full, ten more networks were added to the set each time the network set was run for additional training, This rate of growth was acceptable for two reasons. First, the networks were relatively small because the amount of data wais limited in this proof of concept project. Second, only a few 57 variations to the Base Condition neural networks were run, which kept the number of networks in the set reasonable. These variations were in fact the three other training strategies, developed as potential improvements to the Base Condition. The same data preprocessed for the Base Condition was used to train the networks for each of these strategies: 1. Shuffled Condition: The input data was shuffled once or twice to ensure the assignments of the sample cases to the subsets was not biased by redistributing the sample cases assigned to train, verify and test the network. This strategy was applied by using the 'randomly reassign cases' option in the advanced version of the IPS. 2. Pruned Condition: The input variables were pruned to see how the baseline error is affected by dropping the input variables with low sensitivity ratios. A low sensitivity ratio indicated that the input variable was less important in the neural network (Statsoft 1999). The sensitivity analysis from a Base Condition network was used to indicate which variables to drop from the input data set. The networks were then trained without those input variables, using the basic version of the IPS. 3. Extended Duration Condition: The training duration was extended to see if increasing the time spent designing an effective network for the classification problem improved the performance of the neural networks produced. The basic version of the IPS was used to train the networks, but the duration of the design process chosen was 'Thorough' in order to conduct an extensive search. 58 Results Data Validation N o C h a n g e in A p p a r a t u s The purpose of these tests was to check that the measuring apparatus, specifically the lasers, remained constant throughout the process of measuring the board samples. The calibration measurements taken before and after this process were tested with paired t-tests, using a sample size of 900 data points and an alpha level of 0.05 (Appendix D.l). The null hypothesis in the first test, a two-tailed t-test, is that there is no difference between the initial run of calibration measurements and the follow-up run. However, the results indicate that there is a significant difference between the two runs at an alpha level of 0.05 for all six lasers (Table 7). Therefore, each of the lasers changed during the board measurement process, likely from the vibration of conveying the sample boards through the measuring apparatus. A second set of tests is required to gauge the extent of this movement. The second test, a one-tailed t-test, hypothesises that the difference between the initial run and the follow-up run of calibration measurements is less than the Within Laser Tolerance of 42.3 divisions (0.103 mm). Of the six, Laser B2 was the only laser found to have a difference between runs significantly greater. This difference is a drop in value of 296.06 divisions from the initial calibration measurements to the follow-up calibration measurements, amounting to a 0.723 mm offset. Table 7. Summary o * results for detecting a change in the measuring apparatus Laser Significant difference between runs Difference exceeds tolerance Tl Yes No T2 Yes No T3 Yes No BI Yes No B2 Yes Yes B3 Yes No L a s e r M e a s u r e m e n t R e p e a t a b i l i t y The purpose of this test was to check that the lasers are measuring the data consistently and that their performance can be repeated. Five runs of 900 data points were collected for each of the 59 six lasers by measuring the calibration block five consecutive times. The repeatability of the lasers is tested with paired t-tests, using each of these five sets (Appendix D.2). The first test, a two-tailed t-test, hypothesises that difference between runs within each laser measurement set is zero (Table 8). Since each of the lasers were found to have significant differences between at least one of their five runs, a second set of tests is required to ascertain the size of this difference. The null hypothesis in the second test, a one-tailed t-test, is that the difference between runs is less than the Within Laser Tolerance of 42.3 divisions (0.103 mm). None of the differences between runs significantly exceeded the Within Laser Tolerance of the measurement apparatus. Therefore, the repeatability of the lasers is deemed acceptable. Table 8. Summary o 'results for testing repeatability Laser Significant difference between runs Difference exceeds tolerance Tl Yes No T2 Yes No T3 Yes No B l Yes No B2 Yes No B3 Yes No Data Preprocessing The preprocessed board sample data for training the neural networks was formatted for importing into Statistica Neural Networks (Appendix C l ) . There were 103 rows representing the number of sample boards and 13 columns representing the number of variables. Each row was comprised of a set of twelve numeric input variables and one nominal output variable. The multiple linear regression analysis showed that the model was significant for every sample board measured in the data set (Appendix C.2). The F test values from the regression analysis ranged between 11.31 and 9973.1 for the top surface and between 13.89 and 3141.6 for the bottom surface (Figures 29 and 30). Just less than half of the F values fell between 300 and 600 for the bottom surface, while over half of the F values fell between 300 and 600 for the top surface. The t-test values were not analysed because the purpose was to study the shape of surface, not to develop an optimal regression model. Whether they were significant or not depended on the topography of the board's surface, not the accuracy of the model. 60 A near-zero value for the coefficient of multiple determination (R ) indicated that the surface was flat. The coefficient of multiple determination was significant when the slope of the surface was significant, since the purpose of the model was to detect changes in the surface relief: the higher the value, the steeper the slope of the surface. The values for the coefficient of multiple determination ranged between 0.0271 and 0.9334 for the top surface, and between 0.0378 and 0.8158 for the bottom surface (Figures 31 and 32). For the top surface, just over 40% of the R2 values were concentrated between 0.050 and 0.225, while just over 30% of the R2 values for the bottom surface were between 0.225 and 0.375. These higher R2 values for the bottom surface show that the sniped surface was more often face down as the sample board was fed through the measuring apparatus. This orientation of the boards does not emulate the process in a sawmill. 70 60 5" 50 S 40 §• 30 £ 20 10 TOP: F test va lues o o CO o o CD O O o o CN O O LO o o 0 0 o o OJ F values o 2 Figure 29. Top Surface: F test values by class 60 50 » 40 a 30 cr £ 20 LL 10 0 BOTTOM: F test va lues o o CO o o CD O O O o CN o o o o o o ID 0 0 T- 1 - 1 - CN F values CD \— O Figure 30. Bottom surface: F test values by class 61 TOP: R 2 va lues 30 25 g 20 S 15 cr £ 10 L L 5 0 O O L O O L O O L O O L O U O C N O r ^ - L O C N O o C N co co o o o o o o o o LO O LO N m N LO <q co cd o ci ci d ^ R2 values Figure 31. Top surface: coefficients of multiple determination by class BOTTOM: R2 va lues 20 , O O L O O L O O L O O L O O I O CP i O L n c \ j o r - ~ L O c N O i ^ L O c N !=: O T - C N I C O C O T t L n c O C D r - - : C O ° O O O O O O O O O O O ^ R2 values Figure 32. Bottom surface: coefficients of multiple determination by class Neural Network Trials Interpretation of Resu l t s These results are split into two main sections (Figure 28). The first section looks at the Two Class Networks problem where the network is trained to classify the input data into Snipe or No Snipe categories. The second section looks at the Three Class Networks problem, which has the Combination Snipe category in addition to the Snipe and No Snipe output categories (p.56). Each section shows the results from the four training strategies, which are the Base Condition, the Shuffled Condition, Pruned Condition, and Extended Duration Condition (p.58). For each Condition, a series of ten networks are summarised in a table and the highlights are mentioned. Each network is represented by a set of characteristics reported from Statistica Neural Networks' Basic and Verbose Network Set Datasheets (Table 9 - 17). This set of characteristics 6 2 is split in order to distinguish the characteristics that describe the neural network from those that evaluate the neural network. The characteristics are described briefly below in terms of their meaning and interpretation and discussed at length in the Neural Network Trials of the Discussion (p.71). Description of Neural Networks Network: Each network was assigned a name for clarity and convenience. The network nomenclature describes the position of the network in the set, the training strategy, and the number of classes. Specifically, the first letter and number show where the network is situated in the network set. The next two letters indicate the training strategy: Base Condition (BC), Shuffled Condition (SC), Pruned Condition (PC) or Extended Duration Condition (EC). The last number and letter indicate whether the network solves a two or three class problem. Using N1BC2C for an example, 'NT signifies that it is the first network saved in the set; 'BC stands for Base Condition; and '2C means that it solves a two-class network problem. Type: The three types of networks reported in the results for this snipe classification problem were multilayer perceptrons (MLP), radial basis function (RBF) and linear networks (Linear). Each type of network has a different network architecture, or pattern recognition algorithm, with its own set of merits. Inputs: The number of input variables used by each neural network was reported in the results. These inputs were chosen from the twelve independent variables that model the top and bottom surfaces of the sample boards. Hidden: The number of hidden units was reported to describe the complexity of the network. Essentially, hidden units connect the input units to the output units using transformations. These transformations are designed to optimise the decisions and to minimise error (Bishop 1995). Evaluation of Neural Networks TrError, VeError, & TeError: Statistica Neural Networks reported the training error (TrError), the verification error (VeError) and the testing error (TeError). Each error was the root mean square error (RMS) summarised over its subset. The verification error was obtained from the Basic Datasheet, while the test and training errors were obtained from the Verbose Datasheet. Of these three errors, the verification error was the most significant, since it gave the best indication of the network's ability to make predictions on 63 new data (Statsoft 1999). The test error was used for a final check of the network performance and also required consideration because the two errors were helpful in diagnosing training problems (Statsoft 1999). For example, if the verification error and the test error had similar values, then overfitting had likely not occurred. In Statistica Neural Networks, the training algorithms searched for network solutions that minimised the training error during the training process, so it was not a concern if the training error was much lower than the other errors. Therefore, the training error was not reported in the results. At this early stage of applying neural networks to this snipe classification problem, the best possible error rate was not known. Comparison of these results to similar classification problems was difficult and likely misleading because they were highly dependent on a particular problem. Performance: The performance reported in the results was the verification performance, which represented the proportion of correctly classified sample cases in the verification subset of the data set and was referred to as the performance in this report. For example, a performance of 0.692 meant that 69.2% of the sample cases were correctly classified in the verification subset. Also known as the correct classification rate, it was the best indicator of whether a neural network was suited to the classification problem or not. However, the performance should not be interpreted alone, since other parameters also play an important role in assessing the capabilities of a neural network. At this early stage of applying neural networks to snipe classification, it was difficult to know what the best possible performance rate was for this problem. Although Statistica Neural Networks' Intelligent Problem Solver described verification performance values of 0.731, 0.692 and 0.654 as 'ok performance' on a scale ranging from extremely good to extremely poor, its comments were treated with caution as the achievable level of accuracy depends on the problem (Statsoft 1999). Like the other characteristics, performance of the neural networks is further explored in the Discussion (p. 77). Two Class Network Problem: No Snipe & Snipe Base Condition - Two Class Networks 64 Description: Ten neural networks were saved from the network set trained using the data preprocessed for two output categories (Table 9). Three of the networks were linear, and three were RBF, while the remaining four were MLP networks. Evaluation: N10BC2C had the lowest error at 0.473 and the highest performance at 0.692 for two classes. N6BC2C also had a performance of 0.692, but its error was higher at 0.501. Table 9. Base Condition - Two Class Networks Network Type VeError TeError Inputs Hidden Performance N1BC2C Linear 0.512 0.353 8 - 0.500 N2BC2C RBF 0.511 0.449 1 2 0.385 N3BC2C Linear 0.508 0.358 9 - 0.577 N4BC2C Linear 0.508 0.361 10 - 0.538 N5BC2C RBF 0.504 0.435 1 1 0.308 N6BC2C RBF 0.501 0.437 1 1 0.692 N7BC2C MLP 0.494 0.466 1 1 0.615 N8BC2C MLP 0.484 0.526 12 7 0.654 N9BC2C MLP 0.480 0.424 10 6 0.615 N10BC2C MLP 0.473 0.461 12 7 0.692 Shuffled Condition - Two Class Networks Description: The same data preprocessed for two output categories was used to train these networks as for the Base Condition. However, the sample cases were redistributed, or shuffled, between the training, verification and testing subsets (Table 10). As with the Base Condition, ten networks for the Shuffled Condition were saved from the network set. Four of the networks were linear, three were RBF and three were MLP. Evaluation: The network with the lowest error at 0.383 was N10SC2C, an MLP with a performance of 0.731. However, several linear networks had the highest performance at 0.808 with six to eight input variables and no hidden units. These performance results were unusually high as compared to the Base Condition networks (Table 9). In addition, the large discrepancy between the verification and test errors indicated that the networks may not be very reliable. Therefore, a second round of shuffling was required to double-check that the distribution of the sample cases was not biased to achieve high performance ratings by fluke. Description: Ten networks for the second round of shuffling were saved from the network set (Table 11). Three of the networks were linear, three were RBF and four were MLP. Evaluation: N20SC2C was the network with the lowest error at 0.464 . It was one of two networks with the highest performance at 0.692. The other network, NI 1SC2C, had the same performance, but its error was higher at 0.543. 65 Table 10. Shuffled Condition - Two Class Networks Network Type VeError TeError Inputs Hidden Performance N1SC2C MLP 0.475 0.501 1 1 0.615 N2SC2C RBF 0.474 0.521 12 6 0.615 N3SC2C RBF 0.438 0.556 12 7 0.692 N4SC2C RBF 0.437 0.534 12 6 0.654 N5SC2C Linear 0.423 0.596 5 - 0.692 N6SC2C MLP 0.416 0.526 12 7 0.731 N7SC2C Linear 0.404 0.575 8 0.808 N8SC2C Linear 0.397 0.572 7 - 0.808 N9SC2C Linear 0.397 0.573 6 0.808 N10SC2C MLP 0.383 0.562 12 9 0.731 Table 11. Shuffled Condition Second Round- Two Class Networks Network Type VeError TeError Inputs Hidden Performance N11SC2C RBF 0.543 0.631 10 6 0.692 N12SC2C RBF 0.514 0.663 10 9 0.577 N13SC2C RBF 0.492 0.538 10 10 0.654 N14SC2C Linear 0.485 0.455 9 - 0.615 N15SC2C Linear 0.483 0.455 8 0.577 N16SC2C MLP 0.475 0.493 1 1 0.423 N17SC2C MLP 0.475 0.492 11 8 0.500 N18SC2C Linear 0.473 0.461 10 . 0.615 N19SC2C MLP 0.469 0.498 12 6 0.577 N20SC2C MLP 0.464 0.576 12 6 0.692 Pruned Condition - Two Class Networks Description: The Base Condition network with the lowest error and highest performance, N10BC2C, was used to determine which input variables to drop. The sensitivity analysis indicated that T i l , T2S, BIS and B3S had low sensitivity ratios, with a threshold of 1.05, showing that these four input variables were the least important to the neural network. Therefore, they were dropped from the input data set and the networks were trained again with the remaining eight input variables using the data preprocessed for two output categories. The result was that an additional ten networks were saved in the network set (Table 12). Five of the networks were MLP, while four were RBF and only one was linear. Although each of the networks had eight input variables, the number of hidden units varied. 66 Evaluation: N10PC2C had the lowest error at 0.472, but the second highest performance at 0.615. Three networks had the highest performance at 0.654. They were N3PC2C, N8PC2C and N9PC2C, of which N9PC2C had the lowest error at 0.476. Table 12. Pruned Condition - Two Class Networks Network Type VeError TeError Inputs Hidden Performance N1PC2C RBF 0.507 0.434 8 1 0.462 N2PC2C RBF 0.504 0.436 8 2 0.538 N3PC2C Linear 0.498 0.400 8 . 0.654 N4PC2C RBF 0.497 0.422 8 4 0.577 N5PC2C MLP 0.487 0.393 8 8 0.577 N6PC2C RBF 0.485 0.440 8 8 0.615 N7PC2C MLP 0.484 0.421 8 8 0.615 N8PC2C MLP 0.479 0.403 8 6 0.654 N9PC2C MLP 0.476 0.394 8 6 0.654 N10PC2C MLP 0.472 0.416 8 8 0.615 Extended Duration Condition - Two Class Networks Description: The same data preprocessed for the Base Condition was used to train these networks, but the time spent designing an effective network for the classification problem was increased. Ten networks for the Extended Duration Condition were saved from the network set (Table 13). The first six networks were alternating RBF and linear networks, while the remaining four were MLP networks. Evaluation: N10EC2C had the lowest error at 0.462, and second highest performance at 0.615. The highest performance at 0.692 was held by N5EC2C, with an error of 0.501. Table 13. Extended Duration Condition - Two Class Networks Network Type VeError TeError Inputs Hidden Performance N1EC2C RBF 0.510 0.429 4 1 0.577 N2EC2C Linear 0.504 0.431 3 0.462 N3EC2C RBF 0.504 0.435 1 1 0.308 N4EC2C Linear 0.504 0.427 1 0.538 N5EC2C RBF 0.501 0.437 1 1 0.692 N6EC2C Linear 0.501 0.428 2 0.577 N7EC2C MLP 0.493 0.468 1 1 0.538 N8EC2C MLP 0.487 0.491 3 1 0.577 N9EC2C MLP 0.474 0.435 3 3 0.577 N10EC2C MLP 0.462 0.412 5 4 0.615 67 T h r e e C l a s s N e t w o r k P r o b l e m : N o S n i p e , S n i p e & C o m b i n a t i o n S n i p e Base Condition - Three Class Networks Description: Ten neural networks were saved from the network set trained using the data preprocessed for three output categories (Table 14). The first three networks were linear, followed by an RBF network and four consecutive MLP networks. Two more RBF networks completed the set. Evaluation: N10BC3C, an RBF network with 4 input variables and 1 hidden unit, had the lowest error at 0.411 and the highest performance at 0.654 for three classes. Table 14. Base Condition - Three Class Networks Network Type VeError TeError Inputs Hidden Performance N1BC3C Linear 0.468 0.447 5 - 0.577 N2BC3C Linear 0.464 0.444 6 0.577 N3BC3C Linear 0.457 0.447 7 - 0.577 N4BC3C RBF 0.426 0.421 4 1 0.615 N5BC3C MLP 0.426 0.421 1 1 0.615 N6BC3C MLP 0.425 0.424 12 8 0.615 N7BC3C MLP 0.422 0.421 12 8 0.615 N8BC3C MLP 0.422 0.422 12 6 0.615 N9BC3C RBF 0.414 0.413 4 2 0.615 N10BC3C RBF 0.411 0.414 4 1 0.654 Shuffled Condition - Three Class Networks Description: The same data was used to train the ten saved networks for the Shuffled Condition as for the Base Condition (Table 15). However, the sample cases were redistributed between the training, verification and testing subsets. Three of the resulting networks were linear, three were MLP, while four were RBF networks. Evaluation: Networks N6SC3C, N9SC3C and N10SC3C had the highest performance at 0.615, but N10SC3C had the lowest error at 0.452. Table 15. Shuffled Condition - Three Class Networks Network Type VeError TeError Inputs Hidden Performance N1SC3C Linear 0.471 0.380 5 0.538 N2SC3C Linear 0.470 0.373 7 - 0.538 68 N3SC3C MLP 0.470 0.426 1 1 0.423 N4SC3C MLP 0.467 0.419 11 2 0.385 N5SC3C Linear 0.465 0.373 6 - 0.538 N6SC3C RBF 0.462 0.509 10 28 0.615 N7SC3C RBF 0.459 0.482 10 9 0.538 N8SC3C MLP 0.455 0.419 12 8 0.577 N9SC3C RBF 0.454 0.461 10 16 0.615 N10SC3C RBF 0.452 0.468 10 18 0.615 Pruned Condition - Three Class Networks Description: The Base Condition network with the lowest error and highest performance for three classes, N10BC3C, was used to determine which input variables to prune. As the sensitivity analysis showed that T3I, T2S and T3S had low sensitivity ratios for a threshold of 1.05, they were dropped, leaving T2I as the remaining input variable. The networks were trained again using the data preprocessed for three output categories with one input variable selected. The resulting additional ten networks were saved in the network set (Table 16). Five of the networks were MLP, while four were RBF and only one was linear. Although each of the networks had one input variable, the number of hidden units varied. Evaluation: N10PC3C had the lowest error at 0.412, but shared the highest performance at 0.654 with four other networks. These other networks were N4PC3C, N6PC3C, N8PC3C and N9PC3C. Table 16. Pruned Condition - Three Class Networks Network Type VeError TeError Inputs Hidden Performance N1PC3C RBF 0.425 0.432 1 4 0.615 N2PC3C Linear 0.425 0.422 1 - 0.615 N3PC3C MLP 0.423 0.420 1 8 0.615 N4PC3C MLP 0.421 0.419 1 20 0.654 N5PC3C MLP 0.421 0.419 1 13 0.615 N6PC3C RBF 0.419 0.411 1 1 0.654 N7PC3C RBF 0.418 0.411 1 2 0.615 N8PC3C MLP 0.418 0.417 1 9 0.654 N9PC3C MLP 0.417 0.419 1 13 0.654 N10PC3C RBF 0.412 0.436 1 3 0.654 Extended Duration Condition- Three Class Networks 69 Description: The same data preprocessed for the Base Condition was also used to train these networks, but the time spent designing an effective network for the classification problem was increased. Ten networks for the Extended Duration Condition were saved in the network set (Table 17). Three of the networks were linear, three were RBF networks, and four were MLP networks. Evaluation: N10EC3C had the lowest error at 0.400, but the second highest performance at 0.654. N9EC3C obtained the highest performance at 0.692 with an error of 0.411. Table 17. Extended Duration Condition - Three Class Networks Network Type VeError TeError Inputs Hidden Performance N1EC3C Linear 0.457 0.447 7 - 0.577 N2EC3C RBF 0.426 0.421 4 1 0.615 N3EC3C Linear 0.425 0.418 2 - 0.615 N4EC3C Linear 0.425 0.422 1 - 0.615 N5EC3C MLP 0.424 0.420 5 3 0.615 N6EC3C MLP 0.415 0.404 8 8 0.615 N7EC3C RBF 0.414 0.413 4 2 0.615 N8EC3C RBF 0.411 0.414 4 1 0.654 N9EC3C MLP 0.411 0.401 11 8 0.692 N10EC3C MLP 0.400 0.423 12 13 0.654 70 Discussion Data Validation It is important to validate the reliability of the laser data, as the neural network models are only as useful as the data employed to train the networks. To use a hackneyed maxim: garbage in = garbage out. As discussed previously, three major aspects of the data validation were tested. The first set of tests showed the lasers were offset by 0.5 mm, likely due to the difficulty aligning them in the unadjustable measuring apparatus. The second set of tests showed whether the lasers and their apparatus remained constant throughout the measuring process. The results for laser B2 indicated a difference between runs significantly greater than the Within Laser Tolerance of the apparatus, signalling a problem with laser B2 during the sample measurement process. Since the shift appeared to be in one direction, laser B2 was likely bumped upward toward the surface of the measuring bed, causing an offset of approximately 0.723 mm. If an offset was the problem, then the surface analysis was likely not seriously affected, since the measurements were on a relative scale. The third set of tests demonstrated that the sets of data measured by the lasers were repeatable within the tolerance limits. Therefore, no problem was found in the consistency of each of the laser's measurements from one set of data to the next. For this proof of concept, the data was shown to be reliable enough to prove whether or not the neural networks application to detecting machine shape defects was viable. It was recognised that the data was not perfect, which in this instance was viewed as an advantage. If such a system is demonstrated to be successful with imperfect data, it is very likely to work well in a sawmill environment where data collection has inherent problems. The vibration, interference from other equipment, dust and sometimes rough treatment in a sawmill often makes for inaccuracies in data gathering. Therefore, a forgiving and robust system for detecting machine shape defects is ideal. Neural Network Trials The ideal neural network had low error and high performance, which reflected good classification accuracy, as well as quick classification time, minimal memory usage and fault tolerance (Cornforth 1993). Many of the networks in the results had these advantages to some extent, as those saved in the network set represented a diverse variety of networks. The difficulty was finding a balance between a model's accuracy and its ability to generalise well since having both qualities was not usually possible with commercial applications (Swingler 1996). A simpler model with a smoother curve through the training data generalised well, but 71 missed a few points, thereby reducing the accuracy. Though a larger network more accurately modelled a more convoluted and complex underlying function, the trade-off was that it was more difficult to train, slower to operate and more prone to overfitting (Statsoft 1999). Checking that the verification error and the test error were similar provided some assurance that overfitting had not occurred. Even so, the simplest model was often the best choice. The merits of the neural networks will be discussed in terms of type, complexity, error and performance, bearing in mind that type and complexity are characteristics used to describe the networks, while error and performance are characteristics used to evaluate the networks. D e s c r i p t i o n o f N e u r a l N e t w o r k s Type of Network Linear networks (linear), multilayer perceptrons (MLP) and radial basis function (RBF) were the three types of networks reported in the training results for this classification problem. The type of network refers to the pattern recognition algorithm or network architecture, of which examples are shown in the illustrations (Figures 33, 34 & 35). Input layer Figure 33. Illustration of N9SC2C, linear network with 6 input variables and 0 hidden units 72 Output layer Hidden layer Input layer Figure 34. Illustration of N10BC2C, MLP network with 12 input variables and 7 hidden units Figure 35. Illustration of N10BC3C, RBF network with 4 input variables and 1 hidden unit The RBF and MLP networks appeared to be the most suited to this application of neural networks, having the lower error and higher performance results. For example, in the Base Condition - Two Class Networks, the highest performance and lowest error combination was achieved by an MLP network, N10BC2C, while in the Base Condition - Three Class Networks, this combination was attained by an RBF network, N10BC3C. In fact, MLP networks provided Hidden layer Output layer Input layer 73 the lowest error solutions for the two-class problems, while RBF networks provided the lowest error solutions for all but the Extended Duration Condition in the three-class problems. This finding was not surprising, since these types of networks were often employed to model non- linear problems like classification which cannot normally be solved simply by drawing a hyperplane through the data (Statsoft 1999; 2000). Nevertheless, a linear model provides a good benchmark against which to judge more complex networks and sometimes linear techniques can solve a problem that appears difficult and non-linear (Statsoft 1999). MLP and RBF networks are compared below. • MLP networks are complex with many layers, complicated connections and a variety of different activation functions, while RBF networks tend to be simple, having two layers - one to contain the parameters and one to generate the outputs (Bishop 1995). • MLP can be very slow to converge in the training process because it depends on many hidden units to determine the value of the output unit and can get hung up on local minima (Bishop 1995). By contrast the RBF is faster since few hidden units have significant activations, making for quicker decisions from fewer possibilities. An activation is the value displayed by each unit, signifying its influence on subsequent units in the network (Swingler 1996). • MLP networks use supervised training to determine all of the parameters simultaneously, whereas an RBF network uses unsupervised training to determine the basis functions, followed by supervised training to find the weights of the hidden layer (Bishop 1995). • MLP networks can make unjustified extrapolations with new input data unlike any data encountered during training, but an RBF network will always have a near-zero response to data from outside the normal range. Complexity The main consideration with regard to complexity is whether to build a robust model which generalises well or a brittle model which is more accurate (Swingler 1996). The number of input variables and hidden units generally describes the complexity of a network. The higher the number of hidden units, the more complex the network and the more powerful its model. A more complex model fits more data points in the training set, but is less resistant to modelling the effects of noise or data idiosyncrasies, resulting in overfitting. Though it can model a more convoluted and complex underlying function, the trade-off is that this larger network is more difficult to train, slower to operate and more prone to overfitting (Statsoft 1999). A simpler model with a smoother curve through the training data generalises well, but misses a few points, 74 reducing the accuracy. The difficulty is finding a balance between a model's accuracy and its ability to generalise well since having both qualities is not usually possible with commercial applications (Swingler 1996). Often, the simplest model is the best choice. Complexity is also influenced by the type of neural network. MLP networks are more complex than RBF networks, while linear networks tend to be the most structurally simple (Swingler 1996). In this classification application, the least complex networks with good performance and low error were RBF networks. For the Three Class Networks, they were N10BC3C with four input variables and one hidden unit, N10PC3C with one input and three hidden, and N8EC3C with four inputs and one hidden. For the Two Class Networks, they were N5EC2C with one input and one hidden and N6BC2C, also with one input variable and one hidden unit. E v a l u a t i o n o f N e u r a l N e t w o r k s Error: Training, Verification & Testing The three errors reported by Statistica Neural Networks were the training error (TrError), the verification error (VeError) and the testing error (TeError). These errors were used to gauge how well a neural network performed during iterative training and execution (Statsoft 2000). Each one was the root mean square (RMS) of the individual sample case errors, summarised over the subset (Statsoft 2000). The network's error function was used for each sample case and varied depending on the type of network. Sum-squared was the standard error function applied in neural networks training. It is the sum of the squared difference between the target and the actual output values on each output unit for the subset (Statsoft 2000). For nominal variables like SNIPE in this application, Statistica Neural Networks prepared the values for input into the neural networks, and then interpreted them for the network output. Therefore, the actual, target and error output values were reported in nominal form, while the individual RMS errors were reported numerically (Table 18). Table 18. Sample excerpt of the output values for network N10EC3C S A M P L E C A S E Actual SNIPE Target SNIPE Error SNIPE RMS Error 1 n n Right 0.340923 2 y y Right 0.199405 3 n n Right 0.24407 4 n n Right 0.159318 5 n n Right 0.192806 6 y y Right 0.0548 7 n n Right 0.255378 75 8 y y Right 0.40217 9 n n Right 0.320534 10 n n Right 0.221628 11 y c Wrong 0.533387 12 y y Right 0.179526 13 n n Right 0.379088 14 y n Wrong 0.452284 15 n y Wrong 0.438431 The verification error was monitored during training to detect overlearning by a rise in its value. Overlearning was a problem for networks because it meant the solution was likely not general enough to make predictions with new data. Another way to check for overlearning was to compare the verification error with the test error. These two errors should be about the same value in order to be confident that overfitting had not occurred and that the network can generalise reliably (Statsoft 1999). A significant difference between the errors indicated that there were too few sample cases for the network performance results to be reliable or that the distribution of sample cases was biased. Troubleshooting a possible bias was done by reshuffling the distribution of the test, verification and training sample cases to see if the verification and test errors converged in the new results. To illustrate, N10BC2C and N9SC2C are drawn for their high performance results from two different network data sets for comparison (Tables 9 & 10). The first, an MLP network, is from Base Condition - Two Classes. Its test and verification errors were 0.461 and 0.473 respectively. The similarity indicates that the network will generalise well. The second, from Shuffled Condition - Two Classes, is a linear network. It had a test error of 0.573 and a verification error of 0.397. This large discrepancy between the test and verification error values signals overlearning and therefore poor generalisation ability. Shuffling again yields a more reliable linear network, N18SC2C, whose test and verification errors had converged to 0.461 and 0.473 respectively. Another network, N20SC2C, had the higher performance, but the difference between its test and verification errors was greater, and therefore, it was a less reliable network. Taking reliability into account, the Three Class Networks with the lowest error were N10EC3C at 0.400, N9EC3C and N10BC3C, both at 0.411. The Two Class Networks with the lowest error were N10PC2C and N10EC2C, at 0.472 and 0.462 respectively. All of these networks were MLP, except for N10BC3C, which was an RBF network. 76 Performance The performance reported in the results accounted for the proportion of sample cases classified correctly by a trained neural network in the verification subset of the data set (Table 9 - 17). Known as the correct classification rate, it is an important indicator of the suitability of a network for the classification application. Having said that, a classification rate considered good for one application may not always be considered a good rate in another application. With the same training data set, the performance should fall within a certain range from one network set to the next. Therefore, unusual performance values in a network set may flag a problem with the reliability of the network results. Troubleshooting the reported errors and/or sample case distributions may help locate the source of the performance problem. This technique was used to assess the results for the Shuffled Condition - Two Class Networks whose networks appeared to have unusually high performances (Table 10). An example is N9SC2C, a linear network with a performance of 0.808. Comparing its verification and test errors of 0.397 and 0.573 revealed a large discrepancy. Since this large difference signalled overlearning as discussed in the previous section, the sample cases were re-distributed to produce the Shuffled Condition Second Round (Table 11). As the second round performance results were in the range of results produced by the other training strategies, the high performance results in the first round were likely a fluke. Therefore, the networks for the Shuffled Condition - Two Class Networks (first round) were not reliable and should not be used in future applications. The networks with the best performances were N10BC2C, N6BC2C, N5EC2C and N9EC3C at 0.692. A dozen networks hac the second best performance of 0.654, with almost half of them in Pruned Condition - Three Class Networks, which had errors ranging from 0.412 to 0.421. C l a s s i f i c a t i o n A n a l y s i s o f S a m p l e C a s e s The overall proportion of correctly classified sample cases was calculated for the top ten neural networks (Table 19). A sample of this calculation is shown in Appendix E. Forty percent of the 103 sample cases contained snipe or a combination of snipe and another defect. These networks were chosen in difference to the discussion regarding type, complexity, error and performance in the previous sections. Five networks were from the Two Class Networks and five from the Three Class Networks. In the Two Class Networks, the networks were trained to classify the input data into Snipe or No Snipe categories, while in the Three Class Networks, they were trained to classify the input data into Snipe, No Snipe or Combination Snipe categories. The performance reported was the verification performance. The discrepancy between the performance and the overall proportion was due to the random distribution of sample cases 77 between the training, verification, and testing subsets of data, resulting in uneven numbers of correctly classified sample cases in the subsets. Again, a high proportion of correctly classified sample cases was desirable, indicating a low number of misclassified sample cases. Generally, a high number of falsely classified sample cases demonstrated poor predictive accuracy, signalling the network's inability to recognise a pattern in the training data set. Table 19. Classification data for top ten neural networks Correctly classif ied sample cases Misclassif ied sample cases Network Overall Performance Type I Type II SnipeCombo N10BC3C 0.612 0.654 1 32 7 N10PC3C 0.612 0.654 1 32 7 N10EC3C 0.718 0.654 8 14 7 N9EC3C 0.718 0.692 2 19 7 N8EC3C 0.612 0.654 0 33 7 N10BC2C 0.748 0.692 17 9 n/a N6BC2C 0.485 0.692 32 21 n/a N10PC2C 0.689 0.615 18 14 n/a N10EC2C 0.689 0.615 21 11 n/a N5EC2C 0.466 0.692 36 19 n/a There were two major ways a sample case can be falsely classified, or misclassified, in this snipe classification problem (Table 20). The first, known as Type I, occurred when a neural network classified a sample case as snipe when it was not. In a sawmill, this type of misclassification would likely result in reduced recovery. The second, known as Type II, occurred when a neural network did not classify a sample case as snipe when it was. This misclassification would result in downgraded lumber, since it would likely not be caught until the next grading chain. A third misclassification category, called SnipeCombo, was used to track the errors for snipe combination sample cases in the three-class problem (Table 20). Misclassification by neural networks occurred for several reasons: the type of training data could be unsuited to the application; the input data could contain too much noise; or not enough training data was available in each of the categories to train the network properly (Swingler 1996). 78 Table 20. Types of Misclassification Actual SNIPE Target SNIPE Misclassification y n Type 1 n y Type II n c SnipeCombo n c SnipeCombo Two neural networks, N6BC2C and N5EC2C, were listed among those with the best performance rates in the Two Class Networks. However, the overall proportion of correctly classified sample cases was below 0.500 for each of them, so they were no longer considered reliable. This difference highlighted the importance of considering a combination of characteristics before selecting neural networks to be employed in the classification application. In the Two Class Networks, the remaining top three networks had a greater number of Type I versus Type 11 misclassified sample cases (Table 19). The snipe/wedge combination sample cases were manually classed as No Snipe (Target SNIPE = n) in the preprocessing stage. Despite their containing snipe, the neural networks correctly classified the majority of these sample cases, meaning they were not recognised as having snipe (Actual SNIPE = n). However, the other snipe combination sample cases were manually classed as Snipe (Target SNIPE = y) in the preprocessing stage. The neural networks also did not recognise these sample cases as having snipe (Actual SNIPE = n), resulting in a Type II misclassification. This consistency demonstrated the neural networks' inability to recognise snipe combination sample cases as containing snipe. These snipe combination sample cases may confuse the training of the Two Class Networks by introducing too much noise in the data, resulting in both Type I and II misclassifications. Using a larger number of'pure' snipe sample cases and separating the combination sample cases from them would alleviate this difficulty. This option was not available for this project, given the limitation on acquiring lumber samples. In the Three Class Networks, the top five networks had a much larger proportion of Type II misclassified sample cases, as well as a consistent number of errors in the SnipeCombo misclassification category (Table 19). Seven was the number of sample cases with snipe/wedge manually classified as Combination Snipe (Target SNIPE = c) during preprocessing. The neural networks consistently failed to recognise all seven sample cases as Combination Snipe (Actual SNIPE = n). Clearly, there were not enough examples of this defect combination to train the neural networks for the third category. Although many other sample cases were snipe combination boards containing snipe plus one or more defects, they were manually classed as 79 Snipe (Target SNIPE = y) in the preprocessing stage, instead of Combination Snipe (Target SNIPE = c). The neural network classified them as No Snipe (Actual SNIPE = n), resulting in a large proportion of type II error. This result indicated that the neural network failed to recognise these sample cases of snipe combined with another defect, as containing snipe, let alone as Combination Snipe (Actual SNIPE = c). Likely, the additional defect(s) caused too much noise in the data for the snipe to be recognisable. Increasing the number of sample cases with no defects would help alleviate the confusion with No Snipe by reducing the noise in the data. Additional examples of each of these snipe combinations would also improve the training accuracy. Aside from N6BC2C and N5EC2C, the overall proportion of correctly classified sample cases was quite satisfactory, ranging from 0.612 to 0.748. This accuracy is not adequate for a stand- alone application of neural networks classification in a sawmill; however, it is enough to show that with improvements to the training process, this approach is viable. Furthermore, the remaining neural network models fared better than the base prediction rate, since the percentage of Type I is less than 60%. Finally, the total number of sample boards was considered adequate to prove this concept of training neural networks to differentiate between a board with snipe and one without snipe. 80 Conclusions This proof of concept demonstrated that neural networks can be applied with limited success to detect machine shape defects, in particular snipe, in random samples of rough green lumber. More work and resources are required to perfect the detection of machine shape defect combinations. However, the robustness of this approach using neural networks is appealing for the type of data-gathering environment encountered in the wood products industry where rugged forgiving instruments and systems are necessary. The ideal neural network has low error and high performance, which reflects good classification accuracy, as well as quick classification time, minimal memory usage and fault tolerance (Cornforth 1993). The difficulty is finding a balance between a model's accuracy and its ability to generalise well since having both qualities is not usually possible with commercial applications (Swingler 1996). A simpler model with a smoother curve through the training data generalises well, but misses a few points, reducing the accuracy. A larger network can more accurately model a more complex underlying function, but it is more difficult to train, slower to operate and more prone to overfitting (Statsoft 1999). Checking that the verification error and the test error are similar provides some assurance that overfitting has not occurred. Even so, the simplest model is often the best choice. Of the three types of networks reported in the training results, RBF and MLP networks were the most suited to this application of neural networks, having the lower error and higher performance results (Tables 9 - 17). That said, linear networks were still considered as they may provide a simpler network solution and they make good benchmarks for comparison. N10BC2C, N9EC3C and N10BC3C were considered the three best networks suited to this snipe classification problem. The first one was a Two Class Network, while the last two were Three Class Networks. N10BC2C was an MLP network with low error and one of the best performances. N10BC3C, an RBF network, was a simpler model than N9EC3C, but N9EC3C, an MLP network, had a higher performance. None of these networks appeared to be prone to overlearning. While the neural networks were able to discern between sample cases with snipe and those without snipe, they were unable to recognise the combination snipe sample cases as anything but No Snipe (Actual SNIPE = n). This problem may be overcome by training the neural networks with many more sample cases of boards containing the snipe defect and by classifying the 'pure' snipe sample cases separately. Increasing the number of board samples containing no defects would also improve the training accuracy of the neural networks. Acquiring and processing this amount of data required many more resources than were available at the time of this project. 81 Future Applications Future applications would classify all six of the common machine shape defects produced in sawmills. The neural networks would differentiate between the machine shape defects and their combinations. As this future application is essentially an extension of this project, it is clear that the increase of output categories will require a magnitude of additional raw data to train the networks. An automated measuring system and tailored programming for the preprocessing step is recommended for such a large volume of data. Specifically, the measuring apparatus should be adjustable, in order to finetune the alignment and accuracy of the lasers. It is also suggested that the training data be obtained from a wide base of sawmills in order to cover the different variations of machine shape defects experienced by those mills surveyed in Part I of this thesis. In future applications, it is recommended to consider modelling the top and bottom surfaces of the board separately for input into the neural networks, instead of modelling the 3-D shape of the board. This strategy would reduce the complexity of the pattern recognition problem by reducing the number of input variables. An added benefit is that the number of sample cases per board would be doubled by using a data set for each surface instead of one for each 3-D shape. Further development of this system would be a troubleshooting guide for analysing various process problems based on machine shape defects and their causes. Ultimately, an automatic or expert system could be built, incorporating neural networks to classify the sample boards by machine center or by defect causes. Factors like defect characteristics and their locations on the board are variables useful for training the neural networks to recognise the machine centre causing the problem. The machine shape defect would be detected and then classified by machine source, triggering, futuristically speaking, a self-diagnostic sequence whereby the machine centre would adjust its settings or notify maintenance. 82 SUMMARY The first part of this thesis determined the machine shape defects that were most important to the BC sawmill industry, while the second part dealt with finding a way to detect and classify these defects. Consequently, the tools used to research each part of the problem differed. The first phase employed a province-wide survey to collect data and statistical probability methods for the analysis, while the second phase employed laser-based measurement to collect data and explored the use of neural networks for the classification analysis. Both of these methods produced results which prove interesting for future endeavours toward improving the production of lumber, both immediate and longterm. The major findings and results are summarised in this section with respect to the main objectives of the thesis. The main objective of the survey of the sawmills in British Columbia was to determine the industrial significance of each of the six major machine shape defects. These defects are wedge, flare, taper, snipe, thin snake and fat snake. The survey results not only established that snipe, taper and thin snake are the top three machine shape defects having the greatest impact on the BC industry, but also identified the most common causes of these and the other machine shape defects. The analysis showed that to reduce the production of machine shape defects, sawmills need to focus on improving three critical areas: Piece Stability, Saw Condition, and Alignment. The main objective of the neural network classification was to establish that neural networks can detect machine shape defects found in rough green lumber, using the snipe defect for the proof of concept experiment. Laser displacement sensors measured the surface of the sample boards, three along the top and three along the bottom. A statistical model was developed to interpret the shape characteristics of the top and bottom surfaces, producing a set of regression coefficients for each sample board. This preprocessing step simplified the training task by reducing the number of input variables to the neural networks. The classification results demonstrated that it is feasible to use neural networks to detect and classify machine shape defects found in rough green lumber. However, it was evident that these preliminary neural networks were unable to recognise sample cases with combination snipe. This type of misclassification is attributed to the lack of data representing snipe combined with other defects. Much more data is required to train and develop neural networks to perform this machine shape defect classification task with a better degree of reliability. Future research would expand the classification to all six machine shape defects commonly found in British Columbia sawmills. It would address the misclassification of the combination snipe, enabling the neural networks to differentiate between the defects and their combinations. 83 BIBLIOGRAPHY Bishop, C M . Neural Networks for Pattern Recognition. New York: Oxford, 1995. Bluman, A.G. Elementary Statistics: A Step by Step Approach. 3rd ed. San Francisco: McGraw- Hill, 1997. Brown, T.D. Quality Control in Lumber Manufacturing. San Francisco: Miller, 1982. Brown, T.D. "Lumber Size Control." Special Publication No 14. Corvallis: Forest Research Lab, Oregon State Univ., 1986. Canadian Wood Council. Wood Design Manual. 3rd ed. Ottawa: CWC, 1997. Cornforth, D.J., and D.G. Elliman. "Modelling probability density functions for classifying using CMAC." Techniques and Applications of Neural Networks. Eds. P.J.D. Lisboa and M.J.Taylor. New York: Ellis, 1993. 36-46. de Silva, C.W. Robot Control: Sensors and Actuators. Vancouver: Univ. of British Columbia, 1995. Dillman, D.A. Mail and Internet Surveys: The Tailored Design Method. 2nd ed. New York: Wiley, 2000. DSM Engineering Plastic Products. Products and Applications Guide. 1999:14. Eggers, M., and T. Khuon. "Neural network data fusion for machine-based decision making." Neural Networks: Concepts. Applications and Implementations. Vol. 4. Eds. P. Antognetti and V. Milutinovic. Englewood Cliffs: Prentice, 1991. 240-274. Fukunaga, K., and J.A. Young. "Pattern recognition and neural engineering." Neural Networks: Concepts, Applications and Implementations. Vol 1. Eds P. Antognetti and V. Milutinovic. Englewood Cliffs: Prentice, 1991. 10-32. Globerman, S., et al. Technological Progress and Competitiveness in the Canadian Forest Products Industry. Ottawa: Science Branch and the Industry, Economics and Programs Branch, Canadian Forest Service, Natural Resources Canada, 1999. Hicks, CR. Fundamental Concepts in the Design of Experiments. 4th ed. New York: Oxford, 1993. Industrial LPS Laser Distance Sensor. LMI Measurement & Control. 29 May. 2000. <http://www.lmint.com/cfm/index.cfm> Kennedy, X.J., D.M. Kennedy and S.A. Holladay. The Bedford Guide for College Writers with Reader and Research Manual. Boston: Bedford, 1999. Lee, L., D. Cohen and T.C. Maness. "Markets and Technology in Western Canadian Sawmills." Forest Products Journal 49.7/8 (1999): 36-42. Lehmann, B.F., and S.G. Hutton. "The alignment, set-up and operation of circular saws and handsaws." Proceedings of the 9th International Wood Machining Seminar. Richmond: Forest Products Laboratory, Univ. of California, 1988. 68-79. Lehmann, B.F. "The cutting behavior of handsaws." Diss. Univ. of British Columbia, 1993. 84 Lehmann, B.F. "Eliminating Barriers to Sawing Performance." Abbotsford: Value-Added Skills Centre, 2000. Lehmann, B.F. Personal interview Oct. 2000. Lemaster, R.L., and J.S. Stewart. "Progress in evaluating surface quality using optical profilometer techniques." Proceedings of the 12th International Wood Machining Seminar. Richmond: Forest Products Laboratory, Univ. of California, 1995. 81-91. Lumber Thickness Measuring. Softac Systems Ltd. 31 Oct. 2000. <http://www.westcanex.ca> Madison's Canadian Lumber Directory. 44th ed. Madsen, B. "Improved lumber Trade through better lumber grading rules." Forest Products Trade: Market Trends and Technical Developments. Eds. J.A. Johnson and W.R. Smith. Seattle: Univ.of Washington Press, 1988. 139-144. Maness, T.C. "Real time quality control system for automated lumbermills." Forest Products Journal. 43.7/8(1992): 18-19. Maness, T.C, and D.H. Cohen. "Quality Control in BC Sawmills: State of the Art." FEPA Working Paper 194. Vancouver: Forest Economics and Policy Analysis Unit, 1993. Maness, T.C, D.H. Cohen and R. Smith. "An assessment of quality control in British Columbia sawmills: production, employment, trends and priorities." Forest Products Journal. 44.3 (1994): 27-33. Maness, T.C, and Y. Lin. "The influence of saw kerf and target size reductions on sawmill revenue and volume recovery." Forest Products Journal. 45.11/12 (1995): 43-50. Maness, T.C, C. Staudhammer and R.A. Kozak. "Statistical considerations for real-time size control systems in wood products manufacturing." Wood and Fiber Science. 34.3 (2002): 476-484. Montgomery, D.C Introduction to Statistical Quality Control. 3rd ed. New York: Wiley, 1996. National Lumber Grades Authority. Standard grading rules for Canadian lumber. New Westminster: NLGA, 2000. National Lumber Grades Authority. Canadian Lumber Grading Manual. New Westminster: NLGA, 1998. Norman, F.T., and J.T. Todd. "The visual perception of a 3-dimensional form." Neural Networks for Vision and Image Processing. Eds. G.A. Carpenter and S. Grossberg. Cambridge: MIT, 1992. 93-110. Ott, E.R. Process Quality Control: Troubleshooting and Interpretation of Data. New York: McGraw-Hill, 1975. Pham, D.T., and R.J. Alcock. "Automated grading and defect detection: a review." Forest Products Journal. 48.4 (1998): 34-42. Rhemrev, J., et al. "Laser scanning for the detection of twist or warp in pencil slats." Forest Products Journal. 43.2 (1993): 29-36. 85 Rosenberg, N., et al. "Understanding the adoption of new technology in the forest products industry." Forest Products Journal. 40.10 (1990): 15-22. Sharkey, N.E., and D. Partridge. "The statistical independence of network generalisation: an application in software engineering." Techniques and Applications of Neural Networks. Eds. P.J.D. Lisboa and M.J.Taylor. New York: Ellis, 1993. 22-33. Skrzypek, J., and J. Hoffman. "Visual recognition of script characters and neural network architectures." Neural Networks: Advances and Applications. Ed. E. Gelenbe. New York: Elsevier, 1991. 109-144. Solla, S.A., and Y. le Cun. "Constrained neural networks for pattern recognition." Neural Networks: Concepts, Applications and Implementations. Vol 4. Eds. P. Antognetti and V. Milutinovic. Englewood Cliffs: Prentice, 1991. 142-161. Statsoft, Inc. Statistica Neural Networks: Addendum for Version 4. Tulsa: Statsoft, 1999. Statsoft, Inc. Statistica Neural Networks. Tulsa: Statsoft, 2000. Swingler, K. Applying Neural Networks: A Practical Guide. San Francisco: Morgan, 1996. Szymani, R., and K.A. McDonald. "Defect Detection in lumber: state of the art" Forest Products Journal. 31.11 (1981): 34-44. Thunell, B. "Some aspects of deviations in shapes and sizes of lumber." Proceedings of the 9 th International Wood Machining Seminar. Richmond: Forest Products Laboratory, Univ. of California, 1988. 55-67. Thomlinson, W.W. "Sawmill headrig and edger optimization." Proceedings of the Application of Advanced Informatics to the Forest Industry. Vancouver: Forest Sector 2000, 1992. Tulunay, E. "Introduction to neural networks and their application to process control." Neural Networks: Advances and Applications. Ed. E. Gelenbe. New York: Elsevier, 1991. 241- 273. Williams, P., and A.W.G. Duller "Identification of lighting flicker sources using a neural network.." Techniques and Applications of Neural Networks. Eds. P.J.D. Lisboa and M.J.Taylor. New York: Ellis,1993. 183-197. Williston, E.M. Lumber Manufacturing - The Design and Operation of Sawmills and Planer Mills. San Francisco: Miller, 1988. 86 A P P E N D I X A - L U M B E R S H A P E D E F E C T S U R V E Y 87 APPENDIX B - TOLERANCE CALCULATION 90 Appendix B - Tolerance Calculations Laser Jig (height tolerance mm)(conversion) = height tolerance divisions (0.1mm)(4095/10) = 40.950 divisions Laser (accuracy mm)(conversion) = accuracy divisions (0.05mm)(4095/10) = 20.475 divisions Calibration Block 2(height tolerance mm)(conversion) = height tolerance divisions 2(0.0254mm)(4095/10) = (0.0508mm)(4095/10) = 20.803 divisions Acquisition Card (precision mV)/(1000V/mm)(conversion) = precision divisions (2.44mV)/(1000V/mm)(4095/10) = (0.00244mm)(4095/10) = 0.999 divisions Within Laser Tolerance Laser accuracy + Calibration Block height tolerance + Acquisition Card precision 0.05mm + 0.0508mm + 0.00244mm = 0.103mm 20.475 divisions + 20.803 divisions + 0.999 divisions = 42.277 divisions Total Tolerance Within Laser Tolerance + Laser Jig height tolerance 0.103mm + 0.1mm = 0.203mm 42.277 divisions + 40.950 divisions = 83.227 divisions APPENDIX C - PREPROCESSED DATA C.1 Preprocessed Training Data Set C.2 R2 and F Values 92 Appendix C.1 - Preprocessed Training Data Set - Three Classes T1I 3066.494 3452.795 3243.287 2813.631 2919.055 3336.534 3137.52 3182.105 3159.124 3122.039 3044.323 3355.118 360.2342 2769.463 3176.456 2355.895 3960.514 111.2042 2990.771 770.2947 2983.182 3059.432 2998.543 3165.577 3226.758 1272.403 3186.603 4403.157 2453.288 3076.036 3610.343 3104.621 3044.426 2999.094 873.4713 2973.932 3182.407 2882.216 2866.922 2701.007 -821.4258 1972.477 3138.694 3685.156 289.5983 3193.108 3331.839 3001.301 •3494.311 2592.718 2806.78 2345.195 3271.978 2845.949 3401.128 2913.478 3038.84 2973.759 3117.089 3686.233 -66.69034 2875.498 1463.092 3308.215 2586.308 T1S 0.021583 -1.96361 0.047226 0.148993 0.226443 -1.047375 -0.330908 -0.01321 -0.148085 -0.298314 -0.834775 -0.953927 0.857917 0.293701 -0.318523 -0.205486 -0.392722 1.211995 0.215286 0.354264 -1.867368 -0.011473 0.226883 -0.269009 -2.293772 2.088657 -0.131006 -1.161466 0.526705 0.068643 -0.476595 -0.128592 0.047286 0.133691 1.32772 -1.648814 -0.467138 0.445825 0.927931 0.075974 3.960132 0.848582 -0.064796 -0.191385 2.630378 0.231101 -0.093813 0.381488 -0.372123 0.458141 0.240831 -1.371111 -0.089434 0.28077 -0.469667 -0.247562 0.082861 0.087115 -0.05493 -0.844569 0.404813 0.22861 1.271166 -0.216598 0.162981 T2I 43.75326 60.46703 342.8759 327.465 351.9855 180.6965 387.1242 -133.4159 304.0406 -260.5891 -1228.508 -187.198 1456.052 130.3985 -21.54919 -93.3854 109.9491 1643.701 273.2027 2199.229 122.9668 -49.85836 44.95706 107.9934 -115.9595 856.6414 280.503 -154.1706 190.6188 45.08449 -54.74629 182.73 168.2047 188.5759 2785.915 -83.14168 -300.3617 63.50887 241.9931 11.39572 3955.201 -231.9433 -1773.186 -27.55075 3300.597 167.615 132.3711 68.81617 8.005995 -150.3885 -137.5161 378.8489 66.24158 119.8714 54.33588 492.3107 -82.74281 51.30448 60.56411 242.0808 1817.635 238.6125 2.923783 83.2304 575.5129 T3I -140.3294 65.59174 266.917 809.438 185.2099 -100.301 657.8254 -426.901 -35.10235 -394.9851 -2582.925 -1038.498 2618.77 345.5373 -395.2475 -201.0755 80.9856 2926.94 144.2092 2249.446 132.3456 130.7968 -70.49982 -222.6145 -194.3108 1917.441 165.4249 -180.288 -223.9976 -28.72558 -91.93777 40.82814 -32.00982 -1131.528 2978.4 125.4307 -624.0426 37.12449 63.6701 -166.9126 3889.307 -1521.234 -3283.062 247.9817 3995.915 2.533339 97.68966 43.52809 -53.10995 -273.0551 -220.9797 193.2207 55.90277 17.49576 -92.81985 83.79434 -279.6155 -1.941471 -250.9598 498.9543 3436.442 1396.823 -176.63 -129.6316 675.7705 T2S 0.147171 0.527401 -0.365171 -0.205288 -0.157902 0.05474 -0.114811 0.188405 -0.03872 0.031949 1.946272 0.002925 -0.485379 -0.067877 0.428539 0.437785 0.053738 -0.666883 -0.195699 -0.678126 -0.263549 0.161058 0.158834 0.03818 0.290787 -1.194334 -0.097446 0.782676 -0.09063 0.347027 0.215254 -0.01156 -0.038681 -0.022112 -3.582136 0.979079 0.450625 -0.053758 -0.712226 0.243096 -3.90014 -0.093156 1.385796 0.054991 -2.774541 -0.499358 0.035701 0.022518 0.226007 0.128027 0.287901 0.559774 0.064982 -0.062911 0.104138 -0.274907 0.324411 0.088964 0.086485 0.291136 -0.463299 -0.155462 -0.023395 0.160107 -0.31044 T3S 0.271966 0.32953 -0.670655 -0.546436 -0.069693 0.333925 -0.293735 0.448738 0.262073 -0.243418 3.165127 0.394192 -0.888149 -0.377713 0.568184 0.789415 0.124727 -1.181393 -0.122784 -0.401341 -0.176896 -0.014901 0.480692 0.231763 0.507826 -2.687156 -0.013255 0.81145 0.267561 0.505623 0.352743 0.098098 0.209839 1.701665 -2.466797 0.985282 0.969668 -0.065123 -0.271561 0.472716 -3.744296 0.314924 2.339668 -1.404559 -3.145692 -1.516066 0.101392 0.098962 0.336449 0.28838 0.347324 0.916753 0.142149 0.12066 0.25099 0.378438 0.848927 0.286396 0.532384 0.206864 -0.963574 -0.729629 -0.000329 0.25476 -0.367901 B1I 2273.698 1318.974 2163.037 2207.778 2050.216 1585.082 -133.1121 2093.618 2052.498 2244.921 2228.404 2223.133 1767.657 1582.346 2035.268 1960.255 1995.079 1472.6 1634.941 1964.375 2310.059 2249.874 2139.585 1787.481 2327.601 1775.726 2212.505 2253.151 1583.647 1956.857 1814.429 2145.363 2297.294 2185.173 1846.719 1994.084 1936.606 2229.217 2083.673 2119.886 1467.704 2229.411 2100.263 2111.214 2058.641 2154.499 1796.648 1990.973 1657.277 2059.934 2245.54 2208.217 2190.181 2278.865 139.218 846.4169 1901.02 1335.558 2219.67 -536.5944 1220.851 2220.253 -501.0328 1938.973 2082.567 B1S -0.199011 0.311067 -0.252669 -0.250582 -0.03621 -0.785388 0.614336 -0.548705 -0.348927 -0.18367 -0.227644 -0.101109 -0.127702 0.748069 -0.029319 -0.810131 0.005266 0.330867 0.150701 0.038632 -0.456059 -0.111209 0.028337 0.018474 -0.403207 0.50419 -0.256456 -0.157409 -0.006892 -0.27604 0.213426 -0.070461 -0.297092 -0.029297 0.229797 -0.083098 -0.187891 -0.495915 0.109561 -1.193946 -1.25955 -0.139725 -0.030496 -0.016417 -0.101289 -0.069726 -1.569155 0.024686 -0.197838 -0.112475 -0.258938 -0.462108 -0.142627 -0.320567 1.165136 -0.064659 0.071104 -0.890981 -0.182129 1.673922 -0.104806 -0.13274 1.251152 -0.011126 -0.416885 B2I 472.1326 234.838 129.7254 391.5199 290.4566 472.9158 1813.104 444.4117 274.1856 233.9773 158.9085 436.2695 494.0686 988.0848 303.3031 378.1692 504.8675 536.7137 963.5601 485.7961 306.2008 145.1393 478.8075 580.2317 376.5429 895.2546 197.5518 150.8417 -160.9124 250.7962 208.2445 223.5057 194.0033 236.7484 366.5344 512.3511 234.6811 239.982 428.6297 292.245 967.9663 48.95832 48.04367 172.6713 398.5227 205.84 881.7852 329.4359 -103.8478 345.8406 180.6266 197.76 127.7882 215.6799 2247.946 862.3344 280.274 1044.302 165.6983 2126.189 560.3743 69.69501 2568.136 144.8974 -59.18926 B3I -89.42501 189.6305 -549.354 -676.7671 -108.2994 -76.66901 2159.017 -84.76856 -109.8918 406.4133 •453.323 -153.1315 56.63664 398.4366 -153.9874 -205.8688 -330.5662 625.7862 435.5234 -19.35454 -321.5172 -256.6824 -86.13334 -71.68633 -199.4951 81.88602 -91.98968 -258.2195 466.8856 3.451152 -61.77981 -30.22996 -367.6941 -569.7604 -45.73176 36.95532 -239.189 -83.82802 40.76183 -129.4256 397.628 -353.8076 -478.8206 -210.5555 1404.213 -127.6699 85.29914 -65.06395 -394.0575 -152.9826 -156.5969 -392.9452 -148.5634 -127.5672 1796.055 812.2396 40.14458 715.2208 -68.3063 2403.093 597.8142 1693.266 1808.263 -55.44943 -285.0873 B2S -0.068606 0.33902 -0.855288 0.22143 -0.004142 -0.173594 -1.186632 -0.173438 0.043765 0.156079 0.231972 -0.064779 0.053043 -0.779268 -0.017137 -0.150372 -0.168605 -0.106174 -0.208574 0.042261 0.179537 0.130216 -0.238906 -0.32454 0.040784 -0.749828 0.086147 -0.026715 0.201777 0.045438 -0.030918 -0.074499 0.135994 -0.013299 -0.247594 -0.158798 -0.274063 -0.002217 -0.687706 -0.084042 0.677921 0.108526 -0.00253 -0.004614 -0.437005 -0.027163 0.487211 -0.103148 0.264652 -0.014893 0.043469 -0.179192 0.071118 -0.185188 -2.739902 -0.2162 -0.232661 0.634137 -0.088674 -1.116596 0.273931 0.112767 -2.134961 -0.076785 0.127896 B3S -0.119765 -1.336823 -0.251344 0.652856 -0.017374 0.221904 -0.94362 -0.174183 -0.017271 -0.084703 0.150154 0.020831 0.025163 -0.621995 -0.099932 0.189568 -0.135575 -0.742884 -0.217322 -0.002422 0.461995 0.116097 -0.174155 -0.306248 0.165038 -0.230938 -0.515798 -0.013327 0.263596 0.005983 -0.16715 -0.331594 0.828758 0.700206 -0.192058 -0.041248 -0.17793 0.122855 -0.555456 0.501428 1.402987 0.225651 -0.007735 0.058801 -0.880104 -0.055166 1.565117 -0.059225 -0.097574 -0.002875 0.107223 0.817587 0.017547 -0.044684 -1.194212 -0.574922 -0.235962 0.588109 -0.228344 -1.609651 0.286254 -0.990086 -1.28713 -0.140372 0.217049 SNIPE n y n n n y n y n n c y n n y y n c y y y n n n y n y n y n y n c n c n y n n y y n c y y y y n n n n y n n y y y n n n n n n n n q3 3618.636 3267.936 -408.2357 3236.276 3413.079 3032.606 2879.532 2530.011 2396.354 3013.274 3021.079 2707.165 2999.26 -135.1969 2385.287 1073.693 2915.228 2827.75 2204.934 2403.185 1226.686 3214.491 2028.533 2558.157 3066.116 1098.356 2971.111 3337.467 -965.5856 3485.21 -728.8021 3173.159 2935.267 2825.562 2691.977 2430.516 2787.666 281 .4155 -0.611472 -0 .415887 2.299462 -0.280051 -1 .833255 -0 .051425 0.104694 -0 .067123 0 .470615 -0 .27673 -0.015551 0.32364 -0 .153453 1.913781 0.756326 2.524186 0.048903 -0.148178 0.520674 0 .331867 1.883873 -0 .204978 0 .841597 0.675001 -0 .703226 1.513392 0 .168355 -0.282611 2 .651999 -0.410314 1.803898 -0 .141253 -0.00402 -0.031904 0.696811 -0 .17465 -1 .604364 3.211378 23 .37526 •96.39268 3192.011 •696.5063 -234.693 99.91801 225.2638 91.20008 117.5967 •1778.956 122.7335 437 .7275 431.2458 882.8714 •30.31878 489.8694 61 .98087 •1530.941 260.6596 •71.81626 165.7344 •195.4185 366 537 .5907 25 .54893 571.262 113.9289 83 .30526 402.3248 550.934 3455.554 50.4302 104.8083 224.6893 188.0627 276.4288 142.5892 2090.784 415.1334 -296.7896 2103 .25 -1379.605 -408.2458 12.41715 263.0184 -86.31038 60 .66123 -3518.585 60.72391 244.1502 717.3981 357.0833 153.5361 273.1182 -107.801 -2859.253 83.72934 -1216.444 416.5182 -840.3809 473.1252 612.4358 39.47881 1200.34 153.699 148.9244 -173.9525 802.7574 3581.278 -28 .89817 -155.7774 432.8791 106.8346 369.4666 -713.9551 1506.114 0.508675 0.266534 •3.595609 0 .578253 1.863674 0.096384 0.006064 0.172432 •0.021699 0.677517 0.0552 •0.413952 •0.224417 0.458391 0.19858 •0.671365 0 .047205 1.06814 -0.16502 0.157764 0.462605 0.550203 -0.661654 •0.455349 0.322498 -1.0352 0 .133409 0.16366 -0.314313 •0.466192 •1.468097 0 .214559 -0.031099 0.058921 -0.083471 -0.145262 1.466209 -2.300303 0 .014569 0 .528099 -2 .468455 1.262629 1.916632 0.136763 0.117845 0.067408 -0 .061054 1.6039 0.103364 -0 .540005 -0 .463354 0.670522 0.139397 0.247138 0 .172129 1.237837 0.898056 1.98733 0 .331293 1.819983 -0 .536688 -0.411041 0.599741 -1 .373052 0.158451 0 .192259 0 .338863 -0 .240393 -1 .320372 0.466437 0.1262 -0 .139338 0.209153 -0 .236277 2.411823 -1 .845429 2417.253 2157.718 2371.119 2496.294 1965.866 2223.311 2089.631 1913.992 2197.187 1487.642 2207.636 1980.354 1938.59 2232.521 2073.586 2083.948 2219.007 2170.505 2125.295 1646.421 1489.343 2149.312 2210.948 2153.558 2227.568 1974.583 2154.545 1855.437 1680.258 2052.401 2380.163 2101.279 2153.88 2213.611 2036.929 2018.172 1826.718 2224.066 •0.848859 •0.067404 •1.304976 •1.656003 0.21284 •0.239275 •0.005124 •0.031362 •0.111182 0 .292023 •0.086048 •0.572799 •0.324484 •0.128361 0.013971 0.090988 •0.110893 •0.045967 0.034452 0.247621 1.019 0 .021963 •0.321076 -0.121457 0.016971 0 .100349 -0.055158 -0.089422 0.154492 -0.043624 -0.419902 0 .107263 -0.024457 -0 .45489 0.112202 0.143504 0 .151275 -0.056789 -223.6597 152.7762 37.0704 -290.5198 429 .8996 39.16746 166.8503 289 .2213 21 .97258 77.85181 97.48312 19.19302 112.8258 169.4307 349.745 270.0674 176.9734 -285.4727 250.824 21 .93823 975.349 ^t45.7112 112.1553 128.2168 118.7556 482 .5844 123.0641 89 .22365 626.6282 -118.0337 38.35556 287 .7293 193.6474 136.4607 49 .84797 308.466 467.0414 46.51321 -1110.583 -1 .100886 -381.9374 -944.7194 149.3236 -303.9865 -224.0508 178.1312 -264.2304 -576.8432 -166.1811 ^ 4 0 . 2 8 5 4 -155.873 -126.263 -44.02498 -10.4459 -89.9286 -577.9301 -42.40317 6 .938246 585.5192 -356.0368 -140.5883 -143.654 -125.6376 166.2423 -98.62788 -166.4936 -180.6823 1566.024 -220.0884 40 .48378 -236.9248 -391.0676 -406.4367 73.33412 257 .8055 -203.1668 0.720121 0.043351 0 .988253 0.131092 •0.304267 0 .188883 •0.065315 •0.043729 0 .147745 •0.010889 0.033811 0 .001259 •0.067829 0.016778 •0.221682 •0.351102 •0.002832 0.085632 •0.207972 0.174502 •1.504776 0.729091 0.029238 0.066804 •0.271669 •0.713891 •0.057564 •0.178288 -0.446744 0.299111 0 .242845 -0.438271 -0.149024 0.162781 0.130087 •0.302766 -0 .18736 0.073222 1.412873 c -0 .174237 n 1.094296 n 0.606054 n -0 .356544 y 0.361951 n 0 .137743 n -0 .150324 n 0 .205615 n 0 .23665 n 0 .063077 n 0.436721 y -0 .533289 y 0 .025662 n -0 .052443 n -0 .27707 n -0 .000767 n 0.195851 n -0 .094234 n -0 .428502 n -1 .396429 n 0 .162135 n 0 .050414 n 0 .107659 y -0 .28709 y -0 .845343 n -0 .052078 y -0 .229292 n 0 .035972 n -0 .799548 n 0 .264528 c -0 .536026 n 0 .025109 y 0.824004 y 0 .586458 n -0 .298335 n -0 .221735 y 0 .053829 y Appendix C.2 - R 2 and F Values by Sample Board Sample Board Bottom Surface Top Surface Snipe Classification F value R 2 value F value R 2 value 1 1409.6 0.6859 85.7 0.1172 n 2 117.7 0.1997 323.1 0.4062 y 3 736.8 0.4908 201.7 0.2088 n 4 3141.6 0.8158 1027 0.5915 n 5 297.0 0.3592 114.7 0.1780 n 6 2360.0 0.7640 298.4 0.2904 y 7 182.8 0.2412 192.5 0.2509 n 8 184.9 0.2453 141 0.1986 y 9 936.4 0.5421 1008.9 0.5606 n 10 83.7 0.1375 399.4 0.4320 n 11 863.4 0.5373 114.3 0.1332 c 12 629.8 0.5273 121.2 0.1767 y 13 359.6 0.3199 2598 0.7727 n 14 1265.9 0.6186 280 0.2640 n 15 555.9 0.5004 374.8 0.4031 y 16 1245.2 0.6143 1144.3 0.5941 y 17 269.0 0.3053 25.5 0.0401 n 18 354.0 0.3217 3610.1 0.8287 c 19 997.3 0.5275 109.5 0.1092 y 20 89.8 0.1353 35.3 0.0579 y 21 296.0 0.3774 61.2 0.1113 y 22 1562.6 0.7934 249.5 0.3802 n 23 508.6 0.4955 210.1 0.2886 n 24 272.8 0.2563 140.9 0.1512 n 25 690.1 0.5856 248.4 0.3371 y 26 759.9 0.5625 67.5 0.1025 n 27 504.0 0.5263 132.9 0.2265 y 28 297.6 0.3545 429 0.4419 n 29 120.9 0.1332 26 0.0320 y 30 492.3 0.4868 39.3 0.0704 n 31 649.5 0.4561 522.4 0.4030 y 32 88.1 0.1691 59 0.1199 n 33 33.0 0.0442 256.8 0.2650 c 34 95.8 0.1094 414.8 0.3473 n 35 694.0 0.4754 1532.8 0.6670 c 36 321.6 0.4333 2246.1 0.8423 n 37 171.5 0.2355 122.1 0.1799 y 38 222.2 0.3767 46.8 0.1129 n 39 266.7 0.3645 56.3 0.1080 n 40 128.8 0.1837 62.4 0.0984 y 41 728.9 0.5414 135.4 0.1798 y 42 342.1 0.3992 465 0.4746 n 43 1045.6 0.5703 819.9 0.5098 c 44 28.6 0.0657 11.3 0.0271 y 45 210.8 0.3329 73.3 0.1478 y 46 1064.4 0.5900 138.6 0.1577 y 47 1029.6 0.5648 472.2 0.3731 y 48 1096.1 0.5837 4744.5 0.8585 n 49 538.7 0.3989 431.4 0.3470 n 50 1224.2 0.6004 3708.2 0.8198 n 51 43.4 0.0918 112.2 0.2070 n 52 153.0 0.2098 189.2 0.2472 y 53 428.7 0.5037 57 0.1180 n 54 172.3 0.2966 39.9 0.0889 n 55 265.1 0.3826 12.8 0.0288 y 56 196.0 0.3040 81.4 0.1536 y 57 55.4 0.1140 274.1 0.3888 y 58 96.8 0.1233 375.3 0.3527 n 59 185.5 0.1961 149.9 0.1647 n 60 760.3 0.4844 2768.8 0.7738 n 61 1572.3 0.6861 3172.2 0.8151 n 62 2929.0 0.8047 9973.2 0.9334 n 63 2697.7 0.8081 1018.9 0.6139 n 64 308.3 0.3532 77.8 0.1212 n 65 460.1 0.5681 18.8 0.0509 n 66 254.7 0.2368 196.4 0.1931 c 67 198.2 0.2993 15.2 0.0318 n 68 594.8 0.4359 864 0.5288 n 69 361.4 0.3659 939.3 0.5999 n 70 330.2 0.3506 348.2 0.3628 y 71 510.7 0.3996 114.9 0.1302 n 72 218.5 0.2225 295.1 0.2787 n 73 384.6 0.4065 535.2 0.4880 n 74 477.7 0.3802 2717.8 0.7773 n 75 134.5 0.1811 492.7 0.4474 n 76 81.8 0.1164 80 0.1142 n 77 355.0 0.3144 174.6 0.1841 y 78 300.1 0.2813 2525.8 0.7671 y 79 373.2 0.3339 333.4 0.3092 n 80 260.5 0.2622 22.1 0.0292 n 81 181.0 0.3298 313.4 0.4600 n 82 578.9 0.6119 120.6 0.2472 n 83 13.9 0.0397 149.1 0.3074 n 84 138.2 0.2111 110.5 0.1762 n 85 259.3 0.2628 982.9 0.5745 n 86 339.1 0.2986 60.3 0.0704 n 87 91.8 0.2068 84.5 0.1936 n 88 1529.8 0.7185 993.7 0.6238 n 89 386.7 0.3119 1080.1 0.5587 y 90 17.3 0.0378 54.8 0.1106 y 91 94.0 0.1814 81.8 0.1616 n 92 78.2 0.2152 32.4 0.1021 y 93 82.6 0.1559 29.8 0.0625 n 94 173.7 0.2837 702.4 0.6156 n 95 66.4 0.1528 18.3 0.0474 n 96 226.7 0.2543 1264.2 0.6554 c 97 781.5 0.4896 380.7 0.3184 n 98 193.7 0.2313 21.8 0.0327 y 99 201.5 0.3741 92.2 0.2147 y 100 157.1 0.2919 85.3 0.1829 n 101 598.6 0.4204 71.2 0.0795 n 102 155.1 0.2896 25.6 0.0630 y 103 136.0 0.1456 3749.2 0.8245 y A P P E N D I X D - D A T A V A L I D A T I O N S U M M A R Y O F R E S U L T S D.1 No Change in Apparatus D.2 Laser Measurement Repeatability 97 Appendix D.1 - No Change in Apparatus 1st test hypothesis 2nd test hypothesis For both tests Ho:uD = 0 Ho: uD < tolerance I statistic = (Rbar - uD)/(sD/SQRT(n)) Ha: uD not equal 0 Ha: uD > tolerance Where alpha = 0.05 tolerance = 42.3 divisions Rbar: Average of Residuals n = 900 alpha = 0.05 uD: expected mean of residuals t crit = 1.962 n = 900 sD: standard deviation of residuals t crit = 1.647 n: number of matched pairs before after (after-before) 'laser T2, A9151145 A1155759 Residual Average 2896.95 2903.29 6.34 Stdev 1.43 1.36 2.04 Var 2.04 1.85 4.16 test t statistic 1st test 93.18 <=== there is a significant difference between the two runs (bad) 2nd test -528.93 <=== the difference between the two runs is significantly less than tolerance of 42.3 (good) before after (after-before) [laser B3- ] A9151145 A1155759 Residual Average 1607.32 1570.93 -36.39 Stdev 1.44 1.45 2.09 Var 2.06 2.10 4.36 test t statistic 1st test -522.93 <=== there is a significant difference between the two runs (bad) 2nd test 84.93 <=== the difference between the two runs is significantly less than -42.3 (good) before after (after-before) i j ^ r B f " ! A9151145 A1155759 Residual "Average 2222.04 1925.98 -296.06 Stdev 1.57 1.53 2.23 Var 2.45 2.34 4.96 test t statistic 1st test -3989.96 < = « there is a significant difference between the two runs (bad) 2nd test -3419.89 <=== the difference between the two runs is NOT significantly less than -42.3 (bad) before after (after-before) [§seFfTJ A9151145 A1155759 Residual Average 1827.99 1822.44 -5.55 Stdev 1.49 1.41 2.09 Var 2.23 2.00 4.38 test t statistic 1st test -79.54 <=== there is a significant difference between the two runs (bad) 2nd test 527.02 <=== the difference between the two runs is significantly less than -42.3 (good) before after (after-before) 'laser tt i A9151145 A1155759 Residual " " ' "Average 2046.746 2020.137 -26.6089 Stdev 2.502664 2.65942 3.243943 Var 6.263328 7.072514 10.52317 test t statistic 1st test -246.08 <=== there is a significant difference between thetwo runs (bad) 2nd test 145.11 <=== the difference between the two runs is significantly less than -42.3 (good) before after (after-before) 'laser T3 '• A9151145 A1155759 Residual Average 2746.872 2773.016 26.14333 Stdev 1.335868 1.370182 1.869047 Var 1.784545 1.8774 3.493337 test t statistic 1st test 419.63 <=== there is a significant difference between the two runs (bad) 2nd test -259.33 <=== the difference between the two runs is significantly less than 42.3 (good) 98 Appendix D.2 • Laser Measurement Repeatability 1st test hypothesis Ho: uD = 0 Ha: uD not equal 0 alpha = 0.05 n = 900 t crit = 1.962 2nd test hypothesis Ho: uD < tolerance Ha: uD > tolerance tolerance = 42.3 divisions alpha = 0.05 n = 900 t crit = 1.647 For both tests t statistic = (Rbar - uD)/(sD/SQRT(n)) Where Rbar: Average of Residuals uD: expected mean of residuals sD: standard deviation of residuals n: number of matched pairs A9151145 A9151347 Residuals A9151547 Residuals A9151746 Residuals A9151947 Residuals Average 2896.95 2897.10 0.14 2896.67 -0.29 2897.15 0.20 2896.40 -0.56 Stdev 1.43 1.34 2.02 1.34 2.09 1.43 2.06 1.43 1.99 Var 2.04 1.81 4.09 1.80 4.38 2.06 4.26 2.03 3.97 t statistic t statistic t statistic t statistic 1st test 2.14 -4.11 2.86 -8.40 significant significant significant significant Result There is a significant difference between the runs (bad) t statistic t statistic t statistic t statistic 2nd test -625.48 602.06 -611.94 ' 628.41 sig'tly less sig'tly less sig'tly less sig'tly less Result The difference between the runs is significantly less than tolerance (good) B9151145 B9151347 Residuals B9151547 Residuals B9151746 Residuals B9151947 Residuals Average Stdev Var 1607.32 1.44 2.06 1606.59 1.53 2.33 -0.73 2.12 4.48 1605.49 1.43 2.03 -1.84 2.06 4.26 1604.73 1.55 2.40 t statistic t statistic 1st test -10.39 -26.71 significant significant There is a significant difference between the runs (bad) t statistic t statistic 2nd test 589.12 588.35 sig'tly less sig'tly It -2.60 2.14 4.57 t statistic -36.43 significant t statistic 557.03 sig'tly less 1604.51 -2.82 1.58 2.04 2.49 4.18 t statistic -41.33 significant t statistic 579.56 sig'tly less Result The difference between the runs is significantly less than tolerance (good) C9151145C9151347 Residuals C9151547 Residuals C9151746 Residuals C9151947 Residuals Average Stdev Var 2222.04 1.57 2.45 2221.94 1.54 2.37 -0.11 2.19 4.80 2220.81 1.55 2.42 -1.23 2.33 5.44 2220.39 1.56 2.44 -1.65 2220.11 -1.93 2.26 1.51 2.18 5.09 2.29 4.73 Result * t statistic t statistic t statistic t statistic Msttest -1.49 -15.85 -21.98 -26.61 NOT sig significant significant significant '• There is a significant difference between the runs (bad) except in the first case. 2nd test 577.72 528.24 540.49 556.58 | sig'tly less sig'tly less sig'tly less, sig'tly less The difference between the runs is significantly less than tolerance (good) Lase7Bj~i D9151145 D9151347 Residuals D9151547 Residuals D9151746 Residuals D9151947 Residuals "Average ,1827.99 1827.67 -0.32 1826.89 -1.10 1826.86 -1.13 1826.57 -1.42 Stdev 1.49'. 1.38 2.09 1.37 2.11 1.39 2.04 Var 2.23 1.90 4.37 1.88 4.47 1.93 4.16 1.43 2.06 2.06 4.26 Average Stdev Var t statistic t statistic t statistic 1st test -4.59 -15.57 -16.55 significant significant significant ' There is a significant difference between the runs (bad) 2nd test 602.21 sig'tly less 584.57 sig'tly If 605.38 sig'tly less t statistic -20.60 significant 594.47 sig'tly less Result The difference between the two runs is significantly less than tolerance (good) E9151145 E9151347 Residuals E9151547 Residuals E9151746 Residuals E9151947 Residuals 2046.75 2060.42 2.50 1.56 6.26. 2.43 13.67 2.84 8.05 t statistic 144.62 significant 2058.31 1.83 3.36 11.56 3.66 13.40 t statistic 94.73 significant 2056.60 1.58 2.51 9.86 2051.48 4.74 2.90 1.61 2.90 8.43 2.60 8.39 Result There is a significant difference between the runs (bad) t statistic 101.86 significant t statistic 49.06 significant 2nd test -302.75 -251.89 -335.19 -389.10 sig'tly less sig'tly less sig'tly less sig'tly less 'Result The difference between the runs is significantly less than tolerance (good) F9151145 F9151347 Residuals F9151547 Residuals F9151746 Residuals F9151947 Residuals Average Stdev Var 2746.87 2747.75 1.34 1.37 1.78 1.88 0.88 2747.71 1.94 1.33 3.78 1.76 0.84 2.02 4.10 2747.96 1.39 1.93 1.08 2749.48 2.61 1.93 1.50 1.97 3.73 2.24 3.88 t statistic t statistic t statistic t statistic 1st test 13.55 12.39 16.85 39.72 significant significant significant significant . There is a significant difference between the runs (bad) 2nd test -639.32 -614.69 -640.38 -604.31 sig'tly less sig'tly less sig'tly less sig'tly less The difference between the runs is significantly less than tolerance (good) A P P E N D I X E - E X A M P L E C L A S S I F I C A T I O N A N A L Y S I S O F S A M P L E C A S E 100 Appendix E - Example Classification Analysis of Sample Case |N 10EC3CI actual target error SNIPE T. SNIPE E. SNIPE Error error type 1 n n Right 0.340923 2 y y Right 0.199405 3 n n Right 0.24407 4 n n Right 0.159318 5 n n Right 0.192806 6 y y Right 0.0548 7 n n Right 0.255378 8 y y Right 0.40217 9 n n Right 0.320534 10 n n Right 0.221628 11 y c Wrong 0.533387 mix 12 y y Right 0.179526 13 n n Right 0.379088 14 y n .Wrong 0.452284 type I 15 n y ;Wrong J 0.438431 type II 16 y y Right 0.208186 17 n n Right 0.35821 18 n c : Wrong 1 0.630405 mix 19 y y Right 0.359706 20 y y Right 0.392373 21 y y Right 0.04002 22 n n Right 0.20806 23 n n Right 0.226634 24 y n [Wrong ____ 0.472627 type I 25 y y Right 0.02415 26 y n 1 Wrong • 0.472587 type I 27 n y Wrong 0.603605 type II 28 y n jwrong 0.687566 type I 29 n y [Wrong ^ 0.798456 type II 30 n n Right 0.328139 31 n y [Wrong 0.452809 type II 32 n n Right 'Wrong 0.284964 33 n c 0.709033 mix 34 n n Right 0.05407 35 n c 'Wrong 0.691766 mix 36 y n •Wrong 0.795369 type I 37 y y Right " 0.402455 38 n n Right 0.220557 39 n n Right 0.09504 40 y y Right 0.2058 41 y y Right 0.152041 42 n n Right 0.04657 43 n c Wrong 0.587575 mix 44 y y Right 0.286373 45 y y Right 0.136288 46 n y Wrong 0.479989 type II 47 y y Right 0.04526 48 n n Right 0.175418 49 n n Right 0.133824 50 n n Right 0.141822 51 n n Right 0.197412 52 y y Right 0.037498 53 n n Right 0.226088 54 n n Right 0.202651 IOI 55 y y Right 0.261267 56 n y Wrong ' 0.455065 type II 57 n y Wrong- 0.584529 type II 58 y n Wrong 0.579702 type I 59 n n Right 0.203476 60 n n Right 0.10434 61 n n Right 0.345018 62 n n Right 0.278102 63 n n Right 0.0834 64 n n Right 0.315366 65 n n Right 0.172147 66 n c Wrong . 0.71425 mix 67 n n Right 0.376877 68 n n Right 0.06551 69 n n Right 0.221209 70 y y Right 0.04299 71 n n Right 0.185419 72 n n Right [Wrong 0.186799 73 y n 0.414113 type I 74 n n Right " * 0.08429 75 n n Right 0.233444 76 n n Right 0.208301 77 n y jWrong ' 0.676377 type II 78 n y [Wrong , 0.506371 type II 79 n n Right 0.08709 80 n n Right 0.168539 81 n n Right 0.04008 82 n n Right 0.230565 83 n n Right 0.21344 84 n n Right 0.087599 85 n n Right 0.06804 86 n n Right 0.209265 87 n n Right 0.06139 88 n n Right 0.0856 89 n y (Wrong 1 0.727109 type II 90 y y Right " ' 0.319268 91 n n Right 0.291721 92 n y [Wrong • 0.626114 type II 93 n n Right 0.318135 94 n n Right 0.06949 95 n n Right 0.262171 96 n c •Wrong 0.650968 mix 97 n n Right 0.348434 98 n y Wrong ; 0.622902 type II 99 n y Wrong 0.436087 type II 100 n n Right 0.04072 101 y n Wrong 0.440075 type I 102 y y Right 0.058678 103 n y Wrong ' 0.795753 type II IN10EG3C total .' . itype'-l).'.'^',' type II ' mix :* 29 incorrectly classified 8 28% of incorrectly classified cases are type I 14 48% of incorrectly classified cases are type II 7 24% of incorrectly classified cases are mix 74 are classified correctly (as snipe or as not having snipe) 71.8% of 103 samples 102

Cite

Citation Scheme:

    

Usage Statistics

Country Views Downloads
China 13 9
United States 8 0
Indonesia 4 0
France 4 0
United Kingdom 3 0
Canada 1 0
India 1 0
City Views Downloads
Unknown 11 0
Hangzhou 8 0
Wilmington 4 0
Jakarta 4 0
Beijing 2 0
Redmond 1 0
Vancouver 1 0
Shenzhen 1 9
Guangzhou 1 0
Ashburn 1 0

{[{ mDataHeader[type] }]} {[{ month[type] }]} {[{ tData[type] }]}

Share

Share to:

Comment

Related Items