Open Collections

UBC Theses and Dissertations

UBC Theses Logo

UBC Theses and Dissertations

An analysis of machine shape defects in British Columbia sawmills and their classification using neural… Rasmussen, Helen Katrina E. 2003

Your browser doesn't seem to have a PDF viewer, please download the PDF to view this item.

Item Metadata

Download

Media
831-ubc_2003-0290.pdf [ 9.81MB ]
Metadata
JSON: 831-1.0074991.json
JSON-LD: 831-1.0074991-ld.json
RDF/XML (Pretty): 831-1.0074991-rdf.xml
RDF/JSON: 831-1.0074991-rdf.json
Turtle: 831-1.0074991-turtle.txt
N-Triples: 831-1.0074991-rdf-ntriples.txt
Original Record: 831-1.0074991-source.json
Full Text
831-1.0074991-fulltext.txt
Citation
831-1.0074991.ris

Full Text

AN ANALYSIS OF MACHINE SHAPE DEFECTS IN BRITISH COLUMBIA SAWMILLS AND THEIR CLASSIFICATION USING NEURAL NETWORKS by  H E L E N K A T R I N A E. R A S M U S S E N  B.A.Sc., T h e University of British C o l u m b i a , 1 9 9 4  A T H E S I S S U B M I T T E D IN P A R T I A L F U L F I L M E N T O F THE REQUIREMENTS FOR THE DEGREE OF MASTER OF APPLIED SCIENCE in  THE FACULTY OF GRADUATE STUDIES ' I  (Faculty of Forestry) W e a c c e p t this thesis a s conforming to the required standard  T H E UNIVERSITY O F BRITISH C O L U M B I A February, 2 0 0 3  © H e l e n Katrina E. R a s m u s s e n , 2 0 0 3  UBC Rare Books and Special Collections - Thesis Authorisation Form  Page 1 of 1  In presenting t h i s t h e s i s i n p a r t i a l f u l f i l m e n t of the requirements f o r an advanced degree at the U n i v e r s i t y of B r i t i s h Columbia, I agree that the L i b r a r y s h a l l make i t f r e e l y a v a i l a b l e f o r reference and study. I f u r t h e r agree that permission f o r extensive copying of t h i s t h e s i s f o r s c h o l a r l y purposes may be granted by the head of my department o r by h i s o r her representatives. I t i s understood that copying o r p u b l i c a t i o n of t h i s t h e s i s f o r f i n a n c i a l gain s h a l l not be allowed without my w r i t t e n permission.  The U n i v e r s i t y of B r i t i s h Columbia Vancouver, Canada  http://www.library.ubc.ca/spcoll/thesauth.html  4/9/03  ABSTRACT Ideally, the edges of lumber are parallel to each other and its ends are rectangular and in line with each other. However, sub-optimal occurrences in the sawing processes cause deviations from this ideal shape. In the sawmill, these deviations are often detected as off-size variations in thickness, and one particular defect shape is not necessarily distinguished from another in the downgrading process. These defects, different from machine defects like torn grain or skip, are referred to as machine shape defects in this thesis. The first part of this thesis implements a survey to analyse machine shape defects in British Columbia sawmills, while the second part employs neural networks as an experimental approach in the classification of these defects. A survey was designed and implemented to determine the industrial significance of machine shape defects in British Columbia sawmills. Completed in 2000, the survey focussed on six machine shape defects commonly caused by the sawing process: snipe, flare, wedge, taper, thin snake and fat snake. Responses came from mills located across BC and from both large and small forest companies responsible for 33% of BC softwood lumber production in 2000. Characterising BC sawmills according to machine shape defects and annual production shows that for each category of mill, with one exception, there is over a 20% probability of producing at least five types of machine shape defects. The most common grade cited for all machine shape defects was No. 2 Structural. By ranking the machine shape defects in terms of occurrence and by determining which ones are most serious in terms of final quality, it was established that thin snake, snipe and taper have the most serious impact on the industry. Neural networks were trained to detect and classify snipe in rough green lumber, using more than one hundred trim ends sampled randomly from a mill experiencing difficulty processing frozen wood. A self-contained measuring apparatus was constructed to support measuring equipment and to convey the sample boards through the measuring range of six lasers at a steady rate, using the automatic feedrollers of a shaper table. A statistical model was developed to interpret the physical characteristics of the board's surface, focussing on its shape. This model was used to preprocess the laser data into a set of variables, simplifying the data set for input into the neural networks. It was demonstrated that neural networks can be applied with limited success to detect machine shape defects, in particular snipe, in random samples of rough green lumber. However, it was established that more training data is required to train the neural networks to classify the sample cases with combination snipe. ii  TABLE OF CONTENTS ABSTRACT  II  TABLE OF CONTENTS  Ill  LIST OF TABLES  ...VI  LIST OF FIGURES  VII  ACKNOWLEDGEMENT  VIII  BACKGROUND  1  TROUBLESHOOTING METHODS  2  REAL-TIME LUMBER SIZE CONTROL SYSTEMS  3  THESIS OBJECTIVES  4  PART I - LUMBER SHAPE DEFECT SURVEY INTRODUCTION  5 5  Objective of the Survey of BC Sawmills  7  METHODOLOGY  7  RESULTS  8  Response  8  Probability of Shape Defect  10  Shape Defects Affecting Quality  15  Grade  16  Causes of Shape Defect  16  Nomenclature  23  DISCUSSION  25  Probability of Shape Defect  25  Shape Defects Affecting Quality  26  Grade  28  Causes of Shape Defect  31  Nomenclature  33  CONCLUSIONS  34  PART II - CLASSIFICATION OF MACHINE SHAPE DEFECTS INTRODUCTION  36 36  Objective of Neural Network Classification  iii  37  METHODOLOGY  3  9  Sample Measurement  39  Measurement Equipment  39  Sample Boards and Measurement Process  43  Data Validation  45  Sources of Error & Tolerance  5  4  Calibration Block  4  5  Validation of the data  4  6  Data Preprocessing  47  Laser Data  4  ^  Modelling the 3-D Shape of the Board  50  Input & Output Variables  53  Neural Network Trials  53  Software  53  Neural Network Training and Testing  54  Neural Network Trial Procedure  55  Training Process: Snipe Classification Approaches & Training Strategies  55  RESULTS  5 9  Data Validation  59  No Change in Apparatus  59  Laser Measurement Repeatability  59  Data Preprocessing  60  Neural Network Trials  62  Interpretation of Results  62  Description of Neural Networks  63  Evaluation of Neural Networks  63  Two Class Network Problem: No Snipe & Snipe  64  Base Condition - Two Class Networks  64  Shuffled Condition - Two Class Networks  65  Pruned Condition - Two Class Networks  66  Extended Duration Condition - Two Class Networks  67  Three Class Network Problem: No Snipe, Snipe & Combination Snipe Base Condition - Three Class Networks  68 •  68  Shuffled Condition - Three Class Networks  68  Pruned Condition - Three Class Networks  69  Extended Duration Condition- Three Class Networks  69  iv  71  DISCUSSION  Data Validation  71  Neural Network Trials  71  Description of Neural Networks  72  Type of Network  72  Complexity  74  Evaluation of Neural Networks  75  Error: Training, Verification & Testing  75  Performance  77  Classification Analysis of Sample Cases  77  CONCLUSIONS  81  FUTURE APPLICATIONS  82  SUMMARY  83  BIBLIOGRAPHY  84  APPENDIX A - LUMBER SHAPE DEFECT SURVEY  87  APPENDIX B - TOLERANCE CALCULATION  90  APPENDIX C - PREPROCESSED DATA  92  C l PREPROCESSEDTRAINING DATA SET  92  C. 2 R AND F VALUES  92  2  APPENDIX D - DATA VALIDATION SUMMARY OF RESULTS D. 1 No CHANGE IN APPARATUS  97 97 97  D.2 LASER MEASUREMENT REPEATABILITY APPENDIX E - EXAMPLE CLASSIFICATION ANALYSIS OF SAMPLE CASE  v  100  LIST OF TABLES Table 1. Comparison of grades with respect to skip and wane Table 2. Top three most common causes of each machine shape defect Table 3. List of Equipment Table 4. Defects found in sample boards Table 5. Equipment Tolerances Table 6. Selection of control parameters for the Base Condition neural networks Table 7. Summary of results for detecting a change in the measuring apparatus Table 8. Summary of results for testing repeatability Table 9. Base Condition - Two Class Networks Table 10. Shuffled Condition - Two Class Networks Table 11. Shuffled Condition Second Round- Two Class Networks Table 12. Pruned Condition - Two Class Networks Table 13. Extended Duration Condition - Two Class Networks Table 14. Base Condition - Three Class Networks Table 15. Shuffled Condition - Three Class Networks Table 16. Pruned Condition - Three Class Networks Table 17. Extended Duration Condition - Three Class Networks Table 18. Sample excerpt of the output values for network N10EC3C Table 19. Classification data for top ten neural networks Table 20. Types of Misclassification  vi  30 31 40 43 45 57 59 60 65 66 66 67 67 68 68 69 70 75 78 79  LIST OF FIGURES Figure 1. Graphical representation of each machine shape defect 6 Figure 2. Annual production of respondents by class 9 Figure 3. Machine shape defect proportion of respondents by class 9 Figure 4. Probability of shape defect for all mills 11 Figure 5. Graphical representation of mismatch 11 Figure 6. Probability of shape defect by defect proportion 12 Figure 7. Probability of shape defect: high vs. low defect proportion mills 13 Figure 8. Probability of shape defect by annual production 14 Figure 9. Probability of shape defect: large vs. small production mills 15 Figure 10. Machine shape defects most serious in terms of the quality of the final product 16 Figure 11. Machine shape defect causes: benchmark results 18 Figure 12. Machine shape defect causes: thin snake 20 Figure 13. Machine shape defect causes: snipe 20 Figure 14. Machine shape defect causes: taper 21 Figure 15. Machine shape defect causes: fat snake 21 Figure 16. Machine shape defect causes: wedge 22 Figure 17. Machine shape defect causes: flare 22 Figure 18. Machine shape defect causes: mismatch 23 Figure 19. Combined ranking of machine shape defects 28 Figure 20. Photos of measurement equipment 40 Figure 21. Diagram of data acquisition hardware 41 Figure 22. Configuration of lasers measuring a sample board 42 Figure 23. Sketch of pushboard in the apparatus 44 Figure 24. Drawing of calibration block 46 Figure 25. Graphs of logged laser data from a No Snipe board sample 49 Figure 26. Graphs of logged laser data from a Snipe board sample 50 Figure 27. Model of regression lines on a sample board 52 Figure 28. Graphical representation of the snipe classification problem 56 Figure 29. Top Surface: F test values by class 61 Figure 30. Bottom surface: F test values by class 61 Figure 31. Top surface: coefficients of multiple determination by class 62 Figure 32. Bottom surface: coefficients of multiple determination by class 62 Figure 33. Illustration of N9SC2C, linear network with 6 input variables and 0 hidden units... 72 Figure 34. Illustration of N10BC2C, MLP network with 12 input variables and 7 hidden units 73 Figure 35. Illustration of N10BC3C, RBF network with 4 input variables and 1 hidden unit.... 73 vn  ACKNOWLEDGEMENT The execution of this project and completion of this thesis would not be possible without the support and assistance from several people. I have categorised them below in an attempt to avoid overlooking anyone. I thank each individual heartily for their time, consideration, patience and in some cases, their sense of humour. Academic Support  Thomas Maness Robert Kozak Shawn Mansfield Dallas Foley Darrell Wong Tom Wray Robert Furst Margaret Graham Gary Schajer Tony Besik Christine Lilley Industrial Support  Bruce Lehmann, Thin Kerf Technologies Ken Del Puppo, Slocan Group Todd Buchanan, CAM Technologies Nick Barrett, CAM Technologies Dennis Stein, Interfor Phil Warnery, Interfor Terry Maloff, Kalesnikoff Personal Support  Marc Sauze Jocelyn Sauze Marian Marinescu Ross Farrell Hauke Chrestin Leah Palmer Kristen Lane viii  BACKGROUND In a 1996 survey of B.C. sawmills, increasing fibre recovery was the primary reason for upgrading technology, while improving product quality was the second most important reason (Lee et al 1999). The results of this survey reflect a trend in the wood products industry that is the result of high raw material costs and a highly competitive commodity market. There is a recognition of the importance that technology plays in reducing production and material costs, controlling processes to meet customer demands, and increasing profits. The majority of sawmills in B.C. have adopted quality control methods to increase sawmill recovery and to maximise the value of output lumber by monitoring the process for inconsistencies in lumber sizes (Maness et al 1994). Once a quality control problem develops, it is essential to detect and rectify the problem as soon as possible in order to reduce waste and maximise production. A good sawing process produces lumber with a smooth flat surface and a uniform width and thickness down the length of each board. Problems with the sawing process can result in changes to the board's profile through the width and thickness. Different types of sawing problems cause various shape defects on a board's surface, resulting in different shape profiles. For instance, snake occurs when the blade teeth move laterally in the cut due to a lack of side stability (Thunell 1988). This instability can be the result of either the wood not being held firmly in place or the blade not being tensioned enough. As the resulting profile of the lumber is not uniform, the board requires further processing to remove the sawing defect, reducing recovery and increasing production costs. If the shape defect cannot be rectified or eliminated, the board is downgraded or it becomes waste. Quality control problems in the sawing process are usually detected by inspecting the board's sawn surface, and the cause of the problem is deduced by examining the board's surface characteristics, often manually. Depending on the quality control supervisor's experience and knowledge, troubleshooting a sawing problem can take a considerable amount of time, materials, effort and money. Fortunately, laser-based non-contact sensing systems can be developed to determine the profile or shape of a board in real-time. These systems would detect when a sawing problem is occurring, reducing the manual labour required in monitoring the sawmill process.  1  This project develops a proof-of-concept method to detect sawing defects by classifying board profiles into shape categories, using neural networks. Ultimately the system would incorporate a troubleshooting function to aid in determining the cause of the detected sawing defect.  Troubleshooting Methods Troubleshooting systems in the sawmill industry are capable of detecting process problems in real-time but currently are not capable of diagnosing them. Process troubleshooting is primarily accomplished by the quality control (QC) supervisor, using manual techniques. The QC supervisor often relies on his/her sawmill experience to recognise the problem and consequently, deduce its cause. If it were possible to set forth the cause and effect relationship to locate operating difficulties, troubleshooting a mill would be an easy matter. Such is not the case. The effects or symptoms of trouble often arise from any one of a number of possible causes, but more often from a combination of causes. Therefore, troubleshooting a sawmill often becomes a difficult task (Williston 1988). Below are several troubleshooting methods that have evolved to mitigate the difficulties with finding the causes of quality problems in the sawmill process. Unfortunately, these existing methods are time-consuming and cumbersome. Furthermore, the valuable information gained using these techniques risks being forgotten or lost, since sawmills rarely keep a database of quality problems and their causes. One of the most common troubleshooting methods in sawmills is measuring lumber with calipers in order to infer the location of a quality problem within the sawmill process. The QC supervisor collects a sample of boards, measures them and then investigates the running equipment to identify the delinquent part relating to the position of the defect on the boards (Lehmann Interview). Although this procedure takes a considerable amount of time and energy to complete, it continues to be used for lack of a more successful and efficient method. As defect problems tend to reoccur, the QC supervisor often recognises lumber defects and corrects the process problem after quickly verifying their cause. This knowledge is accumulated through experience and through discussions with maintenance and production personnel (Brown 1982). Further insight is developed by reviewing maintenance records, since the source of a particular cyclical problem can be deduced by relating the negative change in quality with some maintenance event during the same period. Consequently, the amount of time, material, effort and money spent troubleshooting a sawing problem depends largely on the QC supervisor's experience, knowledge, memory and availability. 2  Real-Time Lumber Size Control Systems Lumber size control is defined as: A systematic procedure that, properly carried out, identifies and locates problems occurring in sawing-machine centers, sawing systems, or setworks systems (Brown 1986). As a result, lumber size control is often used to troubleshoot quality problems caused by equipment malfunction, though "frequently, several remedies may be applied to one problem area" (Brown 1982). A detailed review by Maness et al (2002) discusses lumber size control systems and their statistical implementation in the sawmill industry; hence, only a brief description follows. Digital calipers are used to make several measurements per board for a sample of boards. The variation in the width and/or thickness of the lumber is determined from the data points. This variation is measured in terms of between-board and within-board standard deviation. The former is literally the variation in size from board to board in the process. It is often related to the performance of the setworks (Brown 1986). The latter is the variation between the measurements taken on the board itself for a sample of boards. It is often used to indicate how accurately a saw is cutting (Brown 1986). The difficulty is that the reliability of using within and between standard deviation alone to pinpoint specific manufacturing problems is low because the sawing process is highly complex and is affected by many factors (Lehmann Interview). For example, a saw cutting through a knot has been shown to deflect through the cant for several feet before recovering, due to the effect of the clearance gap. Analysing the within-board standard deviation indicates a problem with the sawing (depending on the number of samples taken), but in effect, the problem is more complex, involving the saw's interaction with natural defects like knots as well as the saw's parameters like thickness and tensioning (Lehmann 1993). Thus, the problem could also be the type or quality of the raw material. Furthermore, the specific problem with the saw is not known; it could be the saw setup or the choice of saw itself for the particular application. Essentially, there is not enough detail in the standard deviation statistic to establish the actual cause of the quality problem. Additional evidence from the lumber and its defects is essential to making this method reliable. While many sawmills rely on manual methods to control the size of lumber, such as caliper measurement, increasingly they are turning to real-time technology to monitor their processes. However, to date, these real-time systems are limited to identifying the fact that there is a problem in the process and, furthermore, do not take the shape of the lumber itself into account. These systems continuously measure lumber with optical scanners and create variability control 3  charts with the data to determine whether the process is operating in control. When the product output begins to show consistent defects, i.e. the variation becomes too great, an alarm sounds to indicate the manufacturing process is out of control (Maness 1992). In effect, these automatic systems are extensions of the manual caliper method and use generalisations based on the variation in the lumber dimensions rather than analysing the shape of the board in a logical manner. While there are several products on the market, which perform measurement and control charting, there is no evidence that any of them consider the lumber shape or include process troubleshooting functions. Therefore, shape defect detection and analysis is a new approach to controlling the sawmill process, as it models the surface of the lumber to logically determine whether a problem in the process has occurred and where it likely originated.  Thesis Objectives The two main objectives of this thesis are to study machine shape defects produced in British Columbia mills and to introduce a method for automatically distinguishing between the different defects. The different classes of machine shape defects in rough green lumber will be determined and defined in terms of their characteristics. In addition, these machine shape defects will be ranked with respect to their industrial significance in British Columbia. Furthermore, an experimental method to detect and to classify these machine shape defects, using neural networks, will be developed as a proof of concept  4  PART I - LUMBER SHAPE DEFECT SURVEY  Introduction Several different sawing processes can be employed by a sawmill. The primary breakdown process, which mainly converts the logs into cants, can be accomplished by a headrig bandsaw, a quad bandsaw, a chip'n'saw or an optimising canter, depending on the sophistication of the mill equipment, the size of the logs, the production goals of the mill, and the desired product mix (Williston, 1988). The secondary breakdown, which converts cants into flitches, is normally done by horizontal arbour or vertical arbour saws, either double or single arboured (one or two sets of saws process the cant). The flitches are then processed into lumber by an optimising edger or a reman edger, using chipper heads and/or saws to square the sides and cut multiple boards from an optimised pattern. This pattern is determined automatically at the optimiser edger and manually at the reman edger. Ideally, after edging, all of the lumber edges are parallel to each other and the ends of the boards are rectangular and in line with each other. However, sub-optimal occurrences in the sawing processes described above cause deviations from this ideal shape. Often these deviations are detected as off-size variations in thickness, and one particular defect shape is not necessarily distinguished from another in the downgrading process. Nevertheless, there are six general lumber shapes caused by sawing problems in the mill process. These defects, different from machine defects like torn grain or skip, are referred to as machine shape defects in this paper. Described below, they include snipe, flare, wedge, taper, thin snake and fat snake (Figure 1). Each machine shape defect is described in terms of a possible reason for its occurrence in order to emphasize not only its physical differences, but also its causal differences. The reason for this approach is that some process problems are suspected to cause certain machine shape defects more often than others. For instance, snipe is often associated with mis-timing of the hold-down rolls. When a feedroll lands on a cant too soon, the cant is forced into the saws at an angle, removing a triangular-shaped section from the end. Flare, the complement of snipe, is likely formed by the same phenomenon, but a triangular-shaped section is added onto the end of the board. When some type of misalignment prevents the cant from passing through the saws on a straight course, taper, a gradual thinning (or thickening) down the length of the piece is often visible on the lumber. Conversely, wedge, a gradual thinning (or thickening) across the width or through the thickness of the piece is often linked to problems with the saws, which prevent them from cutting the piece evenly. Snake is frequently attributed to the saws' movement during the cut; fat snake refers to a thicker board, while thin snake is its complement.  5  a. Thin snake b. Fat snake  e. Taper  f. Wedge  Figure 1. Graphical representation of each machine shape defect Although there are several ways in which machine shape defects can reduce the profitability of a sawmill, the extent to which machine shape defects have an economic impact on the entire industry is unknown. Their deviation from the ideal shape of lumber implies that an increase in machine shape defects will reduce mill recovery because of the additional steps in the process required to compensate for them. These steps include increasing target sizes, trimming, and remanufacturing. Thin snake and taper tend to be undersized boards. Typically, sawmills increase their lumber target sizes to compensate for undersizing problems in the process. Therefore, producing snake and taper in significant quantities is likely to cause increases in lumber target sizes. Sawmill simulations which estimate the value and volume recoveries from changes in lumber target sizes have demonstrated that reductions in target sizes significantly increase mill revenues. For instance, an interior B.C sawmill producing dimension lumber would increase its net revenue by $27,160/month for every 0.010-inch reduction in lumber target size (Maness and Lin 1995). Monitoring and controlling these machine shape defects will reduce problems in achieving target sizes and, therefore, result in economic benefits to the sawmilling industry. 6  Snipe and flare are likely to increase trimloss, since they tend to occur at either end of the boards and are trimmed off to avoid downgrading the lumber or to prevent problems in drying and planing. Although the contribution of these machine shape defects to trimloss has not been quantified, a mill experiencing a trimloss of 42 MMBF/year would increase its revenue by $134,400 with a 2% reduction (Thomlinson 1992). On the other hand, wedge and fat snake are remanufactured in order to remove or reduce the effects of the machine shape defects. In some instances, reprocessing the defective lumber at the trimmers or the remanufacturing edger will not be sufficient to rectify the machine shape defects, resulting in downgraded or rejected lumber. Therefore, the mill not only incurs additional reprocessing costs, but must also suffer a reduction in recovery value. O b j e c t i v e of t h e S u r v e y of B C S a w m i l l s  The objective of this research was to rank the aforementioned machine shape defects found in rough green lumber with respect to their industrial significance. A survey was designed and implemented to determine the industrial significance of each of these machine shape defects in British Columbia sawmills. By ranking these machine shape defects in terms of occurrence and by determining which ones are the most serious in terms of final quality, it can be established which of the machine shape defects have the greatest impact on the industry. This information is important for focussing subsequent research on the classification of machine shape defects. A further benefit of this survey lies in the identification of common causes of each machine shape defect within the sawmill process. This information can be used to develop a troubleshooting guide for analysing various process problems using machine shape defects. Methodology  To determine which of the machine shape defects most affected British Columbia sawmills, a facsimile survey was implemented in the fall of 2000 (Appendix A). The survey focussed on six machine shape defects commonly caused by the sawing process: snipe, flare, wedge, taper, thin snake and fat snake. To avoid a naming bias, the machine shape defects were only graphically represented on the survey. The questionnaire was designed to find out, not only which machine shape defects occur in the sawmill process, but also which ones occur the most frequently, what the defects are likely to be called in the mills, what grades they are typically assigned and what the most common causes are. Furthermore, the questionnaire was designed to identify the most serious machine shape defects with respect to the quality of the final product and to determine which machine shape defects, if any, the survey may have missed.  2  Assuming S200/MBF less the value of S40/MBF in chips.  7  In order to expedite the research process and to reduce costs, the surveys were distributed to British Columbia sawmills by facsimile. The distribution list was compiled from two different forest industry directories. An attempt was made to include every company listing sawmills in British Columbia. The graphical representation of the machine shape defects enabled the survey to be faxed out on one page plus a covering letter. In addition, it was expected that a short and simple questionnaire would encourage a good response rate. The initial survey was followed up three months later by a reminder letter with an identical questionnaire. In the first round, the survey was addressed to the Head Sawfiler &/or Planerman, while in the second round it was addressed to the Quality Control Supervisor &/or Head Sawfiler. The rationale for this decision was that many Planermen may have not responded in the first round because the questionnaire dealt only with the sawmill process.  Results Response  The survey was distributed to 110 mills throughout British Columbia, with 33 mills returning completed questionnaires for a response rate of 30%. These 33 mills were responsible for 33% BC softwood lumber production in 2000 (4.5 billion Board Feet out of a provincial total of 13.6 billion Bdft) . Responses came from mills located across BC and from both large and small forest companies. Figure 2 shows the annual production of responding sawmills, while Figure 3 shows the cross section of the responding mills in terms of proportion of machine shape defects. This proportion is the approximate percentage of machine shape defects produced annually in a respondent's sawmill process. 3  3  Council of Forest Industries (2000), 12/2000 year-to-date total B C softwood lumber production.  8  p 23  78.4  133.8  189.2  244.6  More  MMBF  Figure 2. Annual production of respondents by class  Defect %  Figure 3. Machine shape defect proportion of respondents by class Nonresponse bias was statistically tested by comparing the average production of machine shape defects between late and early respondents. Specifically, the mills were grouped according to whether they had responded to the first or the second faxing. The total production of machine shape defects for each mill was determined. These values were then used to calculate the average production of machine shape defects for each respective group of mills. From a t-test of the equality of two means (independent samples and equal variances), no  9  significant difference was observed at an alpha level of 0.05. This finding indicates that nonresponse bias was likely not present in this analysis.  Probability of Shape Defect  In order to identify the machine shape defects that most affect the industry, the importance of each defect must be established in terms of its probability of occurrence during the sawmill process. Each responding sawmill was given a graphic representation of six machine shape defects and asked to indicate which ones occurred or did not occur in its mill (Figure 1). The mill was then asked to rank the machine shape defects in order of frequency. The responses indicated that snipe and taper each had a 94% chance of occurring in the sawing process, the highest probability by far (Figure 4). Thin and fat snake followed with probabilities of 61% and 58%, respectively. Flare, with a 52% chance of occurring, occurred more frequently than wedge at 33%, while mismatch had the lowest probability of occurrence with 12%. Mismatch was not originally included in the survey questionnaire. However, several respondents indicated that mismatch occurred in their mills by sketching it in the available space, so it was included in the analysis (Figure 5). 4  4  Although the survey cover letter explained that the defects of interest were machine shape defects, and not natural  defects like wane, there was some confusion with wedge in three responses. These specific misinterpretations were not used in the analysis.  10  100% "° S 80% |  | 60%  3o o3 40%  £ O 20% 0%  St®  ^  J-  ^  ^  ^'  Shape Defect  Figure 4. Probability of shape defect for all mills  Figure 5. Graphical representation of mismatch Each mill was also asked to indicate their annual production and to approximate the proportion of their total production with machine shape defects. The size of the mills varied between 0.5 and 300 MMBF annual production, while the estimated proportions of machine shape defects varied between 0% and 10%. This information was used to categorise the types of mills experiencing machine shape defects based on annual production and machine shape defect proportion. To analyse the effect of machine shape defect proportion, the mills were separated into four defect proportion categories of 2.5% gradations, starting from 0%. The probability of each machine shape defect occurring in each category was calculated (Figure 6). A l l but one category of the mills surveyed have a 100% likelihood of producing snipe and taper. However, 1 1  in the remaining category, mills with defect proportions of 2.5% or less, snipe and taper still have over an 80% chance of occurring. Either type of snake is most likely produced in mills with defect proportions between 7.5% to 10.0%, at 100% probability, while flare is most likely to occur in mills with defect proportions between 5.0% and 10.0% (two categories), also at 100% probability. At over 50% probability, wedge has the highest chance of occurring in mills with defect proportions between 5% and 7.5%. Mismatch was only reported to occur in mills with defect proportions between 2.5% and 5%.  100%  •= 5  5  t  O  o  °-  • T h i n Snake  60%  •Snipe • Fat Snake •Flare  20%  •Taper • Wedge  0% 2.5%  5.0%  7.5%  10.0%  • Mismatch  Defect %  Figure 6. Probability of shape defect by defect proportion To simplify the relationship between machine shape defect types and machine shape defect proportions, the mills were divided into high defect and low defect proportion mills. The demarcation between high and low defect proportions was set at 3.75% by observing the results of the histogram combined with the data's measures of central tendency (Figure 7). As expected in this generalisation, all the machine shape defects have a higher probability of occurring in high defect proportion mills than in low defect proportion mills. Snipe and taper are the most likely to occur in each type of mill, followed by thin snake, flare and fat snake which have similar probabilities of occurring. Wedge, followed by mismatch, has the lowest probability of occurring. It is interesting to observe the large difference in probabilities between the two categories of mills for some of the defects. The probability of wedge occurring in a high defect proportion mill is almost four times its probability of occurring in a low defect proportion mill, while for mismatch, this probability is more than doubled. Snipe and taper each have over a 10% higher chance of occurrence in a high defect proportion mill than in a low 12  proportion mill. For thin snake, flare and fat snake, this difference in probabilities is less  Q High •Low  0%  20%  40%  60%  80%  100%  Probability of Occurrence  Figure 7. Probability of shape defect: high vs. low defect proportion mills To analyse the effect of mill size, the mills were separated into six annual production categories of 50 MMBF gradations, and the probability of each machine shape defect occurring for each category was calculated (Figure 8). Mills with less than 50 MMBF have the highest chance of producing flare at 100% probability. Mills, in all cases except those in the 50 to 100 MMBF category, have a 100% chance of producing snipe and taper. Both thin and fat snake are most likely to occur in mills between 150 and 200 MMBF at over 70% probability, while wedge is most likely to occur in mills between 200 to 250 MMBF at just under 35% probability. At 50% probability, mismatch was reported the most frequently by mills with annual productions between 250 to 300 MMBF.  13  MMBF  Figure 8. Probability of shape defect by annual production An interesting trend to note is that the probability of producing either type of snake increases as the annual production increases, until 200 MMBF when it begins to decrease. Flare, on the other hand, shows a downward trend as the annual production increases to 150 MMBF. After this point, its probability increases and then drops off sharply only to increase again at 250 MMBF. Wedge and mismatch both have loose upward trends as the annual production increases, while snipe and taper are consistently at 100% probability except for a dip at 50 MMBF. To simplify the relationship between machine shape defect and annual production, the mills were divided into large and small production mills. The demarcation between large and small production was set at 145 MMBF by combining the results of the histogram with the data's measures of central tendency (Figure 9). In this generalisation, all the machine shape defects have a higher probability of occurring in large production mills than in small production mills, except mismatch. Note that mismatch was not included in the original survey but was reported more often by the small production mills. Snipe and taper are the most likely defects to occur in each type of mill, followed by thin snake, flare and fat snake, which have similar probabilities of occurrence, and finally, by wedge and mismatch.  14  E Large • Small  0%  20%  40%  60%  80%  100%  Probability of Occurrence  Figure 9. Probability of shape defect: large vs. small production mills To determine whether mills with certain defects are more likely to experience other defects, the occurrence of each machine shape defect in mills was analysed with respect to each other. It was found that all of the mills with taper also produce snipe and vice versa. Less than 55% of the mills with snipe (and taper) have flare. 95% of mills with thin snake also have fat snake. Wedge has no discernible pattern with regard to the probability of other defects. Over 58% of the mills with snipe (and taper) experience fat snake, while over 61% of those mills with snipe (and taper) experience thin snake. 65% of mills with thin snake have flare, while over 68% of mills with fat snake have flare. Shape Defects Affecting Quality  In order to ascertain which machine shape defects most affect the industry, the machine shapes that cause the most problems with regard to the quality of the final product must be identified. To that end, each mill was asked to indicate which machine shape defects were the most serious in terms of the quality impacts on the final product (Figure 10). The responses clearly show that thin snake is the most serious machine shape defect in terms of quality for sawmills with 32.8% of the responses, followed by snipe with 24.1%. Taper is the third most serious for mills with 22.4% of the responses, followed by fat snake with 12.2%. Wedge and flare are considered the least serious problems in terms of final product quality, comprising 5.2% and 3.4% of the responses, respectively. No information was obtained on mismatch in this section of the survey.  15  Figure 10. Machine shape defects most serious in terms of the quality of the final product Grade  To determine the grades most associated with machine shape defects, mills were asked to cite the most common grade for each of the machine shape defects depicted in the questionnaire. Based on the responses, the most common grade for all machine shape defects was No. 2 Structural (No. 2). Nevertheless, in the majority of mills, thin snake and snipe also required resawing and trimming, respectively. Should thin snake not make No. 2 or be resawn, the lumber was downgraded to No. 3 Structural (No. 3), Economy or rejected as cull in some mills. The lowest grade assignment for snipe was No. 3 or Utility. Fat snake was planed out or resawn and occasionally rejected in mills where it did not make No. 2, while flare was typically sent to the trimmers or the remanufacturing edger. Taper was planed out, resawn or downgraded to No. 3 or Utility, but wedge was typically resawn or downgraded to Economy. There were no comments on the grade assignments for mismatch. C a u s e s of Shape Defect  To pinpoint critical process problems which produce machine shape defects, the respondents were asked to list the most common causes for the machine shape defects which occur in their sawmills. To facilitate analyses, these reported causes were organised by machine shape defect in a master list and then grouped into eight general categories. These categories are described below:  16  1. Piece Stability - causes associated with the movement and position of the wood piece as it is processed, including mechanisms to hold the piece in place such as linebars and pressrolls. 2. Saw Condition - causes related to the physical condition of the saws themselves, like worn or hot saws. 3. Alignment - causes associated with the machine's alignment, including misaligned rolls, saws and fences. 4. Saw Stability - causes associated with the movement of the saws in the cut, such as worn guides. 5. Feeding - causes related to the wood pieces' entering the saws improperly, like feedroll timing and overfeeding. 6. Piece Condition - causes associated with the physical condition of the wood piece itself, like frozen or bowed cants. 7. Setup - causes associated with the setup of the machine, including clearances. 8. Operator - causes associated with operator intervention or error. These categories of reported causes were analysed to determine the most common causes of machine shape defects. These percentages were calculated by dividing the total tally for each cause category from all the machine shape defects by the total number of causes reported. The results from this general analysis set a benchmark against which to judge the individual machine shape defects. These individual percentages were calculated by dividing the tally from the machine shape defect of interest for each cause category by the total number of causes reported for that machine shape defect. To illustrate, the formulae for Saw Condition and Thin Snake follows: ,„  % Reported Causes =  , _  Total tally of Saw Condition reported Total number of causes reported for all machine shape defects  „ ,.  ,  % Reported Causes of Thin Snake =  Tally of Saw Condition reported for Thin Snake Total number of causes reported for Thin Snake  The benchmark results show the most common process problems causing machine shape defects (Figure 11). Piece Stability and Saw Condition tie for the most prevalent cause of machine shape defects at 19.5% of all reported causes, followed closely by Alignment with 18.1%. Saw Stability and Feeding follow with 14% and 13.5%, respectively, outranking Piece Condition at  17  10.2%. With just over 5% of the reported causes combined, Setup and Operator are the least common causes of machine shape defects, at 3.7% and 1.4%, respectively.  Piece Stability S a w Condition Alignment <D to 3 (j  S a w Stability Feeding Piece Condition Setup Operator  0%  10%  20%  30%  40%  50%  % Reported C a u s e s  Figure 11. Machine shape defect causes: benchmark results Thin Snake: According to the survey results, Saw Condition is by far the most common cause of thin snake with just under 35% of the reported causes, followed by Saw Stability with just over 20% (Figure 12). Both easily exceed their Benchmark scores. Feeding and Piece Condition are fairly common causes with over 15% of the responses each, while Alignment is the least common cause of thin snake with just over 5% of the responses. Setup and Operator are not reported as causes of thin snake. Snipe: In keeping with the general results, Piece Stability is the most common cause of snipe, far exceeding the general score with 42% of the reported causes (Figure 13). In contrast, Saw Condition is the least common cause, since Operator is not reported to cause snipe. Alignment, second after Piece Stability, is less than half as common with 20% of the reported causes. Feeding, at 12% of reported causes, is the third most common cause of snipe. Taper: Alignment is the most common cause of taper at 31.6% of the reported causes (Figure 14). Piece Stability, Saw Condition and Feeding tie for second at 15.8% each, half as common as Alignment. Saw Stability, third, is half as common at 7.9% of the reported causes. Like the benchmark results, Operator is the least common cause.  18  Fat Snake: As with thin snake, Saw Condition is by far the most common cause of fat snake at 34.5% (Figure 15). However, Piece Condition follows with just under 20% of the reported causes. Saw Stability is the third most common cause with 17.1%, while Piece Stability is the least common cause with 4.9%. Setup and Operator are not reported to cause fat snake. Aside from these last two, Piece Stability and Alignment are the only causes to lag behind their respective Benchmark scores. Wedge: Saw Condition is also the leading cause of wedge with 35.7% of the reported causes, followed by Alignment with 28.6% and Saw Stability with 14.3% (Figure 16). Here, the sequence somewhat mimics the Benchmark results. However, Piece Stability, Feeding and Piece Condition are the least common with 7.1% each, given that Setup and Operator do not rate as causes. Flare: Flare, like snipe, has Piece Stability as the most common cause, followed by Alignment. (Figure 17). Saw Stability and Feeding are the third most common causes of flare at 13.8%, while Piece Condition and Operator are the least common at 6.9%. Flare is the only machine shape defect not to have Saw Condition reported as a cause. The causes exceeding their respective Benchmarks significantly are Piece Condition, Setup and Operator. Mismatch: The reported causes for mismatch include Saw Condition, Alignment and Saw Stability (Figure 18). Saw Condition and Alignment are the leading causes with 40% each, while Saw Stability is half as common at 20%. This sequence is somewhat consistent with the Benchmark results, though no other causes are reported and the scores are much higher.  19  Figure 13. Machine shape defect causes: snipe  20  Figure 14. Machine shape defect causes: taper  0%  10%  20%  30%  % Reported C a u s e s  Figure 15. Machine shape defect causes: fat snake  21  40%  50%  Figure 17. Machine shape defect causes: flare  22  0%  10%  20%  30%  40%  50%  % Reported C a u s e s Figure 18. Machine shape defect causes: mismatch Some of the reported common causes grouped into the eight general categories highlight problems with a specific part of the equipment, in addition to a process problem. These equipment-specific causes show up in different forms across the eight general categories. Three machine shape defects show this pattern of equipment-specific cross-category causes. They are snipe, flare and taper. In the case of snipe, the predominant equipment-specific cause is problems with the rolls, including feedroll timing, outfeed roll misalignment or worn bed rolls, for example. These roll problems account for 24.0% of the total reported causes for snipe. Flare is also caused by problems related to the rolls, like roll pressure, feedroll timing and outfeed roll misalignment. These roll equipment problems arise in several of the general categories and account for 24.1% of the total reported causes of flare. For taper, the predominant cross-category cause relates to problems with the linebar. These linebar equipment problems account for 15.8% of the total reported causes of taper and include, but are not limited to, linebar misalignment, pieces not tight to the linebar, and internal linebar failure. Nomenclature To ensure that the nomenclature of the machine shape defects in the survey was consistent with industry terminology and to determine whether it is consistent throughout the industry, the respondents were asked to label the graphical representation of each defect with its common mill name (Figure 1 on p.6). Thin snake and fat snake were both primarily called 'snake' or a variation of snake such as 'snaking' or 'snakey sawn' (Figure 1. a & b). One mill referred to thin snake as 'negative  23  snake' and fat snake as 'positive snake', while two mills differentiated between the two snakes by qualifying them as 'thin' or 'thick' boards. Otherwise, no distinction was made by the mills. Interestingly, two mills named the snake defect with respect to its process cause. The first mill employed the term, 'resaw dip', which relates to problems with the handsaws on the resaw. The second mill used the term, 'linebar deviation', referring to cants not being properly held against the gang saw linebar. Snipe was primarily called 'snipe' or a variation qualified by the term, 'end', such as 'sniped end' or 'end snipe' (Figure l.c). One mill described the end of the board in question by using the terms 'lead end snipe' and 'tail end snipe' to locate the snipe. The two naming exceptions were 'tapered end' and 'bull head'. Flare was mainly called 'flare', though three mills did not differentiate it from snipe (Figure 1. d). The two exceptions were 'bump' and 'club foot end'. Taper was mainly called 'taper'. However, slightly less than a third of the mills referred to it as 'wedge', while three mills used the two terms interchangeably (Figure 1. e). One mill called it 'bevel'. Yet another mill pointed to the cause of the taper defect by naming it 'thin line bar board', referring to problems with the cant at the cant optimizer. Wedge was mainly called 'wedge', although several mills used the term 'bevel', and one mill interchanged these two terms (Figure 1. f). One mill incorrectly referred to this defect as 'snipe', while another referred to it as 'taper'. Three responses demonstrated confusion with the graphical representation of wedge by naming natural defects. Two of these mills called it 'wane', while the third called it 'sidecut', which can result in waned edges. As mentioned previously, mismatch was not originally included in the survey; however, several respondents sketched in the defect and labelled it 'mismatch' or 'saw mismatch' (Figure 5). One mill labelled it 'step'.  24  Discussion Probability of Shape Defect  Respondents ranked machine shape defects with regard to frequency of occurrence as follows: 1. Snipe and Taper 2. Thin Snake 3. Fat Snake 4. Flare 5. Wedge 6. Mismatch From the results of the response analysis, it is clear that snipe and taper are the most frequently occurring machine shape defects in British Columbia sawmills. They are distantly followed by thin and fat snake, which lag behind by over 30% of the responses. However, this evidence alone does not ensure that snipe and taper should be the primary focus of subsequent research because their impact on the quality of the final product must also be taken into account. A machine shape defect that occurs the most frequently, but that can be easily corrected in subsequent processes such as planing, is less threatening to the profitability of a mill than one that occurs frequently, but must be downgraded or scrapped. Nonetheless, this information brings to light the fact that a problem with these types of defects most assuredly exists, highlighting the need to improve certain aspects of the sawing process in BC mills. Characterising BC sawmills according to machine shape defects and annual production shows that, for each category of mill, but one, there is over a 20% probability of producing at least five types of machine shape defects. The mills making up the 5 to 7.5% defect proportion category and those making up the 150 to 200 MMBF annual production category generally have the highest probabilities of shape defect, suggesting that they have the most work ahead of them with respect to controlling their sawing processes and reducing the instances of machine shape defects. Further research is required to pinpoint the reasons why mills in these two particular categories experience more problems with machine shape defects than the others. Simplifying the sawmill categories into high and low defect proportions produces results in keeping with expectations. Lower defect proportion mills have a lower probability of producing machine shape defects, since it is presumed that these mills have better control over the quality of their sawing process. The fact that they still experience problems with some of these defects suggests that the causes are either difficult to control or are not known to the mills. Separating the mills into large and small production mills is also consistent with expectations. Large production mills likely experience more problems with machine shape defects because faster, 25  high volume processes create difficulties in detecting and correcting problems, often before significant volumes of defective lumber are produced. Very few mills, if any, have real-time quality control systems, which would address this difficulty in controlling the process. It was found that most mills experiencing problems with one type of machine shape defect are likely to encounter problems with other types of machine shape defects as well. These cooccurrences are linked to the fact that machine shape defects share common causes with each other (p.18- 19). Shape Defects Affecting Quality  The processes normally following primary and secondary breakdown in a sawmill are edging, trimming, drying and planing. It is in trimming and planing that a machine shape defect is often corrected, although certain defects may only effectively be corrected via resawing or remanufacturing. It is often less desirable to trim rather than to remanufacture for two reasons. First, in trimming a piece, it becomes shorter. This process improves its grade but reduces final product options. On the other hand, remanufacturing removes the undesirable characteristic of the piece, allowing it to remain full length in most instances. Second, trim ends result in trimloss and reduce the recovery of a mill (as discussed above). In contrast, remanufacturing yields higher recovery rates, although an extra step is applied to the piece. The trade-off is the costs associated with the additional steps in remanufacturing. Respondents ranked machine shape defects with regard to quality as follows: 1. Thin Snake 2. Snipe 3. Taper 4. Fat Snake 5. Wedge 6. Flare In light of the above discussion, it is logical that thin snake is the most serious machine defect in terms of the quality of the final product, since it is the most difficult machine shape defect to correct in subsequent processes. If not remanufactured into 1" boards, thin snake results in skip at the planer and is downgraded according to the degree and depth of scantness (NLGA 2000). Snipe, the second most serious defect, also results in skip at the planer, unless the more scant areas of the board are trimmed off. In the case of taper, the third most serious machine shape defect, a scant end is removed at the trimmers, but an over-thick end is either planed out or resawn. Also over-thick in areas, fat snake is corrected through planing or remanufacturing to remove the excess ripples of wood from the piece, while flare is corrected by remanufacturing 26  or by trimming to remove the additional section of wood. The most problematic type of wedge is a gradual thinning (or thickening) across the width of the piece, as opposed to a gradual thinning through the thickness which is easily corrected at the edger. However, wedge across the width can normally be corrected by resawing, unless the thin side is too scant, a problem which results in significant downgrading as wane. Combining the quality rankings with the frequency of occurrence rankings identifies the machine shape defects with the greatest impact on the sawmill process (Figure 19). The machine shape defects in the upper right hand quadrant of the graph appear to have the most impact on the industry in BC. However, a complete assessment must also incorporate the influence of lumber downgrading. The combined results show that thin snake is the most serious in terms of the quality of the final product, but the third most frequently occurring defect. Thus, BC mills have recognised the problems with thin snake and are addressing them. However, snipe also requires attention. It is one of the most frequently occurring defects and the second most serious in terms of the quality of the final product. In addressing the problems causing snipe, mills will increase their recovery by decreasing the trimloss incurred from correcting snipe downstream. Taper, the other most frequently occurring machine shape defect, is the third most serious in terms of the quality of the final product. Likely its reprocessing appears insignificant because a scant end is removed at the trimmers or an over-thick end is planed out or resawn.  27  Most Common Tamper Snipe  T h i n1 ^na Snake  Fat at Sn Snake  Least Serious  Most  •-  Serious  Flan  are Wedge  Least Common Figure 19. C o m b i n e d ranking o f machine shape defects  Grade Grading rules set m i n i m u m guidelines for the final dimensions and characteristics o f each board. The grading rules employed i n this research are set by the National L u m b e r Grades Authority (2000). M a c h i n e shape defects are not specifically characterised by these grading rules, but are limited by certain characteristics specified i n the rules. The m a i n characteristics applied i n grading machine shape defects are skip and wane. Skip accounts for scantness resulting i n areas on the face or edge o f a piece that fail to surface clean at the planer. Wane accounts for a "lack o f w o o d from any cause on the edge or corner o f a piece o f lumber" ( N L G A 2000). Note that wane itself is a natural defect, not a machine shape defect. H o w e v e r , the two are treated the same way i n accordance with the grading rules.  Downgrading o f machine shape defects takes place w h e n the defect characteristics exceed the m i n i m u m guidelines for N o . 2 Structural, the most commonly cited grade. A s some machine shape defects are downgraded to N o . 3 or Utility, and others to E c o n o m y , a short comparison o f the guidelines for these grades is i n order (Table 1). It is immediately apparent that the guidelines for N o . 3 and Utility are identical w i t h respect to skip and wane. A s expected, the tolerances for more skip and larger wane increase as the grade decreases from N o . 2 to N o . 3 (or 28  Utility) to Economy. However, Economy is the only grade with an allowable depth of skip exceeding 1/8". It is also the only grade allowing through portions, which are essentially gaps in the lumber. An example from the survey response highlights the cost of downgrading lumber associated with machine shape defects. Based on the survey data, the average BC sawmill produces approximately 5.65 MMBF of machine shaped defective lumber. If only 3% of the machine shape defects result in downgrading the lumber from No. 2 to No. 3 Structural, the average sawmill loses just over $12,700 every year . 5  That snipe and taper are usually only downgraded to No. 3 or Utility suggests that the mills either correct the machine shape defects in downstream processes or misjudge the pieces' final thicknesses. For the most part, snipe and taper result in hit or miss skip, and in less than 10% of the cases, heavy skip at the planer. Some mills may have the same problem misjudging the final thicknesses of lumber with thin snake, since it is downgraded to No. 3. Other mills may not consider it worthwhile to resaw the snaked lumber into 1" and downgrade it to Economy or even reject it. Wedge, on the other hand, is likely easier to judge since it is graded as a wane characteristic. The piece is either successfully remanufactured or downgraded to Economy, which has a fairly generous wane allowance. Fat snake also appears to be easier to manage as it is generally corrected and infrequently rejected. Neither fat snake nor flare appears to be downgraded, likely because they are simple machine shape defects to correct.  5  (0.03 * 5.65MMBF/year*Sdiff c e / M B F = 170MBF*75=12,712.5$/yr) Based on a $75 difference between No. 2  and N o . 3 random lengths  29  Skip  Wane  No. 2 Structural  No. 3 Structural  Utility  Economy  hit and miss*, with a maximum of 5% of the pieces containing hit or miss** or heavy*** skip 2' or less in length. 1/3 thickness and 1/3 width full length, or equivalent on each face. Wane not to exceed 2/3 thickness or 1/2 width up to 1/4 length.  hit or miss with a maximum of 10% of the pieces containing heavy skips.  hit or miss with a maximum of 10% of the pieces containing heavy skips.  1/4" scant in thickness and/or width. Not limited in length.  1/2 thickness and 1/2 width full length, or equivalent on each face. Wane not to exceed 7/8 or 3/4 width for up to 1/4 length.  1/2 thickness and 1/2 width full length, or equivalent on each face. Wane not to exceed 7/8 or 3/4 width for up to 1/4 length.  3/4 width, full length. If through the edge, equivalent to area of 75% of cross-section. Through portion not to exceed 2' in length. If across the face, 1/2 width must not exceed 1/4" scant in thickness for 1/3 length or, as equivalent longer.  * Hit and miss skip is a series of skips not over 1/16" deep with surfaced areas in between * * Hit or miss skip means completely or partly surfaced or entirely rough. May be 1/16" scant. * * * Heavy skip is not over 1/8" in depth  Table 1. Comparison of grades with respect to skip and wane  30  C a u s e s of Shape Defect  The fact that Piece Stability and Saw Condition, followed closely by Alignment, are the most common causes of machine shape defect draws attention to two issues (Figure 11). First, these process problems are the most difficult to control and, second, they generally require the most attention to reduce or eliminate machine shape defects. However, it is also important to consider which specific shape defects are produced by these problems, since a more frequently occurring defect indicates that its cause is not being addressed or recognised by the mill. Results of the survey indicated that Piece Stability and Saw Condition rate as the most common causes of individual machine shape defects, with the exception of taper (Table 2). Piece Stability is the primary cause of snipe and flare, while Saw Condition is the primary cause of thin snake, fat snake and wedge. Alignment is the primary cause of taper, but both Alignment and Saw Condition are responsible for causing mismatch. Machine Shape Defects Top Three Causes 1  2  3  Thin Snake  Snipe  Taper  Alignment Saw Condition  Saw Condition  Piece Stability  Saw Stability  Alignment Saw Condition, Piece Stability & Feeding Feeding Saw Stability  Piece Condition & Feeding  Fat Snake  Piece Stability  Saw Stability  Wedge  Flare  Mismatc h  Saw Condition & Alignment Alignment Alignment Saw Stability Saw Condition  Piece Stability  Saw Stability  Saw Stability & Feeding  Table 2. Top three most common causes of each machine shape defect That snipe and flare are most commonly caused by problems associated with the stability of the piece follows expectations in two respects. First, they have the same types of causes because they are essentially each other's respective shape complements. Second, they both are the result of problems like a mis-timed roll forcing the piece to move during the sawing process. However, the fact that both types of snake and wedge are most commonly caused by problems relating to the condition of the saw is unexpected, since it was thought that they were created by 31  the saws' moving during cutting. In retrospect, this finding is reasonable because a hot or warped saw does not make a straight cut and results in a wavy surface. It was expected that Alignment problems most commonly cause taper because this defect results from sawing the piece at a steady angle. It is not surprising that mismatch has both Saw Condition and Alignment as its most common causes, since it is often attributed to either improperly filed saws on double-arbour edgers or problems with offset in aligning the two sets of saws (Williston 1988). Snipe and taper are the most frequently occurring machine shape defects, indicating that Piece Stability and Alignment are difficult problems and require attention from the mills. Nonetheless, Saw Condition still needs to be improved in order to reduce the frequency of thin snake. Piece Stability is very difficult to control in the sawing process because of equipment vibration and jarring movements from the hold-down mechanisms and from the logs or cants themselves. Several different methods of controlling the piece through the sawing process have been developed. Generally, most systems rely on some type of hold-down mechanism to steady the piece as well as some type of mechanism to keep it centered as it is processed. On a headrig, the knees or dogs hold the log in place as it is sawn, often against a fence, while a canter usually has a chain or slat bed feed with hold-down rolls interspersed strategically down its length to keep the cant in its optimal position (Williston 1988). One centering device, which is still used on some older models of Chip 'n' Saws, chips a 2x4 key in the bottom of the log to hold the piece centered through the canter. This key acts like a wheel on a rail and is later sawn off to make a low grade 2x4. With the increased demand and value of fibre, this concept was refined to use splines to center the log through the canter. For arbour saws, the cant is transported on a slat bed or narrow chain, using a rollcase and linebar or a rollcase and centre-feed system to position the cant in the saws. A sawmill's best strategy may be to maintain a rigorous preventative maintenance schedule to keep the hold-down mechanisms well maintained, since wear reduces their control over the piece and precision in positioning it. Monitoring the lumber on the outfeed for defect shapes would help to indicate when the stability problem gets out of control. Alignment is also very difficult to control due to machine vibration and jarring from the piece being processed. The taut-string method of aligning the equipment by ensuring that the rolls, saws and/or chipping heads are lined up and centred in the vertical and horizontal planes has long been replaced by optical methods using lasers (Williston 1988). However, the alignment performed on the equipment is still static, whereas a sawing operation is dynamic, and therefore,  32  1  requires some type of dynamic test to ensure that the sawing and chipping actions take place in the manner prescribed by the optimisation system or anticipated by the operator. Because saws are constantly running and their performance is affected by many factors, such as feedspeed and wood species, Saw Condition is difficult to control as well. Most sawmills mitigate problems with their saws by changing them regularly throughout the shift. They are heavily reliant on their Sawfiler's ability to recognise the fundamental cause of sawing quality problems and on his/her expertise to resolve them effectively and economically. Not only do Sawfilers consider the size, shape and metallurgy of the saw required for a particular situation, but they also design the saw tips and gullets to meet specific needs (Lehmann 1993). Problems with rolls and linebars are noteworthy in terms of the number of responses as a cause of snipe, flare and taper. Preventative maintenance measures should, therefore, focus on these particular components of the sawing process in order to reduce incidents of these machine shape defects. From the data, it appears that monitoring the timing and alignment of the rolls, as well as the condition and alignment of the linebar, would prove beneficial. Nomenclature  The nomenclature applied to machine shape defects is fairly consistent throughout BC sawmills. This result was surprising given that there are no set definitions of these defects in the Standard Grading Rules For Canadian Lumber and no other definition sources have been located to date. However, grading competitions and regional lumber grade inspectors may contribute to the uniformity in naming defects. Furthermore, the sawmilling community in British Columbia is fairly close-knit and information is often shared between mills, especially as employees change jobs. Finally, the quality control programs in British Columbia's educational institutions may influence the nomenclature in mills as the number of graduates increases in the sawmill workforce. Despite this consistency, difficulties did arise in labelling the graphical representation of taper. It was frequently confused with wedge. This confusion is likely due to the influence of ordinary expressions rather than a misinterpretation of the sketch. For instance, a tapered piece of lumber resembles a wedge used to block a door open. When this survey was designed, there was some discussion with regard to using the term wedge rather than bevel to describe this machine shape defect. Thus, it is not surprising that several mills used the latter term to describe this shape. However, it is difficult to understand how it was confused with snipe. 33  Conclusions Based on this survey, the majority of BC sawmills produce machine shape defects and recognise that these defects have the potential to affect the qualify of their final products. However, the mills surveyed take different approaches with respect to the machine shape defects they encounter. Often the machine shape defect is corrected by downstream processes, like trimming, or downgraded in the planermill. Unfortunately, these both incur negative consequences to the profitability of a mill through reduced recoveries and decreased product values. An example from the survey response highlights the cost of downgrading lumber. Based on the survey data, the average BC sawmill produces approximately 5.65 MMBF of machine shaped defective lumber annually. If only 3% of the machine shape defects result in downgrading the lumber from No. 2 to No. 3 Structural, the average sawmill loses just over $12,700 every year. While many mills recognise the causes of these machine shape defects, it is not clear whether the difficulty of addressing the processing problem impedes them from preventing the defect or whether preventing the defect is not a priority because it can be corrected downstream. However, the fact that thin snake is considered the most serious in terms of the quality of the final product, but is actually the third most frequently occurring defect, indicates that BC mills have recognised the problems with thin snake and are addressing them. Nevertheless, snipe also requires attention. Not only is it one of the most frequently occurring defects and the second most serious in terms of the quality of the final product, but in correcting snipe downstream, a mill's recovery is decreased through increased trimloss. Taper, the other most frequently occurring machine shape defect, is considered the third most serious in terms of the quality of the final product. Likely, it has not been addressed because, like snipe, it is only downgraded to No. 3 or Utility. In addition, its reprocessing may appear insignificant, since a scant end is removed at the trimmers or an over-thick end is planed out or resawn. Characterising the mills according to their proportions of machine shape defects encountered and annual productions shows that each category of mill, with one exception, has over a 20% probability of producing at least five types of machine shape defects. The mills making up the 5 to 7.5% defect proportion category and those making up the 150 to 200 MMBF annual production category generally have the highest probabilities of machine shape defects, suggesting that they have the most work ahead of them with respect to controlling their sawing processes and reducing the instances of machine shape defects. Further research is required to pinpoint the reasons why sawmills in these two particular categories experience more problems with machine shape defects than others.  34  This survey has identified three priority areas of focus for mills desiring to reduce their production of machine shape defects. These critical areas are Piece Stability, Saw Condition and Alignment. Stabilising the piece will significantly reduce snipe and flare, while improving and monitoring the condition of the saw will significantly reduce thin snake, fat snake and wedge. Concentrating on machine alignment will decrease taper significantly, as well as reduce mismatch, which is also affected by the stability of the piece. Choosing to concentrate on any of these improvements will have ripple effects on reducing the production of machine shape defects in general because they share common causes.  35  PART II - CLASSIFICATION OF MACHINE SHAPE DEFECTS  Introduction Several different sawing processes are employed by a sawmill. The primary breakdown process, which converts the logs into cants, can be accomplished by a headrig bandsaw, a quad bandsaw, a chip'n'saw or an optimising canter, depending on the sophistication of the mill equipment, the size of the logs, the production goals of the mill, and the desired product mix (Williston 1988). The secondary breakdown, which converts cants into flitches, is normally done by horizontal arbour or vertical arbour saws, either double or single arboured (one or two sets of saws process the cant). The flitches are then processed into lumber by an optimising edger or a reman edger, using chipper heads and/or saws to square the sides and cut multiple boards from an optimised pattern. This pattern is determined automatically at the optimiser edger and manually at the reman edger. Ideally, after edging, all of the lumber edges are parallel to each other and after trimming, the ends of the boards are rectangular and in line with each other. However, sub-optimal occurrences in the sawing processes described above cause deviations from this ideal shape. Often these deviations are detected as off-size variations in thickness, and one particular defect shape is not necessarily distinguished from another in the downgrading process. Part I of this thesis identified six general lumber shapes caused by sawing problems in the mill process. These defects, different from machine defects like torn grain or skip, are referred to as machine shape defects. They are snipe, flare, wedge, taper, thin snake and fat snake (Figure 1). Classifying these machine shape defects is advantageous for several reasons. First, as each type of machine shape defect is primarily caused by a particular combination of process problems, their classification facilitates process troubleshooting. For instance, producing large quantities of snipe indicates that there is a piece stability or alignment problem, from a linebar failure or misaligned roll for example (Table 2). Second, the type of machine shape defect prevalent in the mill process shows how the mill is affected by the process problem. With snipe for example, downstream processes like trimming are impacted by increased flow, or the amount of lumber downgraded to No. 3 Structural or Utility is increased. Third, the frequency of occurrence of these machine shape defects in the sawmill process indicates the magnitude of the problem in the mill. For example, if snipe accounts for approximately 23% of the 5.65 MMBF machine shape defective lumber produced by an average BC sawmill every year, the sawmill can estimate the cost of the snipe defect problem. If the mill frequently downgrades sniped 6  6  Based on percent chance of occurring in the sawing process (Figure 4)  36  lumber to No. 3 or Utility, the cost of the problem is based on the difference between selling prices. However, if the mill normally trims off the sniped end, then that cost is calculated based on decreased recovery and increased downstream processing. A neural network is a generalisable model built from a set of training data, using experience rather than explicit rules. The model is actually a set of functions, called units, which are linked together by weights that describe the effect each unit will have on the overall model. For a complete understanding of these functions, please refer to Bishop's Neural Networks for Pattern Recognition (1995). Using neural networks is a desirable way to classify machine shape defects because their pattern recognition capability enables them to classify systems that are too complex to model with rules (Swingler 1996). Neural networks also have the potential to perform these classification tasks in real-time, while maintaining their fault tolerance performance (Skrzypek 1991). They provide better fail safety or fault tolerance than classic sequential computing systems. For instance, rather than stalling the whole system, problems occurring in one part of the neural network can be overcome because the information is distributed throughout the network with mainly locally connected nodes (Tulunay 1991). Experimentation has been done on diagnostic systems similar to grading where there are large sets of rules used to recognise problems. For example, on an experimental basis, neural classifiers performed the recognition task to diagnose system faults in an automotive control system (Williams 1993; Marko 1989). Furthermore, neural networks appear to be relatively easy and inexpensive to implement and to maintain. In this application, additional hardware and software requirements are minimal. However, programming required beyond the off-the-shelf software depends on the level of automation desired by the user or sawmill. Once trained and installed, neural networks demonstrate a certain flexibility for changing conditions and ability to accommodate defective hardware, useful attributes in the sawmill environment (Tulunay 1991). This pilot experiment uses neural networks to classify sample boards by the machine shape defect(s) they contain. The study focusses on the snipe defect because it was determined to have the greatest impact of these six machine shape defects on the Wood Products industry in the results of the survey in Part I of this thesis. Objective of Neural Network Classification  This research focusses on developing a method to detect machine shape defects in the sawmill. The objective is to establish whether neural network classification of the aforementioned machine shape defects found in rough green lumber is feasible. In particular, can neural 37  networks be trained to distinguish between those boards which contain snipe (or a combination of snipe and another defect) and those which do not? This question will be addressed by a proof of concept study, not by a full development. For this reason, this experiment is restricted to one machine shape defect: snipe. As mentioned previously, snipe is the machine shape defect of choice because it was found to be the most important in British Columbia sawmills in Part I of this thesis. The main benefit of employing neural networks to classify machine shape defects is the potential for automated real-time identification of these defects in the sawmill process.  38  Methodology Sample Measurement Measurement Equipment  A self-contained measuring apparatus was constructed to support the measuring equipment and convey the sample boards, driven by an automatic feeder (Figure 20). Three laser displacement sensors (lasers) were positioned above the measuring bed and three were positioned below. A table nearby held the data acquisition hardware, including the data acquisition card and PC (Table 3). Cables ran from the lasers to the data acquisition hardware to the PC.  39  b. Overview of measuring apparatus (centre) and data acquisition hardware (right) Figure 20. Photos of measurement equipment Table 3. List of Equipment Equipment Type  Equipment Description  Laser Data Acquisition Card Desktop PC Automatic Feeder Measuring Apparatus Power Converter Software  LDS Displacement sensor model LDS 80/10 6 1 National Instruments PCI-MIO-16E-4  Qty  1 1 1 . 1 1 1 1 1  Pentium 100MHz Pertio power feeder Custom design draws 110 V DAQ Sim NI DAQ 6.5 Driver Windows NT platform Microsoft Excel  The lasers used were Dynavision LDS Displacement sensors, model LDS 80/10. These lasers were capable of rates up to 500kHz. They automatically compensated for differences in the board's surface and colour by varying the power to the laser diode (LMI). The laser beam emitted from a laser diode struck the board surface and reflected onto a position sensitive detector. The current output signals from this detector were translated into distance by signal processing electronics, in this case, a data acquisition card (Figure 21). The data acquisition card was a National Instruments PCI-MIO-16E-4 board with a NI-DAQ 6.5 driver, using a Windows NT platform. It used custom designed Visual C++ software. The software program, called DAQsim, translated the current output signal from each laser into a distance measurement by dividing the signal into intervals called divisions. In this case, the division size was 2.4 micrometers for the laser measurement range of 10 mm and the data acquisition card resolution of 1 in 4096. The resolution was the smallest change in the signal from the lasers that can be detected by the acquisition card. Therefore, the conversion from divisions to metric used the following formula: measurement range in m * (data acquisition card resolution)" = micrometers/division So, 10"m * (4096)" =2.4 micrometers/division 1  3  1  40  In this application of DAQSim, six channels logged the data from the six lasers. Each channel was set to a channel size of 10, a rate of 600 points per second and a queue size of 10. The laser data was logged into a comma separated file (*.csv) for each sample board and can be imported into Microsoft Access or Microsoft Excel.  DATA  ACQUISITION GRD  -O GRD  G-  -G  OUTPUT  CHANNEL  G"+  PC 5 V  INPUT POWER CONVERTER  LASER  0-  ±15V  BOARD  Figure 21. Diagram of data acquisition hardware Six lasers measured a sample board continuously. Three lasers were lined up across the width of the top of the board and three were lined up across the width of the bottom (Figure 22). The configuration of the lasers enabled both the top and the bottom surfaces of the sample board to be measured in three places: two sides and the center. The lasers were staggered for two reasons. Their size was relatively large compared to the width of the board and the lasers had to be oriented perpendicular to the feed direction of the sample board.  41  FEED  SECTION LOOKING EAST  Figure 22. Configuration of lasers measuring a sample board The lasers were fixed in the measurement apparatus at an operating distance of 80 mm from the surface of the rough green lumber sample. A rise or dip in the surface of the sample was detected by a corresponding change in the data readings. Using the lasers in pairs, this profile measurement was made at the same rate along the board. The thickness of the lumber sample can also be determined from each pair of lasers; however, this research was focussed on analysing the shapes of the lumber surfaces, not the absolute thickness changes in the lumber.  42  Sample Boards and Measurement Process  One hundred and three rough green trim ends were sampled randomly from a mill experiencing difficulty processing frozen wood. The resulting sample boards ranged from 7 5/8" to 24 5/16" in length. The board defects were primarily machine shape defects, although some boards also had natural, seasoning and/or manufacturing defects (Table 4). The sample boards were manually assessed for machine shape defects by the author. The number and types of machine shape defects were recorded with the sample board number for the categorization required in preprocessing. Snipe was the most prevalent defect, found in over 40% of the samples, while very few samples had no defects at all. Table 4. Defects found in sample boards Defect  Classification*  # Samples""  machine shape manufacturing manufacturing natural machine shape manufacturing manufacturing machine shape manufacturing N/A manufacturing manufacturing seasoning  43 31 30 17 16 14 8 4 4 3 2 2 2  1  Characteristic*  snipe machine gouge mismatch wane wedge skip or roughness sawcut taper knot tearout no defect thin pickaroon hole split * (NLGA  1998) except machine shape defects  + Number exceeds total of 103 due to defect combinations in samples  The automatic feeder conveyed the sample boards through the measuring range of the six lasers at a steady rate. From experimentation, it was calculated that the measurement rate was 46 points per inch (18.11 points per cm). Because the automatic feed rollers could not drive the sample boards completely through the lasers, a pushboard followed each sample board through the rollers to ensure each sample board was completely scanned by the lasers (Figure 23). The pushboard was also designed to signal the end of each sample board. The 5 mm notches on the top and bottom surfaces of the pushboard signalled the end of the sample board and the beginning of the pushboard by spiking the data to the out-of-range value, 4095 divisions. The 43  pushboard also doubled as a spare calibration block throughout the data acquisition, having been set in the same orientation following every sample piece. Though the lasers were not wired to signal out of range, the leading edge of the sample piece can be detected by a sudden change in the data from the steady reading of 4095 divisions, the maximum value set by the DAQsim program.  SECTION  LOOKING  NORTH  Figure 23. Sketch of pushboard in the apparatus The data for each sample board consisted of six sets of laser readings paired with a sample number. Each laser measured the board continuously, while the software logged the data from all the lasers into one file for each sample board. Physically, the measurement data was split into top and bottom sets because the lasers were positioned above and below the sample board. The top set of lasers was T l , T2 and T3, while the bottom set was BI, B2 and B3. As the lasers were staggered in the measuring apparatus, the data in the files was recorded in the sequence the lasers detected and measured the sample board. The top lasers detected the sample board in numerical order:firstT l , second T2, and third T3, as did the bottom lasers:firstBI, second B2, and third B3. This sequence of board detection was critical for the analysis of the top surface and of the bottom surface, since the laser data start points were verified using the top and bottom lasers in pairs.  44  Data Validation S o u r c e s of Error & T o l e r a n c e  Sources of error were associated with the construction of the measuring apparatus, the fabrication and design of the calibration block, the capabilities of the lasers and other hardware, as well as the human intervention factor. Tolerances and margins of error were evaluated in order to outline the limitations of this pilot experiment (Table 5). The intention was to enable future endeavours to be built upon this project by setting the results in the context of their limitations. The Total Tolerance was estimated by adding up the tolerance of the individual components of the measuring apparatus, from the jig holding the lasers to the acquisition card (Appendix B). The Within Laser Tolerance did not include the component tolerance from the laser jig because data testing for each laser assumed the laser's position remained constant relative to itself. Results outside the Within Laser Tolerance would show the laser may have moved. Tolerances for the separate components were obtained from several sources including equipment manuals, websites, design drawings and physical measurement. The tolerance was calculated in divisions to facilitate the data verification and testing process. The Total Tolerance was the Within Laser Tolerance plus the component tolerance from the laser jig. Table 5. Equipment Tolerances Equipment  Component  Estimated Tolerance  Tolerance in divisions  Laser jig Laser Calibration Block Acquisition Card  height accuracy height precision accuracy  0.1 mm 0.05 mm +/- 0.0254 mm 2.44 mV 4 microseconds to settle  40.950 20.474 20.803 0.999  Within Laser Tolerance 0.103 mm  42.3 divisions  0.203 mm  83.2 divisions  Total Tolerance  Calibration Block  A calibration block was designed and fabricated for use in testing the validity of the laser measurement data (Figure 24). The thickness of the calibration block was based on the rough green thickness of typical sawmill target sizes. Steps were machined in the block for two major reasons. First, the steps were for testing the lasers' ability to detect the changes anticipated in 45  the machine shape defect lumber samples. From the specifications, the average resolution of the lasers was less than 0.1 micrometers. As this figure was an average, it was important to ensure that the lasers can detect the dimensional changes which distinguish machine shape defects. Second, the steps were for testing the effective measurement range of the lasers. It was important to establish that the measurement data remained accurate throughout the whole range, not just at operating distance, since this information was not available from the manufacturer. The smallest step was 5 thousandths of an inch, while the largest step was 391 thousandths of an inch (9.9 mm).  —  r- "  fc  -  2"  --  2"  --  2"  --  2"  -  j TOP VIEW 4"  I  r 1.72  0.0 1 0"|  0.391  L SIDE VIEW  0.005  NTS  o.u  Figure 24. Drawing of calibration block The material chosen for the calibration block was Acetron GP Acetal for its dimensional stability and machinability. The comments in the product profile for Acetron GP also highlighted such desirable features as low moisture absorption, high strength, stiffness, easy to machine and no centerline porosity (DSM 14). Validation of the Data  The purpose of periodic calibration was to verify that the data was measured consistently by the lasers and was valid based on the calibration block and test measurements taken throughout the experiment. The validation of the data had three testing aspects. 1. Step size detection. 46  2. No change in apparatus before and after measurement. 3. Laser measurement repeatability. The laser data used in these tests was from static measurements of the calibration block designed for this purpose. The first set of tests checked the step size detected by the lasers to ensure their accuracy in measuring changes in the surface relief of the sample boards. Comparing the step measurement averages to the caliper measurements, it appeared that the lasers were offset by 0.5 mm. Likely, the lasers were not aligned perfectly perpendicular to the surface of the measuring bed, resulting in the offset. Unfortunately, the nature of the measurement apparatus made the positioning of the lasers unadjustable. The second set of tests checked that the measuring apparatus did not change during the board measurement process by testing the calibration results from before and after board sampling. The residual means of each matched before-and-after pair were tested against the expected mean of the differences of the matched pairs (Bluman 1997). Two-tailed t-tests were used to test for differences between before and after board sampling, whereby the expected mean of the differences equalled zero. One-tailed t-tests were used to test whether the difference was less than the Within Laser Tolerance of the measurement apparatus. In this instance, the residual means should be less than the expected mean of the differences which was the Within Laser Tolerance. The third set of tests checked the repeatability of each laser's measurements to ensure the lasers were measuring the data consistently. Five sets of data were collected for each of the six lasers by measuring the calibration block five consecutive times. The repeatability of the lasers was tested using paired t-tests for each laser. The difference between the sets of data for each laser should be less than the Within Laser Tolerance of the measurement apparatus. Data Preprocessing  In order to train the Neural Networks to differentiate between shape defects, the data was preprocessed for a set of variables. Preprocessing is afixedtransformation of the variables and often greatly improves the performance of a pattern recognition system (Bishop 1995). It simplifies the classification task by reducing the dimensionality of the input vector, and by minimising the amount of data required in each data set (Williams 1993). The main advantage that preprocessing offers is to decrease the time to train the network, while maintaining the level of information in the training set. In this pilot project, the input variables enabled the neural network to classify board sample data into shape defect categories. These output categories were Snipe, No Snipe and Snipe 47  Combination. The raw measurement data was preprocessed using multiple linear regression. The top and bottom surfaces of the sample boards were modelled to obtain complex slope and intercept characteristics using data from all six lasers. These input values, together with the machine shape defect assessment of the sample board, for the data set were used to train, verify and test the neural networks. Laser Data  In effect, each laser draws an imaginary line down the length of the sample board being measured. These lines are called laser lines in this paper. Laser lines T l , T2 and T3 describe the top surface of a sample board, while laser lines BI, B2 and B3 describe the bottom surface. There was a lag in the laser's detection of the sample board for two reasons. First, the board was conveyed down to the lasers, which were staggered in the feed direction and relative to the fence. Second, the user manually initiated the data recording function because the data acquisition software distinguished between reading and recording the laser data. The start of the board data was determined by reviewing the data manually. The sample board's leading edge was detected by finding the out-of-range values for each of the lasers in the data set. The DAQSim program displayed a reading of 4095 divisions when nothing was detected by lasers Tl, T2, T3, B2 and B3. In the case of laser BI, a reading of 0 divisions indicated that the object was out of range, as this laser was setup differently. A rapid change in the out-of-range values flagged the sample board's leading edge in the laser data. An equal number of data points, beginning from the established start of the readings, was used in the regression analysis for each of the lasers in order to simplify the physical interpretation of the analysed surface. Although the top and bottom surfaces of the sample boards were analysed separately, the laser data start points were matched up in top and bottom pairs to ensure consistency in the interpretation of the results (Figure 22). These laser pairs were Tl and BI, T2 and B2, and T3 and B3 (Figure 25 & 26). Data from samples with no snipe was generally fairly flat with an occasional jag. In contrast, data from samples with snipe was irregular and a slope was easily detected from the graphs. Note that the data from lasers T2 and B2 cross over as a function of how it was graphed. It was important to recognise that the logged laser data may not represent the entire sample board and push board due to the lag between reading the data points and logging the data points. It was for this reason that an equal number of data points, matched up between laser pairs, was analysed, instead of the entire raw data set of each laser.  48  No Snipe  4000 c 3000 | 2000 •| 1000  •laser T1 -laser B1 TT—  CM OO CM CO  •sr ^  IO IO  CO ID  N  00 CO CO O)  Sample Number  No Snipe  5000 » 4000 .2 3000 ~ 2000 Q 1000 0  -laser 12 -laser B2 ooior^-CDv-comr^cn i - ( M C O i f ( O S O ! l < I ) 0 T - ( \ f ) * l O C D S O O O  Sample Number  No Snipe  (A C  4000 n 3000  ;§  2000  ~  1000  -laser T3 -laser B3 n i o s o i i - t o m s o i T - C M C O T j - C D r ^ C O a J O i - C N C O ' 4 - U O C O r ^ O O O  Sample Number  Figure 25. Graphs of logged laser data from a No Snipe board sample  49  Snipe  5000 g 4000 .2 3000 •5 2000 5 1000 0  • laser T2 •laser B2 C M C O ^ - t O t O r ^ O O O O O O C D ^ f C M O O O C O ^ - C O  Sample Number  Snipe  5000 m 4000 .2 3000 •5 2000 O 1000 0  •laser T3 •laser B3  c o m r ^ o ) T - c o m r ^ a > o o i O ' t N T - o i s i o n • > - o \ i r o - ^ - ' < t m c o r ^  Sample Number  Figure 26. Graphs of logged laser data from a Snipe board sample  M o d e l l i n g the 3-D S h a p e o f the B o a r d  A statistical model was developed to interpret physical characteristics of a board's surface, primarily the shape, using data from the lasers. The advantage of this model was that it 50  described the three dimensional shape of the board. Three laser lines were compared on one regression surface with their slopes and intercepts. Equation 1-1 models the top surface, and Equation 1-2 models the bottom surface. Combining the results from the top and bottom surfaces defines the shape of the board, by forming a shape from the three connected laser lines of each surface. For example, connecting lines T l , T2 and T3 from the No Snipe graphs in Figure 25 results in a fairly flat top surface. However, connecting lines T l , T2 and T3 from the Snipe graphs in Figure 26 results in an indented and sloped top surface. This statistical model was also particularly important for the simplification of the data set required for input into the neural networks. Rather than using every laser data point, the regression coefficients will represent the changes in a board's surface. Y =TU + TIS*Z + T2I*D i  2  +T2S*D  2  * Z + T3I * D +T3S * Z) *Z + e, 3  3  (1-1)  Where Yj = i observation of the laser data set T i l = intercept for laser line Tl T1S = coefficient of slope for laser line Tl T21 = intercept for laser line T2 T2S = coefficient of slope for laser line T2 T3I = intercept for laser line T3 T3S = coefficient of slope for laser line T3 Z = sample number (reset to 1 for each laser) D2 = dummy variable for laser line T2 D3 = dummy variable for laser line T3 £j = error component th  Y =BU + BIS*Z + B2I*D +B2S*D *Z + B31*D +B3S*D *Z + e,. i  2  2  }  Where Yj = i observation of the laser data set B l l = coefficient of intercept for laser line Bl B1S = coefficient of slope for laser line B1 B2I = coefficient of intercept for laser line B2 B2S = coefficient of slope for laser line B2 B31 = coefficient of intercept for laser line B3 B3S = coefficient of slope for laser line B3 Z = sample number (reset to 1 for each laser) D2 = dummy variable for laser line B2 D3 = dummy variable for laser line B3 th  51  3  (1-2)  E j = error component Using the dummy variables, DI and D2, gave the possibility of describing three different laser lines in the model for the surface. The dummy variables were valued 0 or 1 depending on which laser line the data came from. Each laser line was represented by an intercept and a slope. For example, the laser line for the top surface, made by laser T l , was represented by the intercept T i l and the slope T1S. A configuration of three laser lines allowed testing for shape across the board width as well as along the length (Figure 27). This option became important for modelling wedge as well as the other defects.  Figure 27. Model of regression lines on a sample board An F-test was used to test the significance of the model for a given sample surface, and t-tests were used to test the significance of the coefficients. However, insignificant variables were not eliminated from the regression model, since this research focusses on analysing the shape of the surface, rather thanfindingthe best model tofitthat shape. Therefore, for every sample board, the multiple linear regression produced six coefficients for the top surface and six for the bottom surface. These coefficients revealed the physical characteristics of the sample board by showing what the regression surface looked like. The slopes and intercepts indicated what one part of the surface was doing relative to the other. For example, if the slope coefficient B3S equalled zero, then laser line B3 was flat or horizontal. A negative value for the slope coefficient predicted a downward slope, while a positive value reflected an upward slope. 52  Therefore, one expected a negative slope coefficient for a sniped sample and a near-zero slope coefficient for a flat board sample with no snipe. Microsoft Excel?, Data Analysis Regression function was used to perform the multiple linear regression tests for the top and bottom surfaces of each sample board. The coefficient values were recorded under each respective input variable in a separate workbook that was formatted for importing into the Statistica Neural Networks software. Input & O u t p u t V a r i a b l e s  In order to build a data set for training the neural networks from the laser data, the number and type of input variables required to model the 3-D shape of the board must be determined and examples of the output categories must be collected and defined. In the workbook formatted for importing into the Statistica Neural Networks software, each row contained a set of regression coefficients for each sample board (Appendix C.1). There were twelve coefficients in each set. These coefficients were, in effect, the values of the input variables for the data set. The six variables for the top surface were Til, T1S, T2I, T2S, T3I and T3S and the six variables for the bottom surface were B1I, BIS, B2I, B2S, B3I and B3S. One half of the coefficients represented slopes and the other half represented intercepts. Input variables were named for the surface and laser line represented and for the associated intercept or slope. For example, T i l indicated the top surface by T , the laser line by '1', while T represented the intercept. These variables were numeric. The last variable in the row was SNIPE, the output variable, and represented the machine shape defect indicator value for the sample board. The three values for SNIPE were 'y', 'n' or 'c', and indicated Snipe, No Snipe or Combination Snipe respectively. Consequently, the output variable was nominal. In order to train the neural network, the value of the output variable was determined manually by visually examining the board samples and categorising them for snipe. Otherwise, the neural network performed this classification task by assigning the value. Neural Network Trials Software  The software choice was based on the classification requirement for pattern recognition algorithms. The decision was to use Statistica Neural Networks because of the software's flexibility and its classification features. Thefivedifferent types of networks that can be applied to classification problems, using this software, were multilayer perceptrons (MLP), radial basis function (RBF), Kohonen Self-Organising Feature Map networks (Kohonen), linear 53  networks (linear) and (Bayesian) probabilistic neural networks (PNN). Each of these network types represented a different pattern recognition algorithm. The Intelligent Problem Solver (IPS) feature enabled training decisions to be controlled on several levels from automatic to advancedfine-tuning.The basic version of the IPS is primarily automated; however, the user can set control parameters which determine how classification is performed, such as the number and types of networks saved in the solution set and the duration of training. In the advanced version of the IPS, the user can specify design parameters such as the classification confidence threshold, the number of hidden units and the network type. Using this version required more experience and familiarity with neural networks and the classification problem itself. Regardless of the version chosen, the software automatically interpreted nominal output variables for classification and generated statistics on overall classification performance (Statsoft 1999). In addition, it had clear and useful graphics, as well as intelligible manuals. Neural Network Training and Testing  Before a neural network can be used to classify machine shape defects, it must be trained using a data set comprised of input data paired with the correct output categories (Eggers 1991). Each pair is called a sample case. Training a neural network is essentially the process of tuning a set of parameters to describe a statistical model of its data set (Swingler 1996). The outcome of this training is actually a series of different types of networks which have a variety of different characteristics and parameters. The user chooses the network most appropriate for the application based on its attributes, striking a balance between classification accuracy, training time, classification time, memory usage and fault tolerance (Cornforth 1993). Low error and high performance demonstrate good classification accuracy in a model. The main goal in pattern recognition was to develop a neural network which generalises well, so that it can successfully predict the correct output category from new data. (Bishop 1995). The simplest way to ensure that a neural network has good generalisation ability was to reserve part of the data set for verification and another part for testing. Verification is an independent check on the performance of the network during training, indicating that overlearning has occurred by an increase in the verification error. Overlearning can occur when the neural network model is overfitted to the training data, resulting in a loss of generalisation ability (Swingler 1996) Testing is a final check for bias in the network's performance results. Statistica Neural Networks splits the data set in three and randomly assigned the sample cases in a 2:1:1 ratio for training, verification and testing respectively (Statsoft 1993). If overlearning or a bias in the performance results was reported, the network can be improved by enlarging the data set, 54  by changing the type of network, or by modifying the training process. These options must be weighed with consideration to factors like cost and availability of data, time constraints for training, as well as the levels of noise, required generalisation ability and network simplicity for the particular application (Swingler 1996). Neural Network Trial P r o c e d u r e  In this pilot project, the major constraints governing the collection of sample pieces and the construction of the measurement apparatus were time and money. Within these limits, a significant effort was made to minimise noise in the data by selecting appropriate lumber samples and by careful measurement of these samples. The lumber samples included a random variety of machine shape defects, though snipe is the primary defect of interest, comprising 40% of the samples. Training Process: Snipe Classification Approaches & Training Strategies  Two major approaches in the training process were used for the snipe classification problem: as a two-class problem or as a three-class problem (Figure 28). The input and output variables were pre-processed the same way for both approaches, but some board samples, namely the snipe combination samples, were classed differently between approaches as explained in the following paragraphs. Four training strategies were used in the process of training the neural networks for both approaches to the snipe classification problem: Base Condition, Shuffled Condition, Pruned Condition, and Extended Duration Condition. Each training strategy  resulted in a series often different networks. The control parameters for the Base Condition were described subsequently, followed by summaries of the other training strategies. Snipe Classification Problem  Two Class Networks  Three Class Networks  Base Condition  Base Condition  - 1 0 networks  - 1 0 networks  Shuffled Condition  Shuffled Condition - 1 0 networks  - 1 0 networks Pruned Condition  Pruned Condition  - 1 0 networks  - 1 0 networks  Extended Duration Condition  Extended Duration Condition - 1 0 networks  - 1 0 networks  55  Figure 28. Graphical representation of the snipe classification problem The first approach, called Two Class Networks, treated the snipe classification as a two-class problem in which the network was trained to distinguish between Snipe and No Snipe categories. The Snipe category consisted of sample boards having an edge or center piece cut out from a sudden movement in the saw blade or the wood. In the data, this cutout, called snipe, was observed as a deviation in the board's profile where the lumber thickness had been reduced. In some instances, snipe resembled wane without the bark because of the nature of the sawing process. Note that the Snipe category in the two-class problem did not include the snipe/wedge combination samples. These snipe/wedge samples were classed in the No Snipe category, which primarily consisted of sample boards which did not contain snipe. The second approach, called Three Class Networks, treated the snipe classification as a threeclass problem, by adding a third category, Combination Snipe. Again, the Snipe category was comprised of all the samples which contained snipe, except the snipe/wedge samples. In the three-class problem, these snipe/wedge samples were classed in the Combination Snipe category, which generally included samples having snipe and at least one other machine shape defect. That said, in this proof of concept, only the snipe/wedge samples were included in the Combination Snipe category, due to the small numbers of the other defect combinations. There were not enough examples for each type of snipe combination, so they were classed in the Snipe category. In the three-class problem, the No Snipe category consisted of sample boards which did not contain any snipe. Therefore, these samples contained other machine shape defects or no defects at all. Identical control parameters were selected for the Two and Three Class Networks, using Statistica Neural Networks' Intelligent Problem Solver (IPS) feature to set up the training for the Base Condition networks (Table 6). The Base Condition networks resulted from selecting the basic version of the IPS, which automated as many of the training decisions as possible. The other training strategies differed by whether the data was shuffled once or twice, the input variables were pruned or the training time extended. They are described in more detail below. The Standard problem treated the sample cases in the data set as independent (Statsoft 1999). The dependent or output variable was SNIPE, while the twelve independent variables, the input variables, were Til, T1S, T2I, T2S, T3I, T3S, B1I, BIS, B2I, B2S, B3I and B3S. The option to search for an effective subset of specified variables was selected to allow the IPS to discard those variables deemed irrelevant for a particular network solution.  56  The default setting in the Basic version automatically determined the single threshold to minimise the misclassification rate. Because the classes overlapped, a threshold was set between the doubt and acceptance regions for each class (Swingler 1996). A value below the threshold meant the sample case was rejected for that class. Positioning the threshold was a balance between minimising the classification error and discarding good data (Swingler 1996). For example, if the threshold was split evenly between two classes, no sample cases were rejected, but the classification error was high. The amount of time for the IPS to spend designing an effective neural network for the application was specified in broad terms as 'medium'. However, the actual duration was relative to the available amount of data in the set and ranged from minutes to hours. Any number of neural networks may be saved in the set, but saving ten networks appeared to give a reasonable variety of types and complexities of networks in the trials. Thus, ten networks were saved each time training took place. To maintain diversity, the selection of networks saved balanced the performance against type and complexity. Basically, 'complexity' refers to the number of hidden units connecting the input units to the output units through transformations and 'performance' gauges the predictive accuracy of the network, while 'type' indicates the network model used for the pattern recognition, i.e. RBF versus MLP (Statsoft 1999). Table 6. Selection of control parameters for the Base Condition neural networks Design Control Parameter  Choice  Version of Intelligent Problem Solver Problem type Output variable selection Input variables selection  Basic Standard SNIPE Til, T1S, T2I, T2S, T3I, T3S, B1I, BIS, B2I, B2S, B3IandB3S Medium 10 Balance performance against type & complexity Increase network set size  Duration of design process Number of networks to save Selection of networks to be saved Action if network set too full  Normally, newer networks replaced existing networks in the set to minimise the quantity of redundant networks. In choosing to increase the network size when the set becomes full, ten more networks were added to the set each time the network set was run for additional training, This rate of growth was acceptable for two reasons. First, the networks were relatively small because the amount of data wais limited in this proof of concept project. Second, only a few 57  variations to the Base Condition neural networks were run, which kept the number of networks in the set reasonable. These variations were in fact the three other training strategies, developed as potential improvements to the Base Condition. The same data preprocessed for the Base Condition was used to train the networks for each of these strategies: 1. Shuffled Condition: The input data was shuffled once or twice to ensure the assignments of the sample cases to the subsets was not biased by redistributing the sample cases assigned to train, verify and test the network. This strategy was applied by using the 'randomly reassign cases' option in the advanced version of the IPS. 2. Pruned Condition: The input variables were pruned to see how the baseline error is affected by dropping the input variables with low sensitivity ratios. A low sensitivity ratio indicated that the input variable was less important in the neural network (Statsoft 1999). The sensitivity analysis from a Base Condition network was used to indicate which variables to drop from the input data set. The networks were then trained without those input variables, using the basic version of the IPS. 3. Extended Duration Condition: The training duration was extended to see if increasing the time spent designing an effective network for the classification problem improved the performance of the neural networks produced. The basic version of the IPS was used to train the networks, but the duration of the design process chosen was 'Thorough' in order to conduct an extensive search.  58  Results Data Validation N o C h a n g e in A p p a r a t u s  The purpose of these tests was to check that the measuring apparatus, specifically the lasers, remained constant throughout the process of measuring the board samples. The calibration measurements taken before and after this process were tested with paired t-tests, using a sample size of 900 data points and an alpha level of 0.05 (Appendix D.l). The null hypothesis in thefirsttest, a two-tailed t-test, is that there is no difference between the initial run of calibration measurements and the follow-up run. However, the results indicate that there is a significant difference between the two runs at an alpha level of 0.05 for all six lasers (Table 7). Therefore, each of the lasers changed during the board measurement process, likely from the vibration of conveying the sample boards through the measuring apparatus. A second set of tests is required to gauge the extent of this movement. The second test, a one-tailed t-test, hypothesises that the difference between the initial run and the follow-up run of calibration measurements is less than the Within Laser Tolerance of 42.3 divisions (0.103 mm). Of the six, Laser B2 was the only laser found to have a difference between runs significantly greater. This difference is a drop in value of 296.06 divisions from the initial calibration measurements to the follow-up calibration measurements, amounting to a 0.723 mm offset. Table 7. Summary o * results for detecting a change in the measuring apparatus Laser  Significant difference between runs  Difference exceeds tolerance  Tl T2 T3 BI B2 B3  Yes Yes Yes Yes Yes Yes  No No No No Yes No  Laser M e a s u r e m e n t Repeatability  The purpose of this test was to check that the lasers are measuring the data consistently and that their performance can be repeated. Five runs of 900 data points were collected for each of the  59  six lasers by measuring the calibration block five consecutive times. The repeatability of the lasers is tested with paired t-tests, using each of these five sets (Appendix D.2). The first test, a two-tailed t-test, hypothesises that difference between runs within each laser measurement set is zero (Table 8). Since each of the lasers were found to have significant differences between at least one of their five runs, a second set of tests is required to ascertain the size of this difference. The null hypothesis in the second test, a one-tailed t-test, is that the difference between runs is less than the Within Laser Tolerance of 42.3 divisions (0.103 mm). None of the differences between runs significantly exceeded the Within Laser Tolerance of the measurement apparatus. Therefore, the repeatability of the lasers is deemed acceptable. Table 8. Summary o 'results for testing repeatability Laser  Significant difference between runs  Difference exceeds tolerance  Tl T2 T3 Bl B2 B3  Yes Yes Yes Yes Yes Yes  No No No No No No  Data Preprocessing  The preprocessed board sample data for training the neural networks was formatted for importing into Statistica Neural Networks (Appendix C l ) . There were 103 rows representing the number of sample boards and 13 columns representing the number of variables. Each row was comprised of a set of twelve numeric input variables and one nominal output variable. The multiple linear regression analysis showed that the model was significant for every sample board measured in the data set (Appendix C.2). The F test values from the regression analysis ranged between 11.31 and 9973.1 for the top surface and between 13.89 and 3141.6 for the bottom surface (Figures 29 and 30). Just less than half of the F values fell between 300 and 600 for the bottom surface, while over half of the F values fell between 300 and 600 for the top surface. The t-test values were not analysed because the purpose was to study the shape of surface, not to develop an optimal regression model. Whether they were significant or not depended on the topography of the board's surface, not the accuracy of the model.  60  A near-zero value for the coefficient of multiple determination (R ) indicated that the surface was flat. The coefficient of multiple determination was significant when the slope of the surface was significant, since the purpose of the model was to detect changes in the surface relief: the higher the value, the steeper the slope of the surface. The values for the coefficient of multiple determination ranged between 0.0271 and 0.9334 for the top surface, and between 0.0378 and 0.8158 for the bottom surface (Figures 31 and 32). For the top surface, just over 40% of the R values were concentrated between 0.050 and 0.225, while just over 30% of the R values for the bottom surface were between 0.225 and 0.375. These higher R values for the bottom surface show that the sniped surface was more often face down as the sample board was fed through the measuring apparatus. This orientation of the boards does not emulate the process in a sawmill. 2  2  2  T O P : F test v a l u e s 70 60 5" 50 S 40 §• 30 £ 20 10 o o  CO  o o  CD  O O  o o  CN  o o  O O  00  LO  o o OJ  F values  o  2  Figure 29. Top Surface: F test values by class B O T T O M : F test v a l u e s 60 50 » 40 a 30  cr  £ 20 LL  10 0  o o  CO  o o  CD  O O  O  o  CN  o o  o o  ID 1  00 -  1  -  o o  T-  CN  F values  Figure 30. Bottom surface: F test values by class  61  CD \— O  TOP: R 2 values 30 25 g 20 S 15 cr  £ 10 LL  5 0  O O L O O L O O L O O L O U O C N O r ^ - L O C N O LO <q o CN co co o  o  o  o  o  o  o  o  LO  N co  ci  O  LO  m cNd  ci d  o ^  R2 values  Figure 31. Top surface: coefficients of multiple determination by class  BOTTOM: R2 values 20 ,  O O L O O L O O L O O L O O I O CP i O L n c \ j o r - ~ L O c N O i ^ L O c N !=: O T - C N I C O C O T t L n c O C D r - - C O ° O O O O O O O O O O O ^ :  R2 values  Figure 32. Bottom surface: coefficients of multiple determination by class Neural Network Trials Interpretation o f R e s u l t s  These results are split into two main sections (Figure 28). The first section looks at the Two Class Networks problem where the network is trained to classify the input data into Snipe or No Snipe categories. The second section looks at the Three Class Networks problem, which has the Combination Snipe category in addition to the Snipe and No Snipe output categories (p.56). Each section shows the results from the four training strategies, which are the Base Condition, the Shuffled Condition, Pruned Condition, and Extended Duration Condition (p.58). For each Condition, a series of ten networks are summarised in a table and the highlights are mentioned. Each network is represented by a set of characteristics reported from Statistica Neural Networks' Basic and Verbose Network Set Datasheets (Table 9 - 17). This set of characteristics 62  is split in order to distinguish the characteristics that describe the neural network from those that evaluate the neural network. The characteristics are described briefly below in terms of their meaning and interpretation and discussed at length in the Neural Network Trials of the Discussion (p.71). Description of Neural Networks Network: Each network was assigned a name for clarity and convenience. The network nomenclature describes the position of the network in the set, the training strategy, and the number of classes. Specifically, the first letter and number show where the network is situated in the network set. The next two letters indicate the training strategy: Base Condition (BC), Shuffled Condition (SC), Pruned Condition (PC) or Extended Duration Condition (EC). The last number and letter indicate whether the network solves a two or three class problem. Using N1BC2C for an example, 'NT signifies that it is the first network saved in the set; 'BC stands for Base Condition; and '2C means that it solves a two-class network problem. Type: The three types of networks reported in the results for this snipe classification problem were multilayer perceptrons (MLP), radial basis function (RBF) and linear networks (Linear). Each type of network has a different network architecture, or pattern recognition algorithm, with its own set of merits. Inputs: The number of input variables used by each neural network was reported in the results. These inputs were chosen from the twelve independent variables that model the top and bottom surfaces of the sample boards. Hidden: The number of hidden units was reported to describe the complexity of the network. Essentially, hidden units connect the input units to the output units using transformations. These transformations are designed to optimise the decisions and to minimise error (Bishop 1995). Evaluation of Neural Networks TrError, VeError, & TeError: Statistica Neural Networks reported the training error  (TrError), the verification error (VeError) and the testing error (TeError). Each error was the root mean square error (RMS) summarised over its subset. The verification error was obtained from the Basic Datasheet, while the test and training errors were obtained from the Verbose Datasheet. Of these three errors, the verification error was the most significant, since it gave the best indication of the network's ability to make predictions on 63  new data (Statsoft 1999). The test error was used for a final check of the network performance and also required consideration because the two errors were helpful in diagnosing training problems (Statsoft 1999). For example, if the verification error and the test error had similar values, then overfitting had likely not occurred. In Statistica Neural Networks, the training algorithms searched for network solutions that minimised the training error during the training process, so it was not a concern if the training error was much lower than the other errors. Therefore, the training error was not reported in the results. At this early stage of applying neural networks to this snipe classification problem, the best possible error rate was not known. Comparison of these results to similar classification problems was difficult and likely misleading because they were highly dependent on a particular problem. Performance: The performance reported in the results was the verification performance, which represented the proportion of correctly classified sample cases in the verification subset of the data set and was referred to as the performance in this report. For example, a performance of 0.692 meant that 69.2% of the sample cases were correctly classified in the verification subset. Also known as the correct classification rate, it was the best indicator of whether a neural network was suited to the classification problem or not. However, the performance should not be interpreted alone, since other parameters also play an important role in assessing the capabilities of a neural network. At this early stage of applying neural networks to snipe classification, it was difficult to know what the best possible performance rate was for this problem. Although Statistica Neural Networks' Intelligent Problem Solver described verification performance values of 0.731, 0.692 and 0.654 as 'ok performance' on a scale ranging from extremely good to extremely poor, its comments were treated with caution as the achievable level of accuracy depends on the problem (Statsoft 1999). Like the other characteristics, performance of the neural networks is further explored in the Discussion (p. 77). Two Class Network Problem: No Snipe & Snipe  Base Condition - Two Class Networks  64  Description: Ten neural networks were saved from the network set trained using the data preprocessed for two output categories (Table 9). Three of the networks were linear, and three were RBF, while the remaining four were MLP networks. Evaluation: N10BC2C had the lowest error at 0.473 and the highest performance at 0.692 for two classes. N6BC2C also had a performance of 0.692, but its error was higher at 0.501. Table 9. Base Condition - Two Class Networks Network  Type  VeError  TeError Inputs  N1BC2C N2BC2C N3BC2C N4BC2C N5BC2C N6BC2C N7BC2C N8BC2C N9BC2C N10BC2C  Linear RBF Linear Linear RBF RBF MLP MLP MLP MLP  0.512 0.511 0.508 0.508 0.504 0.501 0.494 0.484 0.480 0.473  0.353 0.449 0.358 0.361 0.435 0.437 0.466 0.526 0.424 0.461  8 1 9 10 1 1 1 12 10 12  Hidden -  2 -  1 1 1 7 6 7  Performance  0.500 0.385 0.577 0.538 0.308 0.692 0.615 0.654 0.615 0.692  Shuffled Condition - Two Class Networks  Description: The same data preprocessed for two output categories was used to train these networks as for the Base Condition. However, the sample cases were redistributed, or shuffled, between the training, verification and testing subsets (Table 10). As with the Base Condition, ten networks for the Shuffled Condition were saved from the network set. Four of the networks were linear, three were RBF and three were MLP. Evaluation: The network with the lowest error at 0.383 was N10SC2C, an MLP with a performance of 0.731. However, several linear networks had the highest performance at 0.808 with six to eight input variables and no hidden units. These performance results were unusually high as compared to the Base Condition networks (Table 9). In addition, the large discrepancy between the verification and test errors indicated that the networks may not be very reliable. Therefore, a second round of shuffling was required to double-check that the distribution of the sample cases was not biased to achieve high performance ratings by fluke. Description: Ten networks for the second round of shuffling were savedfromthe network set (Table 11). Three of the networks were linear, three were RBF and four were MLP. Evaluation: N20SC2C was the network with the lowest error at 0.464 . It was one of two networks with the highest performance at 0.692. The other network, NI 1SC2C, had the same performance, but its error was higher at 0.543. 65  Table 10. Shuffled Condition - Two Class Networks Network  Type  VeError  TeError  Inputs  Hidden  Performance  N1SC2C N2SC2C N3SC2C N4SC2C N5SC2C N6SC2C N7SC2C N8SC2C N9SC2C N10SC2C  MLP RBF RBF RBF Linear MLP Linear Linear Linear MLP  0.475 0.474 0.438 0.437 0.423 0.416 0.404 0.397 0.397 0.383  0.501 0.521 0.556 0.534 0.596 0.526 0.575 0.572 0.573 0.562  1 12 12 12 5 12 8 7 6 12  1 6 7 6 -  9  0.615 0.615 0.692 0.654 0.692 0.731 0.808 0.808 0.808 0.731  7 -  Table 11. Shuffled Condition Second Round- Two Class Networks Network  Type  VeError  TeError  Inputs  Hidden  Performance  N11SC2C N12SC2C N13SC2C N14SC2C N15SC2C N16SC2C N17SC2C N18SC2C N19SC2C N20SC2C  RBF RBF RBF Linear Linear MLP MLP Linear MLP MLP  0.543 0.514 0.492 0.485 0.483 0.475 0.475 0.473 0.469 0.464  0.631 0.663 0.538 0.455 0.455 0.493 0.492 0.461 0.498 0.576  10 10 10 9 8 1 11 10 12 12  6 9 10  0.692 0.577 0.654 0.615 0.577 0.423 0.500 0.615 0.577 0.692  -  1 8 .  6 6  Pruned Condition - Two Class Networks  Description: The Base Condition network with the lowest error and highest performance, N10BC2C, was used to determine which input variables to drop. The sensitivity analysis indicated that Til, T2S, BIS and B3S had low sensitivity ratios, with a threshold of 1.05, showing that these four input variables were the least important to the neural network. Therefore, they were dropped from the input data set and the networks were trained again with the remaining eight input variables using the data preprocessed for two output categories. The result was that an additional ten networks were saved in the network set (Table 12). Five of the networks were MLP, while four were RBF and only one was linear. Although each of the networks had eight input variables, the number of hidden units varied. 66  Evaluation: N10PC2C had the lowest error at 0.472, but the second highest performance at 0.615. Three networks had the highest performance at 0.654. They were N3PC2C, N8PC2C and N9PC2C, of which N9PC2C had the lowest error at 0.476. Table 12. Pruned Condition - Two Class Networks Network  Type  VeError  TeError  Inputs  Hidden  Performance  N1PC2C N2PC2C N3PC2C N4PC2C N5PC2C N6PC2C N7PC2C N8PC2C N9PC2C N10PC2C  RBF RBF Linear RBF MLP RBF MLP MLP MLP MLP  0.507 0.504 0.498 0.497 0.487 0.485 0.484 0.479 0.476 0.472  0.434 0.436 0.400 0.422 0.393 0.440 0.421 0.403 0.394 0.416  8 8 8 8 8 8 8 8 8 8  1 2  0.462 0.538 0.654 0.577 0.577 0.615 0.615 0.654 0.654 0.615  .  4 8 8 8 6 6 8  Extended Duration Condition - Two Class Networks  Description: The same data preprocessed for the Base Condition was used to train these networks, but the time spent designing an effective network for the classification problem was increased. Ten networks for the Extended Duration Condition were saved from the network set (Table 13). The first six networks were alternating RBF and linear networks, while the remaining four were MLP networks. Evaluation: N10EC2C had the lowest error at 0.462, and second highest performance at 0.615. The highest performance at 0.692 was held by N5EC2C, with an error of 0.501. Table 13. Extended Duration Condition - Two Class Networks Network  Type  VeError  TeError  Inputs  Hidden  Performance  N1EC2C N2EC2C N3EC2C N4EC2C N5EC2C N6EC2C N7EC2C N8EC2C N9EC2C N10EC2C  RBF Linear RBF Linear RBF Linear MLP MLP MLP MLP  0.510 0.504 0.504 0.504 0.501 0.501 0.493 0.487 0.474 0.462  0.429 0.431 0.435 0.427 0.437 0.428 0.468 0.491 0.435 0.412  4 3 1 1 1 2 1 3 3 5  1  0.577 0.462 0.308 0.538 0.692 0.577 0.538 0.577 0.577 0.615  67  1 1 1 1 3 4  Three C l a s s Network Problem: N o Snipe, Snipe & Combination Snipe  Base Condition - Three Class Networks Description: Ten neural networks were saved from the network set trained using the data preprocessed for three output categories (Table 14). The first three networks were linear, followed by an RBF network and four consecutive MLP networks. Two more RBF networks completed the set. Evaluation: N10BC3C, an RBF network with 4 input variables and 1 hidden unit, had the lowest error at 0.411 and the highest performance at 0.654 for three classes. Table 14. Base Condition - Three Class Networks Network  Type  VeError  TeError  Inputs  Hidden  Performance  N1BC3C N2BC3C N3BC3C N4BC3C N5BC3C N6BC3C N7BC3C N8BC3C N9BC3C N10BC3C  Linear Linear Linear RBF MLP MLP MLP MLP RBF RBF  0.468 0.464 0.457 0.426 0.426 0.425 0.422 0.422 0.414 0.411  0.447 0.444 0.447 0.421 0.421 0.424 0.421 0.422 0.413 0.414  5 6 7 4 1 12 12 12 4 4  -  0.577 0.577 0.577 0.615 0.615 0.615 0.615 0.615 0.615 0.654  1 1 8 8 6 2 1  Shuffled Condition - Three Class Networks  Description: The same data was used to train the ten saved networks for the Shuffled Condition as for the Base Condition (Table 15). However, the sample cases were redistributed between the training, verification and testing subsets. Three of the resulting networks were linear, three were MLP, while four were RBF networks. Evaluation: Networks N6SC3C, N9SC3C and N10SC3C had the highest performance at 0.615, but N10SC3C had the lowest error at 0.452. Table 15. Shuffled Condition - Three Class Networks Network  Type  VeError  TeError  Inputs  N1SC3C N2SC3C  Linear Linear  0.471 0.470  0.380 0.373  5 7  68  Hidden  Performance  -  0.538 0.538  N3SC3C N4SC3C N5SC3C N6SC3C N7SC3C N8SC3C N9SC3C N10SC3C  MLP MLP Linear RBF RBF MLP RBF RBF  0.470 0.467 0.465 0.462 0.459 0.455 0.454 0.452  0.426 0.419 0.373 0.509 0.482 0.419 0.461 0.468  1 11 6 10 10 12 10 10  1 2 -  28 9 8 16 18  0.423 0.385 0.538 0.615 0.538 0.577 0.615 0.615  Pruned Condition - Three Class Networks  Description: The Base Condition network with the lowest error and highest performance for three classes, N10BC3C, was used to determine which input variables to prune. As the sensitivity analysis showed that T3I, T2S and T3S had low sensitivity ratios for a threshold of 1.05, they were dropped, leaving T2I as the remaining input variable. The networks were trained again using the data preprocessed for three output categories with one input variable selected. The resulting additional ten networks were saved in the network set (Table 16). Five of the networks were MLP, while four were RBF and only one was linear. Although each of the networks had one input variable, the number of hidden units varied. Evaluation: N10PC3C had the lowest error at 0.412, but shared the highest performance at 0.654 with four other networks. These other networks were N4PC3C, N6PC3C, N8PC3C and N9PC3C. Table 16. Pruned Condition - Three Class Networks Network  Type  VeError  TeError  Inputs  Hidden  Performance  N1PC3C N2PC3C N3PC3C N4PC3C N5PC3C N6PC3C N7PC3C N8PC3C N9PC3C N10PC3C  RBF Linear MLP MLP MLP RBF RBF MLP MLP RBF  0.425 0.425 0.423 0.421 0.421 0.419 0.418 0.418 0.417 0.412  0.432 0.422 0.420 0.419 0.419 0.411 0.411 0.417 0.419 0.436  1 1 1 1 1 1 1 1 1 1  4  0.615 0.615 0.615 0.654 0.615 0.654 0.615 0.654 0.654 0.654  Extended Duration Condition- Three Class Networks  69  -  8 20 13 1 2 9 13 3  Description: The same data preprocessed for the Base Condition was also used to train these networks, but the time spent designing an effective network for the classification problem was increased. Ten networks for the Extended Duration Condition were saved in the network set (Table 17). Three of the networks were linear, three were RBF networks, and four were MLP networks. Evaluation: N10EC3C had the lowest error at 0.400, but the second highest performance at 0.654. N9EC3C obtained the highest performance at 0.692 with an error of 0.411. Table 17. Extended Duration Condition - Three Class Networks Network  Type  VeError  TeError  Inputs  Hidden  Performance  N1EC3C N2EC3C N3EC3C N4EC3C N5EC3C N6EC3C N7EC3C N8EC3C N9EC3C N10EC3C  Linear RBF Linear Linear MLP MLP RBF RBF MLP MLP  0.457 0.426 0.425 0.425 0.424 0.415 0.414 0.411 0.411 0.400  0.447 0.421 0.418 0.422 0.420 0.404 0.413 0.414 0.401 0.423  7 4 2 1 5 8 4 4 11 12  -  0.577 0.615 0.615 0.615 0.615 0.615 0.615 0.654 0.692 0.654  70  1 -  3 8 2 1 8 13  Discussion Data Validation  It is important to validate the reliability of the laser data, as the neural network models are only as useful as the data employed to train the networks. To use a hackneyed maxim: garbage in = garbage out. As discussed previously, three major aspects of the data validation were tested. Thefirstset of tests showed the lasers were offset by 0.5 mm, likely due to the difficulty aligning them in the unadjustable measuring apparatus. The second set of tests showed whether the lasers and their apparatus remained constant throughout the measuring process. The results for laser B2 indicated a difference between runs significantly greater than the Within Laser Tolerance of the apparatus, signalling a problem with laser B2 during the sample measurement process. Since the shift appeared to be in one direction, laser B2 was likely bumped upward toward the surface of the measuring bed, causing an offset of approximately 0.723 mm. If an offset was the problem, then the surface analysis was likely not seriously affected, since the measurements were on a relative scale. The third set of tests demonstrated that the sets of data measured by the lasers were repeatable within the tolerance limits. Therefore, no problem was found in the consistency of each of the laser's measurements from one set of data to the next. For this proof of concept, the data was shown to be reliable enough to prove whether or not the neural networks application to detecting machine shape defects was viable. It was recognised that the data was not perfect, which in this instance was viewed as an advantage. If such a system is demonstrated to be successful with imperfect data, it is very likely to work well in a sawmill environment where data collection has inherent problems. The vibration, interference from other equipment, dust and sometimes rough treatment in a sawmill often makes for inaccuracies in data gathering. Therefore, a forgiving and robust system for detecting machine shape defects is ideal. Neural Network Trials  The ideal neural network had low error and high performance, which reflected good classification accuracy, as well as quick classification time, minimal memory usage and fault tolerance (Cornforth 1993). Many of the networks in the results had these advantages to some extent, as those saved in the network set represented a diverse variety of networks. The difficulty was finding a balance between a model's accuracy and its ability to generalise well since having both qualities was not usually possible with commercial applications (Swingler 1996). A simpler model with a smoother curve through the training data generalised well, but 71  missed a few points, thereby reducing the accuracy. Though a larger network more accurately modelled a more convoluted and complex underlying function, the trade-off was that it was more difficult to train, slower to operate and more prone to overfitting (Statsoft 1999). Checking that the verification error and the test error were similar provided some assurance that overfitting had not occurred. Even so, the simplest model was often the best choice. The merits of the neural networks will be discussed in terms of type, complexity, error and performance, bearing in mind that type and complexity are characteristics used to describe the networks, while error and performance are characteristics used to evaluate the networks. Description of Neural Networks  Type of Network  Linear networks (linear), multilayer perceptrons (MLP) and radial basis function (RBF) were the three types of networks reported in the training results for this classification problem. The type of network refers to the pattern recognition algorithm or network architecture, of which examples are shown in the illustrations (Figures 33, 34 & 35).  Input layer  Figure 33. Illustration of N9SC2C, linear network with 6 input variables and 0 hidden units  72  Output layer  Hidden layer Input layer  Figure 34. Illustration of N10BC2C, MLP network with 12 input variables and 7 hidden units  Input layer  Hidden layer  Output layer  Figure 35. Illustration of N10BC3C, RBF network with 4 input variables and 1 hidden unit The RBF and MLP networks appeared to be the most suited to this application of neural networks, having the lower error and higher performance results. For example, in the Base Condition - Two Class Networks, the highest performance and lowest error combination was achieved by an MLP network, N10BC2C, while in the Base Condition - Three Class Networks, this combination was attained by an RBF network, N10BC3C. In fact, MLP networks provided 73  the lowest error solutions for the two-class problems, while RBF networks provided the lowest error solutions for all but the Extended Duration Condition in the three-class problems. This finding was not surprising, since these types of networks were often employed to model nonlinear problems like classification which cannot normally be solved simply by drawing a hyperplane through the data (Statsoft 1999; 2000). Nevertheless, a linear model provides a good benchmark against which to judge more complex networks and sometimes linear techniques can solve a problem that appears difficult and non-linear (Statsoft 1999). MLP and RBF networks are compared below. •  •  •  •  MLP networks are complex with many layers, complicated connections and a variety of different activation functions, while RBF networks tend to be simple, having two layers - one to contain the parameters and one to generate the outputs (Bishop 1995). MLP can be very slow to converge in the training process because it depends on many hidden units to determine the value of the output unit and can get hung up on local minima (Bishop 1995). By contrast the RBF is faster since few hidden units have significant activations, making for quicker decisions from fewer possibilities. An activation is the value displayed by each unit, signifying its influence on subsequent units in the network (Swingler 1996). MLP networks use supervised training to determine all of the parameters simultaneously, whereas an RBF network uses unsupervised training to determine the basis functions, followed by supervised training to find the weights of the hidden layer (Bishop 1995). MLP networks can make unjustified extrapolations with new input data unlike any data encountered during training, but an RBF network will always have a near-zero response to data from outside the normal range.  Complexity  The main consideration with regard to complexity is whether to build a robust model which generalises well or a brittle model which is more accurate (Swingler 1996). The number of input variables and hidden units generally describes the complexity of a network. The higher the number of hidden units, the more complex the network and the more powerful its model. A more complex modelfitsmore data points in the training set, but is less resistant to modelling the effects of noise or data idiosyncrasies, resulting in overfitting. Though it can model a more convoluted and complex underlying function, the trade-off is that this larger network is more difficult to train, slower to operate and more prone to overfitting (Statsoft 1999). A simpler model with a smoother curve through the training data generalises well, but misses a few points, 74  reducing the accuracy. The difficulty is finding a balance between a model's accuracy and its ability to generalise well since having both qualities is not usually possible with commercial applications (Swingler 1996). Often, the simplest model is the best choice. Complexity is also influenced by the type of neural network. MLP networks are more complex than RBF networks, while linear networks tend to be the most structurally simple (Swingler 1996). In this classification application, the least complex networks with good performance and low error were RBF networks. For the Three Class Networks, they were N10BC3C with four input variables and one hidden unit, N10PC3C with one input and three hidden, and N8EC3C with four inputs and one hidden. For the Two Class Networks, they were N5EC2C with one input and one hidden and N6BC2C, also with one input variable and one hidden unit. Evaluation of Neural Networks  Error: Training, Verification & Testing  The three errors reported by Statistica Neural Networks were the training error (TrError), the verification error (VeError) and the testing error (TeError). These errors were used to gauge how well a neural network performed during iterative training and execution (Statsoft 2000). Each one was the root mean square (RMS) of the individual sample case errors, summarised over the subset (Statsoft 2000). The network's error function was used for each sample case and varied depending on the type of network. Sum-squared was the standard error function applied in neural networks training. It is the sum of the squared difference between the target and the actual output values on each output unit for the subset (Statsoft 2000). For nominal variables like SNIPE in this application, Statistica Neural Networks prepared the values for input into the neural networks, and then interpreted them for the network output. Therefore, the actual, target and error output values were reported in nominal form, while the individual RMS errors were reported numerically (Table 18). Table 18. Sample excerpt of the output values for network N10EC3C SAMPLE CASE  Actual SNIPE  Target SNIPE  Error SNIPE  RMS Error  1  n  n  Right  0.340923  2  y  y  Right  0.199405  3  n  n  Right  0.24407  4  n  n  Right  0.159318  5  n  n  Right  0.192806  6  y  y  Right  0.0548  7  n  n  Right  0.255378  75  8  y  y  Right  0.40217  9  n  n  Right  0.320534  10  n  n  Right  0.221628  11  c  Wrong  0.533387  12  y y  y  Right  0.179526  13  n  n  Right  0.379088  14  y  n  Wrong  0.452284  15  n  y  Wrong  0.438431  The verification error was monitored during training to detect overlearning by a rise in its value. Overlearning was a problem for networks because it meant the solution was likely not general enough to make predictions with new data. Another way to check for overlearning was to compare the verification error with the test error. These two errors should be about the same value in order to be confident that overfitting had not occurred and that the network can generalise reliably (Statsoft 1999). A significant difference between the errors indicated that there were too few sample cases for the network performance results to be reliable or that the distribution of sample cases was biased. Troubleshooting a possible bias was done by reshuffling the distribution of the test, verification and training sample cases to see if the verification and test errors converged in the new results. To illustrate, N10BC2C and N9SC2C are drawn for their high performance results from two different network data sets for comparison (Tables 9 & 10). The first, an MLP network, is from Base Condition - Two Classes. Its test and verification errors were 0.461 and 0.473 respectively. The similarity indicates that the network will generalise well. The second, from Shuffled Condition - Two Classes, is a linear network. It had a test error of 0.573 and a verification error of 0.397. This large discrepancy between the test and verification error values signals overlearning and therefore poor generalisation ability. Shuffling again yields a more reliable linear network, N18SC2C, whose test and verification errors had converged to 0.461 and 0.473 respectively. Another network, N20SC2C, had the higher performance, but the difference between its test and verification errors was greater, and therefore, it was a less reliable network. Taking reliability into account, the Three Class Networks with the lowest error were N10EC3C at 0.400, N9EC3C and N10BC3C, both at 0.411. The Two Class Networks with the lowest error were N10PC2C and N10EC2C, at 0.472 and 0.462 respectively. All of these networks were MLP, except for N10BC3C, which was an RBF network.  76  Performance The performance reported in the results accounted for the proportion of sample cases classified correctly by a trained neural network in the verification subset of the data set (Table 9 - 17). Known as the correct classification rate, it is an important indicator of the suitability of a network for the classification application. Having said that, a classification rate considered good for one application may not always be considered a good rate in another application. With the same training data set, the performance should fall within a certain range from one network set to the next. Therefore, unusual performance values in a network set may flag a problem with the reliability of the network results. Troubleshooting the reported errors and/or sample case distributions may help locate the source of the performance problem. This technique was used to assess the results for the Shuffled Condition - Two Class Networks whose networks appeared to have unusually high performances (Table 10). An example is N9SC2C, a linear network with a performance of 0.808. Comparing its verification and test errors of 0.397 and 0.573 revealed a large discrepancy. Since this large difference signalled overlearning as discussed in the previous section, the sample cases were re-distributed to produce the Shuffled Condition Second Round (Table 11). As the second round performance results were in the range of results produced by the other training strategies, the high performance results in the first round were likely a fluke. Therefore, the networks for the Shuffled Condition - Two Class Networks (first round) were not reliable and should not be used in future applications. The networks with the best performances were N10BC2C, N6BC2C, N5EC2C and N9EC3C at 0.692. A dozen networks hac the second best performance of 0.654, with almost half of them in Pruned Condition - Three Class Networks, which had errors ranging from 0.412 to 0.421. Classification A n a l y s i s of S a m p l e C a s e s  The overall proportion of correctly classified sample cases was calculated for the top ten neural networks (Table 19). A sample of this calculation is shown in Appendix E. Forty percent of the 103 sample cases contained snipe or a combination of snipe and another defect. These networks were chosen in difference to the discussion regarding type, complexity, error and performance in the previous sections. Five networks were from the Two Class Networks and five from the Three Class Networks. In the Two Class Networks, the networks were trained to classify the input data into Snipe or No Snipe categories, while in the Three Class Networks, they were trained to classify the input data into Snipe, No Snipe or Combination Snipe categories. The performance reported was the verification performance. The discrepancy between the performance and the overall proportion was due to the random distribution of sample cases 77  between the training, verification, and testing subsets of data, resulting in uneven numbers of correctly classified sample cases in the subsets. Again, a high proportion of correctly classified sample cases was desirable, indicating a low number of misclassified sample cases. Generally, a high number of falsely classified sample cases demonstrated poor predictive accuracy, signalling the network's inability to recognise a pattern in the training data set. Table 19. Classification data for top ten neural networks Correctly classified sample  Misclassified sample cases  cases  Network  SnipeCombo  Overall  Performance  Type I  Type II  N10BC3C  0.612  0.654  1  32  7  N10PC3C  0.612  0.654  1  32  7  N10EC3C  0.718  0.654  8  14  7  N9EC3C  0.718  0.692  2  19  7  N8EC3C  0.612  0.654  0  33  7  N10BC2C  0.748  0.692  17  9  n/a  N6BC2C  0.485  0.692  32  21  n/a  N10PC2C  0.689  0.615  18  14  n/a  N10EC2C  0.689  0.615  21  11  n/a  N5EC2C  0.466  0.692  36  19  n/a  There were two major ways a sample case can be falsely classified, or misclassified, in this snipe classification problem (Table 20). Thefirst,known as Type I, occurred when a neural network classified a sample case as snipe when it was not. In a sawmill, this type of misclassification would likely result in reduced recovery. The second, known as Type II, occurred when a neural network did not classify a sample case as snipe when it was. This misclassification would result in downgraded lumber, since it would likely not be caught until the next grading chain. A third misclassification category, called SnipeCombo, was used to track the errors for snipe combination sample cases in the three-class problem (Table 20). Misclassification by neural networks occurred for several reasons: the type of training data could be unsuited to the application; the input data could contain too much noise; or not enough training data was available in each of the categories to train the network properly (Swingler 1996).  78  Table 20. Types of Misclassification Actual SNIPE  Target SNIPE  Misclassification  y  n  Type 1  n  y  Type II  n  c  SnipeCombo  n  c  SnipeCombo  Two neural networks, N6BC2C and N5EC2C, were listed among those with the best performance rates in the Two Class Networks. However, the overall proportion of correctly classified sample cases was below 0.500 for each of them, so they were no longer considered reliable. This difference highlighted the importance of considering a combination of characteristics before selecting neural networks to be employed in the classification application. In the Two Class Networks, the remaining top three networks had a greater number of Type I versus Type 11 misclassified sample cases (Table 19). The snipe/wedge combination sample cases were manually classed as No Snipe (Target SNIPE = n) in the preprocessing stage. Despite their containing snipe, the neural networks correctly classified the majority of these sample cases, meaning they were not recognised as having snipe (Actual SNIPE = n). However, the other snipe combination sample cases were manually classed as Snipe (Target SNIPE = y) in the preprocessing stage. The neural networks also did not recognise these sample cases as having snipe (Actual SNIPE = n), resulting in a Type II misclassification. This consistency demonstrated the neural networks' inability to recognise snipe combination sample cases as containing snipe. These snipe combination sample cases may confuse the training of the Two Class Networks by introducing too much noise in the data, resulting in both Type I and II misclassifications. Using a larger number of'pure' snipe sample cases and separating the combination sample cases from them would alleviate this difficulty. This option was not available for this project, given the limitation on acquiring lumber samples. In the Three Class Networks, the top five networks had a much larger proportion of Type II misclassified sample cases, as well as a consistent number of errors in the SnipeCombo misclassification category (Table 19). Seven was the number of sample cases with snipe/wedge manually classified as Combination Snipe (Target SNIPE = c) during preprocessing. The neural networks consistently failed to recognise all seven sample cases as Combination Snipe (Actual SNIPE = n). Clearly, there were not enough examples of this defect combination to train the neural networks for the third category. Although many other sample cases were snipe combination boards containing snipe plus one or more defects, they were manually classed as 79  Snipe (Target SNIPE = y) in the preprocessing stage, instead of Combination Snipe (Target SNIPE = c). The neural network classified them as No Snipe (Actual SNIPE = n), resulting in a large proportion of type II error. This result indicated that the neural network failed to recognise these sample cases of snipe combined with another defect, as containing snipe, let alone as Combination Snipe (Actual SNIPE = c). Likely, the additional defect(s) caused too much noise in the data for the snipe to be recognisable. Increasing the number of sample cases with no defects would help alleviate the confusion with No Snipe by reducing the noise in the data. Additional examples of each of these snipe combinations would also improve the training accuracy. Aside from N6BC2C and N5EC2C, the overall proportion of correctly classified sample cases was quite satisfactory, ranging from 0.612 to 0.748. This accuracy is not adequate for a standalone application of neural networks classification in a sawmill; however, it is enough to show that with improvements to the training process, this approach is viable. Furthermore, the remaining neural network models fared better than the base prediction rate, since the percentage of Type I is less than 60%. Finally, the total number of sample boards was considered adequate to prove this concept of training neural networks to differentiate between a board with snipe and one without snipe.  80  Conclusions This proof of concept demonstrated that neural networks can be applied with limited success to detect machine shape defects, in particular snipe, in random samples of rough green lumber. More work and resources are required to perfect the detection of machine shape defect combinations. However, the robustness of this approach using neural networks is appealing for the type of data-gathering environment encountered in the wood products industry where rugged forgiving instruments and systems are necessary. The ideal neural network has low error and high performance, which reflects good classification accuracy, as well as quick classification time, minimal memory usage and fault tolerance (Cornforth 1993). The difficulty is finding a balance between a model's accuracy and its ability to generalise well since having both qualities is not usually possible with commercial applications (Swingler 1996). A simpler model with a smoother curve through the training data generalises well, but misses a few points, reducing the accuracy. A larger network can more accurately model a more complex underlying function, but it is more difficult to train, slower to operate and more prone to overfitting (Statsoft 1999). Checking that the verification error and the test error are similar provides some assurance that overfitting has not occurred. Even so, the simplest model is often the best choice. Of the three types of networks reported in the training results, RBF and MLP networks were the most suited to this application of neural networks, having the lower error and higher performance results (Tables 9 - 17). That said, linear networks were still considered as they may provide a simpler network solution and they make good benchmarks for comparison. N10BC2C, N9EC3C and N10BC3C were considered the three best networks suited to this snipe classification problem. Thefirstone was a Two Class Network, while the last two were Three Class Networks. N10BC2C was an MLP network with low error and one of the best performances. N10BC3C, an RBF network, was a simpler model than N9EC3C, but N9EC3C, an MLP network, had a higher performance. None of these networks appeared to be prone to overlearning. While the neural networks were able to discern between sample cases with snipe and those without snipe, they were unable to recognise the combination snipe sample cases as anything but No Snipe (Actual SNIPE = n). This problem may be overcome by training the neural networks with many more sample cases of boards containing the snipe defect and by classifying the 'pure' snipe sample cases separately. Increasing the number of board samples containing no defects would also improve the training accuracy of the neural networks. Acquiring and processing this amount of data required many more resources than were available at the time of this project. 81  Future Applications Future applications would classify all six of the common machine shape defects produced in sawmills. The neural networks would differentiate between the machine shape defects and their combinations. As this future application is essentially an extension of this project, it is clear that the increase of output categories will require a magnitude of additional raw data to train the networks. An automated measuring system and tailored programming for the preprocessing step is recommended for such a large volume of data. Specifically, the measuring apparatus should be adjustable, in order to finetune the alignment and accuracy of the lasers. It is also suggested that the training data be obtained from a wide base of sawmills in order to cover the different variations of machine shape defects experienced by those mills surveyed in Part I of this thesis. In future applications, it is recommended to consider modelling the top and bottom surfaces of the board separately for input into the neural networks, instead of modelling the 3-D shape of the board. This strategy would reduce the complexity of the pattern recognition problem by reducing the number of input variables. An added benefit is that the number of sample cases per board would be doubled by using a data set for each surface instead of one for each 3-D shape. Further development of this system would be a troubleshooting guide for analysing various process problems based on machine shape defects and their causes. Ultimately, an automatic or expert system could be built, incorporating neural networks to classify the sample boards by machine center or by defect causes. Factors like defect characteristics and their locations on the board are variables useful for training the neural networks to recognise the machine centre causing the problem. The machine shape defect would be detected and then classified by machine source, triggering, futuristically speaking, a self-diagnostic sequence whereby the machine centre would adjust its settings or notify maintenance.  82  SUMMARY Thefirstpart of this thesis determined the machine shape defects that were most important to the BC sawmill industry, while the second part dealt withfindinga way to detect and classify these defects. Consequently, the tools used to research each part of the problem differed. The first phase employed a province-wide survey to collect data and statistical probability methods for the analysis, while the second phase employed laser-based measurement to collect data and explored the use of neural networks for the classification analysis. Both of these methods produced results which prove interesting for future endeavours toward improving the production of lumber, both immediate and longterm. The majorfindingsand results are summarised in this section with respect to the main objectives of the thesis. The main objective of the survey of the sawmills in British Columbia was to determine the industrial significance of each of the six major machine shape defects. These defects are wedge, flare, taper, snipe, thin snake and fat snake. The survey results not only established that snipe, taper and thin snake are the top three machine shape defects having the greatest impact on the BC industry, but also identified the most common causes of these and the other machine shape defects. The analysis showed that to reduce the production of machine shape defects, sawmills need to focus on improving three critical areas: Piece Stability, Saw Condition, and Alignment. The main objective of the neural network classification was to establish that neural networks can detect machine shape defects found in rough green lumber, using the snipe defect for the proof of concept experiment. Laser displacement sensors measured the surface of the sample boards, three along the top and three along the bottom. A statistical model was developed to interpret the shape characteristics of the top and bottom surfaces, producing a set of regression coefficients for each sample board. This preprocessing step simplified the training task by reducing the number of input variables to the neural networks. The classification results demonstrated that it is feasible to use neural networks to detect and classify machine shape defects found in rough green lumber. However, it was evident that these preliminary neural networks were unable to recognise sample cases with combination snipe. This type of misclassification is attributed to the lack of data representing snipe combined with other defects. Much more data is required to train and develop neural networks to perform this machine shape defect classification task with a better degree of reliability. Future research would expand the classification to all six machine shape defects commonly found in British Columbia sawmills. It would address the misclassification of the combination snipe, enabling the neural networks to differentiate between the defects and their combinations.  83  BIBLIOGRAPHY Bishop, C M . Neural Networks for Pattern Recognition. New York: Oxford, 1995. Bluman, A.G. Elementary Statistics: A Step by Step Approach. 3rd ed. San Francisco: McGrawHill, 1997. Brown, T.D. Quality Control in Lumber Manufacturing. San Francisco: Miller, 1982. Brown, T.D. "Lumber Size Control." Special Publication No 14. Corvallis: Forest Research Lab, Oregon State Univ., 1986. Canadian Wood Council. Wood Design Manual. 3rd ed. Ottawa: CWC, 1997. Cornforth, D.J., and D.G. Elliman. "Modelling probability density functions for classifying using CMAC." Techniques and Applications of Neural Networks. Eds. P.J.D. Lisboa and M.J.Taylor. New York: Ellis, 1993. 36-46. de Silva, C.W. Robot Control: Sensors and Actuators. Vancouver: Univ. of British Columbia, 1995. Dillman, D.A. Mail and Internet Surveys: The Tailored Design Method. 2nd ed. New York: Wiley, 2000. DSM Engineering Plastic Products. Products and Applications Guide. 1999:14. Eggers, M., and T. Khuon. "Neural network data fusion for machine-based decision making." Neural Networks: Concepts. Applications and Implementations. Vol. 4. Eds. P. Antognetti and V. Milutinovic. Englewood Cliffs: Prentice, 1991. 240-274. Fukunaga, K., and J.A. Young. "Pattern recognition and neural engineering." Neural Networks: Concepts, Applications and Implementations. Vol 1. Eds P. Antognetti and V. Milutinovic. Englewood Cliffs: Prentice, 1991. 10-32. Globerman, S., et al. Technological Progress and Competitiveness in the Canadian Forest Products Industry. Ottawa: Science Branch and the Industry, Economics and Programs Branch, Canadian Forest Service, Natural Resources Canada, 1999. Hicks, CR. Fundamental Concepts in the Design of Experiments. 4th ed. New York: Oxford, 1993. Industrial LPS Laser Distance Sensor. LMI Measurement & Control. 29 May. 2000. <http://www.lmint.com/cfm/index.cfm> Kennedy, X.J., D.M. Kennedy and S.A. Holladay. The Bedford Guide for College Writers with Reader and Research Manual. Boston: Bedford, 1999. Lee, L., D. Cohen and T.C. Maness. "Markets and Technology in Western Canadian Sawmills." Forest Products Journal 49.7/8 (1999): 36-42. Lehmann, B.F., and S.G. Hutton. "The alignment, set-up and operation of circular saws and handsaws." Proceedings of the 9 International Wood Machining Seminar. Richmond: Forest Products Laboratory, Univ. of California, 1988. 68-79. Lehmann, B.F. "The cutting behavior of handsaws." Diss. Univ. of British Columbia, 1993. th  84  Lehmann, B.F. "Eliminating Barriers to Sawing Performance." Abbotsford: Value-Added Skills Centre, 2000. Lehmann, B.F. Personal interview Oct. 2000. Lemaster, R.L., and J.S. Stewart. "Progress in evaluating surface quality using optical profilometer techniques." Proceedings of the 12th International Wood Machining Seminar. Richmond: Forest Products Laboratory, Univ. of California, 1995. 81-91. Lumber Thickness Measuring. Softac Systems Ltd. 31 Oct. 2000. <http://www.westcanex.ca> Madison's Canadian Lumber Directory. 44 ed. Madsen, B. "Improved lumber Trade through better lumber grading rules." Forest Products Trade: Market Trends and Technical Developments. Eds. J.A. Johnson and W.R. Smith. Seattle: Univ.of Washington Press, 1988. 139-144. Maness, T.C. "Real time quality control system for automated lumbermills." Forest Products Journal. 43.7/8(1992): 18-19. Maness, T.C, and D.H. Cohen. "Quality Control in BC Sawmills: State of the Art." FEPA Working Paper 194. Vancouver: Forest Economics and Policy Analysis Unit, 1993. Maness, T.C, D.H. Cohen and R. Smith. "An assessment of quality control in British Columbia sawmills: production, employment, trends and priorities." Forest Products Journal. 44.3 (1994): 27-33. Maness, T.C, and Y. Lin. "The influence of saw kerf and target size reductions on sawmill revenue and volume recovery." Forest Products Journal. 45.11/12 (1995): 43-50. Maness, T.C, C. Staudhammer and R.A. Kozak. "Statistical considerations for real-time size control systems in wood products manufacturing." Wood and Fiber Science. 34.3 (2002): 476-484. Montgomery, D.C Introduction to Statistical Quality Control. 3rd ed. New York: Wiley, 1996. National Lumber Grades Authority. Standard grading rules for Canadian lumber. New Westminster: NLGA, 2000. National Lumber Grades Authority. Canadian Lumber Grading Manual. New Westminster: NLGA, 1998. Norman, F.T., and J.T. Todd. "The visual perception of a 3-dimensional form." Neural Networks for Vision and Image Processing. Eds. G.A. Carpenter and S. Grossberg. Cambridge: MIT, 1992. 93-110. Ott, E.R. Process Quality Control: Troubleshooting and Interpretation of Data. New York: McGraw-Hill, 1975. Pham, D.T., and R.J. Alcock. "Automated grading and defect detection: a review." Forest Products Journal. 48.4 (1998): 34-42. Rhemrev, J., et al. "Laser scanning for the detection of twist or warp in pencil slats." Forest Products Journal. 43.2 (1993): 29-36. th  85  Rosenberg, N., et al. "Understanding the adoption of new technology in the forest products industry." Forest Products Journal. 40.10 (1990): 15-22. Sharkey, N.E., and D. Partridge. "The statistical independence of network generalisation: an application in software engineering." Techniques and Applications of Neural Networks. Eds. P.J.D. Lisboa and M.J.Taylor. New York: Ellis, 1993. 22-33. Skrzypek, J., and J. Hoffman. "Visual recognition of script characters and neural network architectures." Neural Networks: Advances and Applications. Ed. E. Gelenbe. New York: Elsevier, 1991. 109-144. Solla, S.A., and Y. le Cun. "Constrained neural networks for pattern recognition." Neural Networks: Concepts, Applications and Implementations. Vol 4. Eds. P. Antognetti and V. Milutinovic. Englewood Cliffs: Prentice, 1991. 142-161. Statsoft, Inc. Statistica Neural Networks: Addendum for Version 4. Tulsa: Statsoft, 1999. Statsoft, Inc. Statistica Neural Networks. Tulsa: Statsoft, 2000. Swingler, K. Applying Neural Networks: A Practical Guide. San Francisco: Morgan, 1996. Szymani, R., and K.A. McDonald. "Defect Detection in lumber: state of the art" Forest Products Journal. 31.11 (1981): 34-44. Thunell, B. "Some aspects of deviations in shapes and sizes of lumber." Proceedings of the 9 International Wood Machining Seminar. Richmond: Forest Products Laboratory, Univ. of California, 1988. 55-67. Thomlinson, W.W. "Sawmill headrig and edger optimization." Proceedings of the Application of Advanced Informatics to the Forest Industry. Vancouver: Forest Sector 2000, 1992. Tulunay, E. "Introduction to neural networks and their application to process control." Neural Networks: Advances and Applications. Ed. E. Gelenbe. New York: Elsevier, 1991. 241273. Williams, P., and A.W.G. Duller "Identification of lighting flicker sources using a neural network.." Techniques and Applications of Neural Networks. Eds. P.J.D. Lisboa and M.J.Taylor. New York: Ellis,1993. 183-197. Williston, E.M. Lumber Manufacturing - The Design and Operation of Sawmills and Planer Mills. San Francisco: Miller, 1988. th  86  APPENDIX A - LUMBER SHAPE DEFECT  87  SURVEY  APPENDIX B - TOLERANCE CALCULATION  90  Appendix B - Tolerance Calculations Laser Jig (height tolerance mm)(conversion) = height tolerance divisions (0.1mm)(4095/10) = 40.950 divisions Laser (accuracy mm)(conversion) = accuracy divisions (0.05mm)(4095/10) = 20.475 divisions Calibration Block 2(height tolerance mm)(conversion) = height tolerance divisions 2(0.0254mm)(4095/10) = (0.0508mm)(4095/10) = 20.803 divisions Acquisition Card (precision mV)/(1000V/mm)(conversion) = precision divisions (2.44mV)/(1000V/mm)(4095/10) = (0.00244mm)(4095/10) = 0.999 divisions Within Laser Tolerance Laser accuracy + Calibration Block height tolerance + Acquisition Card precision 0.05mm + 0.0508mm + 0.00244mm = 0.103mm 20.475 divisions + 20.803 divisions + 0.999 divisions = 42.277 divisions Total Tolerance Within Laser Tolerance + Laser Jig height tolerance 0.103mm + 0.1mm = 0.203mm 42.277 divisions + 40.950 divisions = 83.227 divisions  APPENDIX C - PREPROCESSED DATA  C.1 Preprocessed Training Data Set C.2 R and F Values 2  92  Appendix C.1 - Preprocessed Training Data Set - Three Classes T1I T1S T2I T3I 3066.494 0.021583 43.75326 -140.3294 3452.795 -1.96361 60.46703 65.59174 3243.287 0.047226 342.8759 266.917 2813.631 0.148993 327.465 809.438 2919.055 0.226443 351.9855 185.2099 3336.534 -1.047375 180.6965 -100.301 3137.52 -0.330908 387.1242 657.8254 3182.105 -0.01321 -133.4159 -426.901 3159.124 -0.148085 304.0406 -35.10235 3122.039 -0.298314 -260.5891 -394.9851 3044.323 -0.834775 -1228.508 -2582.925 3355.118 -0.953927 -187.198 -1038.498 360.2342 0.857917 1456.052 2618.77 2769.463 0.293701 130.3985 345.5373 3176.456 -0.318523 -21.54919 -395.2475 2355.895 -0.205486 -93.3854 -201.0755 3960.514 -0.392722 109.9491 80.9856 111.2042 1.211995 1643.701 2926.94 2990.771 0.215286 273.2027 144.2092 770.2947 0.354264 2199.229 2249.446 2983.182 -1.867368 122.9668 132.3456 3059.432 -0.011473 -49.85836 130.7968 2998.543 0.226883 44.95706 -70.49982 3165.577 -0.269009 107.9934 -222.6145 3226.758 -2.293772 -115.9595 -194.3108 1272.403 2.088657 856.6414 1917.441 3186.603 -0.131006 280.503 165.4249 4403.157 -1.161466 -154.1706 -180.288 2453.288 0.526705 190.6188 -223.9976 3076.036 0.068643 45.08449 -28.72558 3610.343 -0.476595 -54.74629 -91.93777 3104.621 -0.128592 182.73 40.82814 3044.426 0.047286 168.2047 -32.00982 2999.094 0.133691 188.5759 -1131.528 873.4713 1.32772 2785.915 2978.4 2973.932 -1.648814 -83.14168 125.4307 3182.407 -0.467138 -300.3617 -624.0426 2882.216 0.445825 63.50887 37.12449 2866.922 0.927931 241.9931 63.6701 2701.007 0.075974 11.39572 -166.9126 -821.4258 3.960132 3955.201 3889.307 1972.477 0.848582 -231.9433 -1521.234 3138.694 -0.064796 -1773.186 -3283.062 3685.156 -0.191385 -27.55075 247.9817 289.5983 2.630378 3300.597 3995.915 3193.108 0.231101 167.615 2.533339 3331.839 -0.093813 132.3711 97.68966 3001.301 0.381488 68.81617 43.52809 •3494.311 -0.372123 8.005995 -53.10995 2592.718 0.458141 -150.3885 -273.0551 2806.78 0.240831 -137.5161 -220.9797 2345.195 -1.371111 378.8489 193.2207 3271.978 -0.089434 66.24158 55.90277 2845.949 0.28077 119.8714 17.49576 3401.128 -0.469667 54.33588 -92.81985 2913.478 -0.247562 492.3107 83.79434 3038.84 0.082861 -82.74281 -279.6155 2973.759 0.087115 51.30448 -1.941471 3117.089 -0.05493 60.56411 -250.9598 3686.233 -0.844569 242.0808 498.9543 -66.69034 0.404813 1817.635 3436.442 2875.498 0.22861 238.6125 1396.823 1463.092 1.271166 2.923783 -176.63 3308.215 -0.216598 83.2304 -129.6316 2586.308 0.162981 575.5129 675.7705  T2S T3S 0.147171 0.271966 0.527401 0.32953 -0.365171 -0.670655 -0.205288 -0.546436 -0.157902 -0.069693 0.05474 0.333925 -0.114811 -0.293735 0.188405 0.448738 -0.03872 0.262073 0.031949 -0.243418 1.946272 3.165127 0.002925 0.394192 -0.485379 -0.888149 -0.067877 -0.377713 0.428539 0.568184 0.437785 0.789415 0.053738 0.124727 -0.666883 -1.181393 -0.195699 -0.122784 -0.678126 -0.401341 -0.263549 -0.176896 0.161058 -0.014901 0.158834 0.480692 0.03818 0.231763 0.290787 0.507826 -1.194334 -2.687156 -0.097446 -0.013255 0.782676 0.81145 -0.09063 0.267561 0.347027 0.505623 0.215254 0.352743 -0.01156 0.098098 -0.038681 0.209839 -0.022112 1.701665 -3.582136 -2.466797 0.979079 0.985282 0.450625 0.969668 -0.053758 -0.065123 -0.712226 -0.271561 0.243096 0.472716 -3.90014 -3.744296 -0.093156 0.314924 1.385796 2.339668 0.054991 -1.404559 -2.774541 -3.145692 -0.499358 -1.516066 0.035701 0.101392 0.022518 0.098962 0.226007 0.336449 0.128027 0.28838 0.287901 0.347324 0.559774 0.916753 0.064982 0.142149 -0.062911 0.12066 0.104138 0.25099 -0.274907 0.378438 0.324411 0.848927 0.088964 0.286396 0.086485 0.532384 0.291136 0.206864 -0.463299 -0.963574 -0.155462 -0.729629 -0.023395 -0.000329 0.160107 0.25476 -0.31044 -0.367901  B1I 2273.698 1318.974 2163.037 2207.778 2050.216 1585.082 -133.1121 2093.618 2052.498 2244.921 2228.404 2223.133 1767.657 1582.346 2035.268 1960.255 1995.079 1472.6 1634.941 1964.375 2310.059 2249.874 2139.585 1787.481 2327.601 1775.726 2212.505 2253.151 1583.647 1956.857 1814.429 2145.363 2297.294 2185.173 1846.719 1994.084 1936.606 2229.217 2083.673 2119.886 1467.704 2229.411 2100.263 2111.214 2058.641 2154.499 1796.648 1990.973 1657.277 2059.934 2245.54 2208.217 2190.181 2278.865 139.218 846.4169 1901.02 1335.558 2219.67 -536.5944 1220.851 2220.253 -501.0328 1938.973 2082.567  q3  B1S -0.199011 0.311067 -0.252669 -0.250582 -0.03621 -0.785388 0.614336 -0.548705 -0.348927 -0.18367 -0.227644 -0.101109 -0.127702 0.748069 -0.029319 -0.810131 0.005266 0.330867 0.150701 0.038632 -0.456059 -0.111209 0.028337 0.018474 -0.403207 0.50419 -0.256456 -0.157409 -0.006892 -0.27604 0.213426 -0.070461 -0.297092 -0.029297 0.229797 -0.083098 -0.187891 -0.495915 0.109561 -1.193946 -1.25955 -0.139725 -0.030496 -0.016417 -0.101289 -0.069726 -1.569155 0.024686 -0.197838 -0.112475 -0.258938 -0.462108 -0.142627 -0.320567 1.165136 -0.064659 0.071104 -0.890981 -0.182129 1.673922 -0.104806 -0.13274 1.251152 -0.011126 -0.416885  B2I 472.1326 234.838 129.7254 391.5199 290.4566 472.9158 1813.104 444.4117 274.1856 233.9773 158.9085 436.2695 494.0686 988.0848 303.3031 378.1692 504.8675 536.7137 963.5601 485.7961 306.2008 145.1393 478.8075 580.2317 376.5429 895.2546 197.5518 150.8417 -160.9124 250.7962 208.2445 223.5057 194.0033 236.7484 366.5344 512.3511 234.6811 239.982 428.6297 292.245 967.9663 48.95832 48.04367 172.6713 398.5227 205.84 881.7852 329.4359 -103.8478 345.8406 180.6266 197.76 127.7882 215.6799 2247.946 862.3344 280.274 1044.302 165.6983 2126.189 560.3743 69.69501 2568.136 144.8974 -59.18926  B3I -89.42501 189.6305 -549.354 -676.7671 -108.2994 -76.66901 2159.017 -84.76856 -109.8918 406.4133 •453.323 -153.1315 56.63664 398.4366 -153.9874 -205.8688 -330.5662 625.7862 435.5234 -19.35454 -321.5172 -256.6824 -86.13334 -71.68633 -199.4951 81.88602 -91.98968 -258.2195 466.8856 3.451152 -61.77981 -30.22996 -367.6941 -569.7604 -45.73176 36.95532 -239.189 -83.82802 40.76183 -129.4256 397.628 -353.8076 -478.8206 -210.5555 1404.213 -127.6699 85.29914 -65.06395 -394.0575 -152.9826 -156.5969 -392.9452 -148.5634 -127.5672 1796.055 812.2396 40.14458 715.2208 -68.3063 2403.093 597.8142 1693.266 1808.263 -55.44943 -285.0873  B2S -0.068606 0.33902 -0.855288 0.22143 -0.004142 -0.173594 -1.186632 -0.173438 0.043765 0.156079 0.231972 -0.064779 0.053043 -0.779268 -0.017137 -0.150372 -0.168605 -0.106174 -0.208574 0.042261 0.179537 0.130216 -0.238906 -0.32454 0.040784 -0.749828 0.086147 -0.026715 0.201777 0.045438 -0.030918 -0.074499 0.135994 -0.013299 -0.247594 -0.158798 -0.274063 -0.002217 -0.687706 -0.084042 0.677921 0.108526 -0.00253 -0.004614 -0.437005 -0.027163 0.487211 -0.103148 0.264652 -0.014893 0.043469 -0.179192 0.071118 -0.185188 -2.739902 -0.2162 -0.232661 0.634137 -0.088674 -1.116596 0.273931 0.112767 -2.134961 -0.076785 0.127896  SNIPE B3S -0.119765 n -1.336823 y -0.251344 n 0.652856 n -0.017374 n 0.221904 y -0.94362 n -0.174183 y -0.017271 n -0.084703 n 0.150154 c 0.020831 y 0.025163 n -0.621995 n -0.099932 y 0.189568 y -0.135575 n -0.742884 c -0.217322 y -0.002422 y 0.461995 y 0.116097 n -0.174155 n -0.306248 n 0.165038 y -0.230938 n -0.515798 y -0.013327 n 0.263596 y 0.005983 n -0.16715 y -0.331594 n 0.828758 c 0.700206 n -0.192058 c -0.041248 n -0.17793 y 0.122855 n -0.555456 n 0.501428 y 1.402987 y 0.225651 n -0.007735 c 0.058801 y -0.880104 y -0.055166 y 1.565117 y -0.059225 n -0.097574 n -0.002875 n 0.107223 n 0.817587 y 0.017547 n -0.044684 n -1.194212 y -0.574922 y -0.235962 y 0.588109 n -0.228344 n -1.609651 n 0.286254 n -0.990086 n -1.28713 n -0.140372 n 0.217049 n  3618.636  -0.611472  23.37526  415.1334  0.508675  0.014569  2417.253  •0.848859  -223.6597  -1110.583  0.720121  1.412873 c  3267.936  -0.415887  •96.39268  -296.7896  0.266534  0.528099  2157.718  •0.067404  152.7762  -1.100886  0.043351  -0.174237 n  -408.2357  2.299462  3192.011  2103.25  •3.595609  -2.468455  2371.119  •1.304976  37.0704  -381.9374  0.988253  1.094296 n  3236.276  -0.280051  •696.5063  -1379.605  0.578253  1.262629  2496.294  •1.656003 -290.5198  -944.7194  0.131092  0.606054 n  3413.079  -1.833255  -234.693  -408.2458  1.863674  1.916632  1965.866  0.21284  429.8996  149.3236  •0.304267  -0.356544 y  3032.606  -0.051425  99.91801  12.41715  0.096384  0.136763  2223.311  •0.239275  39.16746  -303.9865  0.188883  0.361951 n  2879.532  0.104694  225.2638  263.0184  0.006064  0.117845  2089.631  •0.005124  166.8503  -224.0508  •0.065315  0.137743 n  2530.011 -0.067123 2396.354 0.470615  91.20008  -86.31038  0.172432  0.067408  1913.992  •0.031362  289.2213  178.1312  •0.043729  -0.150324 n  •0.021699 -0.061054 0.677517 1.6039  2197.187  •0.111182  21.97258  -264.2304  0.147745  0.205615 n  1487.642  0.292023  77.85181  -576.8432  •0.010889  0.23665 n  2207.636  •0.086048  97.48312  -166.1811  0.033811  0.063077 n  117.5967  60.66123  -0.27673  •1778.956  -3518.585  3021.079 -0.015551 2707.165 0.32364  122.7335  60.72391  0.0552  437.7275  244.1502  •0.413952  431.2458  717.3981  •0.224417 0.458391  3013.274  2999.26  -0.153453  -135.1969  1.913781  2385.287  0.756326  1073.693  2.524186  2915.228  0.048903  2827.75  -0.148178  2204.934  0.520674  2403.185  0.331867  1226.686  1.883873  3214.491  -0.204978  2028.533  0.841597  2558.157  0.675001  3066.116  -0.703226  0.103364  1980.354  •0.572799  19.19302  ^40.2854  -0.463354  1938.59  •0.324484  112.8258  -155.873  0.670522  2232.521  •0.128361  169.4307  -126.263  2073.586  0.013971  349.745  -44.02498  -0.540005  882.8714  357.0833  •30.31878  153.5361  489.8694  0.19858  0.139397  273.1182  •0.671365  0.247138  2083.948  0.090988  270.0674  61.98087  -10.4459  -107.801  0.047205  0.172129  2219.007  •0.110893  176.9734  -89.9286  -285.4727  -577.9301  250.824  -42.40317  21.93823  6.938246  975.349  585.5192  •1530.941 260.6596 •71.81626 165.7344 •195.4185 366 537.5907 25.54893  -2859.253  1.06814  1.237837  2170.505  •0.045967  83.72934  -0.16502  0.898056  2125.295  0.034452  -1216.444  0.157764  1.98733  1646.421  0.247621  1489.343  1.019  416.5182  0.462605  0.331293  -840.3809  0.550203  1.819983  2149.312  0.021963  473.1252  -0.661654  -0.536688  2210.948  •0.321076  612.4358  •0.455349  -0.411041  2153.558  -0.121457  39.47881  0.322498  0.599741  2227.568  0.016971  -1.0352 -1.373052  1974.583  0.100349  1098.356  1.513392  571.262  1200.34  2971.111  0.168355  113.9289  153.699  0.133409  0.158451  2154.545  -0.055158  3337.467  -0.282611  83.30526  148.9244  0.16366  0.192259  1855.437  -0.089422  -965.5856  2.651999  402.3248  -173.9525  -0.314313  0.338863  1680.258  0.154492  •0.466192  2052.401  -0.043624  •1.468097 -1.320372  2380.163  -0.419902  2101.279  0.107263  3485.21  -0.410314  550.934  802.7574  -728.8021  1.803898  3455.554  3581.278  -0.240393  3173.159  -0.141253  50.4302  -28.89817  0.214559  0.466437  2935.267  -0.00402  104.8083  -155.7774  -0.031099  0.1262  2153.88  2825.562  -0.031904  224.6893  432.8791  0.058921 -0.139338  2213.611  0.696811  188.0627  2691.977  106.8346  -0.083471  0.209153  112.1553  -140.5883  128.2168  -143.654  118.7556  -125.6376  482.5844  166.2423  123.0641  -98.62788  89.22365  -166.4936  626.6282  -180.6823  -118.0337  1566.024  38.35556  -220.0884  0.436721 y -0.533289 y  0.016778  0.025662 n  •0.221682  -0.052443 n  •0.351102  -0.27707 n  •0.002832  -0.000767 n  0.085632  0.195851 n  •0.207972 -0.094234 n 0.174502 -0.428502 n •1.504776  -1.396429 n  0.729091  0.162135 n  0.029238  0.050414 n  0.066804  0.107659 y  •0.271669  -0.28709 y  •0.713891  -0.845343 n  •0.057564  -0.052078 y  •0.178288  -0.229292 n  -0.446744  0.035972 n  0.299111  -0.799548 n  0.242845  0.264528 c  -0.438271  -0.536026 n  287.7293  40.48378  -0.149024  0.025109 y  193.6474  -236.9248  0.162781  0.824004 y  -0.45489  136.4607  -391.0676  0.130087  0.586458 n  0.112202  49.84797  -406.4367  •0.302766 -0.298335 n  -0.024457  2036.929  ^t45.7112 -356.0368  0.001259 •0.067829  2430.516  -0.17465  276.4288  369.4666  -0.145262  -0.236277  2018.172  0.143504  308.466  73.33412  -0.18736  -0.221735 y  2787.666  -1.604364  142.5892  -713.9551  1.466209  2.411823  1826.718  0.151275  467.0414  257.8055  0.073222  0.053829 y  281.4155  3.211378  2090.784  1506.114  -2.300303  -1.845429  2224.066  -0.056789  46.51321  -203.1668  Appendix C.2 - R and F Values by Sample Board 2  Sample  Bottom Surface  Top Surface  Board  F value  F value  1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50  1409.6 117.7 736.8 3141.6 297.0 2360.0 182.8 184.9 936.4 83.7 863.4 629.8 359.6 1265.9 555.9 1245.2 269.0 354.0 997.3 89.8 296.0 1562.6 508.6 272.8 690.1 759.9 504.0 297.6 120.9 492.3 649.5 88.1 33.0 95.8 694.0 321.6 171.5 222.2 266.7 128.8 728.9 342.1 1045.6 28.6 210.8 1064.4 1029.6 1096.1 538.7 1224.2  R value 2  0.6859 0.1997 0.4908 0.8158 0.3592 0.7640 0.2412 0.2453 0.5421 0.1375 0.5373 0.5273 0.3199 0.6186 0.5004 0.6143 0.3053 0.3217 0.5275 0.1353 0.3774 0.7934 0.4955 0.2563 0.5856 0.5625 0.5263 0.3545 0.1332 0.4868 0.4561 0.1691 0.0442 0.1094 0.4754 0.4333 0.2355 0.3767 0.3645 0.1837 0.5414 0.3992 0.5703 0.0657 0.3329 0.5900 0.5648 0.5837 0.3989 0.6004  85.7 323.1 201.7 1027 114.7 298.4 192.5 141 1008.9 399.4 114.3 121.2 2598 280 374.8 1144.3 25.5 3610.1 109.5 35.3 61.2 249.5 210.1 140.9 248.4 67.5 132.9 429 26 39.3 522.4 59 256.8 414.8 1532.8 2246.1 122.1 46.8 56.3 62.4 135.4 465 819.9 11.3 73.3 138.6 472.2 4744.5 431.4 3708.2  R value 2  0.1172 0.4062 0.2088 0.5915 0.1780 0.2904 0.2509 0.1986 0.5606 0.4320 0.1332 0.1767 0.7727 0.2640 0.4031 0.5941 0.0401 0.8287 0.1092 0.0579 0.1113 0.3802 0.2886 0.1512 0.3371 0.1025 0.2265 0.4419 0.0320 0.0704 0.4030 0.1199 0.2650 0.3473 0.6670 0.8423 0.1799 0.1129 0.1080 0.0984 0.1798 0.4746 0.5098 0.0271 0.1478 0.1577 0.3731 0.8585 0.3470 0.8198  Snipe Classification  n y n n n y n y n n c y n n y y n c y y y n n n y n y n y n y n c n c n y n n y y n c y y y y n n n  51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103  43.4 153.0 428.7 172.3 265.1 196.0 55.4 96.8 185.5 760.3 1572.3 2929.0 2697.7 308.3 460.1 254.7 198.2 594.8 361.4 330.2 510.7 218.5 384.6 477.7 134.5 81.8 355.0 300.1 373.2 260.5 181.0 578.9 13.9 138.2 259.3 339.1 91.8 1529.8 386.7 17.3 94.0 78.2 82.6 173.7 66.4 226.7 781.5 193.7 201.5 157.1 598.6 155.1 136.0  0.0918 0.2098 0.5037 0.2966 0.3826 0.3040 0.1140 0.1233 0.1961 0.4844 0.6861 0.8047 0.8081 0.3532 0.5681 0.2368 0.2993 0.4359 0.3659 0.3506 0.3996 0.2225 0.4065 0.3802 0.1811 0.1164 0.3144 0.2813 0.3339 0.2622 0.3298 0.6119 0.0397 0.2111 0.2628 0.2986 0.2068 0.7185 0.3119 0.0378 0.1814 0.2152 0.1559 0.2837 0.1528 0.2543 0.4896 0.2313 0.3741 0.2919 0.4204 0.2896 0.1456  112.2 189.2 57 39.9 12.8 81.4 274.1 375.3 149.9 2768.8 3172.2 9973.2 1018.9 77.8 18.8 196.4 15.2 864 939.3 348.2 114.9 295.1 535.2 2717.8 492.7 80 174.6 2525.8 333.4 22.1 313.4 120.6 149.1 110.5 982.9 60.3 84.5 993.7 1080.1 54.8 81.8 32.4 29.8 702.4 18.3 1264.2 380.7 21.8 92.2 85.3 71.2 25.6 3749.2  0.2070 0.2472 0.1180 0.0889 0.0288 0.1536 0.3888 0.3527 0.1647 0.7738 0.8151 0.9334 0.6139 0.1212 0.0509 0.1931 0.0318 0.5288 0.5999 0.3628 0.1302 0.2787 0.4880 0.7773 0.4474 0.1142 0.1841 0.7671 0.3092 0.0292 0.4600 0.2472 0.3074 0.1762 0.5745 0.0704 0.1936 0.6238 0.5587 0.1106 0.1616 0.1021 0.0625 0.6156 0.0474 0.6554 0.3184 0.0327 0.2147 0.1829 0.0795 0.0630 0.8245  n y n n y y y n n n n n n n n c n n n y n n n n n n y y n n n n n n n n n n y y n y n n n c n y y n n y y  APPENDIX D - DATA VALIDATION S U M M A R Y O F RESULTS  D.1 No Change in Apparatus D.2 Laser Measurement Repeatability  97  Appendix D.1 - No Change in Apparatus 1st test hypothesis  2nd test hypothesis  For both tests  Ho:uD = 0 Ha: uD not equal 0 alpha = 0.05 n = 900 t crit = 1.962  Ho: uD < tolerance Ha: uD > tolerance tolerance = 42.3 divisions alpha = 0.05 n = 900 t crit = 1.647  I statistic = (Rbar - uD)/(sD/SQRT(n)) Where Rbar: Average of Residuals uD: expected mean of residuals sD: standard deviation of residuals n: number of matched pairs  'laser T2, Average Stdev Var  test  1st test 2nd test  [laser B3- ] Average Stdev Var  test 1st test 2nd test  ij^rBf"! "Average Stdev Var  test  before after (after-before) A9151145 A1155759 Residual 2896.95 2903.29 6.34 1.43 1.36 2.04 2.04 1.85 4.16  t statistic  93.18 <=== there is a significant difference between the two runs (bad) -528.93 <=== the difference between the two runs is significantly less than tolerance of 42.3 (good) before after (after-before) A9151145 A1155759 Residual 1607.32 1570.93 -36.39 1.44 1.45 2.09 2.06 2.10 4.36  t statistic -522.93 <=== there is a significant difference between the two runs (bad) 84.93 <=== the difference between the two runs is significantly less than -42.3 (good) before after (after-before) A9151145 A1155759 Residual 2222.04 1925.98 -296.06 1.57 1.53 2.23 2.45 2.34 4.96  t statistic  1st test  -3989.96 < = « there is a significant difference between the two runs (bad)  2nd test  -3419.89 <=== the difference between the two runs is NOT significantly less than -42.3 (bad)  [§seFfTJ Average Stdev Var  test  before after (after-before) A9151145 A1155759 Residual 1827.99 1822.44 -5.55 1.49 1.41 2.09 2.23 2.00 4.38  t statistic  1st test  -79.54 <=== there is a significant difference between the two runs (bad)  2nd test  527.02 <=== the difference between the two runs is significantly less than -42.3 (good)  'laser tt i ""'"Average Stdev Var test 1st test 2nd test  before after (after-before) A9151145 A1155759 Residual 2046.746 2020.137 -26.6089 2.502664 2.65942 3.243943 6.263328 7.072514 10.52317 t statistic -246.08 <=== there is a significant difference between thetwo runs (bad) 145.11 <=== the difference between the two runs is significantly less than -42.3 (good)  Average Stdev Var  before after (after-before) A9151145 A1155759 Residual 2746.872 2773.016 26.14333 1.335868 1.370182 1.869047 1.784545 1.8774 3.493337  test 1st test  t statistic 419.63 <=== there is a significant difference between the two runs (bad)  'laser T3 '•  2nd test  -259.33 <=== the difference between the two runs is significantly less than 42.3 (good)  98  Appendix D.2 • Laser Measurement Repeatability 2nd test hypothesis Ho: uD < tolerance Ha: uD > tolerance tolerance = 42.3 divisions alpha = 0.05 n = 900 t crit = 1.647  1st test hypothesis Ho: uD = 0 Ha: uD not equal 0 alpha = 0.05 n = 900 t crit = 1.962  Average Stdev Var  For both tests t statistic = (Rbar - uD)/(sD/SQRT(n)) Where Rbar: Average of Residuals uD: expected mean of residuals sD: standard deviation of residuals n: number of matched pairs  A9151145 A9151347 Residuals A9151547 Residuals A9151746 Residuals A9151947 Residuals 0.20 2896.40 -0.56 2896.95 2897.10 0.14 2896.67 -0.29 2897.15 1.43 1.34 2.02 1.34 2.09 1.43 2.06 1.43 1.99 1.80 4.38 2.06 4.26 2.03 3.97 2.04 1.81 4.09 t statistic t statistic t statistic 2.14 -4.11 2.86 significant significant significant There is a significant difference between the runs (bad)  t statistic -8.40 significant  t statistic t statistic t statistic -611.94 ' 602.06 -625.48 sig'tly less sig'tly less sig'tly less The difference between the runs is significantly less than tolerance (good)  t statistic 628.41 sig'tly less  1st test Result  2nd test Result Average Stdev Var  B9151145 B9151347 Residuals B9151547 Residuals B9151746 Residuals B9151947 Residuals 1607.32 1606.59 -0.73 1605.49 -1.84 1604.73 -2.60 1604.51 -2.82 1.44 1.53 2.12 1.43 2.06 1.55 2.14 1.58 2.04 2.06 2.33 4.48 2.03 4.26 2.40 4.57 2.49 4.18 t statistic t statistic t statistic -36.43 -10.39 -26.71 significant significant significant There is a significant difference between the runs (bad) t statistic t statistic t statistic 2nd test 589.12 588.35 557.03 sig'tly less sig'tly It sig'tly less The difference between the runs is significantly less than tolerance (good) 1st test  Result Average Stdev Var  t statistic -41.33 significant t statistic 579.56 sig'tly less  C9151145C9151347 Residuals C9151547 Residuals C9151746 Residuals C9151947 Residuals 2222.04 2221.94 -0.11 2220.81 -1.23 2220.39 -1.65 2220.11 -1.93 1.57 2.19 1.55 2.33 1.56 2.26 1.51 2.18 1.54 2.45 4.80 2.42 5.44 2.44 5.09 2.29 4.73 2.37 t statistic t statistic t statistic t statistic -1.49 -15.85 -21.98 -26.61 NOT sig significant significant significant '• There is a significant difference between the runs (bad) except in the first case.  Msttest  Result *  Lase7Bj~i "Average Stdev Var  2nd test 577.72 528.24 540.49 | sig'tly less sig'tly less sig'tly less, The difference between the runs is significantly less than tolerance (good)  556.58 sig'tly less  D9151145 D9151347 Residuals D9151547 Residuals D9151746 Residuals D9151947 Residuals ,1827.99 1827.67 -0.32 1826.89 -1.10 1826.86 -1.13 1826.57 -1.42 1.49'. 1.38 2.09 1.37 2.11 1.39 2.04 1.43 2.06 2.23 1.90 4.37 1.88 4.47 1.93 4.16 2.06 4.26 t statistic t statistic t statistic -4.59 -15.57 -16.55 significant significant significant ' There is a significant difference between the runs (bad) 1st test  t statistic -20.60 significant  2nd test Result Average Stdev Var  594.47 602.21 584.57 605.38 sig'tly less sig'tly less sig'tly If sig'tly less The difference between the two runs is significantly less than tolerance (good)  E9151145 E9151347 Residuals E9151547 Residuals E9151746 Residuals E9151947 Residuals 9.86 2051.48 4.74 2046.75 2060.42 13.67 2058.31 11.56 2056.60 1.58 2.90 1.61 2.90 2.50 1.56 2.84 1.83 3.66 2.51 8.43 2.60 8.39 6.26. 2.43 8.05 3.36 13.40  Result  t statistic t statistic t statistic 144.62 94.73 101.86 significant significant significant There is a significant difference between the runs (bad) 2nd test  'Result Average Stdev Var  -302.75 -251.89 -335.19 sig'tly less sig'tly less sig'tly less The difference between the runs is significantly less than tolerance (good)  t statistic 49.06 significant -389.10 sig'tly less  F9151145 F9151347 Residuals F9151547 Residuals F9151746 Residuals F9151947 Residuals 2746.87 2747.75 0.88 2747.71 0.84 2747.96 1.08 2749.48 2.61 1.34 1.37 1.94 1.33 2.02 1.39 1.93 1.50 1.97 1.78 1.88 3.78 1.76 4.10 1.93 3.73 2.24 3.88 t statistic t statistic t statistic 13.55 12.39 16.85 significant significant significant . There is a significant difference between the runs (bad) 1st test  2nd test  -639.32 -614.69 -640.38 sig'tly less sig'tly less sig'tly less The difference between the runs is significantly less than tolerance (good)  t statistic 39.72 significant -604.31 sig'tly less  APPENDIX E - E X A M P L E CLASSIFICATION ANALYSIS OF S A M P L E CASE  100  Appendix E - Example Classification Analysis of Sample Case |N 10EC3CI actual SNIPE 1 n 2y 3 n 4 n 5 n 6y 7 n 8y 9 n 10 n 11 y 12 y 13 n 14 y 15 n 16 y 17 n 18 n 19 y 20 y 21 y 22 n 23 n 24 y 25 y 26 y 27 n 28 y 29 n 30 n 31 n 32 n 33 n 34 n 35 n 36 y 37 y 38 n 39 n 40 y 41 y 42 n 43 n 44 y 45 y 46 n 47 y 48 n 49 n 50 n 51 n 52 y 53 n 54 n  target error T. SNIPE E. SNIPE Right n Right y n Right Right n n Right Right y n Right Right y n Right n Right Wrong c Right y n Right .Wrong n ;Wrong y Right y n Right : Wrong c Right y Right y Right y n Right n Right n [Wrong ____ Right y n 1 Wrong • Wrong y jwrong n [Wrong ^ y n Right [Wrong y n Right c 'Wrong Right n c 'Wrong •Wrong n Right " y Right n Right n Right y Right y n Right Wrong c Right y Right y Wrong y Right y n Right n Right n Right Right n Right y n Right Right n J  1  Error 0.340923 0.199405 0.24407 0.159318 0.192806 0.0548 0.255378 0.40217 0.320534 0.221628 0.533387 0.179526 0.379088 0.452284 0.438431 0.208186 0.35821 0.630405 0.359706 0.392373 0.04002 0.20806 0.226634 0.472627 0.02415 0.472587 0.603605 0.687566 0.798456 0.328139 0.452809 0.284964 0.709033 0.05407 0.691766 0.795369 0.402455 0.220557 0.09504 0.2058 0.152041 0.04657 0.587575 0.286373 0.136288 0.479989 0.04526 0.175418 0.133824 0.141822 0.197412 0.037498 0.226088 0.202651  error type  mix  type I type II  mix  type I type type type type  I II I II  type II mix mix type I  mix  type II  IOI  55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 IN10EG3C  y n n y n n n n n n n n n n n y n n y n n n n n n n n n n n n n n n n y n n n n n n n n n n y y n total .' . itype'-l).'.'^',' type II ' mix :*  y y y n n n n n n n n c n n n y n n n n n n y y n n n n n n n n n n y y n y n n n c n y y n n y y  Right Wrong WrongWrong Right Right Right Right Right Right Right Wrong Right Right Right Right Right Right [Wrong Right " * Right Right jWrong [Wrong Right Right Right Right Right Right Right Right Right Right (Wrong Right " Right [Wrong Right Right Right •Wrong Right Wrong Wrong Right Wrong Right Wrong  0.261267 ' 0.455065 0.584529 0.579702 0.203476 0.10434 0.345018 0.278102 0.0834 0.315366 0.172147 . 0.71425 0.376877 0.06551 0.221209 0.04299 0.185419 0.186799 0.414113 0.08429 0.233444 0.208301 ' 0.676377 , 0.506371 0.08709 0.168539 0.04008 0.230565 0.21344 0.087599 0.06804 0.209265 0.06139 0.0856 0.727109 ' 0.319268 0.291721 • 0.626114 0.318135 0.06949 0.262171 0.650968 0.348434 ; 0.622902 0.436087 0.04072 0.440075 0.058678 ' 0.795753 1  type II type II type I  mix  type I  type II type II  type II  type II  mix type II type II type I type II  29 incorrectly classified 8 28% of incorrectly classified cases are type I 14 48% of incorrectly classified cases are type II 7 24% of incorrectly classified cases are mix 74 are classified correctly (as snipe or as not having snipe) 71.8% of 103 samples  102  

Cite

Citation Scheme:

        

Citations by CSL (citeproc-js)

Usage Statistics

Share

Embed

Customize your widget with the following options, then copy and paste the code below into the HTML of your page to embed this item in your website.
                        
                            <div id="ubcOpenCollectionsWidgetDisplay">
                            <script id="ubcOpenCollectionsWidget"
                            src="{[{embed.src}]}"
                            data-item="{[{embed.item}]}"
                            data-collection="{[{embed.collection}]}"
                            data-metadata="{[{embed.showMetadata}]}"
                            data-width="{[{embed.width}]}"
                            async >
                            </script>
                            </div>
                        
                    
IIIF logo Our image viewer uses the IIIF 2.0 standard. To load this item in other compatible viewers, use this url:
http://iiif.library.ubc.ca/presentation/dsp.831.1-0074991/manifest

Comment

Related Items