Applied Science, Faculty of
Engineering, School of (Okanagan)
DSpace
UBCO
Kazemi Nojadeh, Sorayya
Master of Applied Science - MASc
University of British Columbia
Industry 4.0 is becoming a next-generation R&D area in the advanced composites manufacturing sector, to fully automate fabrication processes and increase the production quality and agility. Forming woven fabric reinforced composites on to complex 3D shapes is particularly of interest under this R&D trend, as these composites have become materials of choice in high-performance aerospace, energy, and automotive structures. Defects during the forming of long-fiber fabrics can affect the mechanical properties of the product. A vision under Industry 4.0 would be to develop AI-based tools that can predict, detect, and mitigate such defect during manufacturing. This thesis presents a prototype example of intelligent fabric composite manufacturing by conducting a series of hemisphere draping trials on a thermoplastic fiberglass/polypropylene weave, followed by the digitization of quality check. An image acquisition system was designed and integrated into a robotic arm to facilitate data collection. The quality function for each product was defined based on the height and area of formed wrinkles, as well as the sphericity and roundness of the formed hemisphere. By gathering data from around 70 samples and over 1200 images from different forming set-ups, machine learning algorithms were trained and consequently, the quality class of each product was predicted. Furthermore, to optimize the product quality, machine learning tools recommend the desired input variable of forming. Overall, a higher number of clamps and a higher number of reinforcement layers, with a higher and uniformly distributed torque magnitudes across clamps yielded the least visible wrinkles on the product. As another imaginary scenario in a smart factory, an on-line quality check tool was developed and tested. Namely, when the forming process is finished, a smart image acquisition system scans the part and checks the quality of different formed areas of the part and informs the AI tool of the ‘wrinkle’ or ‘no wrinkle’ continuously. It was found that the K-nearest neighbors (KNN) and Support Vector Machine (SVM) machine learning models could detect the wrinkles with an error rate of less than 5%, regardless of the background noise in the images such as an external object, marks, etc.
A FIRST STEP TOWARD INTELLIGENT FORMING OF FABRIC COMPOSITES: ADAPTING BASIC INDUSTRY 4.0 CONCEPTS by Sorayya Kazemi Nojadeh A THESIS SUBMITTED IN PARTIAL FULFILLMENT OF THE REQUIREMENTS FOR THE DEGREE OF MASTER OF APPLIED SCIENCE in THE COLLEGE OF GRADUATE STUDIES (Mechanical Engineering) THE UNIVERSITY OF BRITISH COLUMBIA (Okanagan) May 2020 © Sorayya Kazemi Nojadeh, 2020 ii The following individuals certify that they have read, and recommend to the Faculty of Graduate and Postdoctoral Studies for acceptance, the dissertation entitled: A FIRST STEP TOWARD INTELLIGENT FORMING OF FABRIC COMPOSITES: ADAPTING BASIC INDUSTRY4.0 CONCEPTS submitted by Sorayya Kazemi Nojadeh in partial fulfillment of the requirements for the degree of Master of Applied Science In Mechanical Engineering Examining Committee: Dr. Abbas S Milani, School of Engineering, UBC Okanagan Supervisor Dr. Homayoun Najjaran, School of Engineering, UBC Okanagan Supervisory Committee Member Dr. Zheng Liu, School of Engineering, UBC Okanagan Supervisory Committee Member Dr. Warren Hare, IKBarber School of Arts and Science, UBC Okanagan University Examiner iii Abstract Industry 4.0 is becoming a next-generation R&D area in the advanced composites manufacturing sector, to fully automate fabrication processes and increase the production quality and agility. Forming woven fabric reinforced composites on to complex 3D shapes is particularly of interest under this R&D trend, as these composites have become materials of choice in high-performance aerospace, energy, and automotive structures. Defects during the forming of long-fiber fabrics can affect the mechanical properties of the product. A vision under Industry 4.0 would be to develop AI-based tools that can predict, detect, and mitigate such defect during manufacturing. This thesis presents a prototype example of intelligent fabric composite manufacturing by conducting a series of hemisphere draping trials on a thermoplastic fiberglass/polypropylene weave, followed by the digitization of quality check. An image acquisition system was designed and integrated into a robotic arm to facilitate data collection. The quality function for each product was defined based on the height and area of formed wrinkles, as well as the sphericity and roundness of the formed hemisphere. By gathering data from around 70 samples and over 1200 images from different forming set-ups, machine learning algorithms were trained and consequently, the quality class of each product was predicted. Furthermore, to optimize the product quality, machine learning tools recommend the desired input variable of forming. Overall, a higher number of clamps and a higher number of reinforcement layers, with a higher and uniformly distributed torque magnitudes across clamps yielded the least visible wrinkles on the product. As another imaginary scenario in a smart factory, an on-line quality check tool was developed and tested. Namely, when the forming process is finished, a smart image acquisition system scans the iv part and checks the quality of different formed areas of the part and informs the AI tool of the ‘wrinkle’ or ‘no wrinkle’ continuously. It was found that the K-nearest neighbors (KNN) and Support Vector Machine (SVM) machine learning models could detect the wrinkles with an error rate of less than 5%, regardless of the background noise in the images such as an external object, marks, etc. v Lay Summary Industry 4.0 and Artificial Intelligence (AI) are two intertwined concepts that are fast finding high interest in advanced manufacturing sectors. The composites manufacturing industry, however, has been fairly slow to adapt this revolution, owing to the underlying complex material forming steps that make the data collection challenging. The wide range of combinations of fibers and thermoset/thermoplastic resin materials; multiple tooling and mold options; energy-intensive autoclave and oven operations; and integrated design and manufacturing requirements, are examples of factors contributing to the latter challenge. However, at the same time, this would make the composite manufacturing sector potentially most benefit from Industry 4.0 implementation, namely by bringing more automated, smart processes in the factories that can increase the production quality, while reducing waste and energy consumption. As a proof of concept, we designed and experimented a lab-scale smart forming process of composite woven fabrics, using AI models and Industry 4.0 concepts. vi Preface A summary of Chapter 2 has been published as a poster as follows: • S. Kazemi, A. Rashidi, A.S. Milani, “Towards intelligent forming of woven fabrics: Adapting basic Industry 4.0 concepts”, The 3rd Industry 4.0 symposium, June 5th, 2019, Western University, London, Ontario. A version of Chapter 2 was also presented in the 2nd Annual Engineering Graduate Symposium, Feb 11th, School of Engineering, UBC Okanagan as follows: • S. Kazemi, A.S. Milani, “Towards intelligent forming of woven fabrics: Adapting basic Industry 4.0 concepts”. Feb 11th, 2019, School of Engineering Graduate Symposium UBC Okanagan. The above presentation won the first-place prize (The best Mechanical Engineering-Oral presentation) in the event. A summary of the entire thesis was presented in 3 Minute Thesis competition as follows, and is also under submission to a journal: • S. Kazemi, “AI is here to help”, March 10th, 2020, UBC Okanagan. Through the event, I was selected as the top candidate in the 3MT Faculty Heats on behalf of the School of Engineering. A portion of the work, with highlighting additional applications in oil and gas upstream, has been accepted in a conference as follows: • S. Kazemi, A.S. Milani “Toward the 4th industrial revolution of oil and gas upstream”, GeoConvection2020, Calgary, Canada vii Table of Contents Abstract ......................................................................................................................................... iii Lay Summary .................................................................................................................................v Preface ........................................................................................................................................... vi Table of Contents ........................................................................................................................ vii List of Figures .............................................................................................................................. xii List of Tables ............................................................................................................................ xviii Acknowledgment ........................................................................................................................ xix Dedication .....................................................................................................................................xx Background and thesis organization ........................................................................1 Literature Review .......................................................................................................6 2.1.1 Additive Manufacturing .............................................................................................. 9 2.1.2 Simulation ................................................................................................................... 9 2.1.3 Virtualization Technology (Virtual Reality (VR) and Augmented Reality (AR)) ..... 9 2.1.4 Industrial Internet of Things ..................................................................................... 11 2.1.5 Robotics .................................................................................................................... 11 viii 2.1.6 Embedded Systems or Cyber-Physical Systems (CPS) ............................................ 12 2.1.7 Cloud Technology ..................................................................................................... 13 Fabric Forming Experiments ..................................................................................15 3.5.1 Shear angles calculation ............................................................................................ 20 3.5.2 Roundness characterization ...................................................................................... 23 3.5.3 Sphericity characterization........................................................................................ 26 3.5.4 Center displacement .................................................................................................. 27 3.5.5 Wrinkles .................................................................................................................... 29 Methodology for Data Modeling .............................................................................34 4.1.1 What does machine learning mean? ......................................................................... 34 4.1.2 When should machine learning be used? .................................................................. 34 4.1.3 Large datasets, promising solutions .......................................................................... 35 ix 4.1.4 The process of finding the right model to fit the data ............................................... 36 4.1.5 A general look to machine learning algorithms ........................................................ 36 4.1.6 Supervised learning ................................................................................................... 36 4.1.7 Unsupervised learning .............................................................................................. 37 4.1.8 Which one suits the application of our project? (Supervised or Unsupervised) ....... 37 4.1.9 Selecting among supervised learning techniques ..................................................... 38 Common classification algorithms ................................................................... 39 Common regression algorithms ........................................................................ 42 4.2.1 KNN K Nearest Neighbor ......................................................................................... 43 4.2.2 Support Vector Machine ........................................................................................... 47 Basic Theory of SVM ....................................................................................... 47 4.2.3 Linear regression with multiple variables ................................................................. 50 4.3.1 R-Squared ................................................................................................................. 51 4.3.2 RMSE ........................................................................................................................ 52 4.3.3 Receiver Operating Characteristic (ROC) ................................................................ 53 4.3.4 Area under the Curve (AUC): ................................................................................... 61 Results and Discussion .............................................................................................63 x 5.1.1 Hypothetical cyber-physical forming factory ........................................................... 63 5.2.1 Training procedure .................................................................................................... 66 5.2.2 Algorithms ................................................................................................................ 69 Step one _ Import the data ................................................................................ 69 Step two – Training ........................................................................................... 70 Step three _ Comparing the algorithms ............................................................ 72 5.2.3 Results ....................................................................................................................... 76 5.3.1 Training procedure .................................................................................................... 80 Step one – training-set images .......................................................................... 81 Step two – imageSet function ........................................................................... 85 Step three – Bag Of Features ............................................................................ 85 Step four- training set table ............................................................................... 86 5.3.2 Machine learning model ........................................................................................... 86 Importing the data ............................................................................................. 86 Results ............................................................................................................... 87 Conclusion and Future Work ..................................................................................95 6.1.1 Experimental (manufacturing parameters effect based on Chapter 3): .................... 95 6.1.2 Modeling (based on Chapter 3): ............................................................................... 96 xi 6.3.1 Auto vehicles ............................................................................................................ 98 6.3.2 Variables set-up ...................................................................................................... 100 6.3.3 Manufacturing barcode ........................................................................................... 100 6.3.4 Laser beam .............................................................................................................. 101 6.3.5 Orientation execution .............................................................................................. 101 6.3.6 Forming ................................................................................................................... 101 6.3.7 Curing process ........................................................................................................ 102 6.3.8 Quality check .......................................................................................................... 102 6.3.9 Other future work ideas towards advanced C4.0 .................................................... 102 References ...................................................................................................................................105 Appendices ..................................................................................................................................108 xii List of Figures Figure 1.1 Examples of fiber wrinkling in practice: (a) a no-wrinkled sample; (b) a wrinkled sample with severe in-plane waviness of yarns (c, d); waviness of yarns along with different levels of local out-of-plane fiber forms; (e, f) Different levels of yarn spreading with local wrinkles. .......................................................................................................................................... 2 Figure 1.2 Thesis framework ......................................................................................................... 5 Figure 2.1 The four historical industrial revolutions ...................................................................... 7 Figure 2.2 Pillars of Industry 4.0 [6]............................................................................................... 8 Figure 2.3 CANCOM 2019 Conference, UBC Okanagan; Microsoft HoloLens ......................... 10 Figure 3.1 (a) Experimental Set up of forming fabric (courtesy Armin Rashidi from CRN), (b) INSTRON testing machine (# 9569 50kN) .................................................................................. 16 Figure 3.2 Curing the formed fabric using a heat gun, without removing the paunch ................. 17 Figure 3.3 (a), (b) and (c) Orientation in a single layer and (d), (e) and (f) are orientation for two-layers; angles are with respect to the edge of the blank holder ............................................. 18 Figure 3.4 Torque amount (Lb-In) distribution according to the number of clamps .................... 18 Figure 3.5 Inputs and outputs of the forming experiments ........................................................... 20 Figure 3.6 Formed fabric samples. Photos are taken from the top (a) and front (b). The hemisphere part of the sample is shown in red. ............................................................................ 21 Figure 3.7 Shear angle calculation in Matlab ............................................................................... 21 Figure 3.8 Shear angle graphs in four different regions of the sample ......................................... 22 xiii Figure 3.9 Extracting points' coordination in the hemisphere part of the samples ....................... 23 Figure 3.10 Shear angle calculation on the hemisphere region .................................................... 23 Figure 3.11 The process of calculating the roundness ............................................................... 24 Figure 3.12 Roundness of different samples corresponding to three different forming set-ups; The first sample is formed using 4 clamps, in which the amount of torque applied to each of them is [3,3,7,7] (Lb-In); the orientation was 45 degrees and it was made of one layer. The roundness of this samples' base was 86.98%. The second sample was formed using 4 clamps. The amount of torque applied to each clamp was [5, 5, 5, 5] (Lb-In). The orientation was 0 degree and this sample was made of two layers. The roundness of this sample's base was 91.442%. The third sample was made using 4 clamps, in which the amount of torque applied to each clamp was: [7, 7, 7, 3] (Lb-In). The orientation was 60 degrees. The roundness percentage of this sample was 90.936. ............................................................................................................ 25 Figure 3.13 Centre displacement calculation in Matlab, These circles show the inner circle of the blank holder before and after forming .......................................................................................... 28 Figure 3.14 Fabric regions slide and form differently based on the applied amount of torque at each clamp on the blank holder .................................................................................................... 29 Figure 3.15 Examples of different affected wrinkling states under different forming set-ups; In the first forming sample, the maximum height of wrinkling has been 3.2 cm and it has occupied 27% of the hemisphere's area. The second sample’s affected height is 1.9 cm and wrinkling has occupied 24% of the hemisphere's area. And, in the third sample, the height is 0.8cm and the occupied area is 10% of the hemisphere. ...................................................................................... 30 xiv Figure 3.16 Samples' quality function variation based on the height and area of wrinkling state in the samples .................................................................................................................................... 32 Figure 4.1 Examples of where machine learning is heavily used [28] ......................................... 35 Figure 4.2 Different algorithms of supervised machine learning. ................................................ 38 Figure 4.3 (a) Logistic regression concept; the two colors show the two class of binary responses;....................................................................................................................................................... 39 Figure 4.4 Support vector machine concept; (a) there are several possible linear classifiers but, (b) there is only one that maximizes the margin. .......................................................................... 48 Figure 4.5 The better the linear regression (b) fits the data in comparison to the simple average (a), the closer the value of R2 is to 1. The areas of the blue squares represent the squared residuals with respect to the linear regression. The areas of the red squares represent the squared residuals with respect to the average value [47] ........................................................................... 52 Figure 4.6 ROC conceptual example; based on two groups of mice with different weights in two categories ‘obese’ and ‘not obese’ ................................................................................................ 54 Figure 4.7 Fitting logistic regression to the illustrative dataset of Figure 34. .............................. 54 Figure 4.8 Logistic regression classification with 0.5 cut-off ...................................................... 55 Figure 4.9 Confusion matrix of the illustrative example [48]. ..................................................... 56 Figure 4.10 ROC curve schematic; The green diagonal line shows where True positive Rate= False Positive Rate [48]. ............................................................................................................... 59 xv Figure 4.11 ROC curve sensitivity to threshold variation (shown as dash line), in the mice obesity example; Point (1, 1) means even though we correctly classified all of the obese samples, we incorrectly classified all of the samples that were not obese [48]. .......................................... 60 Figure 4.12 Area Under the Curve (AUC) for comparison of two predictive models (represented via two ROCs) [48]. ...................................................................................................................... 61 Figure 5.1 Set up variables on a hypothetical HMI screen in the smart forming factory ............ 65 Figure 5.2 Hypothetical fabric forming quality check points in the smart factory ....................... 66 Figure 5.3 Inputs and outputs of forming experiments ................................................................. 67 Figure 5.4 Workflow used for training models in the Regression Learner app ............................ 69 Figure 5.5 Importing the data........................................................................................................ 70 Figure 5.6 Regression learning algorithms in the Learner app ..................................................... 71 Figure 5.7 Trained algorithms results ........................................................................................... 73 Figure 5.8 Predicted vs actual data plot for the linear regression model (Left plot) and for the SVM regression model (Right plot). In these plots, the predicted response of the models is plotted against the actual/true response. A perfect regression model has a predicted response lying on the 45-degree diagonal line. The vertical distance from the line at any point represents the error of the prediction at that set-up. A good model would have the errors scattered around, and near the line. ........................................................................................................................... 74 Figure 5.9 Residual plot of linear regression and SVM regression models. A good model would have its residuals scattered symmetrically around zero, with no obvious pattern (i.e. no sign of auto-correlation). ........................................................................................................................... 75 xvi Figure 5.10 Normal probability plot of linear regression and linear SVM models. Normal Probability plot is a graphical technique for assessing whether a data set is approximately normally distributed. The data are plotted against a theoretical normal distribution in such a way that the points should form an approximate straight line. ...................................... 76 Figure 5.11 Examples of different input combinations effect on the sample quality, based on the prediction of linear regression model. .......................................................................................... 79 Figure 5.12 (a) A camera is attached to the 3-axis robotic arm (Dobot Robotic arm/ Magician); (b) The user is scanning the samples with camera ........................................................................ 80 Figure 5.13 Online wrinkle classification code developed (see details of the steps 1-4 in the next section). ......................................................................................................................................... 81 Figure 5.14 Samples of ‘wrinkled’-labeled set of images, taken during the on-line quality monitoring of the formed parts; notice the presence of external objects/marks in some of the images (which were included intentionally). ................................................................................ 83 Figure 5.15 Samples of ‘not-wrinkled’-labeled set of images, taken during the on-line quality monitoring of the formed parts; notice the presence of external objects/marks in some of the images (which were included intentionally). ................................................................................ 84 Figure 5.16 Steps of Bag Of Features method. ............................................................................. 85 Figure 5.17 Importing the data to the Classification Learner App ............................................... 87 Figure 5.18 Trained algorithms for case study 2 and the models’ accuracies .............................. 88 Figure 5.19 Scatter plots (left column) and Parallel coordinate plots (right column) of three models (kNN, SVM, and Tree). These plots can help to investigate features to include or exclude xvii in the final model. The user can visualize training data and the misclassified points on the scatter plot. ............................................................................................................................................... 89 Figure 5.20 ROC curves (left column) and Confusion Matrix (right column) of the kNN, SVM, and Tree algorithms. ..................................................................................................................... 90 Figure 5.21 Testing the trained online wrinkle quality machine; this example is for the kNN model............................................................................................................................................. 91 Figure 5.22 Comparing SVM and kNN performances in action. ................................................. 92 Figure 5.23 The Tree algorithm performance in action; note the false predictions. ..................... 93 Figure 5.24 kNN and Tree algorithms performance comparison for a two-layer sample. ........... 94 Figure 6.1 Experimental results summary of the thesis work. ...................................................... 95 Figure 6.2 An imaginary C.40 smart composite manufacturing factory ...................................... 99 xviii List of Tables Table 1 Satisfaction percentage based on height and area of wrinkles ......................................... 31 Table 2 Experimental input/output of the forming fabric process ................................................ 68 Table 3 R-squared and RMSE comparison of linear regression and linear SVM algorithms ...... 74 Table 4 Representation of test data set for case study 1 ............................................................... 77 xix Acknowledgment I cannot express enough my thanks to my supervisor, Dr. Abbas Milani for his unwavering support and guidance throughout my Master’s studies. I was always impressed with his genius suggestions and I am also very lucky that he always had a profound belief in my work, which gave me the motivation and energy to work even harder every day. His patience and help throughout this project cannot be overestimated. I would like to extend my sincere thanks to all my committee members for their valuable guidance and comments. I had also the opportunity to pass ‘Applied Machine Learning’ course with Dr. Homayoun Najjaran, which was a great help for me during this project, as well for my long-term career goals. I would like to acknowledge the assistance of my dear friend; Armin Rashidi, who never hesitated to help me throughout the project. I would also like to extend my gratitude to my office mates; Tina olfatbakhsh, Reeghan Osmond, Connor Keegan for their supports and lovely chats. I wish also to thank my lab-mates; Bryn Crawford, Milad Ramezankhani, Reza Sourki, and Danviir Karan Dhir who make the CRN Okanagan lab a wonderful place to work at. I cannot leave UBC Okanagan without mentioning that, this success would not have been possible without the support and nurturing of my beloved family including my dear brother Dr. Nasser Kazemi. They have been particularly helpful to me during this journey abroad, and as always, I am very lucky to have them in my life. xx Dedication ...To my husband and best friend Arash Dalili. Sharing our life and love along this journey is a blessing beyond words. With love… 1 Background and thesis organization 1.1 Introduction Long fabric-reinforced composites are widely used in various industries, especially in the transportation and construction sector, as they provide the possibility to fabricate light and at the same time strong structures. Wrinkling often occurs during composite fabrics forming and is a known major problem for manufacturers. The prediction of this defect is, therefore, of high importance for the design and optimization of associated composite structures. The quality of preforms of fabrics depends on several parameters, such as forming punch geometry, the relative orientation of punch/fabric-layers and blank-holders pressure, and the number of clamps, etc. These parameters play a major role in the quality of the final product shape, in terms of defects appearance [1]. Many past investigations have shown different parameters that cause the wrinkling defect in composite fabrics and also potential solutions have been offered for industrial applications. For instance, experimental and numerical analyses of textile reinforcement forming of a tetrahedral shape were studied in [2]. Buckling and misalignment defect was analyzed in a complex shape forming of flax woven fabrics [3]. Buckling and misalignment defect along with their interaction mechanisms during the forming of multilayer carbon fabrics were analyzed in [4]. Later, different experimental means to minimize the preforming defect of multi-layered interlock dry fabrics were studied [1]. Figure 1.1 exemplifies different kinds of wrinkles seen in fabric forming processes. 2 Figure 1.1 Examples of fiber wrinkling in practice: (a) a no-wrinkled sample; (b) a wrinkled sample with severe in-plane waviness of yarns (c, d); waviness of yarns along with different levels of local out-of-plane fiber forms; (e, f) Different levels of yarn spreading with local wrinkles. 1.2 Motivation and objectives Wrinkles in composite fabrics are often identified as a deviation of the fiber preform’s shape from a smooth pattern upon draping (Figure 1.1). The improvements during forming to avoid wrinkling can be achieved by different strategies like the substitution of the given fabric weave by another tighter/stiffer fabric with better formability, optimizing the manufacturing process parameters (a)(c) (d)(e) (f)(b) 3 through trial and error on-site or through simulations off-line, modifying the ply orientations and parts geometries, etc. From an industrial point of view, however, certain strategies could be more costly and time-consuming (e.g. change of reinforcement type or performing off-line highly complex and time-consuming simulations to predict winkles). In this thesis, we aim to take a preliminary step toward testing the idea of an intelligent fabric forming process, based on enabling technologies of Industry 4.0 such as visualization, automated image-based quality monitoring, as well as machine learning. The fourth industrial revolution is already happening and the composites manufacturing, especially the fabric forming sector, should be at the same pace as others that are gradually adapting this revolution. With such adaptation, mitigations of the forming defects such as wrinkling may be automated and take place on-line during manufacturing with minimal time and cost. Following this motivation, the following objectives were defined for this thesis: • Design and develop a preliminary smart prototype for fabric forming process; • Control and monitor the forming event; • Integrate the system with machine learning algorithms to predict the vulnerable zones for wrinkling defects and provide recommendations. 1.3 Thesis framework This thesis has been arranged in six chapters. Chapter 2 presents a general literature review on Industry 4.0 and its pillars. In Chapter 3, the case study of this research is presented. The experimental set-up, fabric material, and different configurations of fabric forming are explained in detail. The final products’ quality is also experimentally analyzed in this chapter. Later in Chapter 4, we discuss the methodology of different machine learning algorithms for the current system, based on data collected from the experiments. In Chapter 5, we explain how designers could use the machine learning to detect and analyze the wrinkles using SVM (Support vector machines) and kNN (k-nearest neighbor) algorithms on the final formed product. In this chapter, we also explained how one could predict the overall quality of the final product using an interval- 4 based definition of sub-quality measures along with a regression approach. Finally, in Chapter 6, we summarize the main findings of preceding chapters and the perceived processes of intelligent fabric forming in a digital composite factory, called C4.0. The chapter ends with some recommendations for future research. Figure 1.2 illustrates the summary of the topics presented in each chapter and the organizational framework of the thesis. 5 TOWARD INTELLIGENT FORMING OF FABRIC COMPOSITES: ADAPTING BASIC INDUSTRY4.0 CONCEPTSChapter 1Background and thesis organization Chapter 2 Literature review Composite manufacturing ChallengesFourth Industrial revolution I4.0Chapter 3ExperimentsExperiments DataExperimental set upMaterial of experimentsProcess of experimentsChapter 4MethodologyChapter 5Intelligent fabric formingMachine learning algorithms • Wrinkle detection• Best quality predictionChapter 6Conclusion Figure 1.2 Thesis framework 6 Literature Review In this chapter, a literature review is provided on the overall pillars of Industry 4.0 and how different processes in composite manufacturing may be modified by applying such pillars. 2.1 Industry 4.0 Historically, there have been three revolutions in the manufacturing world before the emergence of the fourth industrial revolution, also known as Industry 4.0 (Figure 2.1). Industry 1.0 was born in the 18th century which was mainly focused on mechanical production powered by steam and water. Later in the 19th century, the second industrial revolution occurred by the emerging mass production through assembly lines. Mechanization began in the 18th and 19th Century where steam and waterpower were harnessed to run vast factories. Operatives were required to run these factories, so populations changed from primarily agrarian to metropolitan. As a relevant side note, the primary product that these factories created were textiles, a technology that is still used today in the production of fabrics made from advanced fiber[5]. The third industrial revolution happened in the 20th century as it was the first use of computers and automation in factories. During industry 3.0 huge advancements happened in factories. For example, Programmable Logic Controllers (PLCs) were used in factories to automate mechanical processes. In the third industrial revolution, robotics started to have a great role in automation; however, robots were controlled by a central controller. The fourth industrial revolution takes one step further and focuses on smart objects and robots. In Industry 4.0 robots and objects are smart enough to make decisions by themselves and communicate with each other efficiently. The fourth industrial revolution is bringing about extraordinary levels of connectivity in everyday life. It has made many cultural changes over the last decade which some of them are even missed. The On-Demand Economy is an example of this, which means the economics brought about by technology being able to fulfill demand instantly [5]. There are many examples of this, such as being able to order takeaway, bank services, watch whole seasons of TV programs, all immediately from apps on a mobile device. All these technological advantages were not possible just 10 years ago. Industry 4.0 technologies have been 7 subcategorized into 8 classes, as shown in Figure 2.2: Robotics, Embedded systems (CPS), Simulation, Data analytics and artificial intelligence (Big data analytics), Cloud Systems, Internet of things (IoT), Additive manufacturing, and Virtualization technology. It is important to note that there are many examples of these individual technologies, but the real benefit is gained when they are connected [5]. Figure 2.1 The four historical industrial revolutions INDUSTRY 1.0 Mechanical production. Equipment powered by steam and waterINDUSTRY 2.0Mass production, assembly lines requiring labor and electrical energyINDUSTRY 3.0Automated production using electronic and ITINDUSTRY 4.0intelligent production incorporated with Big data, IOT, CPS 8 Figure 2.2 Pillars of Industry 4.0 [6] In Industry 4.0 robots and objects are smart enough to make decisions by themselves and communicate with each other efficiently. At this stage, industry 4.0 is nothing more than a vision, but by dividing it to its pillars and focusing on each of them separately, it has the potential to change the fourth industrial revolution’s dream to reality. Essential technologies for Industry 4.0 are Cyber-Physical System (CPS), Internet of Things, Virtualization technology, Cloud Systems, and Robotics. Information technology has changed considerably. Over the past few decades. Hence, the companies that are resisting using it in a globally competitive environment may easily lose their power in a foreseeable future. Many of the new technologies continue to surprise users by providing vital support as well as enabling access to different and high-quality processes and products. In Industry 4.0, not only the way of producing products are becoming completely smart, but the products themselves are becoming smart too [7]. To implement Industry 4.0 transformation successfully, supportive technologies continue to be required in different sectors. TechnologiesRoboticsEmbeded Systems (CPS)SimulationData analytics and artificial intelligence Cloud systemsCommunication and Networking/ IOTAdditive manufacturingVirtualization technology 9 2.1.1 Additive Manufacturing Additive manufacturing is a set of developing technologies that produce three-dimensional objects directly from digital models by an additive process. This process happens especially by storing and joining the products with proper polymers, ceramics, or metals[8]. Composite materials inherently suit additive manufacturing due to the flexibility of the material components (different layers, fibers, resin, etc) used in composite manufacturing. Since layers of composite material are built up during production, instead of being removed or subtracted, advanced composites can be considered as additively manufactured from the start. In this respect, composites are ahead of other industries in development towards the fourth industrial revolution[9]. 2.1.2 Simulation Before practical operations, it is better to test the process in a virtual twin and analyze potential problems and defects. Simulations help to simulate the real operation in computers and change inputs, outputs, conditions and analyze their effect on the whole process. To improve the robustness and reliability of process engineers use simulations that reflect the what-if scenarios. By using the simulation solutions, a "digital twin" of the manufacturing process can be produced. A digital twin is a digital view of physical entities, like the production line, process, etc. The digital view is then linked to the actual object to create a cyber-physical system. The digital twin is the key part of how Industry 4.0 can provide the benefits it promises. 2.1.3 Virtualization Technology (Virtual Reality (VR) and Augmented Reality (AR)) Virtualization technologies’ tools are the reflection of the real world with the help of computers and software. Virtual Reality (VR) replaces your existing view of the world with a new digital experience [10]. Augmented Reality (AR) adds new visual or auditory layers to your existing reality. Today, the manufacture of composites is in large a manual process, especially in small-to-medium-sized enterprises. Since the manufacturing processes for composites are fairly complex, digitizing the human part of the manufacture is a challenge. There are already several examples of augmented reality and virtual reality which are either unique composites or which might be applied 10 to composites. Research at the University of Patras [11] has shown that AR in combination with smart tools (tools with RFID tags, or tools which can monitor usage, such as the torque applied on bolts) can be used to both guide operators and collect data from an automotive assembly line. Progress of AR is shown with equipment such as the Microsoft HoloLens being used for inspections on the shop floor. In Figure 2.3 I am using Microsoft HoloLens and I could complete a circuit following some structures. Figure 2.3 CANCOM 2019 Conference, UBC Okanagan; Microsoft HoloLens The HoloLens is Microsoft’s take on augmented reality, which they call “mixed reality”. Using multiple sensors, advanced optics, and holographic processing that melds seamlessly with its environment, these holograms can be used to display information, blend with the real world, or even simulate a virtual world. The HoloLens has a plethora of optical sensors, with two on each side for peripheral "environment understanding" sensing, a main downward-facing depth camera to pick up hand motions, and specialized speakers that simulate sound from anywhere in the room. 11 The HoloLens also has several microphones, an HD camera, an ambient light sensor, and Microsoft's custom "Holographic Processing Unit" that they claim has more processing power than the average laptop. All this comes together to sense the spatial orientation of the unit in the room, track walls and objects in the room, and blend holograms into the environment. Examples of where AR is already available in composites are solutions from Aligned Vision that can both project ply locations onto the tool surface and inspect the location of the plies after they have been laid up. 2.1.4 Industrial Internet of Things The Internet of Things, or IoT, is described as the connecting devices, systems, and services technology that extends beyond simple machine-to-machine communication. The methods of monitoring— data collection, analysis, and decision-making— do not necessarily require human intervention at the moment, but include monitoring not only of devices but also of materials, equipment, and process parameters [5]. 2.1.5 Robotics Manufacturing of different products is getting easier by the use of adaptive and flexible robots with a combination of artificial intelligence. In order to make a complex product, adaptive robots recognize the lower segment of each part. This segmentation should be in a way that reduces the final production cost and the waste time during operations and increases the speed of the process [12]. Adaptive robotics applies to manufacturing systems. Three main areas are; design, manufacturing, and assembly stages. For example, each problem is divided into several sub-problems. To solve each sub-problem, they are divided into simpler sub-problems and then organized in a group of modules. To reach an optimal solution, all modules should be integrated. Robots have first appeared in the third industrial revolution, however, in the fourth industrial revolution robots are smart enough to communicate with humans and other robots. They transfer online date with other robots and make a decision based on current conditions. This is a huge difference between robotic technology in the third and fourth industrial revolutions [13]. There are 12 currently many applications of robotics in composite manufacturing such as pick and place, automatic tape laying, automated fiber positioning, filament winding. Traditional robotics generally requires control and testing as there are no feedback loops to ensure that the automated process was satisfactorily carried out. An example of this is AFP, where it is important to check a ply and ensure that all tows had been properly laid before the next ply is started. It has been stated that the automated solution is only active for 25% to 35% of the time the device is under the machine in such a process [14]. The remaining time is composed of analysis. The next step of robotic development is to add sensors within the device to provide feedback, so the robot can correct itself if a problem occurs. 2.1.6 Embedded Systems or Cyber-Physical Systems (CPS) Embedded systems known as Cyber-physical systems stand for the physical objects that have cyber features too. In Cyber-physical objects, the physical and cyber side are intertwined, meaning that a simple physical actuator which is used in the third industrial revolution changes to a smart actuator in Industry 4.0. The cyber side is added to the actuator and makes it a smart object. This smart actuator has its own special IP and provides data. It is an intelligent object now, in a way that can send data to other sensors and actuators. Auto-cars are a good example of CPS. In Industry 3.0 there have been modern cars, but they all need to be driven by a human. They are not smart enough to be driven by themselves. However, smart cars called auto-cars are now being produced. This is a step toward the fourth industrial revolution. In an auto-car, the central controller of the car can get the address from the passenger. The controller follows the direction toward the destination. Cyber-physical objects should be able to send and receive data from other smart objects. In this case, the car should communicate with smart traffic cameras, to know the online condition of streets. The car should be able to choose the nearest and less crowded ways to go to the destination. In Industry 4.0 all objects are cyber-physical systems. The cyber side of these systems means that they all have a special IP address. CPS produces online data and they communicate with other CPS near to them [15], [16]. Embedded systems have some features mentioned as follows: 13 1. They can detest the safety-critical status regarding their importance level and in this way can increase operational safety. 2. Using CPS eases the monitoring of the process. It includes conditions with and without sensors. 3. Control and monitor the whole process using feedback loops. 4. Data analysis directly and interactively on the local control or in the public cloud system. 5. Machines and other parts of the systems are flexible and reconfigurable. 2.1.7 Cloud Technology As addressed above, every single object in the fourth industrial revolution is assumed to be smart and producing data. This huge amount of data is then gathered in the Cloud, often referred to as “Big Data” storage. In order to analyze this huge amount of data, traditional ways cannot be helpful. Cloud computing is the new modern way to compute the Big Data. In the absence of Cloud computing CPS can just send and receive data. They are not able to analyze them and make a decision based on them. Accordingly, cloud technology is one of the most essential technologies needed for the fourth industrial revolution’s transformation [16]. 2.2 Forming fabrics in composite manufacturing The manufacture of textile fiber composites includes a forming stage. In this stage, the dry or impregnated fabric reinforcement takes a desired, often doubly curved, shape as specified by the design of the final part. It occurs under the mixed action of clamp holding pressure and forces due to the contact of the fabric with flexible or rigid tooling [17]. However, many different faults such as wrinkles, fiber misalignment, and variations in local fiber fraction due to fiber spreading or bunching may happen during forming. In turn, they can cause significant degradation of the performance characteristics of the final product [18], [19]. Among these defects, wrinkling is perhaps the most severe one arising during both automated and manual forming processes of 14 composite materials [20]. Wrinkles can lead to unexpected failures during the manufacturing, or cause a significant reduction in strength and damage tolerance of the final product [21], [22]. It is imperative to gain a better understanding of the effect of processing parameters and material properties on the forming output, in order to resolve the above-described defects. The improvements can be achieved by different strategies like the substitution of fabric by another one with better formability, changing of the manufacturing process, applying optimum manufacturing process parameters, and/or modifying of ply orientations and parts geometry, etc. From an industrial point of view, however, certain strategies could be costly and/or time-consuming (e.g. change of process or change of reinforcement). In this thesis, we conducted several trials of forming of a typical woven fabric, under a hemisphere shape design. Like other forming investigations in the past, we also encountered flaws, especially wrinkles. In order to overcome this issue, we aimed to define and implement an intelligent forming process framework. Eventually, using machine learning algorithms we could predict and optimize the hemisphere part’s quality. 2.3 Summary In this chapter, the definition of Industry 4.0 and a brief history of three previous industrial revolutions were discussed. Next, we broke down the overwhelming expression of Industry 4.0 to eight main pillars including Robotics, Embedded systems (CPS), Simulation, Data analytics and artificial intelligence (Big data analytics), Cloud Systems, Internet of things (IoT), Additive manufacturing, and Virtualization technology. We explained each of these pillars and gave an example of their potential application in composites manufacturing. We also provided an overview of the fabric forming processes in composite manufacturing and identified potential problems/defects that can occur during forming. In the next chapters, we will cover how using an intelligent forming process approach we can omit the flaws in the final (draped) fabric composite product, using basic enabling technologies in Industry 4.0 such as visualization image-based monitoring, as well as machine learning. 15 Fabric Forming Experiments In this chapter, we present the studied lab-scale fabric forming experiments. We also discuss the general experimental results and outcomes of the formed fabrics from a quality perspective. Detailed quantitative modeling, using machine learning, will then follow in the subsequent chapters. 3.1 Experimental set up for fabric forming As demonstrated in Figure 3.1 we used an INSTRON tensile machine (model #9569 50kN) at the CRN Laboratory at UBC Okanagan Campus, to form the fabric samples. The samples were made of fiberglass/polypropylene (Twintex) weft and warp yarns, woven into each other under a twill weave layout. The main parts of the set up over the INSTRON device were two blank holders, clamps, and paunch. We put one of the blank holders on the bars and then put the square-cut fabric on it. Next, we placed the second blank holder and then put the clamps on predefined spots. Then we fastened the clamps using Torque Meter. In this way, we placed the fabric between two blank holders and fastened them all against each other, using C-clamps. Next, the fabric was then ready to be formed utilizing the induced force from the INSTRON head. 16 Figure 3.1 (a) Experimental Set up of forming fabric (courtesy Armin Rashidi from CRN), (b) INSTRON testing machine (# 9569 50kN) 3.2 The process of experiments We applied the required data acquisition inputs in "Blue Valley" software, and the paunch smoothly pushed down the fabric, while sliding out through the blank holder. In a certain distance (8 cm), the paunch was stopped. Next, we fixed the current shaped fabric (i.e. did not unload or open the blank holders) and used a "Heat Gun" to freeze the shape of the formed fabric. By applying heat to fabric, the inner resin in the fabric melts and hardens/cures. By curing the fabric composite we fix the formed shape and then it is ready to be analyzed further (Figure 3.2). (a) (b) 17 Figure 3.2 Curing the formed fabric using a heat gun, without removing the paunch 3.3 Input variables in the analysis Inputs (controlled variables) in the conducted experiments were the number of clamps, number of layers, orientation and torque amount 1. Number of clamps: In some tests, we have formed fabric using 3 clamps and, in some others, we used 4 clamps. 2. Number of layers: After we placed the first blank holder, we put either one or two square-cut fabrics. So, as the input for this factor, we have 'single-layer' or 'two layers'. 3. Orientation: The single-layer was set to have 0o, 45o, or 60o degrees orientation. However, in the two-layer test, we refer to 0o orientation, to the amount of degree between the second layer and first layer's placement. So, as an orientation input, we have three different conditions: 0o, 45o, and 60o degrees Figure 3.3. 18 Figure 3.3 (a), (b) and (c) Orientation in a single layer and (d), (e) and (f) are orientation for two-layers; angles are with respect to the edge of the blank holder 4. Torque amount As mentioned, before we have chosen sample set-ups with 3 as well as 4 clamps. But another variable here is the amount of torque applied to every single clamp. By using Torque Meter we applied the following amounts to the clamps Figure 3.4. Figure 3.4 Torque amount (Lb-In) distribution according to the number of clamps 19 Before defining the levels of variables for inputs of the process, we performed preliminary trial and errors to ensure the forming is feasible within the selected ranges and available equipment. For example, we attempted to form the fabric using 5 clamps, three layers. We also applied 9 pound-inch of torque to clamps. However, we could imply that the latter variable states produce any meaningful sample. 3.4 Outputs of analysis A desired output of a forming fabric process in practice would be simply a well-formed fabric (i.e. with no major defects). But how we can define the term 'good' for e.g. here a hemisphere formed sample? As discussed in Chapter 2, one of the most important issues/defects that happens in forming fabric is “wrinkling”. Samples with fewer wrinkles are considered of higher quality. Not only the number of wrinkles is important but how severe they are, is also another problem that the designers take into account. To judge the quality of a hemisphere sample, here we assessed its Sphericity and base’s roundness. The sphericity of a perfect hemisphere is 84.0% [23] and the base's roundness is 1. In the next section, we explain in detail how we measured these properties for the samples. Figure 3.5 summarizes the inputs and outputs of the fabric forming experiments in this research. 20 Figure 3.5 Inputs and outputs of the forming experiments 3.5 Analyzing the tests 3.5.1 Shear angles calculation Before forming each fabric sample, the angle between weft and warp yarns was nominally 90 degrees. After forming the fabric, yarns rotate relative to each other due to the shearing. The shear angle stands for the difference between the primary and the current angle between the warp and weft yarns. Inputs OutputsSphericity and Circulirity RoundnessWrinklesShear anglesFabric OrientationNumber of ClampsTorque amountNumber of layers 21 Figure 3.6 Formed fabric samples. Photos are taken from the top (a) and front (b). The hemisphere part of the sample is shown in red. We developed a MATLAB code to calculate the shear angle both on the "Flat part" and "Hemisphere part" of samples Figure 3.6. On the flat part of the sample, MATLAB detects the frame and then draws two cross lines on the image. Then signs point on equal distances over the lines. Then using 'Getpoints' command, the user can select desired yarns to find their shear angle as shown in Figure 3.7. Figure 3.7 Shear angle calculation in Matlab 22 In Figure 3.8, for illustration purposes, we show four shear angle graphs measured in four flat regions of a sample. Figure 3.8 Shear angle graphs in four different regions of the sample The more important part of each forming, especially from Industry perspective, would be the hemisphere region. To find shear angles on the hemisphere part of the samples, we had to take two photos of every region. One photo from the top (top view) and another one from the front (front view). In each photo, we extracted the coordinates of each desired point. As we needed (X, Y, Z) for 3D hemisphere points, the 2D coordinates taken from each view were amended in the code. Namely, the 'X' and 'Y' were extracted from the photo taken from the top. Then, the vertical coordinate in the photo that is taken from the front gave us 'Z' value of the point. Figure 3.9 shows the steps followed to extract the coordinates of the points on yarns and calculating the shear angles in the hemisphere region. For example, if the angle between yarns after forming is 83.5o the shear angle is 90-83.5= 6o. 5 23 Figure 3.9 Extracting points' coordination in the hemisphere part of the samples After extracting every point's coordinates, MATLAB calculated the shear angle on the selected point on the hemisphere using Equation 3-1 see Figure 3.10: Figure 3.10 Shear angle calculation on the hemisphere region 3.5.2 Roundness characterization To analyze the inner part of the samples, we calculated the roundness of the base of the hemispheres. To do this we used Equation 3-2 [23]: 𝑅𝑜𝑢𝑛𝑑𝑛𝑒𝑠𝑠 = ( 4 ∗ 𝜋 ∗ 𝐴𝑟𝑒𝑎 (𝑃𝑒𝑟𝑖𝑚𝑒𝑡𝑒𝑟)2⁄ ) ∗ 100 . Equation 3-2 We separated the base of the hemisphere from other parts in its image. Next, we converted it into a binary image. Figure 3.11 shows a sample binary image of the formed fabric’s inner round shape. Figure 3.12 shows the roundness values obtained through different select setups. A B C 𝜃 = csc−1 𝑃. 𝑄 |𝑝||𝑄|⁄ Shear angle = 90- 𝜃 Equation 3-1 24 Figure 3.11 The process of calculating the roundness After forming before forming The outer part of blank holder marked before forming has been started. The inner part of blank holder marked before forming has been started. The inner part of blank holder marked after forming has been finished. The outer part of blank holder marked after forming has been finished 25 Figure 3.12 Roundness of different samples corresponding to three different forming set-ups; The first sample is formed using 4 clamps, in which the amount of torque applied to each of them is [3,3,7,7] (Lb-In); the orientation was 45 degrees and it was made of one layer. The roundness of this samples' base was 86.98%. The second sample was formed using 4 clamps. The amount of torque applied to each clamp was [5, 5, 5, 5] (Lb-In). The orientation was 0 degree and this sample was made of two layers. The roundness of this sample's base was 91.442%. The third sample was made using 4 clamps, in which the amount of torque applied to each clamp was: [7, 7, 7, 3] (Lb-In). The orientation was 60 degrees. The roundness percentage of this sample was 90.936. 26 By comparing the roundness of the three samples we could see that samples with less variety in the amount of torque have had roundness more than 90%. For example, the roundness of the sample with a torque amount of T=[3,5,7,0] (Lb.-In) was found to be less than a sample with T=[5,5,5,5]. 3.5.3 Sphericity characterization Sphericity is a measure of how closely an object's shape resembles that of a perfect sphere; it is, in fact, a specific example of a measure of shape compactness. Defined by Wadell in 1935 [23], the Sphericity of a particle is the ratio of the surface area of a sphere with the same volume as the given particle, to the surface area of the particle, see Equation 3-3: 𝑆𝑝ℎ𝑒𝑟𝑖𝑐𝑖𝑡𝑦 =𝜋13⁄ (6𝑉𝑝)23⁄ 𝐴𝑝 ⁄ Equation 3-3 To find the volume of the samples, we poured water inside the hemisphere and calculated the approximate volume. We calculated the area of the samples' surface by covering them with a transparent tissue and marking the perimeter of samples. It gave the approximate area of the sample that we could image and calculate using Matlab-image processing. Having in mind that the sphericity of a perfect hemisphere is 84%, we could imply that all samples have had a good amount of sphericity. Namely, they were mostly close to 80% ± 5%. However, most samples that had less sphericity were the ones that were made of fewer clamps, and the majority of samples with higher sphericities were made of two layers of the fabric. 27 3.5.4 Center displacement Before forming the fabric, we marked the inner and outer premiers of the blank holder, as well as the center of the circle under blank holder (see also Figure 3.11 ). We then did the same markings after forming. After forming, marking inside and outside of the blank holder gave us two perfect circles. However, as seen in Figure 3.13 the primer of the first marked circle has deformed after forming. We also can see that the center of the hemisphere is not in the middle of the sample upon forming. By analyzing all samples and calculating the central displacement in MATLAB, we could observe that samples can move a longer distance under the clamps that have had less torque. 28 Figure 3.13 Centre displacement calculation in Matlab, These circles show the inner circle of the blank holder before and after forming For example, in Figure 3.14 if we compare the distance between blue and red circles in the sample with 3 clamps with the amounts of torque of [3, 5, 7] lb. Ft, we definitely can see that the longest distance has been for the torque amount of 3lb.ft and the shortest one is for the clamp with the torque amount of 7lb.ft. Comparing all samples' center-displacement, we can say that samples with more clamps and torque amounts of the same value applied to clamps have been formed approximately in the middle of the sample and had the least center displacement. 29 Figure 3.14 Fabric regions slide and form differently based on the applied amount of torque at each clamp on the blank holder The complete table of samples’ sphericity, roundness and center movement is provided in the appendix. 3.5.5 Wrinkles A wrinkle is a much-undesired defect for designers, happening during the forming of fabrics. In this section, we more closely define what we mean by a wrinkle in our samples. Figure 3.15 shows misalignments in yarns during forming. The distance between yarns that caused open areas (also called yarn spreading), waviness of yarns, and out of plane forms mean that in general wrinkle has happened in that hemisphere area. 30 Figure 3.15 Examples of different affected wrinkling states under different forming set-ups; In the first forming sample, the maximum height of wrinkling has been 3.2 cm and it has occupied 27% of the hemisphere's area. The second sample’s affected height is 1.9 cm and wrinkling has occupied 24% of the hemisphere's area. And, in the third sample, the height is 0.8cm and the occupied area is 10% of the hemisphere. Upon all the forming tests, we observed that different shapes and sizes of wrinkles happened on the hemisphere. Some of them were severe some others were in a very small area and did not much affect the final shape of the hemisphere. In order to categorize the quality of the observed wrinkling states to 'A (Good)', 'B (Medium)', ‘C (Poor)’ and ‘D (Rejected)’ quality measures, we calculated the maximum height and the area the wrinkling has occupied on the hemisphere, as shown in Figure 3.15. Table 1 shows the categorized samples' qualities based on the area and height of wrinkles. We defined that the quality (satisfaction %) for A (Perfect) is (80%-100%), B (Medium) is (60%-80%) and C (Poor) is (45-60) and D (Rejected) is below 45%. Needless to add, that the definition of the quality categories would depend on the application and experience of the designers. Figure 3.16 also demonstrates the satisfaction percentage of samples’ quality graphically Height(cm) Area (%) Quality Quality (%) 𝟎 ≤ 𝑯 ≤ 𝟏 𝟎 ≤ 𝐴 ≤ 𝟏5 A (Perfect) 𝟖𝟎 ≤ 𝑄 ≤ 𝟏00 𝟏 < 𝑯 ≤ 𝟐 15 < 𝐴 ≤ 25 B (Medium) 𝟔𝟎 ≤ 𝑄 < 80 𝟐 < 𝑯 ≤ 𝟑 25 < 𝐴 ≤ 35 C (Poor) 45 ≤ 𝑄 < 60 31 𝑯 > 𝟑 𝑨 > 𝟑5 D (Rejected) 𝟎 ≤ 𝑸 < 𝟒𝟓 Table 1 Satisfaction percentage based on height and area of wrinkles Next, to find a predictive equation of quality variation according to area and height's change per table above, we used a regression model in MATLAB. Using 'fitlm’ command returned a linear model fit, with a root mean squared error of 0.826 and 0.999 goodness of fit. Equation 3-4 is the final fit model obtained for the quality quantification using the forming data. It noted that the regression coefficients of the Height and Area variable are negative, which confirms negative sensitivity of the quality function for the formed part with regards to any increase in these variables. 𝑄𝑢𝑎𝑙𝑖𝑡𝑦 = (−0.16.83) 𝐻𝑒𝑖𝑔ℎ𝑡 + (−0.14) 𝐴𝑟𝑒𝑎 + 98.788 Equation 3-4 32 Figure 3.16 Samples' quality function variation based on the height and area of wrinkling state in the samples Comparing the result of wrinkles' quality values, we can say that samples with the orientation of 60 degrees and made of the single layer have had severe wrinkles; i.e. wrinkles with maximum height and occupying the biggest area of the surface of samples. However, samples with 0 degrees of orientation had overall very good quality. Comparing samples that had the same amount of torque applied to their clamps, those with a greater amount of torque have had a better quality. Note that although an excessive amount of torque would cause less/no wrinkles, but it may cause other problems like yarn spreading or tension/breakage. In this project the function of samples’ 01234501020304050607080901000 15 25 35 40 5010080604500MAXIMUM WRINKLES' HEIGHT(CM)QUALITY %MAXIMUM WRINKLES' AREA %Samples' quality based on Area and height of wrinkles 33 quality is deemed general (i.e. includes concurrent defect as a whole: wrinkles, waviness, bunching etc, as seen in Figure 1.1). 3.6 Summary In this chapter, we explained how we performed and analyzed the fabric forming experiments, and how we could judge the final quality of samples. Namely, by imaging and analyzing the inner part of the samples, we were able to calculate the roundness, sphericity, central displacement; and by analyzing the outer part of samples we could find the quality of samples based on the height and area of formed wrinkles and the shear angle degrees on different regions. We could also identify some key input settings that would yield overall higher quality in the hemisphere formed parts. 34 Methodology for Data Modeling In this chapter, we overview machine learning techniques that could be potentially used for data modeling of the presented forming experiments in Chapter 3, while discussing factors that one should take into account when using each algorithm. Finally, we select two algorithms and explain them in more detail, the results of which will be presented in the next chapter. 4.1 Machine learning Machine learning is considered one of today's most innovative technologies for companies and a powerful competitive advantage enabler for moving to Industry 4.0. Although machine learning has been around for over 60 years, it has shown significant potential to disrupt economies and societies only recently. Healthcare, finance, manufacturing, and transportation are just a few of the sectors that have been influenced by machine learning [24]. 4.1.1 What does machine learning mean? Machine learning teaches computers to do what comes naturally to humans and animals: learn from experience. Machine learning algorithms use computational methods to directly learn knowledge, without relying as a guide on a prearranged equation [24]. As the number of samples available for learning increases, the efficiency of these algorithms is improved adaptively. 4.1.2 When should machine learning be used? We can use machine learning to model and predict complex systems, especially the ones with a large number of data and multiple variables for which there are no explicit equations or formula 35 to connect them to the system output(s). Examples of such cases have been shown in Figure 4.1 [24]. Figure 4.1 Examples of where machine learning is heavily used [24] 4.1.3 Large datasets, promising solutions Machine learning algorithms identify naturally occurring trends in large datasets, that can provide new insight and help to make better decisions and predictions. They are nowadays used on daily basis to make critical decisions in the data-rich fields such as medical diagnosis, stock trading, energy prediction, and more [25]. As another example, social media sites heavily rely on machine learning to sift through millions of choices, to give suggestions to viewers e.g. for songs or movies. Hand written rules and equations are too complex; as in face recognition and speech recognition. The rules of a task are constantly changing; as in fraud detection from transaction records. The nature of the data keeps changing, and the program needs to frequently adapt; as in automated trading, energy demand forecasting, and predicting shopping trends. 36 Real-world databases can be often disordered, incomplete, and available in various formats. In particular applications, we may encompass simple numeric data from e.g. material testing. In some other applications, we may need to combine different types of data, such as sensor signals, text, and camera streaming images, etc. 4.1.4 The process of finding the right model to fit the data Choosing the right pattern is a balancing act. Highly flexible models tend to overfit data by modeling small noise-causing variations [25]. On the other hand, it is likely that simple models presume too much. There are always tradeoffs between model speed exactness and the complexity of the chosen model. Bearing in mind that the essence of machine learning is trial and error— if one method or algorithm does not work, we should simply try another. But a structured process would obviously help us get off to a smooth start. 4.1.5 A general look to machine learning algorithms Machine learning uses two types of techniques in general: supervised learning, which trains a model on labeled input and output data so that it can predict future outputs; and unsupervised learning, which finds hidden patterns or intrinsic structures in unlabeled data. 4.1.6 Supervised learning A supervised learning algorithm takes a known set of input data and known responses to the data (output) and trains a model to generate reasonable predictions for the response to new input data. Supervised learning uses classification and regression techniques to develop such predictive models. Classification techniques predict discrete responses—for example, whether an email is 37 genuine or spam, or whether a tumor is cancerous or benign. Typical applications include medical imaging, speech recognition, and credit scoring. Regression techniques predict continuous responses— for example, changes in temperature or fluctuations in power demand. Typical applications include electricity load forecasting and algorithmic trading. 4.1.7 Unsupervised learning Unsupervised learning finds hidden patterns or intrinsic structures in data. It is used to draw inferences from datasets consisting of input data without labeled responses. Clustering is the most common unsupervised learning technique. It is used for exploratory data analysis to find hidden patterns or groupings in data. Applications for clustering include gene sequence analysis, market research, and object recognition. 4.1.8 Which one suits the application of our project? (Supervised or Unsupervised) Choosing the right algorithm may seem overwhelming in practice. There are hundreds of supervised and unsupervised machine learning algorithms, and each takes a different approach to learning. There is no best method or one size fits all. Finding the right algorithm for a given application is partly trial and error. Even a highly experienced data scientist may not tell whether an algorithm will certainly work without trying it out. But algorithm selection also depends on the size and type of data we are working with, the insights we want to get from the data, and how those insights will be used [26]. In the current project, given the known labels of input and output variables (per Chapter 3; Figure 3.5), supervised learning should be used [27]. 38 4.1.9 Selecting among supervised learning techniques As mentioned earlier, most supervised learning techniques are in the form of classification or regression. Figure 4.2 shows different algorithms of supervised learning techniques. Figure 4.2 Different algorithms of supervised machine learning. Selecting the right algorithm is a trade-off between specific characteristics of the algorithm, such as [26]: • Speed of training; • Memory usage; • Predictive accuracy on new data; and • Transparency or interpretability (how easily a user can understand the reasons an algorithm makes its predictions). Supervised Machine learning Classification Nearest Neighbor Discriminant AnalysisDesicion Tree Naive BayesSupport Vector Machine Regression Linear RegressionRegression TreesSupport Vector MachineGaussian Process Regression 39 Here we discuss some of the most commonly used classification and regression algorithms for supervised learning, as pertinent to our project. Common classification algorithms Figure 4.3 (a) Logistic regression concept; the two colors show the two class of binary responses; (b) K-Nearest Neighbor method schematic’ (c) Support Vector Machine schematic, with nonlinear boundary. (d) Neural Network; (e) A decision tree schematic’ (f) Linear regression schematics’ (g) Nonlinear regression; (h) SVM regression schematic; the error margin is shown as dashed lines. (i) A regression tree. (a) (b) (c) (d) (e) (f) (g) (h) (i) 40 • Logistic regression This algorithm fits a model that can predict the probability of a binary response belonging to one class or the other (e.g. pass/fail quality prediction in a manufacturing process). Because of its simplicity, logistic regression is commonly used as a starting point for binary classification problems [28]. It is best used [26]: • When data can be clearly separated by a single, linear boundary; • As a baseline for evaluating more complex classification methods. • k Nearest Neighbor (kNN) kNN categorizes objects based on the classes of their nearest neighbors in the dataset. kNN predictions assume that objects near each other are similar. Distance metrics, such as Euclidean, city block, cosine, and Chebychev, are used to find the nearest neighbors. It is best used [26, 29]: • When you need a simple algorithm to establish benchmark learning rules; • When memory usage of the trained model is a lesser concern; • When the prediction speed of the trained model is a lesser concern. • Support Vector Machine (SVM) SVM classifies data by finding the decision boundary (hyperplane) that separates all data points of one class from those of the other class. The best hyperplane for an SVM is the one with the largest margin between the two classes when the data is linearly separable. If the data is not linearly separable, a loss function is used to penalize points on the wrong side of the hyperplane. It is best used [26, 30]: • For data that has exactly two classes; 41 • For high-dimensional, nonlinearly separable data. • When you need a classifier that’s simple, easy to interpret, and accurate • Neural Networks Inspired by the human brain, a neural network consists of a highly connected network of neurons that relate the inputs to the desired outputs. The network is trained by iteratively modifying the strengths of the connections so that the given inputs map to the correct response, with minimum prediction error. It is best used [26, 31]: • For modeling highly nonlinear systems; • When data is available incrementally and you wish to constantly update the model ; • When there could be unexpected changes in your input data; • When model interpretability is not a key concern. • Decision trees A decision tree lets you predict responses to data by following the decisions in the tree from the root (beginning) down to a leaf node. A tree consists of branching conditions where the value of a predictor is compared to a trained weight. The number of branches and the values of weights is determined in the training process. Additional modification, or pruning, may be used to simplify the model. It is best used [26, 32]: •When you need an algorithm that is easy to interpret and fast to fit; • To minimize memory usage; • When high predictive accuracy is not a requirement. 42 Common regression algorithms • Linear regression Linear regression is a statistical modeling technique used to describe a continuous response variable as a linear function of one or more predictor variables. Because linear regression models are simple to interpret and easy to train, they are often the first model to be fitted to a new dataset. It is best used [26, 33]: • As a baseline for evaluating other, more complex, regression models. • Nonlinear regression Nonlinear regression is a statistical modeling technique that helps describe nonlinear relationships in experimental data. Nonlinear regression models are generally assumed to be parametric, where the model is described as a nonlinear equation. “Nonlinear” refers to a fit function that is a nonlinear function of the fit parameters. For example, if the fitting parameters are b0, b1, and b2: the equation 𝑦 = 𝑏0 + 𝑏1𝑥1 + 𝑏2𝑥2 is a linear function of the fitting parameters, whereas 𝑦 =𝑏0𝑏1𝑏2 +𝑋 is a nonlinear function of the fitting parameters. It is best used [34, 35]: • When data has strong nonlinear trends and cannot be easily transformed into a linear space; • For fitting custom models to data. • SVM regression SVM regression algorithms work like SVM classification algorithms but are modified to be able to predict a continuous response. Instead of finding a hyperplane that separates data, SVM 43 regression algorithms find a model that deviates from the measured data by error value no greater than a predefined small amount (to minimize sensitivity to error). It is best used [34, 36]: • For high-dimensional data (where there will be a large number of predictor variables). • Regression trees Decision trees for regression are similar to decision trees for classification, but they are modified to be able to predict continuous responses. It is best used [34, 37]: • When predictors are categorical (discrete) or behave nonlinearly. 4.2 Details of algorithms used this project 4.2.1 KNN K Nearest Neighbor The k-nearest neighbors algorithm (k-NN) is a non-parametric method used for classification and regression . In both cases, the input consists of the k closest training examples in the feature space. The output depends on the type of given problem [38]: • In k-NN classification, the output is a class membership. An object is classified by a plurality vote of its neighbors, with the object being assigned to the class most common among its k nearest neighbors. If 𝑘 = 1, then the object is simply assigned to the class of that single nearest neighbor. • In k-NN regression, the output is the property value for the object. This value is the average of the values of k nearest neighbors. A useful strategy for both classification and regression can be to assign weights to neighboring inputs so that the nearby neighbors contribute more to the average than the more distant ones. A 44 common weighting scheme, for example, is to give each neighbor a weight of 1/𝑑, where 𝑑 is the distance from the neighbor [39]. • Algorithm The training examples are vectors in a multidimensional feature space, each with a class label. The training phase of the algorithm consists only of storing the feature vectors-for example features of images can be the pixel numbers, contrast, brightness, etc- and class labels of the training samples. In the classification phase, k is a user-defined constant, and an unlabeled vector (a query or test point) is classified by assigning the label which is most frequent among the k training samples nearest to that query point. A disadvantage of the basic "majority voting" classification occurs when the class distribution is skewed. That is, examples of a more frequent class tend to dominate the prediction of the new example because they tend to be common among the k nearest neighbors due to their large number. One way to overcome this problem is to weight the classification, taking into account the distance from the test point to each of its k nearest neighbors. The class (or value, in regression problems) of each of the k nearest points is multiplied by a weight proportional to the inverse of the distance from that point to the test point. • Distances We can compute the distance between two points (design scenarios) using some pre-defined distance function (𝑥 , 𝑦) where 𝑥, 𝑦 are scenarios composed of 𝑁 features, such that 𝑥 ={𝑥1, … . , 𝑥𝑁 } , 𝑦 = { 𝑌1, … . , 𝑌𝑁 }. Common distance functions are Equation 4-1 and Equation 4-2: • Absolute distance: 45 𝑑𝐴(x, y) = ∑ |𝑋𝑖 − 𝑌𝑖 |.𝑁𝑖=1 Equation 4-1 • Euclidean distance: 𝑑𝐴(x, y) = ∑ √𝑋𝑖2 _𝑌𝑖2𝑁𝑖=1. Equation 4-2 Because the distance between two scenarios is dependent on the intervals, it is recommended that resulting distances be scaled such that the arithmetic means across the dataset is 0, and the standard deviation 1. This can be accomplished by replacing the scalars 𝑥, 𝑦 with 𝑥’, 𝑦’ according to the following transformation (see Equation 4-3 ): 𝑥′ =𝑥−?̅?𝛿(𝑥) . Equation 4-3 Where x is the unscaled value, ?̅? is the arithmetic mean of feature x across the data set (see Equation 3.4), 𝛿(𝑥) is its standard deviation (see Equation 3.5), and 𝑥′ is the resulting scaled value. The arithmetic mean is defined as Equation 4-4: ?̅? =𝑁𝑖=1∑ 𝑥𝑖 . Equation 4-4 The standard deviation is calculated as follows (see Equation 4-5): 𝜹(𝒙) = √𝟏𝑵 ∑ (𝒙𝒊 − ?̅?)𝟐𝑵𝒊=𝟏 . 46 Equation 4-5 • K-Nearest-Neighbor Now that we have established a measure in which to determine the distance between two scenarios, we can pass through the data set, one scenario at a time, and compare it to the query scenario. We can represent our dataset as a matrix 𝐷 = 𝑁 × 𝑃 containing 𝑃 scenarios 𝑠1 … 𝑠𝑝 , where each scenario 𝑠𝑖 contains N features 𝑠𝑖 ={𝑠1𝑖 , … . . , 𝑠𝑁𝑖 }. A vector o with length 𝑃 of output values, 𝑜 ={𝑜1 , … . , 𝑜𝑝} accompanies this matrix, listing the output value 𝑜𝑖 for each scenario 𝑠𝑖. It should be noted that the vector can also be seen as a column matrix; if multiple output values are desired, the width of the matrix may be expanded. • Summary of the steps kNN can be run in these steps: 1. Store the output values of the ‘𝑀’ nearest neighbors to query scenario ‘𝑞’ in vector 𝑟 = {𝑟 1 … .. 𝑟𝑀 } by repeating the following loop M times: a. Go to the next scenario 𝑠𝑖 on the data set, where 𝐼 is the current iteration within the domain {1, … , 𝑃} b. If 𝑞 is not set or 𝑞 < 𝑑(𝑞, 𝑠𝑖): 𝑞 ← 𝑑(𝑞, 𝑠𝑖), 𝑡 ← 𝑜𝑖 c. Loop until we reach the end of the data set (i.e., 𝑖 = 𝑃) d. Store q into vector c and t into vector r 2. Calculate the arithmetic mean output across r as follows in Equation 4-6: 47 ?̅? = 1𝑀∑ 𝑟𝑖𝑀𝑖=1 . Equation 4-6 3. Return ?̅? as the output value for the query scenario 𝑞 4.2.2 Support Vector Machine Basic Theory of SVM The goal of the two-class classification problems is to have a mathematical automated tool to distinguish between the two classes. Consider the example in Figure 4.4a, where there are several possible linear classifiers that can distinguish the data, but there is only one (Figure 4.4b) that maximizes the margin (the distance between the hyperplane and the nearest data point of each class). This linear classifier is called the Optimum Separating Hyperplane (OSH). Intuitively, we would expect this boundary to generalize well, as compared to the other potential limits [40]. 48 Figure 4.4 Support vector machine concept; (a) there are several possible linear classifiers but, (b) there is only one that maximizes the margin. Assume we have two separate classes, (𝑥1 , 𝑦1) , . . . , (𝑥1 , 𝑦𝑚) where xi ∈ Rn , yi ∈ {-1,+1} with a hyperplane 𝑤𝑥 + 𝑏 = 0. The set of vectors is said to be optimally separated by the hyperplane if it is separated without error and the margin is maximal. A canonical hyperplane has the constraint for parameters w and 𝑏: 𝑚𝑖𝑛 𝑥𝑖 𝑦𝑖(𝑊. 𝑋𝑖 + 𝑏) = 1. A separating hyperplane in the canonical form must satisfy the following constraints (Equation 4-7), 𝑌𝑖[(𝑊. 𝑋𝑖) + 𝑏] ≥ 1, 𝑖 = 1, … . . , 𝑚. Equation 4-7 The distance of a point 𝑥 from the hyperplane is ( Equation 4-8 ), 𝑑(𝑤, 𝑏; 𝑥) =|𝑤.𝑥+𝑏|||𝑤||. Equation 4-8 n m l Hyperplane Margin Support vectors (b) (a) 49 The margin is 2||𝑤|| according to its definition. Hence the hyperplane that optimally separates the data is the one that minimizes Equation 4-9 𝜑(𝑤) =12||𝑤||2. Equation 4-9 The solution to the optimizing problem (3) under the constraints of (1) is given by the saddle point of the Lagrange functional (see Equation 4-10), 𝐿(𝑤, 𝑏, 𝑎) =12||𝑤||2− ∑ 𝛼𝑖𝑚𝑖=1{𝑦𝑖 [(𝑤. 𝑥𝑖) + 𝑏] − 1}. Equation 4-10 Where 𝛼𝑖 are the Lagrange multipliers. The Lagrangian has to be minimized with respect to 𝑤, 𝑏 had maximized with respect to 𝛼𝑖 ≥ 0 . Classical Lagrangian duality enables the 𝑝𝑟𝑖𝑚𝑎𝑙 problem (4) to be transformed to its 𝑑𝑢𝑎𝑙 problem, which is easier to solve. The 𝑑𝑢𝑎𝑙 problem is given by Equation 4-11, 𝑊(𝛼) =𝛼 𝑚𝑎𝑥 { 𝐿(𝑤, 𝑏, 𝛼)𝑤,𝑏𝑚𝑖𝑛 }𝛼𝑚𝑎𝑥 . Equation 4-11 The solution to the 𝑑𝑢𝑎𝑙 problem is given by Equation 4-12, ?̅? = 𝐚𝐫𝐠 ∑ 𝜶𝒊 𝒎𝒊=𝟏𝜶 𝒎𝒊𝒏−𝟏𝟐∑ ∑ 𝜶𝒊𝜶𝒋𝒚𝒊𝒚𝒋𝑿𝒊 𝒎𝒊=𝟏𝒎𝒊=𝟏. 𝑿𝒋 . Equation 4-12 With constraints (Equation 4-13), 𝛼𝑖 ≥ 0 𝑖 = 1, … , 𝑚 50 ∑ 𝛼𝑖𝑦𝑖𝑚𝑖=1 = 0. Equation 4-13 Solving Equation 4-11 with constraints Equation 4-12 and Equation 4-13 determines the Lagrange multipliers, and the OSH (optimal separating hyperplane) is given by Equation 4-14 and Equation 4-15, ?̅? = ∑ ?̅?𝑖𝑦𝑖𝑋𝑖𝑚𝑖=1 . Equation 4-14 ?̅? = −12𝑤. [𝑥𝑟 + 𝑥𝑠]. Equation 4-15 Where 𝑥𝑟 and 𝑥𝑠 are support vectors, satisfying (see Equation 4-16), ?̅?𝑟 , ?̅?𝑠 > 0 𝑦𝑟 = 1 𝑦𝑠 = −1 Equation 4-16 For a new data point 𝑥, the classification is then (see Equation 4-17), 𝑓(𝑥) = 𝑠𝑖𝑔𝑛 (?̅?. 𝑥 + 𝑏). Equation 4-17 4.2.3 Linear regression with multiple variables Multiple linear regressions (𝑀𝐿𝑅), also known simply as multiple regression, is a statistical technique that uses explanatory variables (𝑥𝑖; 𝑖 = 1, . . . , 𝑛) to predict the outcome of a response variable ℎ𝜃 , by minimizing the sum of squares of the difference between the model and actual observations. The eventual goal of 𝑀𝐿𝑅 is to provide a linear relationship between the explanatory 51 (𝑖𝑛𝑑𝑒𝑝𝑒𝑛𝑑𝑒𝑛𝑡) variables and response (𝑑𝑒𝑝𝑒𝑛𝑑𝑒𝑛𝑡) variable in the form of (see Equation 4-18) [41]: ℎ𝜃 (𝑥) = 𝜃0𝑥0 + 𝜃1𝑥1 + ⋯ + 𝜃𝑛𝑥𝑛 . Equation 4-18 4.3 Measures to determine the goodness of fit 4.3.1 R-Squared A dataset has n values marked 𝑦1, . . . , 𝑦𝑛 (collectively known as 𝑦𝑖 or as a vector 𝑦 = [𝑦1, . . . , 𝑦𝑛]𝑇), each associated with a fitted (or modeled, or predicted) value 𝑓1, . . . , 𝑓𝑛 (known as fi, or sometimes ŷi, as a vector 𝑓 ). We define the residuals as 𝑒𝑖 = 𝑦𝑖 − 𝑓𝑖 (forming a vector 𝑒). 𝑦 ̅is the mean of the observed data (see Equation 4-19): ?̅? =1𝑛∑ 𝑦𝑖𝑛𝑖=1 . Equation 4-19 Then the variability of the dataset can be measured using three sums of squares formulas: The total sum of squares (proportional to the variance of the data) [42] (Equation 4-20): 𝑆𝑆𝑡𝑜𝑡 = ∑ (𝑦𝑖 − ?̅?)2𝑛𝑖=1 . Equation 4-20 • The regression sum of squares also called the explained sum of squares (Equation 4-21): • 𝑆𝑆𝑟𝑒𝑔 = ∑ (𝑓𝑖 − ?̅?)2𝑛𝑖=1 . Equation 4-21 • The sum of squares of residuals, also called the residual sum of squares (Equation 4-22): 52 𝑆𝑆𝑟𝑒𝑠 = ∑ (𝑦𝑖 − 𝑓𝑖)2 =𝑛𝑖=1 𝑒𝑖2 . Equation 4-22 The most general definition of the coefficient of determination (or R-Squared) is (Equation 4-23): 𝑅2 ≡ 1 −𝑆𝑆𝑟𝑒𝑠𝑆𝑆𝑡𝑜𝑡 . Equation 4-23 Figure 4.5 shows the concept of R-squared visually. Figure 4.5 The better the linear regression (b) fits the data in comparison to the simple average (a), the closer the value of 𝑹𝟐 is to 1. The areas of the blue squares represent the squared residuals with respect to the linear regression. The areas of the red squares represent the squared residuals with respect to the average value [43] 4.3.2 RMSE Root Mean Square Error (RMSE) is the standard deviation of the residuals (prediction errors). Residuals are a measure of how far from the regression line data points are; RMSE is a measure of how to spread out these residuals are. In other words, it shows how concentrated the data is around the line of the best fit. Root mean square error is commonly used in climatology, 53 forecasting, and regression analysis to verify experimental results. Equation 4-24 shows the RMSE calculation. 𝑅𝑆𝑀𝐸 = √(𝑓 − 𝑦)2̅̅ ̅̅ ̅̅ ̅̅ ̅̅ ̅. Equation 4-24 f = forecasts (expected values or unknown results), y = observed values (known results). In comparing different fitting models, the one with less RMSE would be mostly preferred because the data set is more concentrated on the best fit linear line. 4.3.3 Receiver Operating Characteristic (ROC) Here we explain receiver operating characteristic (𝑅𝑂𝐶) concept and how it helps us to compare different classification, prediction models. Figure 4.6 presents example data showing mice in two categories, ‘Obese’ and ‘not obese’ [44]. As seen in this graph 𝑀1 (mice number one) does not weigh that much, but it is still considered obese for its size and 𝑀9 is not obese even though it weighs highly. 54 Figure 4.6 ROC conceptual example; based on two groups of mice with different weights in two categories ‘obese’ and ‘not obese’ As shown in Figure 4.7 one could fit a logistic regression to the data. While doing that, the Y axis changes from 0 to 1. 𝑌 axis presents the probability of obesity. So e.g. if a mouse weighs 4𝑘𝑔 we can say that it is obese and can assure that a 1kg-mouse is not obese. Therefore, this logistic Regression gives the probability that a mouse is obese based on its weight. Figure 4.7 Fitting logistic regression to the illustrative dataset of Figure 34. Weight 55 However, if we want to classify the mouse as obese or not obese, we need a way to turn probability values to classification. One way is to set a threshold of 0.5 and classify all mice with a probability larger than 0.5 as "Obese" and classify all mice with a probability of less than 0.5 as "Not obese". Figure 4.8 shows how we could classify new mice groups using the logistic regression curve with the 0.5 cut off. We see that some mice are not classified correctly. Figure 4.8 Logistic regression classification with 0.5 cut-off Here we can use the confusion matrix to analyze the classification accuracy even better (see Figure 4.9). 56 Figure 4.9 Confusion matrix of the illustrative example [44]. By having a confusion matrix, we can now calculate two important terms, ‘Sensitivity’ -which is also called ‘Recall’ in machine learning- and ‘Specificity’ using the following equations (see Equation 4-25, Equation 4-26, Equation 4-27 and Equation 4-28): 𝑇𝑟𝑢𝑒 𝑃𝑜𝑠𝑖𝑡𝑖𝑣𝑒 𝑅𝑎𝑡𝑒 = 𝑆𝑒𝑛𝑠𝑖𝑡𝑖𝑣𝑖𝑡𝑦 =𝑇𝑟𝑢𝑒 𝑃𝑜𝑠𝑖𝑡𝑖𝑣𝑒𝑠𝑇𝑟𝑢𝑒 𝑃𝑜𝑠𝑖𝑡𝑖𝑣𝑒𝑠 + 𝐹𝑎𝑙𝑠𝑒 𝑁𝑒𝑔𝑎𝑡𝑖𝑣𝑒𝑠 Equation 4-25 And, 57 𝐹𝑎𝑙𝑠𝑒 𝑃𝑜𝑠𝑖𝑡𝑖𝑣𝑒 𝑅𝑎𝑡𝑒 = (1 − 𝑆𝑝𝑒𝑐𝑖𝑓𝑖𝑐𝑖𝑡𝑦) =𝐹𝑎𝑙𝑠𝑒 𝑃𝑜𝑠𝑖𝑡𝑖𝑣𝑒𝑠𝐹𝑎𝑙𝑠𝑒 𝑃𝑜𝑠𝑖𝑡𝑖𝑣𝑒𝑠 + 𝑇𝑟𝑢𝑒 𝑁𝑒𝑔𝑎𝑡𝑖𝑣𝑒𝑠 Equation 4-26 For this specific example: 𝑇𝑟𝑢𝑒 𝑃𝑜𝑠𝑖𝑡𝑖𝑣𝑒 𝑅𝑎𝑡𝑒 = 𝑆𝑒𝑛𝑠𝑖𝑡𝑖𝑣𝑖𝑡𝑦 =33 + 1= 0.75 Equation 4-27 𝐹𝑎𝑙𝑠𝑒 𝑃𝑜𝑠𝑖𝑡𝑖𝑣𝑒 𝑅𝑎𝑡𝑒 = (1 − 𝑆𝑝𝑒𝑐𝑖𝑓𝑖𝑐𝑖𝑡𝑦) =13 + 1= 0.25 Equation 4-28 Calculating "True Positive Rate" and "False Positive Rate" help us draw the ROC curve to analyze the accuracy of a machine learning model. Let us show how varying the threshold would affect the confusion matrix. For example, if it was highly important to correctly classify every obese sample, we could set the threshold to 0.1. This would result in the correct classification for all 4 obese mice. However, it would also increase the number of False-positives. The lower threshold would also reduce the number of False-negatives because all of the obese mice were correctly classified, and it would reduce the number of True Negatives because two of the mice that were not obese were incorrectly classified as obese. The importance of varying thresholds makes more sense when the data is presenting a subject of high sensitivity, like samples as infected with Ebola and not infected with Ebola. In this case, it is absolutely essential to correctly classify every sample infected with Ebola in order to minimize the risk of an outbreak and that means lowering a threshold, even if that results in more False Positives. 58 On the other hand, we could set the threshold to 0.9. In this case, we would correctly classify the same number of obese samples as when the threshold was set to 0.5; but we would not have any positives. We would correctly classify one more sample that was not obese and has the same number of false negatives as before. Based on this data, the higher threshold does a better job classifying samples as obese or not obese. The threshold could be set to anything between 0 and 1. Receiver Operator Characteristic (ROC) helps analysts monitor all information and decide what threshold to select. ROC graphs provide a simple way to summarize all of the information. Figure 4.10 represents the ROC curve schematic. The Y-axis shows the true positive rate, which is the same thing as sensitivity. The X-axis shows the False Positive rate, which is the same as 1-Specificity. The False Positive rate tells you the proportion of not obese samples that were incorrectly classified and are False Positives. To make this example clearer, in Figure 4.11 we show achieving the ROC curve out of calculating True Positive Rate and False Positive rate from the confusion matrix above. We start with a threshold that classifies all of the samples as obese and finds out the confusion matrix of that threshold. In the next steps, we do the math to find the True Positive Rate, and False Positive Rate. Going back to the Logistic regression, we increase the threshold so that we find all points coordinates on the ROC curve. 59 Figure 4.10 ROC curve schematic; The green diagonal line shows where True positive Rate= False Positive Rate [44]. In the current example, any point on this line means that the proportion of correctly classified obese samples is the same as the proportion of incorrectly classified samples that are not obese. 60 Figure 4.11 ROC curve sensitivity to threshold variation (shown as dash line), in the mice obesity example; Point (1, 1) means even though we correctly classified all of the obese samples, we incorrectly classified all of the samples that were not obese [44]. Threshold Variation Confusion Matrix Points on ROC curve 𝑇𝑟𝑢𝑒 𝑃𝑜𝑠𝑖𝑡𝑖𝑣𝑒 𝑅𝑎𝑡𝑒 =44 + 0= 1 𝐹𝑎𝑙𝑠𝑒 𝑃𝑜𝑠𝑖𝑡𝑖𝑣𝑒 𝑅𝑎𝑡𝑒 =44 + 0 = 1 Point is (1, 1) 𝑇𝑟𝑢𝑒 𝑃𝑜𝑠𝑖𝑡𝑖𝑣𝑒 𝑅𝑎𝑡𝑒 =44 + 0= 1 𝐹𝑎𝑙𝑠𝑒 𝑃𝑜𝑠𝑖𝑡𝑖𝑣𝑒 𝑅𝑎𝑡𝑒 =33 + 1= 0.75 Point is (1, 0.75) 𝑇𝑟𝑢𝑒 𝑃𝑜𝑠𝑖𝑡𝑖𝑣𝑒 𝑅𝑎𝑡𝑒 =44 + 0= 1 𝐹𝑎𝑙𝑠𝑒 𝑃𝑜𝑠𝑖𝑡𝑖𝑣𝑒 𝑅𝑎𝑡𝑒 =22 + 2= 0.5 Point is (1, 0.5) 𝑇𝑟𝑢𝑒 𝑃𝑜𝑠𝑖𝑡𝑖𝑣𝑒 𝑅𝑎𝑡𝑒 =33 + 1= 0.75 𝐹𝑎𝑙𝑠𝑒 𝑃𝑜𝑠𝑖𝑡𝑖𝑣𝑒 𝑅𝑎𝑡𝑒 =11 + 3= 0.25 Point is (0.75, 0.25) 𝑇𝑟𝑢𝑒 𝑃𝑜𝑠𝑖𝑡𝑖𝑣𝑒 𝑅𝑎𝑡𝑒 =33 + 1= 0.75 𝐹𝑎𝑙𝑠𝑒 𝑃𝑜𝑠𝑖𝑡𝑖𝑣𝑒 𝑅𝑎𝑡𝑒 =00 + 4= 0 Point is (0.75, 0) 61 4.3.4 Area under the Curve (AUC): 𝐴𝑈𝐶 is the area under the 𝑅𝑂𝐶 curve. In the current specific example, the 𝐴𝑈𝐶 is 0.9. The AUC makes it easy to compare one 𝑅𝑂𝐶 curve to another. In Figure 4.12 Area Under the Curve (AUC) for comparison of two predictive models (represented via two ROCs)𝐴𝑈𝐶 of the red 𝑅𝑂𝐶 curve is greater than the AUC for the blue ROC curve, suggesting that the red curve is better. So, e.g., if the red 𝑅𝑂𝐶 curve represents logistic regression and the blue 𝑅𝑂𝐶 curve represents a Random Forest model, we would use the logistic regression model. To summarize, 𝑅𝑂𝐶 curves make it easy to identify the best threshold for making a decision and 𝐴𝑈𝐶 can help on deciding which model is preferred. Figure 4.12 Area Under the Curve (AUC) for comparison of two predictive models (represented via two ROCs) [44]. 4.4 Summary In this chapter, we provided a brief review of common machine learning methods. We explained two main approaches called 'Supervised learning' and 'Unsupervised learning'. Supervised 62 learning is a method that predicts output based on input/inputs. However, unsupervised learning is a method that discovers the hidden patterns in given dataset. Supervised learning uses classification and regression techniques to develop predictive models. Based on a given training set and application, we can decide which method to use; 'Classification' or 'Regression'. Classification is used for a training set that the output is labeled in groups, like 'Obese' or 'Not obese', 'Wrinkle' or 'Not wrinkle', etc. Regression is used when the output in the training set can be a continuous number; for example, the 'Price of a house', 'the modulus of a formed material’, etc. In this thesis, the input and outputs are labeled. Thus, we aim to use supervised learning. More specifically, in one case our output will be the quality of formed fabric samples, which can vary between 0-100% as discussed in Chapter 3. For this case, we will use regression algorithms. In the other case, we aim to use the visual inspections to assess the existence of wrinkles on the formed samples. Therefore, the output of the training set is labeled as 'Wrinkle' and 'Not wrinkle', and we will use classification algorithms. There are many algorithms for both classification and regression methods. To be able to select an optimum algorithm for the training in a given dataset, we can use statistical measures such as 'R-squared' in Regression models, and 'ROC' and 'AUC' in classification models. Based on these factors, we will explain and compare in detail in the next chapter how Linear Regression, k-Nearest Neighbor, and Support Vector Machine can predict the quality outputs for the performed experiments. 63 Results and Discussion In this chapter, we first show how we applied machine learning algorithms on the extracted and quantified visual data from formed fabrics explained in Chapter 3, for quality check/analysis at the on-set of a forming line in factor (i.e. before forming; named Case study 1 in the following sections). Next, we explain the details of steps used to develop a live (on-line) machine learning tool for monitoring (e.g. by an operator or robot) the wrinkle/part quality “after forming”. The latter scenario has been called Case study 2. 5.1 Preliminaries To position both case studies of this chapter under Industry 4.0, let us imagine a hypothetical cyber-physical composite forming factory. 5.1.1 Hypothetical cyber-physical forming factory As we studied in Chapter 3, here we draped the composite prepreg fabrics in the form of a hemisphere part. The formed fabric was then partially cured using the heat gun to perform visual inspections. But this would be just the beginning in a related cyber-physical factory. In actual manufacturing, there are several there steps to pass like full curing, adding other layers of different materials, surface finishing, etc. The final product can be part of an airplane, submarine or vehicle's structure. Obviously, here we want to have a perfectly shaped hemisphere in the first step, otherwise doing all other steps on an imperfectly formed sub-structure can cause waste of time, money and labor to repair the part, if it does not pass the minimum quality measures. Per Chapter 3, by a perfect hemisphere forming, we mean a formed shape that has a base roundness very close to 1 and the sphericity of 84%. The other important quality issue in forming fabrics is to avoid 64 'wrinkles'. Designers must prevent wrinkle existence on the formed fabric as much as possible, often by tuning the forming set-up parameters via trial and error. To mimic that, here we have done multiple forming tests and analyzed the quality of ensuing samples. The number of clamps, the amount of applied torque to each clamp, orientation, and the number of layers varies from sample to sample. Once data is gathered, including visual inspection and quantification of output quality variables (see also Figure 3.5), now we want to train reliable machine learning model(s) to predict how different inputs yield a different amount of roundness, sphericity, number of wrinkles, wrinkles' height and the occupied area by wrinkles on the hemisphere's surface. After training the model, for example, a user/designer may input the number of clamps, amount of torque applied to clamps, orientation, and the number of layers, and predict the quality of the final sample. User can vary these inputs to get the desired sample based on the given application. Note that such a design decision making stage before the forming of fabric can be performed by the user on the HMI (Human Machine Interface) pages of a PLC (Programmable Logic Controller) as shown in Figure 5.1. Then, the underlying digital twin should give the quality percentage of the final product to the user. Now it is the user's decision to end up with the setting and pushing the production button to start or apply further revisions. The above-described scenario (i.e. before the draping stars) will constitute our Case study 1 philosophy. Next, under another scenario, assume the quality check is needed after the fabric is formed. We should inspect the prefrom's shape to see if it is qualified to move to the next steps in the manufacturing line (i.e. on-line quality monitoring). In a fully automated factory, imagine samples are formed, gradually cured and are on a moving conveyor. There is a robot (or an operator) in the 65 middle of the conveyor belt checking the quality of samples at each stage. This robot/operator has a smart camera in its end effector/hand. The conveyor belt is stopped every minute and the camera moves all around the sample and checks if there is a wrinkle on the sample or not. One can also further customize/automate such smart factory set-up. For example, the robot can be programmed to buzz every time it finds a wrinkle on a sample. Then we can set the buzzing time to 𝑛. Now moving conveyor belt stops, robot checks the quality of the sample, if it buzzes n times the sample is not qualified to move forward to the next steps so the robot picks and places the sample on another conveyor belt which is caring the waste samples. Otherwise, if the robot buzzes less than n times, the sample is ready to go forward to the next steps on the current conveyor. Figure 5.2 gives a visual look of this process, and the above-described scenario will constitute our Case study 2 philosophy. Figure 5.1 Set up variables on a hypothetical HMI screen in the smart forming factory 66 Figure 5.2 Hypothetical fabric forming quality check points in the smart factory 5.2 Case study I: Part quality prediction (before draping) 5.2.1 Training procedure As outlined in Chapter 3, for the current forming process we have selected four variables/ inputs see (Figure 5.3). 67 Figure 5.3 Inputs and outputs of forming experiments We defined the ensuing quality of samples based on the height and area of occupied wrinkles. Using a camera, we inspected the wrinkled region all-around each sample separately and then selected the maximum height and area values. Then we defined a four-region quality (Perfect/medium/low/rejected) membership function per Section 3.5.5 (Equation 3-4). Finally, we arrived with the input/output dataset as in Table 2. 68 INPUTS OUTPUT Set up number Orientation (degree) number of layers number of clamps T1* (lb.-inch) T2* (lb-inch) T3* (lb-inch) T4* (lb-inch) Height (Cm) Area (%) Quality (Out of 100) 1 0 1 4 5 5 5 5 0.6 10 88 2 0 1 4 7.5 7.5 3 3 1 22 65.5 3 0 1 4 7.5 3 3 3 1.1 17 76 4 45 1 4 5 5 5 5 0.7 10.5 86 5 45 1 4 7.5 7.5 3 3 1.5 20 69 6 45 1 4 7.5 3 3 3 1.2 18 74.5 7 60 1 4 5 5 5 5 0.8 12 84 8 60 1 4 7.5 7.5 3 3 1.6 22 66.5 9 60 1 4 7.5 3 3 3 1.3 18 73 10 0 1 3 5 5 5 0 1.7 20 64.7 11 0 1 3 3 5 7.5 0 2 25 59 12 45 1 3 5 5 5 0 1.8 22 65.5 13 45 1 3 3 5 7.5 0 2.1 26 57 14 60 1 3 5 5 5 0 1.6 23 66 15 60 1 3 3 5 7.5 0 2 25 60 16 0 2 4 5 5 5 5 0.25 3.75 95 17 0 2 4 7.5 7.5 3 3 1.3 19 73.5 18 0 2 4 7.5 3 3 3 0.9 13.5 82 19 0 2 3 5 5 5 0 1.6 21 66.7 20 0 2 3 3 5 7.5 0 2 25 60 21 45 2 4 5 5 5 5 0.35 5.25 93 22 45 2 4 7.5 7.5 3 3 1.4 20 70 23 45 2 4 7.5 3 3 3 1 15 80 24 45 2 3 5 5 5 0 1.6 21.5 67.5 25 45 2 3 3 3 7.5 0 1.1 17.6 76 26 60 2 4 5 5 5 5 0.5 7.5 90.1 27 60 2 4 7.5 7.5 3 3 1.5 21 67.56 28 60 2 4 7.5 3 3 3 1.2 17.8 75 29 60 2 3 5 5 5 0 1.5 20 68.4 30 60 2 3 3 5 7.5 0 1.8 23.5 63 Table 2 Experimental input/output of the forming fabric process 69 5.2.2 Algorithms Using data in Table 2, we used the 'regression learner app' in MATLAB to predict the ‘quality’ (as single output variable) based on inputs of the forming set-up. This app allowed the user to choose among various algorithms to train and validate regression models. After training multiple models, we can compare their validation (test) errors, and then choose the best model. The flowchart in Figure 5.4 shows a common workflow used for training regression models in the Regression Learner app. Figure 5.4 Workflow used for training models in the Regression Learner app Below, we explain the process of training and selecting the selected machine learning algorithms step by step. Step one _ Import the data This is the first step in training the machine where we define and import the inputs (predictors) and the output (response). We can specify holdout validation option, to use a percentage of the training set as the test dataset. For example, in Figure 5.5, the holdout validation is set to 25%. It means that the machine randomly selects and uses 25% of whole data later to validate the trained algorithm. 70 Figure 5.5 Importing the data Step two – Training In this step, we can select all or any of the algorithms that we want to train the machine with. Here we use all algorithms to be able to compare them and select the best one (see Figure 5.6). 71 Figure 5.6 Regression learning algorithms in the Learner app 72 Step three _ Comparing the algorithms Figure 5.7 shows different algorithms that were trained by the fabric forming quality dataset. We could compare the RMSE of each algorithm and select the one with the least RMSE. In this case, we could say that linear regression is the best algorithm for the given dataset; its R-squared is 0.89 and RMSE is 3.3615 (which means only ~%3.4 of the quality in the final product would be lost if the trained model is used as part of the hypothetical smart factory line in Figure 5.1). 73 Figure 5.7 Trained algorithms results To further dissect the composition of the algorithms, let us compare 'Linear Regression' and 'Linear SVM' performance based on the given dataset. Table 3 shows the comparison based on the R-squared and RMSE measures. 74 R-Squared RMSE Linear Regression 0.89 3.53 Linear SVM 0.54 7.10 Table 3 R-squared and RMSE comparison of linear regression and linear SVM algorithms In Figure 5.8 , Figure 5.9 and Figure 5.10, we can see the comparison of 'predicted vs actual data plot', 'Residuals plot', and ‘normality plot' of the linear regression and linear SVM trained models. Based on these graphs, the linear regression has met the normality assumption fairly well, with a minimum difference between the actual and predicted data, as well as a random scatter of residuals (i.e. no apparent trend/auto-correlation of errors). Residuals have been scattered more widely, and the prediction error at different point are larger in the linear SVM model. Figure 5.8 Predicted vs actual data plot for the linear regression model (Left plot) and for the SVM regression model (Right plot). In these plots, the predicted response of the models is plotted against the actual/true response. A perfect regression model has a predicted response lying on the 45-degree diagonal line. The vertical distance from the line at any point represents the error of the prediction at that set-up. A good model would have the errors scattered around, and near the line. Predicted vs actual data (linear Regression) True Predicted True Predicted response Predicted vs actual data (SVM Regression) 75 Figure 5.9 Residual plot of linear regression and SVM regression models. A good model would have its residuals scattered symmetrically around zero, with no obvious pattern (i.e. no sign of auto-correlation). True response True response Residuals (Quality) Residuals (Quality) Residual plot (SVM Regression) Residual plot (Linear Regression) 76 Figure 5.10 Normal probability plot of linear regression and linear SVM models. Normal Probability plot is a graphical technique for assessing whether a data set is approximately normally distributed. The data are plotted against a theoretical normal distribution in such a way that the points should form an approximate straight line. 5.2.3 Results Table 4 shows the predicted quality values by the model, as a function of different forming set-ups. Very matching results are observed here with the qualitative analysis results in Chapter 2. First, three rows in Table 4 are the inputs of the samples that we had tested already and defined their quality and we know that their quality in the training set (from Chapter 3) was 'Perfect', 'Medium', 'Poor'; Here we predicted 86.71, 77.83 and 65.70 which make a good agreement with the experimental observations, referring to definition of quality percentage in Table 1 We have created the test-dataset in a way that we can analyze the independent variables' effects on the 77 quality. For example, to analyze the effect of the applied torque, in rows 4, 5 and 6, the orientation, layer and number of clamps are constant but the magnitude of the torque varies. By checking the trained model results, we can infer that the greater value of torque has resulted in fewer wrinkles (higher quality) on the sample. This is what we had seen before in experiments as well. Based on experiments we could say that when we apply small torques, the clamps are fastened loosely and hence there is a slight gap between the blank holders and fabric while forming, This allows the fabric to slide before being fully sheared and hence it wrinkles the fabric, especially near the clamp areas. Table 4 Representation of test data set for case study 1 78 To analyze the effect of orientation, in rows 7, 8 and 9 all inputs are the same but orientation. The orientations are 45, 60 and 0 degrees. Based on the algorithm's prediction, the qualities of these are very close. This was seen in real samples as well. To check the effect of the number of layers on samples' quality, let us consider set-ups in rows 7-9, as compared to rows 10-12. We can see that decreasing the number of layers reduces the quality of the formed fabric. The quality of samples with two layers are (86% -87%), but when the number of layer changes to one, the quality decreases to (82%-83%). A greater number of layers ends up with better samples with fewer wrinkles. Having two layers distributes the applied forces between the layers as they are forming together. To find the effect of the number of clamps on the final samples' quality, we changed the number of clamps from 4 to 3 (from rows 10, 11, 12 to rows 13, 14, 15). As seen in Table 4 Representation of test data set the quality of samples has been reduced from (82% - 83%) to (68% - 70%), respectively. Figure 5.11 demonstrates examples of how linear regression model predicted the quality of samples in different set-ups. 79 Figure 5.11 Examples of different input combinations effect on the sample quality, based on the prediction of linear regression model. 80 5.3 Case study 2: Part quality monitoring online (after draping) In this section, we clarify how we can use machine learning to check the quality of samples after they have been formed. 5.3.1 Training procedure We opted to use a camera to take track of samples and check their quality and their form. We can attach this camera to the end effector of a robot, or it can be simply via a handheld camera of an operator Figure 5.12. The robot/operator arm moves around the sample and shows the result online. To train the machine learning/decision aid tool, we used the forming process of the same samples explained in Chapter 3, and then trained an on-line wrinkle quality classification machine. We will explain the steps (1-4) of the process of collecting data and the training in the next sub-section (the corresponding code is also shown in Figure 5.13) Figure 5.12 (a) A camera is attached to the 3-axis robotic arm (Dobot Robotic arm/ Magician); (b) The user is scanning the samples with camera (a) (b) 81 Figure 5.13 Online wrinkle classification code developed (see details of the steps 1-4 in the next section). Step one – training-set images Here we took a video of samples right upon forming, and we coded MATLAB to take screenshots via the camera every 0.5 seconds. By taking a 5-minute video, we could save ~600 image frames. When the code starts, a camera window pops up and the operator (or robot) can move the camera all around the sample and label (in the software) the 'wrinkle' regions. After taking a video of wrinkles, we close the camera window and another camera window pops up and now we take 1 4 3 2 82 another video, and this time mark/label the areas that have no wrinkle. In both cases, we can move the camera on all areas (the flat and hemisphere parts) of the samples; the user can also zoom in and out as needed. We can even take images of the sample when there are some other objects on the image frame, like the user's hand, background, etc; i.e. we do not need to restrict images just for the sample. By doing this, later when the designer would use the camera in future cases, he/she is not supposed to put the camera under a certain configuration. Figure 5.14 and Figure 5.15 present the examples of such images taken for the wrinkle and not-wrinkle parts of a sample, respectively. As seen in Figure 5.14, images of wrinkles have taken from different views. In (a) we can see a normal image of wrinkle. Some images are taken from very close and further distance of the wrinkle. We can see the whole sample including some marks on the sample and a piece of paper on it as an external object (c). In the image (d), we put a marker to cover part of the sample and there are other objects in the background but still, we can recognize the wrinkle and by taking this photo and putting it in the training set, we can train the machine to learn the wrinkle existence in this general image. Also, Figure 5.15 presents some images from training–set which was labeled as not wrinkle. There are images from the flat and hemisphere part of the sample. These images are taken from different views too. There are written numbers of the sample but still, there is no wrinkle in that area. That's why we labeled them as not wrinkle. There are images that are blurring (d). Variant images are useful because the machine learns to detect wrinkles or not wrinkle in multiple possible configurations. This would make the final AI-based decision aid tool more user-friendly since there is no need for setting a camera at a certain angle etc. Besides, there is no restriction to have images in a certain distance or color. The machine could still perform accurately as seen in the results section to follow. 83 Figure 5.14 Samples of ‘wrinkled’-labeled set of images, taken during the on-line quality monitoring of the formed parts; notice the presence of external objects/marks in some of the images (which were included intentionally). 84 Figure 5.15 Samples of ‘not-wrinkled’-labeled set of images, taken during the on-line quality monitoring of the formed parts; notice the presence of external objects/marks in some of the images (which were included intentionally). 85 Step two – imageSet function We used ‘𝑖𝑚𝑎𝑔𝑒𝑆𝑒t’ that returns a vector of image sets found through a recursive search starting from the folder called ‘training images’. The ‘𝑖𝑚𝑎𝑔𝑒𝑆𝑒𝑡’ output is a 1 − 𝑏𝑦 − 𝑁𝑢𝑚𝐹𝑜𝑙𝑑𝑒𝑟𝑠 vector, where 𝑁𝑢𝑚𝐹𝑜𝑙𝑑𝑒𝑟𝑠 is the number of folders that contain at least one image. Here in this example, we have two folders 'wrinkle' and 'not wrinkle' in the '𝑡𝑟𝑎𝑖𝑛𝑖𝑛𝑔𝐼𝑚𝑎𝑔𝑒’ folder. Step three – Bag Of Features We used the ‘Bag Of Features’ method to encode each image from the training set. This function detects and extracts features from the image and then uses the approximate nearest neighbor algorithm to construct a feature histogram for each image. The function then increments histogram bins based on the proximity of the descriptor to a particular cluster center. The histogram length corresponds to the number of visual words that the ‘𝐵𝑎𝑔 𝑂𝑓 𝐹𝑒𝑎𝑡𝑢𝑟𝑒𝑠’ object constructed. The histogram becomes a feature vector for the image (see Figure 5.16). Figure 5.16 Steps of Bag Of Features method. 86 Step four- training set table Now that features are extracted, we can define the training set table including the inputs/predictors and the output/response. The predictors are extracted features and response is the label of images ‘wrinkle’ and ‘not wrinkle’. 5.3.2 Machine learning model After preparing the training data set as explained in the previous section, we trained a machine learning model. To this end, we used the 'Classification Learner App' in Matlab. Below are the steps followed to train the classifier. Importing the data First, we imported the data of the training set and identified the predictors and responses. Predictors are vectors called image features, and the response labels are wrinkle and not wrinkle (see Figure 5.17). 87 Figure 5.17 Importing the data to the Classification Learner App Results Next, we ran different algorithms and compared their accuracies. As seen in Figure 5.18, the SVM and kNN were found to be more accurate algorithms in this case. In Figure 5.19 and Figure 5.20 we presented the 'scatter plot', 'Confusion Matrix', 'ROC Curve' and 'AUC magnitude' of the select models. Comparing these plots help the analyst better understand why one algorithm has performed overall better than the others. For better comparison purposes, next to SVM and kNN, we also included the 'Tree' model, which has been one of the low performing models based on Figure 5.18. The scatter plot presents the feature vector components of collected image data (see 88 also Section 5.3.1.2 for feature vector extraction) and the predictions in the same plot. By dissecting this plot, we can interpret how a given algorithm could predict the wrinkle and not wrinkle areas. In this plot, false predictions are shown by the 'cross' sign. As we discussed in detail in Chapter 3, the ROC curve can help to identify which threshold would give better accuracy for classification. The magnitude of the AUC which would help to compare different algorithms’ performance; the greater the AUC, the better the algorithm performance. Besides, the 'Parallel coordination plot' of the three algorithms can help to notice how the machine has used different features to detect the response and which algorithm could do it with less false predictions. Figure 5.18 Trained algorithms for case study 2 and the models’ accuracies 89 Figure 5.19 Scatter plots (left column) and Parallel coordinate plots (right column) of three models (kNN, SVM, and Tree). These plots can help to investigate features to include or exclude in the final model. The user can visualize training data and the misclassified points on the scatter plot. 90 Figure 5.20 ROC curves (left column) and Confusion Matrix (right column) of the kNN, SVM, and Tree algorithms. 91 To test the final performance of the machine learning models, we moved the camera around randomly selected new samples. The screen in Matlab code showed the online video and the prediction of the wrinkle presence/absence, per Figure 5.21. Two bars named 'wrinkle' and 'not-wrinkle' gave another visual view of the machine's challenge to predict the quality correctly. While moving the camera, these bars were updated continuously based on the camera's current view. Figure 5.21 Testing the trained online wrinkle quality machine; this example is for the kNN model. 92 To exemplify the comparison of the performance of the three algorithms, we called these algorithms in the codes and moved the camera around the same sample. kNN and SVM predicted equal outcome and correctly Figure 5.22. However, we can see a false prediction using Tree classifier Figure 5.23 and Figure 5.24. Figure 5.22 Comparing SVM and kNN performances in action. 93 Figure 5.23 The Tree algorithm performance in action; note the false predictions. 94 Figure 5.24 kNN and Tree algorithms performance comparison for a two-layer sample. Figure 5.24 shows another example of a comparison between e.g. Tree classifier’ and ‘k-NN classifier’ for a two-later forming sample. There is no wrinkle in the outer layer in this forming. Note that here we can see the number marks on the sample (3 and 7) as well as some parts of the background table etc.; yet, kNN has been accurate enough to detect that there is no wrinkle. Tree could correctly predict the obvious wrinkles only and failed when the wrinkles were not obvious or when there were some other external subjects on the sample image as shown above. 95 Conclusion and Future Work 6.1 Summary The main conclusions of this study regarding its experimental and modeling contents can be summarized as follows. 6.1.1 Experimental (manufacturing parameters effect based on Chapter 3): Figure 6.1 Experimental results summary of the thesis work. Clamps•The more clamps, overall the better quality (i.e. less chance of wrinkles)•Using four clamps helped the paunch-force to be distributed equally. Torque•Equal torques / less amount of Torque is preferred.•Equal amount of torque causes the equal distribution of forces in the forming process. Besides when there is less amount of torque, the clamp is fastened looser and so the fabric can slide easier during forming, which prevents stretching of yearn. Layer• Multi-layer layup was better than single layer, in terms of final apparent wrinkles.•Having two layers helps to distribute the forces during the forming. Our tests for modeling analyses were for single or two-layers forming. We also made an out of order sample using three layers, which compared to two layers had even less wrinkles and also had very less shear angles at the outer surface. Orientation• Overall, samples made of 45 degree orientation had better result (less wrinkles).• In multi-layer forming, the less orientation difference between layers, would cause less deformation distortion and hence less wrinkles. We specially observed this for 0 degree samples. 96 6.1.2 Modeling (based on Chapter 3): We applied machine learning to the training-set extracted from formed fabric samples. We explained two different approaches (quality control scenarios based on a hypothetical smart factory). In the case study 1, we explained how we used the data to train the machine to predict the quality of samples before being formed; e.g. useful for production design. The user can vary the forming variables to get the desired product quality and then start the main production line. We used the 'linear regression' model to predict the quality function, which was composed of different indices representing the wrinkling state. Also, we used a 'linear SVM' model for comparisons. The R-squared of the linear SVM was less than the linear regression. We could validate how these two different algorithms perform while they were trained based on the same training set. Finally, we could validate the prediction results with the actual observed quality of formed samples from experiments. • Results recommended that a higher number of clamps and layers, with a more and equal amount of torque on clamps, and the zero-fabric orientation produce least wrinkles and consequently higher quality samples. This was again perfectly aligned with experimental observations in Chapter 3. Next, as case study 2, we explained the application of machine learning in the on-line visual processing of samples (i.e. after the part is draped in the factory). We went through the steps to train the machine based on video scans of samples. In the training set, images were labeled as 'wrinkle' and 'not-wrinkle'. Comparing the 'ROC curve', 'AUC ' and the confusion matrix of three different algorithms (kNN classifier, SVM classifier, and Tree classifier), showed that the kNN 97 and SVM are very comparable and appropriate for this application. They both had an error rate of less than 5%. However, the Tree classifier's prediction accuracy was only 89% (i.e. over 10% error rate). In the final test, we could validate the above models’ performances in action, using new samples. In this test, we moved the camera around each sample and could see the online video on the user’s screen along with each algorithm's prediction (indicating 'wrinkle' or 'not wrinkle'). Next, we compared the algorithms performances two by two. First, we had SVM and kNN on the screen and we could see how they predict the same area of different test samples. We could see that the results of kNN and SVM were mostly the same and they predicted the wrinkles correctly. However, comparing Tree with kNN, we noted that Tree could predict the obvious wrinkles, but could not predict correctly when there were some other subjects on the sample; for example, a pen mark, a piece of paper, or background table image, etc. 6.2 A future work perspective Let us conclude this thesis by giving a big picture of an imaginary basic composites manufacturing factory called 'C4.0'. We then focus on just the process of forming in this factory and explain steps of production. Next, we clarify how such factory may benefit from some of the pillars and results discussed in this thesis, as future work. 6.3 C4.0 Figure 6.2 gives a view of the C4.0's fabric-forming production line. In this view of the factory, there are three main conveyor belts. There different rolls of fabric are the source of the process ('Twintex', 'Fiberglass' and 'Carbon fiber' fabric rolls, each with barcode for digital identification). 98 These fabrics are pre-formed and after passing through many other steps become part of the final structure/application. 6.3.1 Auto vehicles A mini autonomous vehicle (carrier), that is connected to other vehicles, moves forward the source. These vehicles are connected to each other by WI-Fi and know each other's positions online. This prevents the accident and gives them the possibility to decide better routes through the factory. These vehicles know the position of the source as well and they move toward it. If there would be an obstacle in their way, a smart camera on them detects it, and depending on the distance, the vehicle either reduces the speed or stops. This is exactly where e.g. the SVM/kNN algorithms may be used to train the vehicle on how to react to external unknown objects. 99 Figure 6.2 An imaginary C.40 smart composite manufacturing factory 100 The auto vehicle drives toward the source and uploads the certain fabric roll and drives to the beginning of the production line and drops it. As all auto vehicles are connected to each other, if there are two auto vehicles carrying the same material at the same time, there must be an error somewhere. Since there are three auto vehicles, and there are three production lines specified for certain fabric type, each vehicle should carry just one kind of fabric. It is very improbable to happen because there are barcodes on fabrics and auto vehicles carry smart barcode readers that detect the fabric's type. 6.3.2 Variables set-up Before starting the production process, the chief production engineer sets the variable for the production line. Here we suppose there is a touch screen PLC. Users can set the variables like discussed in Chapter 3; i.e. the number of forming clamps and composite layers, as well as the amount of torque applied to different clamp, and the orientation. User can select the appropriate paunch (tooling) shape too. As soon as variables are set, the set-up gets ready to start. When the user sets the variable, he/she can see the final product quality ‘prediction’ on the screen. They can vary the variables until the desired quality percentage is achieved, assuming also meeting the product design specifications. 6.3.3 Manufacturing barcode After variables are set. A robot sticks a barcode that includes historical detail about the manufacturing of the sample. For instance, the date and time of production, the production 101 engineer name, and the test variables; the number of clamps and layers, orientation degree and amount of torque can be extracted using the barcode. 6.3.4 Laser beam Here laser beam shows the edges of the square that must be cut off the fabric. The size of this square can be changed by programming the laser beam. Later this fabric is cut in squared shape based on defined size and moves on conveyor toward the next steps. 6.3.5 Orientation execution Orientation has been set by the user in the beginning. When squared cut fabric moves on the conveyor, a rotational disk changes the orientation of the fabric. For example, in Figure 6.2 the orientation of the first fabric is 45 degrees, the second one is 0 degree and the third one is 60 degrees. We have an orientation setting for both one layer and two-layer set-ups. When it is just one layer, the disk rotates for every squared-cut fabric. However, if the set-up is for two layers, the rotational disk rotates just for the second layer. 6.3.6 Forming Now that the layup is ready, the forming process can start. In this step, clamps are fastened, and the desired paunch stands on top of the forming set-up ready to form the fabric. Paunch moves downward and forms the fabric. The start time, the duration, and stop time all are programmed and set beforehand. 102 6.3.7 Curing process As soon as the fabric is formed. The heating system starts to heat-up and slightly cures the (thermoplastic) fabric. By curing the fabric, the form of the fabric fixes and is ready to be sent to the next processes. 6.3.8 Quality check This step is after the fabric is formed, cured and ready to be analyzed. Here a smart camera takes an online video of the product and checks the existence of wrinkles on the hemisphere. The alarm system buzzes/turns on every time it sees the wrinkle. And based on the program if it sees, for example, more than two wrinkles, the robot picks and places the product on the other conveyor which is shown as number 15 in Figure 6.2. This conveyor moves toward waste bins. Undesired products are shipped to the analysis center first, and then the recycling center. However, other samples that are good to continue to the production process move on the main conveyor belt and go to other stations of production. The quality check system sends data to the cloud continually. By monitoring the products daily and taking videos, not only the current production line benefits but also every time more data is added to the central digital library, it improves the machine learning accuracy for future predictions. Later the rejected samples with low quality are separated and analyzed by the R&D team. 6.3.9 Other future work ideas towards advanced C4.0 When we are thinking of using AI in a C4.0 factory, we should compare it (at least) with the human brain of a highly expert composites engineer. In case study 1 in Chapter 5, we showed how a trained machine learning model can predict an upcoming event (here the product quality based on 103 the state of wrinkles) based on past data. In case study 2 (as an on-line monitoring tool), we showed many wrinkles to the AI machine, and now every time that it sees a new wrinkle during the production line, it recognizes it. Future work perspective specific to these cases studies may be: Training the machine to hear the user We can train the machine in case study 1 to hear the voice of the production engineer and accordingly set the manufacturing variables. For example, instead of setting the variables on PLC, the user can read them for the machine. Immersive technology The trained models in the thesis may be used in a virtual technology device, like HoloLens. Using HoloLens, users can set the variables on a virtual screen in every stage of the process, and also see the predicted “finished product”. Training a hybrid AI machine learning model to predict (before forming) and detect (after forming) the quality of wrinkles and then update itself for next use; As we explained in case study 1, we could predict four quality levels for each forming set-up; ‘Perfect’, ‘Medium’, ‘Poor’, and ‘Rejected’. It can be a worthwhile future study to establish an advanced (self-updating/adaptive) AI to detect and measure the area and height of wrinkles during each new process and then update its training to not only classify the quality of samples in its prediction, but also to specify the predicted height, area, and location of wrinkles. This in fact would be a hybrid of case study 1 and 2 presented in Chapter 5. Modify the fabric type/ experimental set-up 104 Different types of fabrics, set-ups for example (different shapes of paunches) can be tested against different machine learning models to see if their performance is generalizable. Optimization Trained and validated model(s) can be used for optimization of the set-up as well. 105 References [1] A. Shanwan, S. Allaoui, Different experimental ways to minimize the preforming defects of multi-layered interlock dry fabric, International Journal of Material Forming 12(1) (2019) 69-78. [2] S. Allaoui, P. Boisse, S. Chatel, N. Hamila, G. Hivet, D. Soulat, E. Vidal-Salle, Experimental and numerical analyses of textile reinforcement forming of a tetrahedral shape, Composites Part A: Applied Science and Manufacturing 42(6) (2011) 612-622. [3] P. Ouagne, D. Soulat, J. Moothoo, E. Capelle, S. Gueret, Complex shape forming of a flax woven fabric; analysis of the tow buckling and misalignment defect, Composites Part A: Applied Science and Manufacturing 51 (2013) 1-10. [4] F.N. Nezami, T. Gereke, C. Cherif, Analyses of interaction mechanisms during forming of multilayer carbon woven fabrics for composite applications, Composites Part A: Applied Science and Manufacturing 84 (2016) 406-416. [5] J. Snudden, Progression to the next industrial revolution: Industry 4.0 for composites, Reinforced Plastics 63(3) (2019) 136-142. [6] A. Ustundag, E. Cevikcan, Industry 4.0: managing the digital transformation, Springer2017. [7] F. Backlund, D. Chronéer, E. Sundqvist, Project management maturity models–A critical review: A case study within Swedish engineering and construction organizations, Procedia-Social and Behavioral Sciences 119(0) (2014) 837-846. [8] H. Gaub, Customization of mass-produced parts by combining injection molding and additive manufacturing with Industry 4.0 technologies, Reinforced Plastics 60(6) (2016) 401-404. [9] J. Snudden, B. Horn, C. Ward, K. Potter, K. Drechsler, Effects of Automated Patch Placement on the Mechanical Performance of Reformed NCF Carbon Fibre, ECCM17-17th European Conference on Composite Materials, Munich, Germany, 2016, pp. 26-30. [10] V. Paelke, Augmented reality in the smart factory: Supporting workers in an industry 4.0. environment, Proceedings of the 2014 IEEE emerging technology and factory automation (ETFA), IEEE, 2014, pp. 1-4. [11] G. Michalos, P. Karagiannis, S. Makris, Ö. Tokçalar, G. Chryssolouris, Augmented reality (AR) applications for supporting human-robot interactive cooperation, Procedia CIRP 41 (2016) 370-375. [12] C. Wittenberg, Cause the trend Industry 4.0 in the automated industry to new requirements on user interfaces?, International Conference on Human-Computer Interaction, Springer, 2015, pp. 238-245. [13] L. Wang, G. Wang, Big data in cyber-physical systems, digital manufacturing and industry 4.0, International Journal of Engineering and Manufacturing (IJEM) 6(4) (2016) 1-8. [14] J. Snudden, Industry 4.0 for composites, (December 2019). [15] B. Bagheri, S. Yang, H.-A. Kao, J. Lee, Cyber-physical systems architecture for self-aware machines in industry 4.0 environment, IFAC-PapersOnLine 48(3) (2015) 1622-1627. [16] J. Lee, B. Bagheri, H.-A. Kao, A cyber-physical systems architecture for industry 4.0-based manufacturing systems, Manufacturing letters 3 (2015) 18-23. 106 [17] A. Long, A. Skordos, M. Sutcliffe, Optimisation of forming in woven composites manufacturing using variable peripheral pressure, (2006). [18] P. Wang, X. Legrand, P. Boisse, N. Hamila, D. Soulat, Experimental and numerical analyses of manufacturing process of a composite square box part: Comparison between textile reinforcement forming and surface 3D weaving, Composites Part B: Engineering 78 (2015) 26-34. [19] A. Prodromou, J. Chen, On the relationship between shear angle and wrinkling of textile composite preforms, Composites Part A: Applied Science and Manufacturing 28(5) (1997) 491-503. [20] A. Long, C. Rudd, M. Blagdon, P. Smith, Characterizing the processing and performance of aligned reinforcements during preform manufacture, Composites Part A: Applied Science and Manufacturing 27(4) (1996) 247-253. [21] T.G. Gutowski, G. Dillon, S. Chey, H. Li, Laminate wrinkling scaling laws for ideal composites, Composites Manufacturing 6(3-4) (1995) 123-134. [22] R. Arndt, Fabric preforming for structural reaction injection molding, ASM International(USA) (1991) 35-40. [23] H. Wadell, Volume, shape, and roundness of quartz particles, The Journal of Geology 43(3) (1935) 250-280. [24] Mathworks, Machine Learning with MATLAB, (February 2020). [25] Mathworks, Introducing Machine Learning, (February 2020). [26] Mathworks, Applying suppervised learning (January 2020). [27] T.O. Ayodele, Types of machine learning algorithms, New advances in machine learning (2010) 19-48. [28] M.J. Wainwright, J.D. Lafferty, P.K. Ravikumar, High-Dimensional Graphical Model Selection Using Regularized Logistic Regression, Advances in neural information processing systems, 2007, pp. 1465-1472. [29] M. Mejdoub, C.B. Amar, Classification improvement of local feature vectors over the KNN algorithm, Multimedia Tools and Applications 64(1) (2013) 197-218. [30] J. Górriz, C. Puntonet, M. Salmerón, Online algorithm for time series prediction based on support vector machine philosophy, International Conference on Computational Science, Springer, 2004, pp. 50-57. [31] N. Murata, S. Yoshizawa, S.-i. Amari, Network information criterion-determining the number of hidden units for an artificial neural network model, IEEE transactions on neural networks 5(6) (1994) 865-872. [32] M. Sahinoglu, Security meter: A practical decision-tree model to quantify risk, IEEE security & privacy 3(3) (2005) 18-24. [33] S. Ekici, F. Unal, U. Ozleyen, Comparison of different regression models to estimate fault location on hybrid power systems, IET Generation, Transmission & Distribution 13(20) (2019) 4756-4765. [34] Mathwork, Linear Regression, (March 2020). [35] D.A. Ratkowsky, D.E. Giles, Handbook of nonlinear regression models, M. Dekker New York1990. 107 [36] V. Cherkassky, Y. Ma, Practical selection of SVM parameters and noise estimation for SVM regression, Neural networks 17(1) (2004) 113-126. [37] A.M. Prasad, L.R. Iverson, A. Liaw, Newer classification and regression tree techniques: bagging and random forests for ecological prediction, Ecosystems 9(2) (2006) 181-199. [38] S. Matwork, Machine Learning Toolbox User's Guide, MathWork Inc., Natick, MA, USA, 2017. [39] Y. Li, B. Cheng, An improved k-nearest neighbor algorithm and its application to high resolution remote sensing image classification, 2009 17th International Conference on Geoinformatics, Ieee, 2009, pp. 1-4. [40] G. Guo, S.Z. Li, K. Chan, Face recognition by support vector machines, Proceedings fourth IEEE international conference on automatic face and gesture recognition (cat. no. PR00580), IEEE, 2000, pp. 196-201. [41] R.A. Bottenberg, J.H. Ward, Applied multiple linear regression, 6570th Personnel Research Laboratory, Aerospace Medical Division, Air Force Systems Command, Lackland Air Force Base, 1963. [42] N. Nagelkerke, D. 1991." A Note on a General Defini-tion of the Coefficient of Determination.", Biometrika 78(3) 69. [43] A. Ritter, R. Muñoz-Carpena, Performance evaluation of hydrological models: Statistical significance for reducing subjectivity in goodness-of-fit assessments, Journal of Hydrology 480 (2013) 33-45. [44] Sataquest, ROC and AUC clearly explained (December 2019). 108 Appendices Appendix A : Raw Data from Forming Experiments 109 Table A 1 Inputs and outputs data from the fabric forming experiments 110 Figure A1 The variation of the output data based on the forming set-up number as specified in Table A1
Thesis/Dissertation
eng
Mechanical Engineering
Vancouver : University of British Columbia Library
Attribution-NonCommercial-NoDerivatives 4.0 International
Graduate
A first step toward intelligent forming of fabric composites : adapting basic industry 4.0 concepts
Text