International Construction Specialty Conference of the Canadian Society for Civil Engineering (ICSC) (5th : 2015)

Automated monitoring of hardhats wearing for onsite safety enhancement Zhu, Zhenhua; Park, Man-Woo; Elsafty, Nehad Jun 30, 2015

Your browser doesn't seem to have a PDF viewer, please download the PDF to view this item.

Item Metadata

Download

Media
52660-Zhu_Z_et_al_ICSC15_138_Automated Monitoring_of_Hardhats_Wearing.pdf [ 1.07MB ]
52660-Zhu_Z_et_al_ICSC15_138_Automated Monitoring_of_Hardhats_Wearing_slides.pdf [ 1.06MB ]
Metadata
JSON: 52660-1.0076342.json
JSON-LD: 52660-1.0076342-ld.json
RDF/XML (Pretty): 52660-1.0076342-rdf.xml
RDF/JSON: 52660-1.0076342-rdf.json
Turtle: 52660-1.0076342-turtle.txt
N-Triples: 52660-1.0076342-rdf-ntriples.txt
Original Record: 52660-1.0076342-source.json
Full Text
52660-1.0076342-fulltext.txt
Citation
52660-1.0076342.ris

Full Text

5th International/11th Construction Specialty Conference 5e International/11e Conférence spécialisée sur la construction    Vancouver, British Columbia June 8 to June 10, 2015 / 8 juin au 10 juin 2015   AUTOMATED MONITORING OF HARDHATS WEARING FOR ONSITE SAFETY ENHANCEMENT Zhenhua Zhu1, Man-Woo Park2 and Nehad Elsafty1, 1 Department of Building, Civil and Environmental Engineering, Concordia University, Canada 2 Department of Civil and Environmental Engineering, Myongji University, South Korea Abstract: Construction is one of the most dangerous job sectors over the world. Any accidents that happen on the construction sites will bring the sufferings to the workers and their families and incur the delays and costs to the projects. Therefore, it is necessary for the contractors to monitor potential site safety issued and comply with existing safety regulations all the time. One of fundamental safety regulations is hardhat wearing. The wearing of the hardhats is always mandated and should not be violated anytime on the construction sites. In this paper, a novel method is proposed to facilitate the monitoring of whether any persons on the construction sites are wearing hardhats as required by the safety regulations. The method is built upon computer vision techniques. Under the method, human bodies and hardhats are first detected in the video frames captured by real-time on-site construction cameras. Then, their pair-wise matching is found. For those persons without the matching of the hardhats, they are identified as not wearing hardhats. The proposed method has been tested on real site videos. The test results showed that multiple persons could be monitored even if they are not wearing any real-time location sensors or tags. The test results demonstrate the potential of using live streaming or time-lapse construction site videos to facilitate the safety monitoring work on construction sites. 1 INTRODUCTION Construction has been identified as one of the most dangerous job sectors. CBC News (2012) once reported that a total of 700 deaths occurred in the construction industry from 2008 to 2010, which almost accounted for 23.3% of all workplace fatalities. Also, the Association of Workers’ Compensation Boards of Canada (AWCBC) mentioned that there were more than 27,000 time-loss injuries in the construction industry each year from 2009 to 2010 (AWCBC, 2011). In order to provide a healthy and safe working environment for construction workers, several onsite safety regulations have been established by the governments. They specify the contractors' responsibilities and duties on the construction sites, including  the appropriate use of appropriate personal protective equipment (PPE). The contractors must ensure that the regulations are enforced. In Quebec, this job is assigned to onsite safety inspectors, called construction site health and safety management guarantors. The guarantors need to take every necessary measure to ensure the general contractors comply with the safety regulations.  Hardhats are one type of important PPE. Any individual person must wear a hardhat, when they enter to a construction site. For example, the Safety Code for the Construction Industry mandates that "any person on a construction site shall wear a certified safety hat in accordance with CSA Standard" (Quebec, 2014). The similar guideline or regulation could also be found in the OSHA (Occupational Safety & Health Administration). It stipulates that “Employees working in areas where there is a possible danger of head 138-1 injury from impact, or from falling or flying objects, or from electrical shock and burns, shall be protected by protective helmets” (OSHA, 2012). Therefore, it is one of top priorities for the safety inspectors to check and confirm all employees and site visitors wear hardhats all the time on the construction sites. However, the statistic data indicated that that most workers who suffered impact injuries to the head were not wearing hardhats, even when performing their normal jobs on their regular worksites. The Bureau of Labor Statistics once noted that "hardhats were worn by only 16% of those workers who sustained head injuries, although two-fifths were required to wear them for certain tasks at specific locations" (OSHA, 2006). The reasons for not wearing hardhats could be various, such as discomfort while wearing hardhats, disassociation with the safety issues, etc. For example, workers might take off their hardhats to cool off, when the temperature is high. Therefore, it is necessary for the safety inspectors to monitor all persons on the site to wearing hardhats all the time. In order to facilitate the inspectors' monitoring work, this paper proposed a novel method for checking whether persons on the construction sites are wearing hardhats or not. The proposed method is built upon the live streaming or time-lapse construction site videos with computer vision techniques. It mainly include the detection of human bodies and the detection of hardhats. When the human bodies and hardhats are detected, their pair-wise matching is performed, so that those persons without wearing hardhats could be identified.  The effectiveness of the proposed method has been tested with real site videos. The test results showed that the method could monitor the wearing of hardhats on multiple persons simultaneously. These persons do not have to be tagged. This nature makes the method affordable to be used at most construction sites. The proposed method is expected to facilitate the safety inspectors' work and enhance construction site safety. The site safety enhancement could bring several benefits to the contractors, including increasing workers' productivity and reducing project direct and indirect costs due to the cost saving from preventing construction accidents.  2 RELATED WORK 2.1 RTLS-based Site Safety Enhancement The enhancement of onsite construction safety has been increasingly received attentions of researchers and industrial practitioners. Several research studies have been initiated for the purpose of adding an extra level of proactive safety measures using real-time location systems (RTLS)  These RTLS typically relied on the remote sensing techniques, such as radio frequency identification (RFID), global positioning system (GPS), wireless local areas networks (WLAN), and ultra-wide band (UWB).  Most RTLS have been used for proximity warning. Ruff (2007) compared different RTLS in his report for evaluating and implementing proximity warning systems on surface mining equipment. Moreover, Teizer et al. (2010) presented their findings of using Very-High Frequency (VHF) active Radio Frequency (RF) technique to warn ground workers and equipment operators once they are too close in proximity. In the work of Carbonari et al. (2011), they relied on UWB tracking data to implement a proactive virtual fencing system, and demonstrated its capability of reinforcing safety management policies. Also, Chen and Teizer (2013) integrated the real-time GPS and UWB location data into a virtual reality environment representing a construction site for monitoring construction safety.  In addition to the use of RTLS for the proximity warning, the systems could also be adopted for other purposes related to construction safety enhancement. Kelm et al. (2013) designed an RFID-based portal to check whether the workers' personal protective equipment (PPE) complied with the corresponding specifications. Aguilara and Hewageb (2013) developed an IT based safety management system using web cameras, barcodes and RFID tags installed on construction equipment. However, existing RTLS come with limitations, when they are used for monitoring the wearing of hardhats on construction workers. First, in order to check whether construction workers are wearing hardhats, it is at least necessary to attach a tag or sensor on each person and hardhat. In addition, the RTLS are mainly 138-2 used for real-time location tracking. Therefore, it is difficult to use the RTLS to identify whether the hardhats are appropriately used, considering that workers might take off their hardhats and put them aside while doing their jobs.  2.2 Video-based Site Safety Enhancement Live streaming or time-lapse video cameras are increasingly placed on construction sites, considering their acceptable return on investment (Bohn and Teizer, 2010). For example, Hydro-Qubec has recently decided to install video cameras on their construction sites to facilitate the monitoring of daily construction activities.The live streaming or time-lapse videos collected by the cameras record project ongoing construction activities on the construction sites. Therefore, they are useful for monitoring construction site safety issues. Abeid and Arditi (2002) relied on live streaming or time-lapse videos collected on construction sites to investigate construction accidents. Liaw et al. (2012) used the live stream or time-lapse videos to create safety training and education media resources.  So far, several research studies have been initiated to investigate the potentials of live streaming or time-lapse videos to create onsite safety alert systems built upon computing techniques. For example, Steele et al. (2003) once proposed to mount a stereo camera on the rear of an off-highway dump truck, and the stereo videos could help the truck driver identify possible on-site obstacles on the mining site. In addition to mounting stereo cameras on the equipment, the cameras are also placed on the site to capture and analyze workers' unsafe actions, considering these actions might result in accidents. Han and Lee (2013) and Han et al. (2013) relied on Kinect-style cameras to monitor whether construction workers on a ladder are leaning too far to one side or reaching too far overhead to prevent the falls from the ladders.  Similar to the work presented in this paper, Giovanni et al. (2009, 2011) designed a vision-based system to support the workers' safety by checking the presence of those workers without wearing hardhats on construction sites. In their work, they first used a pedestrian classifier with covariance descriptors to recognize construction workers in video frames, and then used the head and hard-hat detectors. However, there is one major limitation in their system. The system could not function correctly, when the workers on construction sites wear white hardhats. The failure of the detection of white hardhats makes the system generate a lot of false alerts. As illustrated in Figure 1,  the safety alert issued by their system (Giovanni et al. 2009, 2011) was false, since the person identified as not wearing a hardhat (red box) was actually wearing a hardhat.  Figure 1: Failure of detecting white hardhats 138-3 4.2 Hardhat Detection Hardhats are typically made of fiberglass and rigid plastic. On the one hand, the colors of the hardhats could be white, brown, green, blue, orange, red, etc. to indicate the roles (e.g. managers, engineers, superintendents, and laborers) of the wearers on construction sites. Considering the colors could vary, it would be difficult to rely on the color information as detection cues. On the other hand, most hardhats have similar cap-style shapes. They are rigid and do not have deformations.  Therefore, hardhat detection here is similar to the detection of human bodies. The HOG features of the hardhats are extracted from the video frames as the detection cue, especially considering that the HOG feature is capable of describing detailed shape information in an efficient way. Moreover, the effectiveness of the HOG features has been well-proven in many research studies for shape-based object detection. The hardhat detection includes two stages. First, hundreds of construction site images with different poses of hardhats are collected as a training database, where the hardhat HOG features are calculated and input into an SVM for the supervised training. The training results are compiled into a hardhat detection model. In order to detect hardhats in a video frame, the HOG features extracted from the test frame are input to the detection model, and the best possible placements of the hardhats could be determined. The detection model here only uses the HOG features without color cues. Therefore, it could detect the hardhats with different colors.  4.3 Human Body and Hardhat Pair-wise Matching After the detection of human bodies and hardhats, the detected hardhats should be matched to the detected human bodies to find out those persons who are not wearing hardhats on the construction sites. The matching here is mainly based on the spatial relationships between the detection regions of the hardhats and human bodies. Typically, the detected hardhat should be located in the upper area of the detected human body region, if the person in the detected human body region is wearing a hardhat. If there is no matching that could be found on that person, it indicates that the person is not wearing a hardhat. Then, a safety alert is produced to ask for the safety inspector's attention. One example of the pair-wise matching between the detected human bodies and hardhats could be found in Figure 3. In the figure, the matches of the hardhats for three persons in the right could be successfully found. As a contrast, the pair-wise matches of the hardhat for the two persons in the left could not be found, since they are not wearing hardhats.   Figure 3: Pair-wise matching between human body and hardhat 5 IMPLEMENTATION AND RESULTS The method was implemented and tested with real site videos captured by a Canon VISXIA HF S100 at a project site in Atlanta. The site was managed by Barton Malow Company. The test videos includes a total 138-5 of five persons. Three of them were wearing hardhats on the site and two were not wearing hardhats for the evaluation purpose. The test video was 166-second long, and recorded with 20 frames per second (fps) containing 3,320 frames. The resolution of the videos is fixed at 768 pixels by 432 pixels. 5.1 Results of Human Body Detection and Hardhats Detection Figure 4 illustrates the results of detecting human bodies and hardhats in the test video frames. Most human body detections made by the method are correct, although the method missed some human body detections (i.e. the persons in the video frames are not detected by the method). The correct detection of human bodies is more important than the missed detection in terms of monitoring whether a person is wearing a hardhat or not on the construction site. If the method could only successfully detect a person every 10 frames, the miss rate is high, but it is still able to monitor whether the person is wearing a hardhat once in a second. On the other hand, if the correct detection rate is low, it increases the chances of false alarms. For example, if the method misrecognizes tree branches as a human body, a false alarm will be issued, since the hardhat could not be detected in the detection region of that 'person'.       Figure 4: The detection of human bodies and hardhats Compared with the human body detection, most hardhats in the test video frames could be detected by the proposed method. However not all the detections are correct. The strategy for the hardhat detection is to reduce the detection miss rate. The low detection miss rate is more important than the correct hardhat detection rate in terms of monitoring whether a person is wearing a hardhat or not on the construction site. In other words, most false hardhat detections do not significantly affect the performance of issuing the final safety alerts, when persons are not wearing hardhats. The reason is mainly due to difficulty in establishing the links of the incorrect hardhat detection results to the detection of any human bodies.  5.2 Performance of Safety Alerts for not Wearing Hardhats When there is no hard detected in a human body detection region, it indicated that the individual person in the human body detection region is not wearing a hardhat. Therefore, a safety alert should be issued o highlight the person for the onsite safety inspector's attention. In order to measure the performance of the safety alerts issued by the proposed method, the issued safety alerts are manually classified as true positive (TP) alerts, false positive (FP) alerts, and false negative (FN) alerts. The TP alerts are the alerts that should be issued in reality. The FP alerts are the alerts that do not have to be issued in reality. The FN alerts are the alerts that should be issued in reality but the alert is not issued by the method. Their definitions have been illustrated in Table 1. Based on the number of TP, FP, and FN safety alerts, the precision and recall for issuing the safety alerts could be calculated using the following equations. [1] Precision = TP / (TP + FP)           [2] Recall = TP / (TP + FN)                138-6 Table 1: Definition of TP, FP, and FN safety alerts Category Issuing the alert based on the method Issuing the alert based on the safety regulation TP YES YES FP YES NO FN NO YES According to the test results, it was found that the precision for issuing the safety alerts reached 94.3%, while the recall for issuing the safety alerts was 89.4%. The 94.3% precision means that 5.7% of the alerts issued by the proposed method are false. The 89.4% recall indicates that nine out of ten individual persons without wearing hardhats could be successfully monitored when applying the proposed method on construction sites. Both high alert precision and recall show the effectiveness of the proposed method on monitoring whether individual persons on the construction site are wearing hardhats as required by existing safety regulations, such as the Safety Code for the Construction Industry and the Safety and Health Regulations for Construction.  6 LIMITATIONS AND DISCUSSION There are several limitations found from the tests. Most limitations are more or less associated with the detection of human bodies on a construction site. First, the human body detection is limited to the detection of the persons with standing or walking postures on the construction sites. So far, it is difficult to detect the persons with other postures. For example, when the workers are crouching down, bending, or sitting, it is difficult for the method to detect them. Therefore, the detection template used by the method should be extended to include more training samples of the workers with different postures.  Second, the occlusion is one of the major obstacles that could affect the performance of the proposed method on monitoring the wearing of the hardhats. If the persons are partially or fully occluded on the construction site, the monitoring with the proposed method will fail. The proposed method could only monitor the workers when they are visible in the camera view. Therefore, it is important to install an appropriate  number of construction cameras and select the appropriate positions for installing the cameras on a construction site.  In addition, the proposed method relied on the background subtraction to speed up the detection of human bodies. The purpose of the background subtraction is to limit the detection of human bodies only in the foreground blobs in motion. This means only moving persons on the construction site could be detected. If they are static without any motions, the workers are filtered out through the background subtraction, since they are considered as the background. Considering the workers could always be detected when they just enter into the camera view, the integration of the detection and tracking of onsite persons might provide one possible solution to continuously monitor the persons even if they are static. 7 CONCLUSIONS AND FUTURE WORK Existing safety policies and regulations could enhance construction site safety; however, these policies and regulations might not be strictly followed on the construction site. For example, construction workers might slip up and not exactly keep the requirements in the policies and regulations due to fatigue, distractions, carelessness, etc. Therefore, it is necessary to monitor any violations of the safety policies and regulations on construction sites. Currently, this safety monitoring job is manually performed by onsite safety inspectors (e.g. construction site health and safety management guarantors in Quebec). It might be difficult for the inspectors to monitor all safety policy and regulation violations on the construction site at any time through manual monitoring. In order to facilitate the safety inspectors' monitoring work, this paper proposed a novel, vision-based method for automatically monitoring whether any individual persons wear hardhats on the construction site as required by existing construction site safety polices and regulations. The method includes three 138-7 main steps: 1) human body detection, 2) hardhat detection, and 3) human body and hardhat pair-wise matching. If there is a human body detection area without the corresponding hardhat detection matched, it means that the person in the area is not wearing the hardhat. Therefore, a safety alert will be issued.  The method has been tested with the video frames captured at a real site. According to the test results, it was found that the overall precision and recall for issuing the safety alerts with the proposed method could reach 94.3% and 89.4%. The high precision and recall indicate the effectiveness of the proposed method on monitoring whether individual persons are wearing hardhats on the construction site. Future work will focus on the following two areas. First, the monitoring of wearing other types of personal protective equipment (safety vests, boots, etc.) on the construction sites will be investigated using vision technologies. Also, more investigations regarding the detection of human bodies with different postures will be performed in the future.  Acknowledgements This paper is based in part upon work supported by the National Science and Engineering Research Council (NSERC) of Canada. Any opinions, findings, and conclusions or recommendations expressed in this paper are those of the author(s) and do not necessarily reflect the views of the NSERC. References Abeid, J. and Arditi, D. (2002). ”Time-Lapse Digital Photography Applied to Project Management.” Journal of Construction Engineering and Management, 128(6), 530–535.  Aguilar, G., and Hewage, K. (2013). "IT based system for construction safety management and monitoring: C-RTICS" Automation in Construction, 35: 217-228. Association of Workers' Compensation Boards of Canada (AWCBC), (2011)."Summary Tables of Accepted Time-loss Injuries/Diseases and Fatalities by Jurisdiction."<http://www.awcbc.org/en/statistics.asp> (August 10, 2013) Bohn, J. and Teizer, J. (2010). "Benefits and Barriers of Construction Project Monitoring Using High-Resolution Automated Cameras." J. Constr. Eng. Manage, 136(6), 632–640. Carbonari, A. and Giretti, A. and Naticchia, B. (2011). "A proactive system for real-time safety management in construction sites." Automation in Construction, 20(6): 686-698. Cheng, T., and Teizer, J. (2013). "Real-time resource location data collection and visualization technology for construction safety and activity monitoring applications." Automation in Construction, 34: 3-15. Dalal, N. and Triggs, B. (2005)."Histograms of Oriented Gradients for Human Detection." In: IEEE Computer Society Conference on Computer Vision and Pattern Recognition, San Diego, CA. Vol. 1, pp. 886-893.  Felzenszwalb, P., Girshick, R., McAllester, D., and Ramanan, D. (2010). "Object Detection with Discriminatively Trained Part Based Models." IEEE Transactions on Pattern Analysis and Machine Intelligence, 32(9): 1627--1645. Gualdi, G., Prati, A., and Cucchiara, R. (2011). "Contextual information and covariance descriptors for people surveillance: an application for safety of construction workers." Journal on Image and Video Processing 2011, Article No. 9, pp. 1 - 16. Gualdi, G., Prati, A., and Cucchiara, R.  (2009). "Covariance descriptors on moving regions for human detection in very complex outdoor scenes." In Distributed Smart Cameras, 2009. ICDSC 2009. Third ACM/IEEE International Conference on, pp. 1-8. IEEE, 2009. Han, S., and Lee, S. (2013) "A vision-based motion capture and recognition framework for behavior-based safety management." Automation in Construction, 35: 131-141. Han, S., Achar, M., Lee, S. and Peña-Mora, F. (2013) "Empirical assessment of a RGB-D sensor on motion capture and action recognition for construction worker monitoring." Visualization in Engineering, No. 1: 1-13. Kelm, A., Laußat, L., Meins-Becker, A., Platz, D., Khazaee, M., Costin, A.,  Helmus, M., and Teizer, J. (2013). "Mobile passive Radio Frequency Identification (RFID) portal for automated and rapid control of Personal Protective Equipment (PPE) on construction sites." Automation in Construction, 36: 38-52. 138-8 Liaw, Y., Lin, K., Li, M., and Chi, N. (2012) Learning Assessment Strategies for an Educational Construction Safety Video Game. Construction Research Congress 2012: pp. 2091-2100.  Mcfarlane, N. and Schofield, C. (1995) “Segmentation and Tracking of Piglets in Images” Machine Vision and Applications, 8(3): 187-193. OSHA (Occupational Safety & Health Administration) (2006) "Module 13 - Personal Protective Equipment." , <https://www.osha.gov/dte/> (Sept. 06, 2014) OSHA (Occupational Safety & Health Administration) (2012) “OSHA Law & Regulations.”<https://www.osha.gov/law-regs.html>  (February 11, 2014) Qubec. (2014). " Safety Code for the construction industry - An Act respecting occupational health and safety." Éditeur officiel du Québec <http://www2.publicationsduquebec.gouv.qc.ca/dynamicSearch/telecharge.php?type=3&file=/S_2_1/S2_1R4_A.HTM> (February 26, 2014). Ruff, T. (2007). "Recommendations for evaluating and implementing proximity warning systems on surface mining equipment, Report of Investigations 9672, National Institute for Occupational Safety and Health. Steele, J., Derunner, C., and Whitehorn, M., (2003). “Stereo images for object detection in surface mine safety applications.”  Western Mining Resource Center Tech Report No. TR20030109. Teizer, J., Allread, B., Fullerton, C., and Hinze, J. (2010). "Autonomous pro-active real-time construction worker and equipment operator proximity safety alert system." Automation in Construction, 19: 630-640.  138-9  5th International/11th Construction Specialty Conference 5e International/11e Conférence spécialisée sur la construction    Vancouver, British Columbia June 8 to June 10, 2015 / 8 juin au 10 juin 2015   AUTOMATED MONITORING OF HARDHATS WEARING FOR ONSITE SAFETY ENHANCEMENT Zhenhua Zhu1, Man-Woo Park2 and Nehad Elsafty1, 1 Department of Building, Civil and Environmental Engineering, Concordia University, Canada 2 Department of Civil and Environmental Engineering, Myongji University, South Korea Abstract: Construction is one of the most dangerous job sectors over the world. Any accidents that happen on the construction sites will bring the sufferings to the workers and their families and incur the delays and costs to the projects. Therefore, it is necessary for the contractors to monitor potential site safety issued and comply with existing safety regulations all the time. One of fundamental safety regulations is hardhat wearing. The wearing of the hardhats is always mandated and should not be violated anytime on the construction sites. In this paper, a novel method is proposed to facilitate the monitoring of whether any persons on the construction sites are wearing hardhats as required by the safety regulations. The method is built upon computer vision techniques. Under the method, human bodies and hardhats are first detected in the video frames captured by real-time on-site construction cameras. Then, their pair-wise matching is found. For those persons without the matching of the hardhats, they are identified as not wearing hardhats. The proposed method has been tested on real site videos. The test results showed that multiple persons could be monitored even if they are not wearing any real-time location sensors or tags. The test results demonstrate the potential of using live streaming or time-lapse construction site videos to facilitate the safety monitoring work on construction sites. 1 INTRODUCTION Construction has been identified as one of the most dangerous job sectors. CBC News (2012) once reported that a total of 700 deaths occurred in the construction industry from 2008 to 2010, which almost accounted for 23.3% of all workplace fatalities. Also, the Association of Workers’ Compensation Boards of Canada (AWCBC) mentioned that there were more than 27,000 time-loss injuries in the construction industry each year from 2009 to 2010 (AWCBC, 2011). In order to provide a healthy and safe working environment for construction workers, several onsite safety regulations have been established by the governments. They specify the contractors' responsibilities and duties on the construction sites, including  the appropriate use of appropriate personal protective equipment (PPE). The contractors must ensure that the regulations are enforced. In Quebec, this job is assigned to onsite safety inspectors, called construction site health and safety management guarantors. The guarantors need to take every necessary measure to ensure the general contractors comply with the safety regulations.  Hardhats are one type of important PPE. Any individual person must wear a hardhat, when they enter to a construction site. For example, the Safety Code for the Construction Industry mandates that "any person on a construction site shall wear a certified safety hat in accordance with CSA Standard" (Quebec, 2014). The similar guideline or regulation could also be found in the OSHA (Occupational Safety & Health Administration). It stipulates that “Employees working in areas where there is a possible danger of head 138-1 injury from impact, or from falling or flying objects, or from electrical shock and burns, shall be protected by protective helmets” (OSHA, 2012). Therefore, it is one of top priorities for the safety inspectors to check and confirm all employees and site visitors wear hardhats all the time on the construction sites. However, the statistic data indicated that that most workers who suffered impact injuries to the head were not wearing hardhats, even when performing their normal jobs on their regular worksites. The Bureau of Labor Statistics once noted that "hardhats were worn by only 16% of those workers who sustained head injuries, although two-fifths were required to wear them for certain tasks at specific locations" (OSHA, 2006). The reasons for not wearing hardhats could be various, such as discomfort while wearing hardhats, disassociation with the safety issues, etc. For example, workers might take off their hardhats to cool off, when the temperature is high. Therefore, it is necessary for the safety inspectors to monitor all persons on the site to wearing hardhats all the time. In order to facilitate the inspectors' monitoring work, this paper proposed a novel method for checking whether persons on the construction sites are wearing hardhats or not. The proposed method is built upon the live streaming or time-lapse construction site videos with computer vision techniques. It mainly include the detection of human bodies and the detection of hardhats. When the human bodies and hardhats are detected, their pair-wise matching is performed, so that those persons without wearing hardhats could be identified.  The effectiveness of the proposed method has been tested with real site videos. The test results showed that the method could monitor the wearing of hardhats on multiple persons simultaneously. These persons do not have to be tagged. This nature makes the method affordable to be used at most construction sites. The proposed method is expected to facilitate the safety inspectors' work and enhance construction site safety. The site safety enhancement could bring several benefits to the contractors, including increasing workers' productivity and reducing project direct and indirect costs due to the cost saving from preventing construction accidents.  2 RELATED WORK 2.1 RTLS-based Site Safety Enhancement The enhancement of onsite construction safety has been increasingly received attentions of researchers and industrial practitioners. Several research studies have been initiated for the purpose of adding an extra level of proactive safety measures using real-time location systems (RTLS)  These RTLS typically relied on the remote sensing techniques, such as radio frequency identification (RFID), global positioning system (GPS), wireless local areas networks (WLAN), and ultra-wide band (UWB).  Most RTLS have been used for proximity warning. Ruff (2007) compared different RTLS in his report for evaluating and implementing proximity warning systems on surface mining equipment. Moreover, Teizer et al. (2010) presented their findings of using Very-High Frequency (VHF) active Radio Frequency (RF) technique to warn ground workers and equipment operators once they are too close in proximity. In the work of Carbonari et al. (2011), they relied on UWB tracking data to implement a proactive virtual fencing system, and demonstrated its capability of reinforcing safety management policies. Also, Chen and Teizer (2013) integrated the real-time GPS and UWB location data into a virtual reality environment representing a construction site for monitoring construction safety.  In addition to the use of RTLS for the proximity warning, the systems could also be adopted for other purposes related to construction safety enhancement. Kelm et al. (2013) designed an RFID-based portal to check whether the workers' personal protective equipment (PPE) complied with the corresponding specifications. Aguilara and Hewageb (2013) developed an IT based safety management system using web cameras, barcodes and RFID tags installed on construction equipment. However, existing RTLS come with limitations, when they are used for monitoring the wearing of hardhats on construction workers. First, in order to check whether construction workers are wearing hardhats, it is at least necessary to attach a tag or sensor on each person and hardhat. In addition, the RTLS are mainly 138-2 used for real-time location tracking. Therefore, it is difficult to use the RTLS to identify whether the hardhats are appropriately used, considering that workers might take off their hardhats and put them aside while doing their jobs.  2.2 Video-based Site Safety Enhancement Live streaming or time-lapse video cameras are increasingly placed on construction sites, considering their acceptable return on investment (Bohn and Teizer, 2010). For example, Hydro-Qubec has recently decided to install video cameras on their construction sites to facilitate the monitoring of daily construction activities.The live streaming or time-lapse videos collected by the cameras record project ongoing construction activities on the construction sites. Therefore, they are useful for monitoring construction site safety issues. Abeid and Arditi (2002) relied on live streaming or time-lapse videos collected on construction sites to investigate construction accidents. Liaw et al. (2012) used the live stream or time-lapse videos to create safety training and education media resources.  So far, several research studies have been initiated to investigate the potentials of live streaming or time-lapse videos to create onsite safety alert systems built upon computing techniques. For example, Steele et al. (2003) once proposed to mount a stereo camera on the rear of an off-highway dump truck, and the stereo videos could help the truck driver identify possible on-site obstacles on the mining site. In addition to mounting stereo cameras on the equipment, the cameras are also placed on the site to capture and analyze workers' unsafe actions, considering these actions might result in accidents. Han and Lee (2013) and Han et al. (2013) relied on Kinect-style cameras to monitor whether construction workers on a ladder are leaning too far to one side or reaching too far overhead to prevent the falls from the ladders.  Similar to the work presented in this paper, Giovanni et al. (2009, 2011) designed a vision-based system to support the workers' safety by checking the presence of those workers without wearing hardhats on construction sites. In their work, they first used a pedestrian classifier with covariance descriptors to recognize construction workers in video frames, and then used the head and hard-hat detectors. However, there is one major limitation in their system. The system could not function correctly, when the workers on construction sites wear white hardhats. The failure of the detection of white hardhats makes the system generate a lot of false alerts. As illustrated in Figure 1,  the safety alert issued by their system (Giovanni et al. 2009, 2011) was false, since the person identified as not wearing a hardhat (red box) was actually wearing a hardhat.  Figure 1: Failure of detecting white hardhats 138-3 4.2 Hardhat Detection Hardhats are typically made of fiberglass and rigid plastic. On the one hand, the colors of the hardhats could be white, brown, green, blue, orange, red, etc. to indicate the roles (e.g. managers, engineers, superintendents, and laborers) of the wearers on construction sites. Considering the colors could vary, it would be difficult to rely on the color information as detection cues. On the other hand, most hardhats have similar cap-style shapes. They are rigid and do not have deformations.  Therefore, hardhat detection here is similar to the detection of human bodies. The HOG features of the hardhats are extracted from the video frames as the detection cue, especially considering that the HOG feature is capable of describing detailed shape information in an efficient way. Moreover, the effectiveness of the HOG features has been well-proven in many research studies for shape-based object detection. The hardhat detection includes two stages. First, hundreds of construction site images with different poses of hardhats are collected as a training database, where the hardhat HOG features are calculated and input into an SVM for the supervised training. The training results are compiled into a hardhat detection model. In order to detect hardhats in a video frame, the HOG features extracted from the test frame are input to the detection model, and the best possible placements of the hardhats could be determined. The detection model here only uses the HOG features without color cues. Therefore, it could detect the hardhats with different colors.  4.3 Human Body and Hardhat Pair-wise Matching After the detection of human bodies and hardhats, the detected hardhats should be matched to the detected human bodies to find out those persons who are not wearing hardhats on the construction sites. The matching here is mainly based on the spatial relationships between the detection regions of the hardhats and human bodies. Typically, the detected hardhat should be located in the upper area of the detected human body region, if the person in the detected human body region is wearing a hardhat. If there is no matching that could be found on that person, it indicates that the person is not wearing a hardhat. Then, a safety alert is produced to ask for the safety inspector's attention. One example of the pair-wise matching between the detected human bodies and hardhats could be found in Figure 3. In the figure, the matches of the hardhats for three persons in the right could be successfully found. As a contrast, the pair-wise matches of the hardhat for the two persons in the left could not be found, since they are not wearing hardhats.   Figure 3: Pair-wise matching between human body and hardhat 5 IMPLEMENTATION AND RESULTS The method was implemented and tested with real site videos captured by a Canon VISXIA HF S100 at a project site in Atlanta. The site was managed by Barton Malow Company. The test videos includes a total 138-5 of five persons. Three of them were wearing hardhats on the site and two were not wearing hardhats for the evaluation purpose. The test video was 166-second long, and recorded with 20 frames per second (fps) containing 3,320 frames. The resolution of the videos is fixed at 768 pixels by 432 pixels. 5.1 Results of Human Body Detection and Hardhats Detection Figure 4 illustrates the results of detecting human bodies and hardhats in the test video frames. Most human body detections made by the method are correct, although the method missed some human body detections (i.e. the persons in the video frames are not detected by the method). The correct detection of human bodies is more important than the missed detection in terms of monitoring whether a person is wearing a hardhat or not on the construction site. If the method could only successfully detect a person every 10 frames, the miss rate is high, but it is still able to monitor whether the person is wearing a hardhat once in a second. On the other hand, if the correct detection rate is low, it increases the chances of false alarms. For example, if the method misrecognizes tree branches as a human body, a false alarm will be issued, since the hardhat could not be detected in the detection region of that 'person'.       Figure 4: The detection of human bodies and hardhats Compared with the human body detection, most hardhats in the test video frames could be detected by the proposed method. However not all the detections are correct. The strategy for the hardhat detection is to reduce the detection miss rate. The low detection miss rate is more important than the correct hardhat detection rate in terms of monitoring whether a person is wearing a hardhat or not on the construction site. In other words, most false hardhat detections do not significantly affect the performance of issuing the final safety alerts, when persons are not wearing hardhats. The reason is mainly due to difficulty in establishing the links of the incorrect hardhat detection results to the detection of any human bodies.  5.2 Performance of Safety Alerts for not Wearing Hardhats When there is no hard detected in a human body detection region, it indicated that the individual person in the human body detection region is not wearing a hardhat. Therefore, a safety alert should be issued o highlight the person for the onsite safety inspector's attention. In order to measure the performance of the safety alerts issued by the proposed method, the issued safety alerts are manually classified as true positive (TP) alerts, false positive (FP) alerts, and false negative (FN) alerts. The TP alerts are the alerts that should be issued in reality. The FP alerts are the alerts that do not have to be issued in reality. The FN alerts are the alerts that should be issued in reality but the alert is not issued by the method. Their definitions have been illustrated in Table 1. Based on the number of TP, FP, and FN safety alerts, the precision and recall for issuing the safety alerts could be calculated using the following equations. [1] Precision = TP / (TP + FP)           [2] Recall = TP / (TP + FN)                138-6 Table 1: Definition of TP, FP, and FN safety alerts Category Issuing the alert based on the method Issuing the alert based on the safety regulation TP YES YES FP YES NO FN NO YES According to the test results, it was found that the precision for issuing the safety alerts reached 94.3%, while the recall for issuing the safety alerts was 89.4%. The 94.3% precision means that 5.7% of the alerts issued by the proposed method are false. The 89.4% recall indicates that nine out of ten individual persons without wearing hardhats could be successfully monitored when applying the proposed method on construction sites. Both high alert precision and recall show the effectiveness of the proposed method on monitoring whether individual persons on the construction site are wearing hardhats as required by existing safety regulations, such as the Safety Code for the Construction Industry and the Safety and Health Regulations for Construction.  6 LIMITATIONS AND DISCUSSION There are several limitations found from the tests. Most limitations are more or less associated with the detection of human bodies on a construction site. First, the human body detection is limited to the detection of the persons with standing or walking postures on the construction sites. So far, it is difficult to detect the persons with other postures. For example, when the workers are crouching down, bending, or sitting, it is difficult for the method to detect them. Therefore, the detection template used by the method should be extended to include more training samples of the workers with different postures.  Second, the occlusion is one of the major obstacles that could affect the performance of the proposed method on monitoring the wearing of the hardhats. If the persons are partially or fully occluded on the construction site, the monitoring with the proposed method will fail. The proposed method could only monitor the workers when they are visible in the camera view. Therefore, it is important to install an appropriate  number of construction cameras and select the appropriate positions for installing the cameras on a construction site.  In addition, the proposed method relied on the background subtraction to speed up the detection of human bodies. The purpose of the background subtraction is to limit the detection of human bodies only in the foreground blobs in motion. This means only moving persons on the construction site could be detected. If they are static without any motions, the workers are filtered out through the background subtraction, since they are considered as the background. Considering the workers could always be detected when they just enter into the camera view, the integration of the detection and tracking of onsite persons might provide one possible solution to continuously monitor the persons even if they are static. 7 CONCLUSIONS AND FUTURE WORK Existing safety policies and regulations could enhance construction site safety; however, these policies and regulations might not be strictly followed on the construction site. For example, construction workers might slip up and not exactly keep the requirements in the policies and regulations due to fatigue, distractions, carelessness, etc. Therefore, it is necessary to monitor any violations of the safety policies and regulations on construction sites. Currently, this safety monitoring job is manually performed by onsite safety inspectors (e.g. construction site health and safety management guarantors in Quebec). It might be difficult for the inspectors to monitor all safety policy and regulation violations on the construction site at any time through manual monitoring. In order to facilitate the safety inspectors' monitoring work, this paper proposed a novel, vision-based method for automatically monitoring whether any individual persons wear hardhats on the construction site as required by existing construction site safety polices and regulations. The method includes three 138-7 main steps: 1) human body detection, 2) hardhat detection, and 3) human body and hardhat pair-wise matching. If there is a human body detection area without the corresponding hardhat detection matched, it means that the person in the area is not wearing the hardhat. Therefore, a safety alert will be issued.  The method has been tested with the video frames captured at a real site. According to the test results, it was found that the overall precision and recall for issuing the safety alerts with the proposed method could reach 94.3% and 89.4%. The high precision and recall indicate the effectiveness of the proposed method on monitoring whether individual persons are wearing hardhats on the construction site. Future work will focus on the following two areas. First, the monitoring of wearing other types of personal protective equipment (safety vests, boots, etc.) on the construction sites will be investigated using vision technologies. Also, more investigations regarding the detection of human bodies with different postures will be performed in the future.  Acknowledgements This paper is based in part upon work supported by the National Science and Engineering Research Council (NSERC) of Canada. Any opinions, findings, and conclusions or recommendations expressed in this paper are those of the author(s) and do not necessarily reflect the views of the NSERC. References Abeid, J. and Arditi, D. (2002). ”Time-Lapse Digital Photography Applied to Project Management.” Journal of Construction Engineering and Management, 128(6), 530–535.  Aguilar, G., and Hewage, K. (2013). "IT based system for construction safety management and monitoring: C-RTICS" Automation in Construction, 35: 217-228. Association of Workers' Compensation Boards of Canada (AWCBC), (2011)."Summary Tables of Accepted Time-loss Injuries/Diseases and Fatalities by Jurisdiction."<http://www.awcbc.org/en/statistics.asp> (August 10, 2013) Bohn, J. and Teizer, J. (2010). "Benefits and Barriers of Construction Project Monitoring Using High-Resolution Automated Cameras." J. Constr. Eng. Manage, 136(6), 632–640. Carbonari, A. and Giretti, A. and Naticchia, B. (2011). "A proactive system for real-time safety management in construction sites." Automation in Construction, 20(6): 686-698. Cheng, T., and Teizer, J. (2013). "Real-time resource location data collection and visualization technology for construction safety and activity monitoring applications." Automation in Construction, 34: 3-15. Dalal, N. and Triggs, B. (2005)."Histograms of Oriented Gradients for Human Detection." In: IEEE Computer Society Conference on Computer Vision and Pattern Recognition, San Diego, CA. Vol. 1, pp. 886-893.  Felzenszwalb, P., Girshick, R., McAllester, D., and Ramanan, D. (2010). "Object Detection with Discriminatively Trained Part Based Models." IEEE Transactions on Pattern Analysis and Machine Intelligence, 32(9): 1627--1645. Gualdi, G., Prati, A., and Cucchiara, R. (2011). "Contextual information and covariance descriptors for people surveillance: an application for safety of construction workers." Journal on Image and Video Processing 2011, Article No. 9, pp. 1 - 16. Gualdi, G., Prati, A., and Cucchiara, R.  (2009). "Covariance descriptors on moving regions for human detection in very complex outdoor scenes." In Distributed Smart Cameras, 2009. ICDSC 2009. Third ACM/IEEE International Conference on, pp. 1-8. IEEE, 2009. Han, S., and Lee, S. (2013) "A vision-based motion capture and recognition framework for behavior-based safety management." Automation in Construction, 35: 131-141. Han, S., Achar, M., Lee, S. and Peña-Mora, F. (2013) "Empirical assessment of a RGB-D sensor on motion capture and action recognition for construction worker monitoring." Visualization in Engineering, No. 1: 1-13. Kelm, A., Laußat, L., Meins-Becker, A., Platz, D., Khazaee, M., Costin, A.,  Helmus, M., and Teizer, J. (2013). "Mobile passive Radio Frequency Identification (RFID) portal for automated and rapid control of Personal Protective Equipment (PPE) on construction sites." Automation in Construction, 36: 38-52. 138-8 Liaw, Y., Lin, K., Li, M., and Chi, N. (2012) Learning Assessment Strategies for an Educational Construction Safety Video Game. Construction Research Congress 2012: pp. 2091-2100.  Mcfarlane, N. and Schofield, C. (1995) “Segmentation and Tracking of Piglets in Images” Machine Vision and Applications, 8(3): 187-193. OSHA (Occupational Safety & Health Administration) (2006) "Module 13 - Personal Protective Equipment." , <https://www.osha.gov/dte/> (Sept. 06, 2014) OSHA (Occupational Safety & Health Administration) (2012) “OSHA Law & Regulations.”<https://www.osha.gov/law-regs.html>  (February 11, 2014) Qubec. (2014). " Safety Code for the construction industry - An Act respecting occupational health and safety." Éditeur officiel du Québec <http://www2.publicationsduquebec.gouv.qc.ca/dynamicSearch/telecharge.php?type=3&file=/S_2_1/S2_1R4_A.HTM> (February 26, 2014). Ruff, T. (2007). "Recommendations for evaluating and implementing proximity warning systems on surface mining equipment, Report of Investigations 9672, National Institute for Occupational Safety and Health. Steele, J., Derunner, C., and Whitehorn, M., (2003). “Stereo images for object detection in surface mine safety applications.”  Western Mining Resource Center Tech Report No. TR20030109. Teizer, J., Allread, B., Fullerton, C., and Hinze, J. (2010). "Autonomous pro-active real-time construction worker and equipment operator proximity safety alert system." Automation in Construction, 19: 630-640.  138-9  Automated Monitoring of Hardhats Wearing for Onsite Safety EnhancementZhenhua Zhu1, ManWoo Park2, and Nehad Elsafty11 Department of Building, Civil, & Environmental EngineeringConcordia University, Montreal, Canada2 Department of Civil and Environmental Engineering, Myongji University, South KoreaMotivation2Motivation The Bureau of Labor Statistics in U.S.: “hardhats were worn by only 16% of those workers who sustained head injuries, although two-fifths were required to wear them for certain tasks at specific locations.” [1] Reasons [2] Discomfort while wearing hardhats Disassociation with the safety issues … … Need to monitor the wearing of hardhats onsite3Current Practice4Related Research Work Sensor based alerts for PPE wearing E.g. RFID-based tags and portal [3] Limitation Checking at the portal onlyPPE wearing control [3]5Related Research Work Vision based alerts for PPE wearing No need to attach tags RGB-D cameras [4] Indoor, small-scale space (5m x 5m x 3m) Video cameras [5] Failure to detect white hardhatsHardhat classifier [4]Incorrectly issuing the alert [5]6 To create an automatic method to detect whether the persons are wearing hardhats on construction sites Open, large-scale construction sites Hardhats with different colorsObjective7Proposed MethodologySite videosHardhat detectionHuman body detectionHardhatsHuman bodiesPair-wise matchingSafety alerts8Human Body/Hardhat DetectionDilation ErosionRectangle fittingDetection using histogram of oriented gradients (HOG) features9Pair-Wise MatchingRegion (1) Region (2) 10 Implementation Microsoft Visual C# .NET Framework 4.0 environment Test: 3320 video frames Evaluation criteria True positive (TP), false positive (FP), and false negative (FN)Implementation and Results11 Human body detection Hardhat detection Safety alertsTP, FP and FN DefinitionsCategory of safety alertSafety alert is issued by the methodSafety alert should be issued in realityTP Yes Yes FP Yes No FN No Yes12Metric Human body                detectionHardhat detectionSituation 1 Situation 2TP 3026 2247 2984FP 61 9 1893FN 291 838 100Precision (%) 98.0 99.6 61.2Recall (%) 91.2 72.8 96.8ResultsMetricSafety alert from the proposed methodSituation 1 Situation 2Precision (%) 53.6 94.3Recall (%) 87.7 89.413Video Demo14Conclusion and Future Work  An automatic vision-based method to detect whether  onsite people are wearing hardhats High precision and recall for issuing safety alerts  Built upon the high recall for hardhat detection  Limitation Background subtraction  moving workers  Other site safety violations with video cameras15Thank you!Questions?16[1] OSHA (Occupational Safety & Health Administration) (2006) "Module 13 - Personal Protective Equipment." , <https://www.osha.gov/dte/> (Sept. 06, 2014)[2] GIS (General Insurance Services) “BLS Says Construction Workers Not Wear ing Hard Hats Despite Obvious Dangers”, <http://www.genins.com/img/~www.genins.com/commercial%20lines%20articles/bls%20says%20construction%20workers%20not%20wearing%20hard%20hats%20despite%20obvious%20dangers.pdf> (Sept. 05, 2014)[3] Kelm, A., Laußat, L., Meins-Becker, A., Platz, D., Khazaee, M., Costin, A.,  Helmus, M., and Teizer, J. (2013). "Mobile passive Radio Frequency Identification (RFID) portal for automated and rapid control of Personal Protective Equipment (PPE) on construction sites." Automation in Construction 36 (2013): 38-52.[4] Gualdi, G., Prati, A., and Cucchiara, R. (2011). "Contextual information and covariance descriptors for people surveillance: an application for safety of construction workers." Journal on Image and Video Processing 2011 (2011), Article No. 9, pp. 1 - 16.[5] Weerasinghe, I. (2013). "Automated Construction Worker Performance and Tool-time Measuring Model Using RGB Depth Camera and Audio Microphone Array System", PhD Thesis, University of Calgary.References17

Cite

Citation Scheme:

        

Citations by CSL (citeproc-js)

Usage Statistics

Share

Embed

Customize your widget with the following options, then copy and paste the code below into the HTML of your page to embed this item in your website.
                        
                            <div id="ubcOpenCollectionsWidgetDisplay">
                            <script id="ubcOpenCollectionsWidget"
                            src="{[{embed.src}]}"
                            data-item="{[{embed.item}]}"
                            data-collection="{[{embed.collection}]}"
                            data-metadata="{[{embed.showMetadata}]}"
                            data-width="{[{embed.width}]}"
                            async >
                            </script>
                            </div>
                        
                    
IIIF logo Our image viewer uses the IIIF 2.0 standard. To load this item in other compatible viewers, use this url:
http://iiif.library.ubc.ca/presentation/dsp.52660.1-0076342/manifest

Comment

Related Items