Medicine, Faculty of
Non UBC
Pathology and Laboratory Medicine, Department of
DSpace
Bioengineering 11 (7): 709 (2024)
International Collaboration on Repair Discoveries
Park, Jinho
Nguyen, Thien
Park, Soongho
Hill, Brian
Shadgan, Babak
Gandjbakhche, Amir
A two-stream convolutional neural network (TCNN) for breathing pattern classification has been devised for the continuous monitoring of patients with infectious respiratory diseases. The TCNN consists of a convolutional neural network (CNN)-based autoencoder and classifier. The encoder of the autoencoder generates deep compressed feature maps, which contain the most important information constituting data. These maps are concatenated with feature maps generated by the classifier to classify breathing patterns. The TCNN, single-stream CNN (SCNN), and state-of-the-art classification models were applied to classify four breathing patterns: normal, slow, rapid, and breath holding. The input data consisted of chest tissue hemodynamic responses measured using a wearable near-infrared spectroscopy device on 14 healthy adult participants. Among the classification models evaluated, random forest had the lowest classification accuracy at 88.49%, while the TCNN achieved the highest classification accuracy at 94.63%. In addition, the proposed TCNN performed 2.6% better in terms of classification accuracy than an SCNN (without an autoencoder). Moreover, the TCNN mitigates the issue of declining learning performance with increasing network depth, as observed in the SCNN model. These results prove the robustness of the TCNN in classifying breathing patterns despite using a significantly smaller number of parameters and computations compared to state-of-the-art classification models.
Citation: Park, J.; Nguyen, T.; Park, S.;Hill, B.; Shadgan, B.; Gandjbakhche, A.Two-Stream Convolutional NeuralNetworks for Breathing PatternClassification: Real-Time Monitoringof Respiratory Disease Patients.Bioengineering 2024, 11, 709.https://doi.org/10.3390/bioengineering11070709Academic Editor: Luca MesinReceived: 29 May 2024Revised: 26 June 2024Accepted: 11 July 2024Published: 12 July 2024Copyright: © 2024 by the authors.Licensee MDPI, Basel, Switzerland.This article is an open access articledistributed under the terms andconditions of the Creative CommonsAttribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).bioengineeringArticleTwo-Stream Convolutional Neural Networks for BreathingPattern Classification: Real-Time Monitoring of RespiratoryDisease PatientsJinho Park 1 , Thien Nguyen 1, Soongho Park 1,2, Brian Hill 1, Babak Shadgan 3,4 and Amir Gandjbakhche 1,*1 Eunice Kennedy Shriver National Institute of Child Health and Human Development, National Institutesof Health, 49 Convent Dr., Bethesda, MD 20894, USA; jinho.park@nih.gov (J.P.); thien.nguyen4@nih.gov (T.N.);soongho.park@nih.gov (S.P.); brian.hill@nih.gov (B.H.)2 National Heart, Lung and Blood Institute, National Institutes of Health, 10 Center Dr.,Bethesda, MD 20892, USA3 Implantable Biosensing Laboratory, International Collaboration on Repair Discoveries,Vancouver, BC V5Z 1M9, Canada; babak.shadgan@ubc.ca4 Department of Pathology & Laboratory Medicine, University of British Columbia,Vancouver, BC V6T 1Z7, Canada* Correspondence: gandjbaa@mail.nih.govAbstract: A two-stream convolutional neural network (TCNN) for breathing pattern classificationhas been devised for the continuous monitoring of patients with infectious respiratory diseases.The TCNN consists of a convolutional neural network (CNN)-based autoencoder and classifier.The encoder of the autoencoder generates deep compressed feature maps, which contain the mostimportant information constituting data. These maps are concatenated with feature maps generatedby the classifier to classify breathing patterns. The TCNN, single-stream CNN (SCNN), and state-of-the-art classification models were applied to classify four breathing patterns: normal, slow, rapid,and breath holding. The input data consisted of chest tissue hemodynamic responses measuredusing a wearable near-infrared spectroscopy device on 14 healthy adult participants. Among theclassification models evaluated, random forest had the lowest classification accuracy at 88.49%, whilethe TCNN achieved the highest classification accuracy at 94.63%. In addition, the proposed TCNNperformed 2.6% better in terms of classification accuracy than an SCNN (without an autoencoder).Moreover, the TCNN mitigates the issue of declining learning performance with increasing networkdepth, as observed in the SCNN model. These results prove the robustness of the TCNN in classifyingbreathing patterns despite using a significantly smaller number of parameters and computationscompared to state-of-the-art classification models.Keywords: COVID-19; deep learning; convolutional neural network; respiratory disease; NIRS;wearable device1. IntroductionIn 2019, the respiratory illness Coronavirus Disease 2019 (COVID-19) caused by SARS-CoV-2 started spreading globally at an unprecedented rate. The virus has spread to over228 countries and, as of May 2024, has affected approximately 775 million people, resultingin over 7.05 million deaths [1]. Common symptoms of COVID-19 include cough, breathingcomplications, wheezing, and tachypnea, which often accompany respiratory distress,fever, headache, fatigue, hypoxia, and lethargy [2–6]. Various methods have been used tomonitor infected patients, such as CT scans [7–9], X-rays [7,10], thermal cameras [11], depthcameras [4], ultrasound technology [12], and radar [13–17]. CT scans and X-ray imagesprovide high precision and resolutions, but they are expensive and expose the patient toharmful radiation. Ultrasound technology is noninvasive and harmless but has limitedsensitivity. Changes in chest displacement can be observed using depth cameras and radar.Bioengineering 2024, 11, 709. https://doi.org/10.3390/bioengineering11070709 https://www.mdpi.com/journal/bioengineeringBioengineering 2024, 11, 709 2 of 16However, these methods are constrained by a person’s movement and position making itdifficult to observe the displacement accurately. To overcome these issues, various wearablemedical devices are being developed that can monitor patients in real time [18]. Amongthese, near-infrared spectroscopy (NIRS) is gaining attention for its simplicity, low cost,and ability to noninvasively monitor tissue hemodynamics [19].Common symptoms experienced by patients with respiratory infectious diseasesinclude difficulties in breathing and changes in respiration rate, which are either rapidand shallow (tachypnea) or slow and deep [2,4,20,21]. Therefore, the respiration rateis an important parameter to be monitored [20–24], and various sensors and deviceshave been developed to measure human respiration rates and patterns. In particular,flexible and wearable sensors have garnered significant interest in terms of respirationmonitoring [25–27]. Many of these sensors also measure the peripheral arterial bloodoxygen saturation (SpO2), which has been used as a key marker for the monitoring ofthe progression of respiratory diseases [28]. Using these sensors to monitor SpO2 canbe convenient but may result in inaccurate readings due to factors like body movement,reduced blood circulation, the skin thickness, and the skin color [29]. While the SpO2levels are a measure of the systemic arterial blood oxygen saturation, NIRS devices canmeasure the more localized tissue oxygen saturation (StO2) within the microvasculatureof tissue [30], which represents an advantage of NIRS devices over pulse oximeters [31].Studies have shown that NIRS devices can detect the effects of hypoxia better than standardpulse oximeters [32]. Therefore, in this study, we use an NIRS device to evaluate thehemodynamic responses and monitor the respiratory patterns in patients with infectiousrespiratory diseases.Infectious respiratory diseases are often transmitted through droplets, person-to-person contact, or vectors [3,33–35]. Therefore, diagnosis through direct contact withinfected individuals increases the risk of infection for both contacts and healthcare work-ers [36]. To minimize the occurrence and prevent the spread of highly contagious respiratorydiseases from infected individuals to those around them, it is crucial to detect infectionsearly, enforce self-isolation, and ensure appropriate treatment for a swift recovery. Toachieve this, non-contact monitoring methods that can check the patient’s condition inreal time without direct contact are necessary [37–39]. An Internet of Things (IoT)-basedmonitoring system serves as a prime example. Shafi et al. proposed a system utilizingwearable devices to measure a patient’s temperature, heart rate, and oxygen saturation,transmitting and storing these data in an IoT cloud. Medical staff can access the clouddata to monitor the patient’s health status without direct contact [40]. While this systemeffectively monitors patients’ conditions without direct contact, clinical decisions must bemade by medical staff. However, medical decision-making poses challenges, as healthcareproviders face time and physical limitations in diagnosing extensive patient data. Therefore,an intelligent method for the automatic detection of abnormalities from monitored signalsis needed.Deep learning and machine learning algorithms have been applied to various medicalfields and clinical decisions, including DNA classification, brain tumor diagnosis, eyediseases, and malaria diagnosis [41,42]. Commonly used algorithms include support vectormachines (SVM), extreme gradient boosting (XGBoost), artificial neural networks, andconvolutional neural networks (CNN) [43]. For example, Cho et al. utilized a thermalcamera to monitor breathing patterns by analyzing temperature fluctuations around thenose regions of individuals. Subsequently, a CNN-based algorithm was employed toautomatically categorize psychological stress levels [11]. Shah et al. introduced a techniqueto detect distinctive patterns associated with COVID-19 from CT scan images utilizinga deep learning algorithm [44]. Hong et al. proposed a method that utilizes mm-wavesensors to measure the displacement of the patient’s chest height and extracts a featurevector using CNN algorithms. The extracted feature vector is compared with pre-learnedfeature vectors stored in a database to classify it according to the most similar featurevector [45]. Kim et al. developed an embroidered textile sensor that measures electricalBioengineering 2024, 11, 709 3 of 16impedance changes depending on the intensity of compression and chest movement. Themeasured electrical impedance changes were classified into respiratory patterns using a1D-CNN. [46]. Conway et al. used capnography to determine ‘breath’ versus ‘no breath’using a CNN based on changes in the CO2 concentration in a patient’s breathing [47].These studies have shown encouraging results in classifying individuals suffering fromrespiratory diseases. However, long-term data collection might cause discomfort to patients,making it unsuitable for continuous monitoring.In our previous study, we used an NIRS device to measure chest tissue hemodynamicresponses and classified simulated breathing patterns using the oxyhemoglobin (O2Hb)signal [48]. The simulated breathing patterns included rest, rapid/shallow, and loadedbreathing. From the O2Hb signals, features including the amplitude, depth, and intervalwere extracted, and the breathing patterns were classified using the random forest clas-sification method, achieving accuracy of 87%. However, this method is unsuitable forreal-time monitoring due to the need for post-processing and feature extraction. Therefore,in another study, we developed a deep learning algorithm based on a 1D-CNN to classifythe three breathing patterns, which resulted in classification accuracy of 91.28% [49]. The1D-CNN model is composed of a single-stream structure. Hence, it faces challenges inparameter learning and optimization as the number of layers increases. As a result, itsperformance may not improve or may even deteriorate beyond a certain number of layerparameters [50–52].To overcome the issues associated with a single-stream CNN (SCNN), this studyproposes a two-stream CNN (TCNN) for breathing pattern classification. The TCNNconsists of an autoencoder for the generation of deep compressed feature maps frombreathing signals and a classifier module for classification. The deep compressed featuremaps are concatenated with feature maps generated by the CNN-based classifier module toultimately assist in classifying breathing patterns. We hypothesize that the combination ofdeep compressed feature maps generated by the autoencoder and feature maps created bythe CNN-based classifier module will enhance the classification performance and overcomethe parameter learning and optimization issues observed in an SCNN.2. Materials and Methods2.1. NIRS DeviceThe NIRS device consists of one light source with LEDs emitting light at three wave-lengths (730 nm, 800 nm, and 850 nm) and two photodetectors, located at 3 cm and 4 cmfrom the light source (prototype provided by Hamamatsu Photonics K.K., Shizuoka, Japan)(Figure 1). The light source emits a constant amount of light into the tissue, which under-goes absorption and scattering. The backscattered light is detected by the photodetectors,which is then used to calculate changes in O2Hb, deoxyhemoglobin (HHb), and totalhemoglobin (THb) at each distance. The absolute tissue oxygen saturation (StO2) is alsomeasured. The NIRS device was connected to a mobile phone (Pixel XL) for power supplyand data recording. A mobile application on the phone converted the raw data measuredby the NIRS device into hemodynamic responses, which included two sets of O2Hb, HHb,and THb from the two photodetector separations and one set of StO2. The data werecollected at a sampling rate of 120 Hz, providing high-resolution data.2.2. Breathing Class Definition and Data CollectionTo facilitate the non-contact monitoring and classification of breathing patterns forthe early detection and monitoring of respiratory disease patients, we define 4 classesof breathing patterns. Patients with respiratory diseases exhibit breathing rates that arefaster or slower than those of healthy individuals [2,4,20,21]. In this study, we defined thefollowing four breathing patterns.• Class 1—normal breathing: Participants breathed normally through their mouth ornose while in a relaxed position (Figure 2a).Bioengineering 2024, 11, 709 4 of 16• Class 2—breath holding: Participants exhaled and held their breath for as long as theycould (Figure 2b).• Class 3—slow breathing: Participants breathed at a rate of 10 times per minute for twominutes (Figure 2c).• Class 4—rapid breathing: Participants breathed at a rate of 30 times per minute fortwo minutes (Figure 2d).Bioengineering 2024, 11, x FOR PEER REVIEW 4 of 16 Figure 1. Data recording system and NIRS device. (a) The signals measured by the NIRS device are converted into O2Hb, HHb, THb, and StO2 in real time and recorded on the connected mobile phone. (b) The NIRS device consists of one light source and two photodetectors. 2.2. Breathing Class Definition and Data Collection To facilitate the non-contact monitoring and classification of breathing patterns for the early detection and monitoring of respiratory disease patients, we define 4 classes of breathing patterns. Patients with respiratory diseases exhibit breathing rates that are faster or slower than those of healthy individuals [2,4,20,21]. In this study, we defined the fol-lowing four breathing patterns. • Class 1―normal breathing: Participants breathed normally through their mouth or nose while in a relaxed position (Figure 2a). • Class 2―breath holding: Participants exhaled and held their breath for as long as they could (Figure 2b). • Class 3―slow breathing: Participants breathed at a rate of 10 times per minute for two minutes (Figure 2c). • Class 4―rapid breathing: Participants breathed at a rate of 30 times per minute for two minutes (Figure 2d). The experiment for data collection was conducted at an outpatient clinic at the Na-tional Institutes of Health (NIH) Clinical Center. During data collection, the NIRS device was attached to the participants’ chests using medical-grade, double-layer tape. To ac-quire data on normal breathing, each participant comfortably sat in a chair in a resting position and breathed normally through their mouth or nose for 5 min. The participants then performed a breath holding task where they were asked to exhale and hold their breath for as long as possible, up to two minutes. After resting for two minutes, the task was repeated twice for a total of three repetitions. The participants then rested for five minutes before starting the slow breathing task. During slow breathing, participants took ten breaths per minute for two minutes, rested for two minutes, and then repeated. The slow breathing task ended with a two-minute rest followed by the rapid breathing task, where the participants breathed thirty times per minute for two minutes. This task was also repeated following a two-minute rest period. Figure 2 shows the breathing signals for each class obtained through the NIRS device. Figure 1. Data recording system and NIRS device. (a) The signals measured by the NIRS device areconverted into O2Hb, HHb, THb, and StO2 in real time and recorded on the connected mobile phone.(b) The NIRS device consists of one light source and two photodetectors.Bioengineering 2024, 11, x FOR PEER REVIEW 5 of 16 Figure 2. Breathing signal in the time domain: (a) normal breathing; (b) breath holding; (c) slow breathing; (d) rapid breathing. 2.3. Participants Twenty-six healthy adult participants (mean age = 34.7 ± 15.7 years old, 18 females) were enrolled in the study. The study enrollment and procedure followed the protocol of a clinical trial and was approved by the Institutional Review Board of the Eunice Kennedy Shriver National Institute of Child Health and Human Development (NICHD protocol #21CH0028) on 31 August 2021. Data from 12 participants were excluded due to motion artifacts and/or low signal-to-noise ratios in one or more tasks. Data from 14 participants who had good signal-to-noise ratios for all four tasks were retained for further analysis. 2.4. Data Preprocessing The data recorded by the NIRS device included impulse noise caused by voltage tran-sients during measurement. To remove the impulse noise, the first derivative of the signal was calculated, and the mean and standard deviation (STD) of the first derivative values were computed. Any signals exceeding the 90% confidence interval were considered noise and removed. Figure 3 shows the signal acquired by the NIRS device (a) and the signal after impulse noise removal (b). Figure 2. Breathing signal in the ti e do ain: (a) normal breathing; (b) breath holding; (c) slowbreathing; (d) rapid breathing.Bioengineering 2024, 11, 709 5 of 16The experiment for data collection was conducted at an outpatient clinic at the NationalInstitutes of Health (NIH) Clinical Center. During data collection, the NIRS device wasattached to the participants’ chests using medical-grade, double-layer tape. To acquire dataon normal breathing, each participant comfortably sat in a chair in a resting position andbreathed normally through their mouth or nose for 5 min. The participants then performeda breath holding task where they were asked to exhale and hold their breath for as long aspossible, up to two minutes. After resting for two minutes, the task was repeated twicefor a total of three repetitions. The participants then rested for five minutes before startingthe slow breathing task. During slow breathing, participants took ten breaths per minutefor two minutes, rested for two minutes, and then repeated. The slow breathing taskended with a two-minute rest followed by the rapid breathing task, where the participantsbreathed thirty times per minute for two minutes. This task was also repeated followinga two-minute rest period. Figure 2 shows the breathing signals for each class obtainedthrough the NIRS device.2.3. ParticipantsTwenty-six healthy adult participants (mean age = 34.7 ± 15.7 years old, 18 females)were enrolled in the study. The study enrollment and procedure followed the protocol of aclinical trial and was approved by the Institutional Review Board of the Eunice KennedyShriver National Institute of Child Health and Human Development (NICHD protocol#21CH0028) on 31 August 2021. Data from 12 participants were excluded due to motionartifacts and/or low signal-to-noise ratios in one or more tasks. Data from 14 participantswho had good signal-to-noise ratios for all four tasks were retained for further analysis.2.4. Data PreprocessingThe data recorded by the NIRS device included impulse noise caused by voltagetransients during measurement. To remove the impulse noise, the first derivative of thesignal was calculated, and the mean and standard deviation (STD) of the first derivativevalues were computed. Any signals exceeding the 90% confidence interval were considerednoise and removed. Figure 3 shows the signal acquired by the NIRS device (a) and thesignal after impulse noise removal (b).Bioengineering 2024, 11, x FOR PEER REVIEW 6 of 16 Figure 3. Impulse noise removal: (a) original signal from NIRS device, (b) signal with impulse noise removed. 2.5. Breathing Pattern Classification Model The proposed TCNN model consists of a CNN-based autoencoder for the generation of deep compressed features and a CNN-based classifier module for the classification of breathing patterns. The autoencoder comprises an encoder that compresses the input sig-nal and a decoder that reconstructs the high-dimensional features back into the original signal. The high-dimensional features generated from the encoder are trained to detect the most important features that make up the signal. Therefore, we assume that the fea-tures generated by the encoder can be helpful in classifying breathing patterns. To test this hypothesis, we designed a breathing pattern classification model, as shown in Figure 4a. The deep compressed features generated by the autoencoder are integrated into the fea-ture model created in the classifier module to classify breathing patterns. Figure 4. Architecture of TCNN. (a) The autoencoder network is used to generate compressed fea-tures from the input data; then, the compressed features are combined with deep features extracted from the classification network. (b) The structure of a residual unit. The autoencoder and classifier module were designed by modifying the pre-activa-tion residual model (Pre-ResNet), which is an algorithm developed for the classification of 2D images [53]. Typically, the parameters in a deep learning model are optimized by Figure 3. Impulse noise removal: (a) original signal from NIRS device, (b) signal with impulsenoise removed.2.5. Breathing Pattern Classification ModelThe proposed TCNN model consists of a CNN-based autoencoder for the generationof deep compressed features and a CNN-based classifier module for the classification ofbreathing patterns. The autoencoder comprises an encoder that compresses the inputBioengineering 2024, 11, 709 6 of 16signal and a decoder that reconstructs the high-dimensional features back into the originalsignal. The high-dimensional features generated from the encoder are trained to detect themost important features that make up the signal. Therefore, we assume that the featuresgenerated by the encoder can be helpful in classifying breathing patterns. To test thishypothesis, we designed a breathing pattern classification model, as shown in Figure 4a.The deep compressed features generated by the autoencoder are integrated into the featuremodel created in the classifier module to classify breathing patterns.Bioengineering 2024, 11, x FOR PEER REVIEW 6 of 16 Figure 3. Impulse noise removal: (a) original signal from NIRS device, (b) signal with impulse noise removed. 2.5. Breathing Pattern Classification Model The proposed TCNN model consists of a CNN-based autoencoder for the generation of deep compressed features and a CNN-based classifier module for the classification of breathing patterns. The autoencoder comprises an encoder that compresses the input sig-nal and a decoder that reconstructs the high-dimensional features back into the original signal. The high-dimensional features generated from the encoder are trained to detect the most important features that make up the signal. Therefore, we assume that the fea-tures generated by the encoder can be helpful in classifying breathing patterns. To test this hypothesis, we designed a breathing pattern classification model, as shown in Figure 4a. The deep compressed features generated by the autoencoder are integrated into the fea-ture model created in the classifier module to classify breathing patterns. Figure 4. Architecture of TCNN. (a) The autoencoder network is used to generate compressed fea-tures from the input data; then, the compressed features are combined with deep features extracted from the classification network. (b) The structure of a residual unit. The autoencoder and classifier module were designed by modifying the pre-activa-tion residual model (Pre-ResNet), which is an algorithm developed for the classification of 2D images [53]. Typically, the parameters in a deep learning model are optimized by Figure 4. Architecture of TCNN. (a) The autoencoder network is used to generate compressed featuresfrom the input data; then, the compressed features are combined with deep features extracted fromthe classification network. (b) The structure of a residual unit.The autoe and clas ifier module were d signed by modifying the pre-activationresidual model (Pre-ResN t), which is an algorithm developed for the classific tion of2D images [53]. Typically, the par meters in a deep learning model are optimized bycomputing the gradient through the differentiation of the error between the predictedvalues from the model and the actual values and then performing the backpropagationof this gradient [54,55]. However, as the depth of the model increases, the vanishinggradient problem occurs because the transmitted gradient values decrease, making trainingdifficult and leading to overfitting [50–52]. To address these issues, the Pre-ResNet modelintroduces residual learning by structuring layers in building block units and addingshortcut connections between blocks [53]. Deep learning models composed of residualunits mitigate gradient loss during backpropagation by skipping one or more layers.Figure 4b shows a residual unit, which consists of a convolution layer, an activationfunction (ReLU) [56], and batch normalization (BN) [57].To apply the residual unit to the 1D signals acquired from the NIRS device, 1Dconvolutional layers were used instead of 2D convolutional layers, and the size of theconvolutional kernels was adjusted to fit the input signal. Detailed information on thenetwork structure and parameters is presented in Tables 1 and 2, with the BN and ReLUfunctions omitted. Each convolution block within the same group has the same kernel sizeand number of kernels. The block structure indicates the kernel size and number of kernelsincluded in each residual block. The block number indicates the number of residual unitscomposing each group. The input size and output size represent the sizes of the inputand output features at each stage. The feature map generated in Stage 3 of the encoder iscombined with the Stage 3 output features of the classifier module and used to train theclassification model.Bioengineering 2024, 11, 709 7 of 16Table 1. Autoencoder architectures.Group Name Input SizeOutput SizeBlock Structure(Kernel Size, Number) Block NumberEncoderStage 0 1 × 3841 × 96 1 × 7, 16 1Stage 1 1 × 961 × 32 1 × 7, 16 1Stage 2 1 × 321 × 321 × 1, 161 × 3, 161 × 1, 645Stage 3 1 × 321 × 161 × 1, 321 × 3, 321 × 1, 1285DecoderStage 0 1 × 161 × 321 × 1, 1281 × 3, 321 × 1, 325Stage 1 1 × 321 × 321 × 1, 641 × 3, 161 × 1, 165Stage 2 1 × 321 × 96 1 × 5, 16 1Stage 3 1 × 961 × 384 1 × 7, 4 1Table 2. Classifier module architectures.Group Name Input SizeOutput SizeBlock Structure(Kernel Size, Number) Block NumberStage 0 1 × 3841 × 96 1 × 7, 16 1Stage 1 1 × 961 × 32 1 × 7, 16 1Stage 2 1 × 321 × 321 × 1, 161 × 3, 161 × 1, 6422Stage 3 1 × 321 × 161 × 1, 321 × 3, 321 × 1, 12811Concatenation 1 × 16→ 1 × 32 - -Stage 4 1 × 321 × 161 × 1, 321 × 3, 321 × 1, 12811Stage 5 1 × 161 × 81 × 1, 641 × 3, 641 × 1, 25622Average pooling 1 × 81 × 256 1 × 8 1Fully connected layer 1 × 256Class number - 1Stride values of 4 and 3 were used to downsample the feature maps in Stage 0 andStage 1 of the classifier module and the encoder, respectively. In the remaining stages,Bioengineering 2024, 11, 709 8 of 16except in Stage 2, a stride value of 2 was used in the first convolutional layer of the firstresidual unit to downsample the feature maps to half their size, and the dimensions of thefeature map were expanded twice. As the depth of the network increased, more complexand comprehensive features were extracted.In the decoder, transposed convolution was used in the first convolutional layer ofeach stage’s first residual unit to upsample the feature maps to twice their size and halvethe dimensionality of the feature maps [58,59]. Stride values of 3 and 4 were used in Stage 2and Stage 3 to increase the size of the feature maps threefold and fourfold, respectively. Asa result, the generated feature maps resembled the input signal more closely as the depthof the network increased.The total objective function of the TCNN is defined asLtotal = LR + αLC (1)where LR and LC represent the reconstruction loss of the autoencoder and the cross-entropyloss of the classifier module, respectively. LR is defined as follows:LR =1NN∑i=1(Xi − Xˆi)2 (2)where Xi and Xˆi represent the input signal and the reconstructed signal, respectively, andN is the number of samples.2.6. DatasetAmong the acquired signals, the changes in O2Hb and HHb were used to classifybreathing patterns. THb, which is the sum of O2Hb and HHb, and StO2, which representsthe relationship between O2Hb and THb, were excluded because they are directly depen-dent on changes in O2Hb and HHb. Signals were cropped into intervals of 384 data points,corresponding to approximately 3.2 s of measurement time. As a result, the dataset ac-quired from the study consisted of a total of 113,456 samples, with 28,364 samples per class.Eleven out of 14 participants were randomly selected to have their data (22,286 samplesper class) used for the training of the classification model, while the dataset obtained fromthe remaining 3 participants (6078 samples per class) was used for model testing.2.7. Data AugmentationThe performance of deep-learning-based algorithms heavily relies on having an ex-tensive training dataset to prevent overfitting. To enhance the size and quality of ourtraining dataset, we applied data augmentation. The main purpose of data augmentationis to increase the quantity and diversity of the training data, and this technique is widelyused in various fields, such as computer vision, image classification, and natural languageprocessing [60–63]. To determine the data augmentation methods, we observed changes inthe data during each participant’s breathing. The participants exhibited similar respiratoryrates while performing respiration corresponding to each class. On the other hand, the am-plitude, shape, and direct current (DC) offset of the signals varied among the participants.Hence, a random amplitude scale, horizontal flipping with 50% probability, and DC offsetwere applied as data augmentation methods to maintain the signal period and increasethe diversity of the signal amplitude, shape, and DC values. Figure 5 shows an example ofdata augmentation.Each data augmentation method was applied before the signal was input into themodel and is defined as follows.• DC offset: performed by adding a DC value β to the original signal.fDCo f f set(Is) = Is + β (3)Bioengineering 2024, 11, 709 9 of 16where I represents the sample data input to the model during model training, and thesubscript s denotes the data source. Let D and d represent the training dataset and thedata sample belonging to the training dataset, respectively; then, the range of β is definedas follows.β ∈[mindis∈Ds(min(dis))−min(Is), maxdjs∈Ds(max(djs))−max(Is)](4)where i, j ∈ [1, 2, . . . , N] represent the indices of the data samples, and N denotes the totalnumber of samples in the training dataset.Bioengineering 2024, 11, x FOR PEER REVIEW 9 of 16 class. Eleven out of 14 participants were randomly selected to have their data (22,286 sam-ples per class) used for the training of the classification model, while the dataset obtained from the remaining 3 participants (6078 samples per class) was used for model testing. 2.7. Data Augmentation The performance of deep-learning-based algorithms heavily relies on having an ex-tensive training dataset to prevent overfitting. To enhance the size and quality of our train-ing dataset, we applied data augmentation. The main purpose of data augmentation is to increase the quantity and diversity of the training data, and this technique is widely used in various fields, such as computer vision, image classification, and natural language pro-cessing [60–63]. To determine the data augmentation methods, we observed changes in the data during each participant’s breathing. The participants exhibited similar respira-tory rates while performing respiration corresponding to each class. On the other hand, the amplitude, shape, and direct current (DC) offset of the signals varied among the par-ticipants. Hence, a random amplitude scale, horizontal flipping with 50% probability, and DC offset were applied as data augmentation methods to maintain the signal period and increase the diversity of the signal amplitude, shape, and DC values. Figure 5 shows an example of data augmentation. Figure 5. Example of the data augmentation methods. (a) The original signals recorded using the mobile phone. (b) The result of adding DC offset values of 0.004, −0.012, −0.007, and 0.005 to the PD1-O2Hb, PD1-HHb, PD2-O2Hb, and PD2-HHb signals, respectively. (c) The result of multiplying the PD1-O2Hb, PD1-HHb, PD2-O2Hb, and PD2-HHb signals by scale values of 1.5, 2, 0.4, and 2.3, respectively. (d) The result of a horizontal flip occurring with a 50% probability. Each data augmentation method was applied before the signal was input into the model and is defined as follows. • DC offset: performed by adding a DC value  to the original signal. ( )DCoffset s sf I I = + (3) Figure 5. Example of the data augmentation methods. (a) The original signals recorded using themobile phone. (b) The result of adding DC offset values of 0.004, −0.012, −0.007, and 0.005 to thePD1-O2Hb, PD1-HHb, PD2-O2Hb, and PD2-HHb signals, respectively. (c) The result of multiplyingthe PD1-O2Hb, PD1-HHb, PD2-O2Hb, and PD2-HHb signals by scale values of 1.5, 2, 0.4, and 2.3,respectively. (d) The result of a horizontal flip occurring with a 50% probability.• Amplitude random scaling: involves reducing or increasing the amplitude of the databy multiplying the scaling value γ with the original data.fScale(Is) = Is × γ (5)γ is determined by the smallest and largest amplitude values among the trainingdataset.γ ∈[mindis∈Ds(max(dis)−min(dis)), maxdjs∈Ds(max(djs)−min(djs))](6)• Horizontal flip fFlip(Is): performed by flipping the original data horizontally andoccurs with a probability of 50%. Horizontal flipping is a simple technique that caneffectively improve a model’s learning performance.Bioengineering 2024, 11, 709 10 of 16In this experiment, the augmented data fAug(Is) are finally defined as follows:fAug(Is) = fFlip(fDC o f f set( fScale(Is)))(7)3. Results3.1. TCNN with and without AutoencoderWe used a computer equipped with a CPU i5-12600k, a GPU RTX-3090, and 64 GBDDR4 RAM for the experiment. We first pre-trained an autoencoder with the same structureas in Figure 4. The weights of the pre-trained autoencoder were used as the initial valuesof the autoencoder of the TCNN. The pre-training of the autoencoder and the training ofthe TCNN were conducted using stochastic gradient descent (SGD) for 140 epochs, withmomentum of 0.9 and a batch size of 128. The initial learning rate for the pre-training ofthe autoencoder was set to 0.1, while the initial learning rate for the training of the TCNNwas set to 0.01. The learning rate was divided by 10 every 30 epochs. In this experiment,the hyper-parameter α was set to 10.Table 3 presents the experimental results to assess the impact of the compressed featuremaps generated from the autoencoder on the classification performance. Each model wastested five times. The SCNN (without an autoencoder) is a model derived from the TCNNexcluding the autoencoder. The SCNN has no concatenation group, and the output size ofStage 3 is 1 × 32. The classification accuracy is defined by the following formula.Accuracy(%) =Number of correct predictionsTotal number of predictions× 100(%). (8)Table 3. Classification performance compared with an SCNN (without autoencoder) and the dataaugmentation effect.Method Mean Accuracy (%) STD Best Accuracy (%)TCNN (201 layers) 94.00 0.449 94.63TCNN (without dataaugmentation)(201 layers)88.51 0.337 89.04SCNN (201 layers) 91.50 0.458 92.04SCNN (264 layers) 91.13 0.443 91.74The experimental results showed that the TCNN models had better performance thanthe SCNN models. In addition, the SCNN with more layers (264 layers) showed slightlyworse performance than the SCNN with less layers (201 layers). Furthermore, applyingdata augmentation techniques during the training process of the TCNN model led to anapproximately 5% improvement in the classification accuracy compared to not applyingdata augmentation.3.2. Confusion MatrixFigure 6 shows the normalized confusion matrix for the breathing pattern classificationresults. The ground truth represents the labels for the data samples, while the predictedlabels indicate the results predicted by the model. Therefore, a larger value in the diagonalmatrix indicates that the predicted label has been correctly classified. It can be observedthat the diagonal matrix is prominent in all three confusion matrices. This indicates thatthe majority of the data samples have been correctly classified into their respective classes.It is worth noting that in all three confusion matrices, the normal class has the lowestclassification accuracy.Bioengineering 2024, 11, 709 11 of 16Bioengineering 2024, 11, x FOR PEER REVIEW 11 of 16 (201 layers) SCNN (201 layers) 91.50 0.458 92.04 SCNN (264 layers) 91.13 0.443 91.74 The experimental results showed that the TCNN models had better performance than the SCNN models. In addition, the SCNN with more layers (264 layers) showed slightly worse performance than the SCNN with less layers (201 layers). Furthermore, applying data augmentation techniques during the training process of the TCNN model led to an approximately 5% improvement in the classification accuracy compared to not applying data augmentation. 3.2. Confusion Matrix Figure 6 shows the normalized confusion matrix for the breathing pattern classifica-tion results. The ground truth represents the labels for the data samples, while the pre-dicted labels indicate the results predicted by the model. Therefore, a larger value in the diagonal matrix indicates that the predicted label has been correctly classified. It can be observed that the diagonal matrix is prominent in all three confusion matrices. This indi-cates that the majority of the data samples have been correctly classified into their respec-tive classes. It is worth noting that in all three confusion matrices, the normal class has the lowest classification accuracy. Figure 6. Confusion matrix from predicted results of the TCNN and SCNN (without autoencoder). 3.3. Performance of Well-Known Classification Algorithms Table 4 shows a performance comparison of several classification methods. To com-pare the classification performance of the proposed method with that of other state-of-the-art classification algorithms, we used random forest [64], EfficientNetV2-M [65], Pyramid-Net [66], a modified 1D-PreResNet [49], and a coarse-to-fine CNN (CF-CNN) [67]. The random forest algorithm consists of multiple decision trees and is an ensemble learning method used for the training and classification of data. For our experiment, we employed a model consisting of 100 decision trees. To train the random forest, we utilized the aver-age amplitude, depth, and interval of the NIRS signals [48]. EfficientNetV2-M and PyramidNet are CNN-based algorithms developed for image classification, renowned for their exceptional performance and stability and widely utilized across various domains [68–71]. PyramidNet consists of 272 layers, and the widening step factor value for increas-ing feature map dimensions is set to 200. CF-CNN consists of a main network for fine classification and several coarse sub-networks with a hierarchical structure for coarse clas-sification. To train a new hierarchical structure for coarse classification, we generated hi-erarchical group labels using the group label augmentation method [67]. The coarse net-work of CF-CNN consists of a downsized version of the base network that constitutes the Figure 6. onfusion atrix fro predicted results of the T and S ( ithout autoencoder).3.3. Perfor ance of ell-Kno n Classification lgorith sTable 4 sho s a performance comparison of several classification methods. To comparethe classification performance of the proposed method with that of other state-of-the-artclassification algorithms, we used random forest [64], EfficientNetV2- [65], Pyramid-Net [66], a modified 1D-PreResNet [49], and a coarse-to-fine CNN (CF-CNN) [67]. Therandom forest algorithm consists of multiple decision trees and is an ensemble learningmethod used for the training and classification of data. For our experiment, we employed amodel consisting of 100 decision trees. To train the random forest, we utilized the averageamplitude, depth, and interval of the NIRS signals [48]. EfficientNetV2-M and Pyramid-Net are CNN-based algorithms developed for image classification, renowned for theirexceptional performance and stability and widely utilized across various domains [68–71].PyramidNet consists of 272 layers, and the widening step factor value for increasing featuremap dimensions is set to 200. CF-CNN consists of a main network for fine classificationand several coarse sub-networks with a hierarchical structure for coarse classification. Totrain a new hierarchical structure for coarse classification, we generated hierarchical grouplabels using the group label augmentation method [67]. The coarse network of CF-CNNconsists of a downsized version of the base network that constitutes the main network.This model is also proposed to overcome the limitations of learning in single-stream net-works. For comparison, the 2D convolutional layers of each network were replaced with1D convolutional layers. For the evaluation of the CF-CNN model, we chose PyramidNetas the base network. The number of layers in each of the Coarse 1, Coarse 2, and mainnetworks was set to 26, 52, and 272, respectively. For Coarse 1 and Coarse 2 classification,the number of group labels was set to 2 and 3, respectively.Table 4. Classification performance compared with state-of-the-art methods.Method Number ofParametersFLOP(for 1 Sample)MeanAccuracy (%) STDBestAccuracy (%)Random Forest [64] - - 88.10 0.321 88.491D Pre-ResNet (200 layers) [49] 1.36 M 14.8 M 90.98 0.428 91.54EfficientNetV2-M [65] 52 M 61.6 M 93.96 0.459 94.45PyramidNet [66] 17 M 198 M 89.74 0.412 90.17CF-CNN [67] 29.7 M 244 M 91.51 0.406 92.01Proposed Method 1.53 M 16.4 M 94.00 0.449 94.63Each model was tested five times, and, for each test, the model was trained fromscratch. To train the 1D Pre-ResNet, EfficientNetV2-M, PyramidNet, and CF-CNN, theinput data were downsampled using the bilinear interpolation method to match the inputBioengineering 2024, 11, 709 12 of 16size of each model at the input layer. While our proposed method had a larger numberof parameters and floating-point operations (FLOP) compared to the 1D Pre-ResNet, itachieved the highest classification performance among the compared methods.4. DiscussionThe aim of our study was to develop a breathing pattern classification method for thereal-time monitoring of patients with infectious respiratory diseases such as COVID-19.First, we defined four common breathing patterns observed in patients with respiratorydiseases: normal, breath holding, slow, and rapid. Then, we collected data from partici-pants using a wearable NIRS device. Second, we proposed the TCNN, consisting of anautoencoder and a classifier module, to classify breathing patterns. We hypothesized thatthe deep compressed features generated by the encoder of the autoencoder would containinformation about the composition of the signal, which could potentially help to improvethe classification performance. To validate this hypothesis, we compared the classifica-tion performance of the proposed TCNN with that of an SCNN model. Comparing thebreathing pattern classification accuracy between the two SCNN models with 201 layersand 264 layers, we observed that the SCNN with less layers exhibited higher classificationaccuracy. This demonstrates the challenge of parameter learning becoming more difficultas the number of parameters increases. Comparing the classification performance of theTCNN and SCNN, we observed that the TCNN achieved higher classification performance.This result suggests that the deep compressed features generated by the encoder of theautoencoder contribute to the improvement in classification performance. Furthermore,as training is performed on networks with different objectives, the diversity of the featuremaps is enhanced, and the burden on parameter optimization is reduced.The results of the TCNN (without data augmentation) showed significantly lowerclassification accuracy compared to the TCNN, primarily due to insufficient training data.Deep learning algorithms can prevent overfitting and enhance the model’s generalizationwhen trained with diverse and abundant data. To improve the model’s generalizationperformance, we applied three data augmentation methods: DC offset, amplitude randomscale, and horizontal flip. As a result, the classification performance was improved byapproximately 5%. This outcome suggests that data augmentation effectively enhanced themodel while preserving the essential features necessary for breathing pattern classification.For all three models in Figure 6, the classification accuracy for normal breathing wasthe lowest. One plausible explanation is that the most distinguishing feature in breathingpattern classification is the respiratory rate. In some instances, a normal breathing rate canbe confused with slow and rapid breathing.Among all classification models used in this study, the proposed TCNN model ex-hibited the best classification accuracy. Moreover, the TCNN model used significantly asmaller number of parameters and FLOP, which are directly related to the hardware storagespace and computational load. FLOP represents the number of floating-point operationsrequired to execute each classification model. Therefore, a lower FLOP count indicates ashorter classification time. Although the TCNN shows slightly more parameters and FLOPthan the 1D-PreResNet, which has the fewest parameters and FLOP, it achieves higherclassification accuracy. When compared to the performance of the EfficientNetV2-M, Pyra-midNet, and CF-CNN models, the proposed method demonstrates the best performancewith significantly fewer parameters and FLOP. The classification accuracy when usingthe random forest algorithm was the lowest among the compared methods. Traditionalmachine learning methods like random forest perform well when the input features arediverse and contain distinct information. However, creating diverse feature extractors man-ually is very challenging for humans. On the other hand, deep learning algorithms showbetter performance than traditional machine learning methods because they automaticallygenerate diverse and meaningful features during the model learning process.Future studies could expand on our results by collecting signals from individuals andpatients with respiratory diseases. We are considering the application of noise removalBioengineering 2024, 11, 709 13 of 16techniques during the data preprocessing stage to enhance the classification performance.Additionally, we are exploring the integration of sensors such as temperature sensors,accelerometers, and NIRS devices to obtain diverse and meaningful signals.5. ConclusionsThis study proposed a two-stream convolutional neural network (TCNN) model forthe classification of breathing patterns to facilitate the evaluation and monitoring of patientswith infectious respiratory diseases. This method utilizes the changes in oxyhemoglobinand deoxyhemoglobin measured using a wearable NIRS device. The proposed TCNNconsists of an autoencoder and a classifier module. The deep compressed feature mapsgenerated in the encoder of the autoencoder are combined with the feature maps generatedin the classifier module to classify breathing patterns. The compressed features generatedby the encoder contain important information that constitutes the signal, unlike the featuresgenerated by the classifier. Therefore, compressed features help to improve the breathingpattern classification performance because they provide additional information for theclassification of breathing patterns. The proposed method alleviates the issue of decreasingparameter learning capabilities as the depth of a typical single-stream network modelincreases. For the breathing pattern classification experiment, we defined four commonbreathing patterns observed in infectious respiratory disease patients: normal breathing,breath holding, slow breathing, and rapid breathing. We then conducted the classificationexperiment. As a result, we achieved classification accuracy of 94.63%. The proposedmethod demonstrates its potential for the remote and real-time monitoring of respiratorydiseases such as COVID-19, pneumonia, and influenza.Author Contributions: Conceptualization, J.P., T.N., B.H. and A.G.; Methodology, J.P., S.P., T.N.and A.G.; Software, J.P.; Validation, J.P. and A.G.; Formal Analysis, J.P.; Investigation, J.P. and A.G.;Resources, A.G.; Data Curation, T.N. and J.P.; Writing—Original Draft Preparation, J.P.; Writing—Review and Editing, T.N., S.P., J.P., B.H., B.S. and A.G.; Visualization, J.P.; Supervision, A.G.; ProjectAdministration, T.N. and A.G.; Funding Acquisition, A.G. All authors have read and agreed to thepublished version of the manuscript.Funding: This research was funded by the Intramural Research Program at the Eunice KennedyShriver National Institute of Child Health and Human Development, National Institutes of Health,grant number [Z01-HD000261].Institutional Review Board Statement: The study was conducted according to the guidelines ofthe Declaration of Helsinki and approved by the Institutional Review Board of the Eunice KennedyShriver National Institute of Child Health and Human Development on 31 August 2021 (NICHDprotocol #21CH0028).Informed Consent Statement: Informed consent was obtained from all subjects involved inthe study.Data Availability Statement: Data supporting the reported results available at https://github.com/dkskzmffps/TCNNforBP, accessed on 11 July 2024.Acknowledgments: We acknowledge the Hamamatsu Photonics company for providing their wear-able NIRS prototype to conduct the experiments.Conflicts of Interest: The authors declare no conflicts of interest.References1. WHO Coronavirus (COVID-19) Dashboard. Available online: https://covid19.who.int/ (accessed on 5 December 2023).2. Pan, L.; Mu, M.; Yang, P.; Sun, Y.; Wang, R.; Yan, J.; Li, P.; Hu, B.; Wang, J.; Hu, C. Clinical characteristics of COVID-19patients with digestive symptoms in Hubei, China: A descriptive, cross-sectional, multicenter study. Am. J. Gastroenterol. 2020,115, 766–773. [CrossRef] [PubMed]3. Gandhi, L.; Maisnam, D.; Rathore, D.; Chauhan, P.; Bonagiri, A.; Venkataramana, M. Respiratory illness virus infections withspecial emphasis on COVID-19. Eur. J. Med. Res. 2022, 27, 236. [CrossRef] [PubMed]Bioengineering 2024, 11, 709 14 of 164. Wang, Y.; Hu, M.; Li, Q.; Zhang, X.-P.; Zhai, G.; Yao, N. Abnormal respiratory patterns classifier may contribute to large-scalescreening of people infected with COVID-19 in an accurate and unobtrusive manner. arXiv 2020, arXiv:2002.05534.5. Tleyjeh, I.M.; Saddik, B.; Ramakrishnan, R.K.; AlSwaidan, N.; AlAnazi, A.; Alhazmi, D.; Aloufi, A.; AlSumait, F.; Berbari, E.F.;Halwani, R. Long term predictors of breathlessness, exercise intolerance, chronic fatigue and well-being in hospitalized patientswith COVID-19: A cohort study with 4 months median follow-up. J. Infect. Public Health 2022, 15, 21–28. [CrossRef] [PubMed]6. Martelletti, P.; Bentivegna, E.; Spuntarelli, V.; Luciani, M. Long-COVID headache. SN Compr. Clin. Med. 2021, 3, 1704–1706.[CrossRef] [PubMed]7. Benmalek, E.; Elmhamdi, J.; Jilbab, A. Comparing CT scan and chest X-ray imaging for COVID-19 diagnosis. Biomed. Eng. Adv.2021, 1, 100003. [CrossRef] [PubMed]8. Bai, H.X.; Hsieh, B.; Xiong, Z.; Halsey, K.; Choi, J.W.; Tran, T.M.L.; Pan, I.; Shi, L.-B.; Wang, D.-C.; Mei, J. Performance of radiologistsin differentiating COVID-19 from non-COVID-19 viral pneumonia at chest CT. Radiology 2020, 296, E46–E54. [CrossRef] [PubMed]9. Ceniccola, G.D.; Castro, M.G.; Piovacari, S.M.F.; Horie, L.M.; Corrêa, F.G.; Barrere, A.P.N.; Toledo, D.O. Current technologies inbody composition assessment: Advantages and disadvantages. Nutrition 2019, 62, 25–31. [CrossRef] [PubMed]10. Wang, L.; Lin, Z.Q.; Wong, A. Covid-net: A tailored deep convolutional neural network design for detection of covid-19 casesfrom chest X-ray images. Sci. Rep. 2020, 10, 19549. [CrossRef]11. Cho, Y.; Bianchi-Berthouze, N.; Julier, S.J. DeepBreath: Deep learning of breathing patterns for automatic stress recognition usinglow-cost thermal imaging in unconstrained settings. In Proceedings of the 2017 Seventh International Conference on AffectiveComputing and Intelligent Interaction (ACII), San Antonio, TX, USA, 23–26 October 2017; pp. 456–463.12. Taylor, W.; Abbasi, Q.H.; Dashtipour, K.; Ansari, S.; Shah, S.A.; Khalid, A.; Imran, M.A. A Review of the State of the Art inNon-Contact Sensing for COVID-19. Sensors 2020, 20, 5665. [CrossRef]13. Rahman, A.; Lubecke, V.M.; Boric–Lubecke, O.; Prins, J.H.; Sakamoto, T. Doppler radar techniques for accurate respirationcharacterization and subject identification. IEEE J. Emerg. Sel. Top. Circuits Syst. 2018, 8, 350–359. [CrossRef]14. Ahmad, A.; Roh, J.C.; Wang, D.; Dubey, A. Vital signs monitoring of multiple people using a FMCW millimeter-wave sensor. InProceedings of the 2018 IEEE Radar Conference (RadarConf18), Oklahoma City, OK, USA, 23–27 April 2018; pp. 1450–1455.15. Prat, A.; Blanch, S.; Aguasca, A.; Romeu, J.; Broquetas, A. Collimated beam FMCW radar for vital sign patient monitoring. IEEETrans. Antennas Propag. 2018, 67, 5073–5080. [CrossRef]16. Podder, P.; Das, S.R.; Mondal, M.R.H.; Bharati, S.; Maliha, A.; Hasan, M.J.; Piltan, F. Lddnet: A deep learning framework for thediagnosis of infectious lung diseases. Sensors 2023, 23, 480. [CrossRef] [PubMed]17. Purnomo, A.T.; Lin, D.-B.; Adiprabowo, T.; Hendria, W.F. Non-contact monitoring and classification of breathing pattern for thesupervision of people infected by COVID-19. Sensors 2021, 21, 3172. [CrossRef] [PubMed]18. Kim, J.; Campbell, A.S.; de Ávila, B.E.-F.; Wang, J. Wearable biosensors for healthcare monitoring. Nat. Biotechnol. 2019,37, 389–406. [CrossRef] [PubMed]19. Nguyen, T.; Khaksari, K.; Khare, S.M.; Park, S.; Anderson, A.A.; Bieda, J.; Jung, E.; Hsu, C.-D.; Romero, R.; Gandjbakhche,A.H. Non-invasive transabdominal measurement of placental oxygenation: A step toward continuous monitoring. Biomed. Opt.Express 2021, 12, 4119–4130. [CrossRef] [PubMed]20. Massaroni, C.; Lo Presti, D.; Formica, D.; Silvestri, S.; Schena, E. Non-contact monitoring of breathing pattern and respiratory ratevia RGB signal measurement. Sensors 2019, 19, 2758. [CrossRef] [PubMed]21. Tobin, M.J.; Chadha, T.S.; Jenouri, G.; Birch, S.J.; Gazeroglu, H.B.; Sackner, M.A. Breathing patterns: 2. Diseased subjects. Chest1983, 84, 286–294. [CrossRef]22. AL-Khalidi, F.Q.; Saatchi, R.; Burke, D.; Elphick, H.; Tan, S. Respiration rate monitoring methods: A review. Pediatr. Pulmonol.2011, 46, 523–529. [CrossRef]23. Nicolò, A.; Massaroni, C.; Schena, E.; Sacchetti, M. The importance of respiratory rate monitoring: From healthcare to sport andexercise. Sensors 2020, 20, 6396. [CrossRef]24. Cretikos, M.A.; Bellomo, R.; Hillman, K.; Chen, J.; Finfer, S.; Flabouris, A. Respiratory rate: The neglected vital sign. Med. J. Aust.2008, 188, 657–659. [CrossRef] [PubMed]25. Rohmetra, H.; Raghunath, N.; Narang, P.; Chamola, V.; Guizani, M.; Lakkaniga, N.R. AI-enabled remote monitoring of vital signsfor COVID-19: Methods, prospects and challenges. Computing 2023, 105, 783–809. [CrossRef]26. Costanzo, I.; Sen, D.; Rhein, L.; Guler, U. Respiratory monitoring: Current state of the art and future roads. IEEE Rev. Biomed. Eng.2020, 15, 103–121. [CrossRef] [PubMed]27. Qi, W.; Aliverti, A. A multimodal wearable system for continuous and real-time breathing pattern monitoring during dailyactivity. IEEE J. Biomed. Health Inform. 2019, 24, 2199–2207. [CrossRef] [PubMed]28. Torjesen, I. COVID-19: Patients to use pulse oximetry at home to spot deterioration. BMJ 2020, 71, m4151. [CrossRef]29. Todd, B. Pulse oximetry may be inaccurate in patients with darker skin. Am. J. Nurs. 2021, 121, 16.30. Takegawa, R.; Hayashida, K.; Rolston, D.M.; Li, T.; Miyara, S.J.; Ohnishi, M.; Shiozaki, T.; Becker, L.B. Near-infrared spectroscopyassessments of regional cerebral oxygen saturation for the prediction of clinical outcomes in patients with cardiac arrest: A reviewof clinical impact, evolution, and future directions. Front. Med. 2020, 7, 587930. [CrossRef] [PubMed]31. Samraj, R.S.; Nicolas, L. Near infrared spectroscopy (NIRS) derived tissue oxygenation in critical illness. Clin. Investig. Med. 2015,38, E285–E295. [CrossRef] [PubMed]Bioengineering 2024, 11, 709 15 of 1632. Cheung, A.; Tu, L.; Macnab, A.; Kwon, B.K.; Shadgan, B. Detection of hypoxia by near-infrared spectroscopy and pulse oximetry:A comparative study. J. Biomed. Opt. 2022, 27, 077001. [CrossRef]33. Koonin, E.V.; Krupovic, M.; Agol, V.I. The Baltimore classification of viruses 50 years later: How does it stand in the light of virusevolution? Microbiol. Mol. Biol. Rev. 2021, 85, e00053-21. [CrossRef]34. Monto, A.S.; Gravenstein, S.; Elliott, M.; Colopy, M.; Schweinle, J. Clinical signs and symptoms predicting influenza infection.Arch. Intern. Med. 2000, 160, 3243–3247. [CrossRef] [PubMed]35. Piret, J.; Boivin, G. Pandemics throughout history. Front. Microbiol. 2021, 11, 631736. [CrossRef] [PubMed]36. Wang, J.; Zhou, M.; Liu, F. Reasons for healthcare workers becoming infected with novel coronavirus disease 2019 (COVID-19) inChina. J. Hosp. Infect. 2020, 105, 100–101. [CrossRef] [PubMed]37. Lewnard, J.A.; Lo, N.C. Scientific and ethical basis for social-distancing interventions against COVID-19. Lancet Infect. Dis. 2020,20, 631–633. [CrossRef] [PubMed]38. Salathé, M.; Althaus, C.L.; Neher, R.; Stringhini, S.; Hodcroft, E.; Fellay, J.; Zwahlen, M.; Senti, G.; Battegay, M.; Wilder-Smith,A. COVID-19 epidemic in Switzerland: On the importance of testing, contact tracing and isolation. Swiss Med. Wkly. 2020,150, w20225. [CrossRef] [PubMed]39. Kim, P.S.; Read, S.W.; Fauci, A.S. Therapy for early COVID-19: A critical need. JAMA 2020, 324, 2149–2150. [CrossRef] [PubMed]40. Shafi, I.; Din, S.; Farooq, S.; Díez, I.d.l.T.; Breñosa, J.; Espinosa, J.C.M.; Ashraf, I. Design and development of patient healthtracking, monitoring and big data storage using Internet of Things and real time cloud computing. PLoS ONE 2024, 19, e0298582.[CrossRef]41. Shafi, I.; Ansari, S.; Din, S.; Jeon, G.; Paul, A. Artificial neural networks as clinical decision support systems. Concurr. Comput.Pract. Exp. 2021, 33, e6342. [CrossRef]42. Fasihfar, Z.; Rokhsati, H.; Sadeghsalehi, H.; Ghaderzadeh, M.; Gheisari, M. AI-driven malaria diagnosis: Developing a robustmodel for accurate detection and classification of malaria parasites. Iran. J. Blood Cancer 2023, 15, 112–124. [CrossRef]43. Ghaderzadeh, M.; Asadi, F.; Ramezan Ghorbani, N.; Almasi, S.; Taami, T. Toward artificial intelligence (AI) applications in thedetermination of COVID-19 infection severity: Considering AI as a disease control strategy in future pandemics. Iran. J. BloodCancer 2023, 15, 93–111. [CrossRef]44. Shah, V.; Keniya, R.; Shridharani, A.; Punjabi, M.; Shah, J.; Mehendale, N. Diagnosis of COVID-19 using CT scan images and deeplearning techniques. Emerg. Radiol. 2021, 28, 497–505. [CrossRef] [PubMed]45. Hong, J.-W.; Kim, S.-H.; Han, G.-T. Detection of multiple respiration patterns based on 1D SNN from continuous human breathingsignals and the range classification method for each respiration pattern. Sensors 2023, 23, 5275. [CrossRef] [PubMed]46. Kim, J.; Kim, J. Classification of Breathing Signals According to Human Motions by Combining 1D Convolutional Neural Networkand Embroidered Textile Sensor. Sensors 2023, 23, 5736. [CrossRef] [PubMed]47. Conway, A.; Goudarzi Rad, M.; Zhou, W.; Parotto, M.; Jungquist, C. Deep learning classification of capnography waveforms:Secondary analysis of the PRODIGY study. J. Clin. Monit. Comput. 2023, 37, 1327–1339. [CrossRef] [PubMed]48. Mah, A.J.; Nguyen, T.; Ghazi Zadeh, L.; Shadgan, A.; Khaksari, K.; Nourizadeh, M.; Zaidi, A.; Park, S.; Gandjbakhche, A.H.;Shadgan, B. Optical Monitoring of breathing patterns and tissue oxygenation: A potential application in COVID-19 screening andmonitoring. Sensors 2022, 22, 7274. [CrossRef] [PubMed]49. Park, J.; Mah, A.J.; Nguyen, T.; Park, S.; Ghazi Zadeh, L.; Shadgan, B.; Gandjbakhche, A.H. Modification of a conventionaldeep learning model to classify simulated breathing patterns: A step toward real-time monitoring of patients with respiratoryinfectious diseases. Sensors 2023, 23, 5592. [CrossRef]50. Bengio, Y.; Simard, P.; Frasconi, P. Learning long-term dependencies with gradient descent is difficult. IEEE Trans. Neural Netw.1994, 5, 157–166. [CrossRef] [PubMed]51. Glorot, X.; Bengio, Y. Understanding the difficulty of training deep feedforward neural networks. In Proceedings of the ThirteenthInternational Conference on Artificial Intelligence and Statistics, Sardinia, Italy, 13–15 May 2010; pp. 249–256.52. Basodi, S.; Ji, C.; Zhang, H.; Pan, Y. Gradient amplification: An efficient way to train deep neural networks. Big Data Min. Anal.2020, 3, 196–207. [CrossRef]53. He, K.; Zhang, X.; Ren, S.; Sun, J. Identity mappings in deep residual networks. In Proceedings of the Computer Vision–ECCV 2016: 14th European Conference, Amsterdam, The Netherlands, 11–14 October 2016; Proceedings, Part IV 14; Springer:Berlin/Heidelberg, Germany, 2016; pp. 630–645.54. LeCun, Y.; Boser, B.; Denker, J.S.; Henderson, D.; Howard, R.E.; Hubbard, W.; Jackel, L.D. Backpropagation applied to handwrittenzip code recognition. Neural Comput. 1989, 1, 541–551. [CrossRef]55. Robbins, H.; Monro, S. A stochastic approximation method. Ann. Math. Stat. 1951, 22, 400–407. [CrossRef]56. Nair, V.; Hinton, G.E. Rectified linear units improve restricted boltzmann machines. In Proceedings of the 27th InternationalConference on Machine Learning (ICML-10), Haifa, Israel, 21–24 June 2010; pp. 807–814.57. Ioffe, S.; Szegedy, C. Batch normalization: Accelerating deep network training by reducing internal covariate shift. In Proceedingsof the International Conference on Machine Learning, Lille, France, 6–11 July 2015; pp. 448–456.58. Dumoulin, V.; Visin, F. A guide to convolution arithmetic for deep learning. arXiv 2016, arXiv:1603.07285.59. Zeiler, M.D.; Krishnan, D.; Taylor, G.W.; Fergus, R. Deconvolutional networks. In Proceedings of the 2010 IEEE Computer SocietyConference on Computer Vision and Pattern Recognition, San Francisco, CA, USA, 13–18 June 2010; pp. 2528–2535.60. Shorten, C.; Khoshgoftaar, T.M. A survey on image data augmentation for deep learning. J. Big Data 2019, 6, 60. [CrossRef]Bioengineering 2024, 11, 709 16 of 1661. Mumuni, A.; Mumuni, F. Data augmentation: A comprehensive survey of modern approaches. Array 2022, 16, 100258. [CrossRef]62. Feng, S.Y.; Gangal, V.; Wei, J.; Chandar, S.; Vosoughi, S.; Mitamura, T.; Hovy, E. A survey of data augmentation approaches forNLP. arXiv 2021, arXiv:2105.03075.63. Wen, Q.; Sun, L.; Yang, F.; Song, X.; Gao, J.; Wang, X.; Xu, H. Time series data augmentation for deep learning: A survey. arXiv2020, arXiv:2002.12478.64. Ho, T.K. Random decision forests. In Proceedings of the 3rd International Conference on Document Analysis and Recognition,Montreal, QC, Canada, 14–16 August 1995; pp. 278–282.65. Tan, M.; Le, Q. Efficientnetv2: Smaller models and faster training. In Proceedings of the International Conference on MachineLearning, Virtual, 18–24 July 2021; pp. 10096–10106.66. Han, D.; Kim, J.; Kim, J. Deep pyramidal residual networks. In Proceedings of the IEEE Conference on Computer Vision andPattern Recognition, Honolulu, HI, USA, 21–26 July 2017; pp. 5927–5935.67. Park, J.; Kim, H.; Paik, J. Cf-cnn: Coarse-to-fine convolutional neural network. Appl. Sci. 2021, 11, 3722. [CrossRef]68. Long, X.; Deng, K.; Wang, G.; Zhang, Y.; Dang, Q.; Gao, Y.; Shen, H.; Ren, J.; Han, S.; Ding, E. PP-YOLO: An effective and efficientimplementation of object detector. arXiv 2020, arXiv:2007.12099.69. Tan, M.; Pang, R.; Le, Q.V. Efficientdet: Scalable and efficient object detection. In Proceedings of the IEEE/CVF Conference onComputer Vision and Pattern Recognition, Seattle, WA, USA, 13–19 June 2020; pp. 10781–10790.70. Kim, S.; Rim, B.; Choi, S.; Lee, A.; Min, S.; Hong, M. Deep learning in multi-class lung diseases’ classification on chest X-rayimages. Diagnostics 2022, 12, 915. [CrossRef]71. Paoletti, M.E.; Haut, J.M.; Fernandez-Beltran, R.; Plaza, J.; Plaza, A.J.; Pla, F. Deep pyramidal residual networks for spectral–spatialhyperspectral image classification. IEEE Trans. Geosci. Remote Sens. 2018, 57, 740–754. [CrossRef]Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individualauthor(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury topeople or property resulting from any ideas, methods, instructions or products referred to in the content.
Article
eng
Reviewed
Vancouver : University of British Columbia Library
Multidisciplinary Digital Publishing Institute
CC BY 4.0
Faculty
Researcher
COVID-19
deep learning
convolutional neural network
respiratory disease
NIRS
wearable device
Two-Stream Convolutional Neural Networks for Breathing Pattern Classification: Real-Time Monitoring of Respiratory Disease Patients
Text