Science, Faculty of
Statistics, Department of
DSpace
UBCV
Sidrow, Evan
Doctor of Philosophy - PhD
University of British Columbia
Hidden Markov models (HMMs) are commonly used to identify latent processes from observed time series, but it is challenging to fit them to large and complex time series collected by modern sensors. Using data from threatened resident killer whales (Orcinus orca) as a case study, this dissertation provides solutions to three common challenges when identifying latent behaviour from biologging data. First, time series are now collected at high frequencies and exhibit fine-scale dependence structures that violate common assumptions of HMMs. I propose a hierarchical approach that divides time series into sequences of curves. A coarse-scale HMM models the sequence of curves while a related fine-scale HMM describes the structure within each curve. The fine-scale model includes autoregression to capture dependence in addition to moving-window summaries and Fourier analysis to capture intricate fine-scale structures. I use simulation and case studies to show that this framework produces more interpretable state estimation and more accurate parameter estimates than existing methods. Second, many modern biologging time series include labels of the latent process of interest. Labels can improve predictive accuracy, but sparse labels can have a negligible influence on parameter estimates. I introduce a weighted likelihood approach that increases the relative influence of labeled observations. I use this approach to develop two HMMs that model the foraging behaviour of killer whales at different scales. Using cross-validated evaluation metrics, I show that my weighted method produces more accurate and understandable models than unweighted methods. Finally, applying HMMs to very large time series is computationally demanding, as expectation-maximization (EM) algorithms typically iterate through the entire data set for every parameter update. I propose a novel optimization algorithm that combines a partial E step with variance-reduced stochastic optimization within the M step. I prove my algorithm converges under standard regularity conditions and test my algorithm empirically using simulations and case studies. My algorithm converges in fewer epochs, with less computation time, and to regions of higher likelihood than standard numerical optimization techniques. In total, this dissertation introduces novel methods to develop and train HMMs for complex biologging time series collected by modern sensors.
Modelling Complex Biologging Datawith Hidden Markov ModelsbyEvan SidrowB.Sc., The University of Colorado Boulder, 2018M.Sc., The University of Colorado Boulder, 2018A DISSERTATION SUBMITTED IN PARTIAL FULFILLMENT OFTHE REQUIREMENTS FOR THE DEGREE OFDOCTOR OF PHILOSOPHYinThe Faculty of Graduate and Postdoctoral Studies(Statistics)THE UNIVERSITY OF BRITISH COLUMBIA(Vancouver)October 2024© Evan Sidrow 2024The following individuals certify that they have read, and recommend to the Faculty ofGraduate and Postdoctoral Studies for acceptance, the dissertation entitled:Modelling Complex Biologging Data with Hidden Markov Modelssubmitted by Evan Sidrow in partial fulfillment of the requirements for the degree ofDoctor of Philosophy in Statistics.Examining Committee:Nancy Heckman, Professor Emeritus, Department of Statistics, UBCCo-supervisorMarie Auger-Méthé, Associate Professor, Department of Statistics, UBCCo-supervisorAlexandre Bouchard-Côté, Professor, Department of Statistics, UBCSupervisory Committee MemberMatías Saibián-Barrera, Professor, Department of Statistics, UBCUniversity ExaminerDaniel Coombs, Professor, Department of Mathematics, UBCUniversity ExaminerRoland Langrock, Professor, Statistics and Data Analysis Group, Bielefeld UniversityExternal ExaminerAdditional Supervisory Committee Members:Jim Zidek, Professor Emeritus, Department of Statistics, UBCSupervisory Committee MemberiiAbstractHidden Markov models (HMMs) are commonly used to identify latent processes from ob-served time series, but it is challenging to fit them to large and complex time series collectedby modern sensors. Using data from threatened resident killer whales (Orcinus orca) asa case study, this dissertation provides solutions to three common challenges when iden-tifying latent behaviour from biologging data. First, time series are now collected at highfrequencies and exhibit fine-scale dependence structures that violate common assumptionsof HMMs. I propose a hierarchical approach that divides time series into sequences ofcurves. A coarse-scale HMM models the sequence of curves while a related fine-scale HMMdescribes the structure within each curve. The fine-scale model includes autoregressionto capture dependence in addition to moving-window summaries and Fourier analysis tocapture intricate fine-scale structures. I use simulation and case studies to show that thisframework produces more interpretable state estimation and more accurate parameter es-timates than existing methods. Second, many modern biologging time series include labelsof the latent process of interest. Labels can improve predictive accuracy, but sparse labelscan have a negligible influence on parameter estimates. I introduce a weighted likelihoodapproach that increases the relative influence of labeled observations. I use this approachto develop two HMMs that model the foraging behaviour of killer whales at different scales.Using cross-validated evaluation metrics, I show that my weighted method produces moreaccurate and understandable models than unweighted methods. Finally, applying HMMsto very large time series is computationally demanding, as expectation-maximization (EM)algorithms typically iterate through the entire data set for every parameter update. I pro-pose a novel optimization algorithm that combines a partial E step with variance-reducedstochastic optimization within the M step. I prove my algorithm converges under standardregularity conditions and test my algorithm empirically using simulations and case studies.My algorithm converges in fewer epochs, with less computation time, and to regions ofhigher likelihood than standard numerical optimization techniques. In total, this disser-tation introduces novel methods to develop and train HMMs for complex biologging timeseries collected by modern sensors.iiiLay SummaryResearchers often want to understand a process that they cannot directly observe. For ex-ample, ecologists want to know when a killer whale hunts, but they cannot see it underwater.Therefore, they use animal-borne tags to collect movement data instead. Statisticians thenuse methods like hidden Markov models (HMMs) to predict the hidden process from thetag data.Modern data sets, including those collected from animal-borne tags, are more complexthan ever before, leading to exciting research opportunities and statistical challenges. Thisdissertation addresses three such challenges for HMMs. First, I develop an HMM to modelvery high-frequency data. Second, modern technology can occasionally directly observepreviously hidden processes, and I incorporate these glimpses into an HMM. Finally, Idevelop a computational method to use HMMs more efficiently for large data sets. In total,this dissertation introduce tools for researchers to infer hidden behaviours from complexdata collected by modern sensors.ivPrefaceThis dissertation is the original work of Evan Sidrow, completed under the joint supervisionof Dr. Nancy Heckman and Dr. Marie Auger-Méthé.The killer whale data were collected under University of British Columbia Animal CarePermit number A19-0053 and Fisheries and Oceans Canada Marine Mammal ScientificLicense for Whale Research number XMMS 6 2019. Tags were deployed by Mike deRoos,the tagging boat skipper was Chris Hall, photo-ID was done by Taryn Scarff.A version of Chapter 2 has been published [Sidrow E, Heckman N, Fortune SME, TritesAW, Murphy I, Auger-Méthé M. Modelling multi-scale, state-switching functional data withhidden Markov models. The Canadian Journal of Statistics. 50(1):327-356, 2022]. The ideaswere jointly developed by Evan Sidrow, Dr. Nancy Heckman and Dr. Marie Auger-Méthéwith the majority of the computational work and manuscript writing conducted by EvanSidrow as lead author. Dr. Sarah Fortune assisted with data collection and manuscriptediting. Dr. Andrew Trites provided helpful comments and edited the manuscript. IanMurphy assisted with some computational work.The ideas in Chapter 3 were jointly developed by Evan Sidrow, Dr. Nancy Heckman andDr. Marie Auger-Méthé with the majority of computational work and manuscript writingconducted by Evan Sidrow. Dr. Sarah Fortune assisted with data collection. Dr. BethVolpov and Tess McRae conducted much of the data processing and most of the datalabelling. Dr. Andrew Trites provided helpful comments and edited the manuscript.A version of Chapter 4 has been published [Sidrow E, Heckman N, Bouchard-Côté A,Fortune SME, Trites AW, Auger-Méthé M. Variance-reduced stochastic optimization for ef-ficient inference of hidden Markov models. Journal of Computational and Graphical Statis-tics. doi:10.1080/10618600.2024.2350476, 2024]. The ideas were jointly developed by EvanSidrow, Dr. Nancy Heckman, Dr. Marie Auger-Méthé, and Dr. Alexandre Bouchard-Côtéwith the majority of the computational work and manuscript writing conducted by EvanSidrow as lead author. Dr. Sarah Fortune assisted with data collection and manuscriptediting. Dr. Andrew Trites provided helpful comments and edited the manuscript.vTable of ContentsAbstract . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . iiiLay Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . ivPreface . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . vTable of Contents . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . viList of Tables . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xList of Figures . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xiiAcknowledgements . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xviiiDedication . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xx1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11.1 Definition of Hidden Markov Models . . . . . . . . . . . . . . . . . . . . . . 31.2 Goals and Advantages of HMMs . . . . . . . . . . . . . . . . . . . . . . . . 31.2.1 Parameter Inference . . . . . . . . . . . . . . . . . . . . . . . . . . . 41.2.2 Hidden State Decoding . . . . . . . . . . . . . . . . . . . . . . . . . 51.3 Advancements in Ecological HMMs . . . . . . . . . . . . . . . . . . . . . . 61.4 Current Challenges for HMMs . . . . . . . . . . . . . . . . . . . . . . . . . 81.5 Dissertation Outline . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 102 Modelling Multi-Scale, State-Switching Biologging Data . . . . . . . . . 122.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 122.2 Models and Parameter Estimation . . . . . . . . . . . . . . . . . . . . . . . 152.2.1 HMMs as a Base Structure . . . . . . . . . . . . . . . . . . . . . . . 152.2.2 Relaxing Conditional Independence with the CarHMM . . . . . . . 172.2.3 Incorporating Multiple Scales with the HHMM . . . . . . . . . . . . 182.2.4 Transforming Fine-scale Observations with the HMM-DFT . . . . . 20viTable of Contents2.2.5 Generalized Hierarchical Markov Models . . . . . . . . . . . . . . . 222.3 Killer Whale Case Study . . . . . . . . . . . . . . . . . . . . . . . . . . . . 232.3.1 Data Collection and Preprocessing . . . . . . . . . . . . . . . . . . . 232.3.2 Model Definition and Selection . . . . . . . . . . . . . . . . . . . . . 252.3.3 Case Study Results . . . . . . . . . . . . . . . . . . . . . . . . . . . 292.3.4 Model Validation . . . . . . . . . . . . . . . . . . . . . . . . . . . . 332.3.5 Comparison with Candidate Models . . . . . . . . . . . . . . . . . . 342.4 Simulation Study . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 352.4.1 Simulation Procedure . . . . . . . . . . . . . . . . . . . . . . . . . . 352.4.2 Simulation Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . 382.5 Discussion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 413 Incorporating Sparse Labels using Weighted Likelihoods . . . . . . . . . 433.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 433.2 Background . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 453.2.1 Hidden Markov Models . . . . . . . . . . . . . . . . . . . . . . . . . 453.2.2 Semi-Supervised Mixture Models . . . . . . . . . . . . . . . . . . . 463.2.3 Weighted Likelihood for Semi-Supervised Mixture Models . . . . . . 483.3 Weighted Likelihood for Semi-Supervised Learning in Hidden Markov Models 493.4 Case Studies . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 523.4.1 Case Study 1: Behavioural Classification of Killer Whale Dives . . . 533.4.2 Case Study 2: Identification of Killer Whale Foraging . . . . . . . . 573.5 Discussion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 634 Efficient Inference using Variance-Reduced Stochastic Optimization . 674.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 674.2 Background . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 694.2.1 Hidden Markov Models . . . . . . . . . . . . . . . . . . . . . . . . . 694.2.2 State Decoding . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 704.2.3 The Baum-Welch Algorithm . . . . . . . . . . . . . . . . . . . . . . 714.2.4 Stochastic Optimization . . . . . . . . . . . . . . . . . . . . . . . . 734.3 Stochastic Optimization for HMM Inference . . . . . . . . . . . . . . . . . 744.3.1 Variance-Reduced Stochastic M Step . . . . . . . . . . . . . . . . . 744.3.2 Partial E Step within the M step . . . . . . . . . . . . . . . . . . . 764.3.3 Full Algorithm . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 784.4 Practical Considerations . . . . . . . . . . . . . . . . . . . . . . . . . . . . 824.4.1 Line Search for Step Size Selection . . . . . . . . . . . . . . . . . . . 824.4.2 Multiple Step Sizes . . . . . . . . . . . . . . . . . . . . . . . . . . . 83viiTable of Contents4.4.3 Adaptive Step Size for Fixed Lipschitz Constants . . . . . . . . . . 844.4.4 Sampling for SAGA and SVRG Without Replacement . . . . . . . . 844.5 Simulation Study . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 854.5.1 Simulation Procedure . . . . . . . . . . . . . . . . . . . . . . . . . . 854.5.2 Optimization Procedure . . . . . . . . . . . . . . . . . . . . . . . . . 854.5.3 Simulation Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . 874.6 Case Study . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 874.6.1 Data Collection and Preprocessing . . . . . . . . . . . . . . . . . . . 914.6.2 Model Formulation . . . . . . . . . . . . . . . . . . . . . . . . . . . 914.6.3 Optimization Procedure . . . . . . . . . . . . . . . . . . . . . . . . . 944.6.4 Case Study Results . . . . . . . . . . . . . . . . . . . . . . . . . . . 954.7 Discussion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 965 Conclusions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 995.1 Contributions to Statistical Methodology . . . . . . . . . . . . . . . . . . . 995.1.1 Modelling Fine-Scale Dependence . . . . . . . . . . . . . . . . . . . 995.1.2 Incorporating Rare Labels . . . . . . . . . . . . . . . . . . . . . . . 1005.1.3 Performing Efficient Inference . . . . . . . . . . . . . . . . . . . . . 1005.2 Impact Beyond Statistics . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1015.3 Future Work . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 103Bibliography . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 106AppendicesA Appendix for Chapter 2 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 123A.1 Detailed Description of the Data Simulation in Section 2.4.1 . . . . . . . . 124A.2 Likelihood of the CarHHMM-DFT . . . . . . . . . . . . . . . . . . . . . . . 126A.3 Additional Results from Chapter 2 Case Study . . . . . . . . . . . . . . . . 128A.3.1 Lag Plots . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 128A.3.2 Likelihood Surface of the CarHHMM-DFT . . . . . . . . . . . . . . 129A.3.3 Parameter Estimates . . . . . . . . . . . . . . . . . . . . . . . . . . 135A.3.4 Decoded Dive States . . . . . . . . . . . . . . . . . . . . . . . . . . 139A.3.5 Model Checking: Dive Duration (nt) . . . . . . . . . . . . . . . . . 147A.3.6 Model Checking: Acceleration (A˜∗t,t˜∗) . . . . . . . . . . . . . . . . . 150A.3.7 Model Checking: “Wiggliness” (W˜ ∗t,t˜∗) . . . . . . . . . . . . . . . . . 154A.4 Additional Results from Chapter 2 Simulation Study . . . . . . . . . . . . 157A.4.1 Simulated Data . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 157viiiTable of ContentsA.4.2 Accuracy Comparisons . . . . . . . . . . . . . . . . . . . . . . . . . 158A.4.3 Empirical Joint Distributions: Dive Duration (nt) . . . . . . . . . . 159A.4.4 Empirical Joint Distributions: Acceleration (A˜∗t,t˜∗) . . . . . . . . . . 164A.4.5 Empirical Joint Distributions: “Wiggliness” (W˜ ∗t,t˜∗) . . . . . . . . . 169A.4.6 Empirical Joint Distributions - Transition Probability Matrices (Γand Γ∗) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 173B Appendix for Chapter 3 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 179B.1 Additional Results from Chapter 3 Case Study 1 . . . . . . . . . . . . . . . 180C Appendix for Chapter 4 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 182C.1 Version 2 of EM-VRSO . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 183C.2 Proof of Theorem 4.1 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 184C.3 Proof of Theorem 4.2 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 189C.4 Additional Results from Chapter 4 Simulation Study . . . . . . . . . . . . 194C.4.1 Simulation Study Results, i = 10+ . . . . . . . . . . . . . . . . . . 194C.4.2 Simulation Study Results, i = 105 . . . . . . . . . . . . . . . . . . 209C.5 Additional Results from Chapter 4 Case Study . . . . . . . . . . . . . . . . 220C.5.1 Expanded Form of Transition Probability Matrices . . . . . . . . . 220C.5.2 Parameter Estimates . . . . . . . . . . . . . . . . . . . . . . . . . . 220C.5.3 Decoded Dive Profiles . . . . . . . . . . . . . . . . . . . . . . . . . . 223ixList of Tables2.1 Parameter estimates and standard errors for parameters associated with diveduration (nt), acceleration (A˜∗t,t˜∗), and “wiggliness” (W˜ ∗t,t˜∗) of killer whalekinematic data. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 292.2 Average decoding accuracies and training times for all models in the simula-tion study. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 38A.1 Estimates and predicted standard errors of emission parameters for the dis-tributions of dive duration, acceleration, and “wiggliness” of the killer whalecase study data under the full CarHHMM-DFT. . . . . . . . . . . . . . . . 135A.2 Estimates and predicted standard errors of emission parameters for the dis-tributions of dive duration, acceleration, and “wiggliness” of the killer whalecase study data under the HHMM-DFT. . . . . . . . . . . . . . . . . . . . . 136A.3 Estimates and predicted standard errors of emission parameters for the dis-tributions of dive duration and acceleration of the killer whale case studydata under the CarHHMM . . . . . . . . . . . . . . . . . . . . . . . . . . . 137A.4 Estimates and predicted standard errors of emission parameters for the dis-tributions of dive duration, acceleration, and “wiggliness” of the killer whalecase study data under the CarHMM-DFT. . . . . . . . . . . . . . . . . . . . 138A.5 Average decoding accuracies and training times for all models used to cate-gorize dive type and subdive state in the simulation study. . . . . . . . . . . 158A.6 Estimates and standard errors for parameters of dive duration across all fourmodels in the simulation study. . . . . . . . . . . . . . . . . . . . . . . . . . 159A.7 Estimates and standard errors for the parameters of acceleration across allfour models in the simulation study. . . . . . . . . . . . . . . . . . . . . . . 164A.8 Estimates and standard errors for the parameters of the distribution of wig-gliness across all four models in the simulation study. . . . . . . . . . . . . . 169A.9 Estimates and standard errors for the coarse-scale transition probability ma-trices across all four models in the simulation study. . . . . . . . . . . . . . 173A.10 Estimates and standard errors for fine-scale transition probability matricesacross all four models in the simulation study. . . . . . . . . . . . . . . . . . 174xList of TablesC.1 Legend connecting my notation to that of Johnson and Zhang (2013). . . . 185C.2 Maximum likelihood parameter estimates from the killer whale case studydescribed in the main text. . . . . . . . . . . . . . . . . . . . . . . . . . . . 221xiList of Figures1.1 Number of publications each year with “hidden Markov model” or “HMM”as an author keyword. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21.2 Graphical representation of an HMM. . . . . . . . . . . . . . . . . . . . . . 42.1 Dependence structure of a standard HMM, CarHMM, HHMM, and HMM-DFT. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 172.2 Dive depth and three-dimensional acceleration from a killer whale over ap-proximately 5 hours. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 252.3 Graphical representation of a CarHHMM-DFT. . . . . . . . . . . . . . . . . 282.4 Estimated gamma densities of killer whale dive duration, estimated normalconditional densities of killer whale acceleration, and estimated gamma den-sities of “wiggliness”. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 302.5 The x-component of acceleration, (y∗t,t∗)x and dive depth of a northern resi-dent killer whale for a sequence of six selected dives. . . . . . . . . . . . . . 322.6 Pseudoresiduals of “wiggliness” and weighted empirical distributions of diveduration. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 342.7 Estimated probabilities that each dive t is of type xt for six selected dives ofa simulated data set of killer whale dive behaviour. . . . . . . . . . . . . . . 402.8 Estimated probabilities that each window (tP t˜∗) corresponds to subdive statex˜∗t,t˜∗ for six selected dives of a simulated data set of killer whale dive behaviour. 413.1 Graphical representation of an HMM. . . . . . . . . . . . . . . . . . . . . . 463.2 Graphical representation of various PHMMs with various likelihood weightings. 513.3 Sensitivity, specificity, and AUC values from case study 1 associated witheach dive type for PHMMs with different values of α. . . . . . . . . . . . . . 553.4 Viterbi-decoded dives for a given sub-profile and several different PHMMsfrom the first case study. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 563.5 Visualization showing how the Markov chain Xs can evolve within the secondcase study of killer whale foraging. . . . . . . . . . . . . . . . . . . . . . . . 59xiiList of Figures3.6 Four-fold, cross-validated area under the ROC curve (AUC) values for fivePHMMs as well as the baseline method of Tennessen et al. (2019b). . . . . 613.7 State-dependent densities for change in depth, heading total variation, andnormalized jerk peak for various PHMMs. . . . . . . . . . . . . . . . . . . . 623.8 Dive profiles, observations, and the decoded probability that each window isthe “capture” state for a selected successful foraging dive in case study 2. . 643.9 Dive profiles, observations, and the decoded probability that each window isthe “capture” state for a selected dive with no foraging in case study 2. . . 654.1 Graphical representation of an HMM. . . . . . . . . . . . . . . . . . . . . . 704.2 Maximum log-likelihood minus the log-likelihood at the given epoch for aselected run of each optimization algorithm in the simulation study. . . . . 884.3 Box plots showing time to terminate, epochs to terminate, and maximumlog-likelihood minus log-likelihood at termination for each optimization algo-rithm in the simulation study. . . . . . . . . . . . . . . . . . . . . . . . . . . 894.4 Maximum log-likelihood minus log-likelihood at termination versus epochsto terminate for each optimization algorithm in the simulation study. . . . . 904.5 Depth profile for a selected killer whale off the coast of British Columbia,Canada. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 924.6 Maximum log-likelihood minus log-likelihood vs epoch and computation timefor the model from the killer whale case study. . . . . . . . . . . . . . . . . 954.7 Maximum log-likelihood minus log-likelihood at termination versus epochsto terminate for the killer whale case study. . . . . . . . . . . . . . . . . . . 96A.1 Collection of five lag plots, where each plot corresponds to a feature in thekiller whale dive data. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 128A.2 Log-likelihood of the CarHHMM-DFT as a function of the dive durationparameters for dive types i = 1P 2. . . . . . . . . . . . . . . . . . . . . . . . . 129A.3 Log-likelihood of the CarHHMM-DFT as a function of the coarse-scale tran-sition probability matrix parameters Γ),2 and Γ2,). . . . . . . . . . . . . . . 129A.4 Log-likelihood of the CarHHMM-DFT as a function of the “wiggliness” pa-rameters for subdive types i∗ = 1P 2P 3. . . . . . . . . . . . . . . . . . . . . . 130A.5 Log-likelihood of the CarHHMM-DFT as a function of the x-accelerationparameters for subdive types i∗ = 1P 2P 3. . . . . . . . . . . . . . . . . . . . . 131A.6 Log-likelihood of the CarHHMM-DFT as a function of the y-accelerationparameters for subdive types i∗ = 1P 2P 3. . . . . . . . . . . . . . . . . . . . . 132A.7 Log-likelihood of the CarHHMM-DFT as a function of the z-accelerationparameters for subdive types i∗ = 1P 2P 3. . . . . . . . . . . . . . . . . . . . . 133xiiiList of FiguresA.8 Log-likelihood of the CarHHMM-DFT as a function of the fine-scale transi-tion probability matrix parameters associated with both dive types. . . . . 134A.9 The x-component of acceleration and dive depth of a northern resident killerwhale for a sequence of six selected dives, colour-coded using the CarHHMM-DFT. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 139A.10 The probability that each dive is of a particular type or that the whale is ina particular subdive state, given the fitted CarHHMM-DFT. . . . . . . . . . 140A.11 The x-component of acceleration and dive depth of a northern resident killerwhale for a sequence of six selected dives, colour-coded using the HHMM-DFT.141A.12 The probability that each dive is of a particular type or that the whale is ina particular subdive state, given the fitted HHMM-DFT. . . . . . . . . . . . 142A.13 The x-component of acceleration and dive depth of a northern resident killerwhale for a sequence of six selected dives, colour-coded using the CarHHMM. 143A.14 The probability that each dive is of a particular type or that the whale is ina particular subdive state, given the fitted CarHHMM. . . . . . . . . . . . . 144A.15 The x-component of acceleration and dive depth of a northern resident killerwhale for a sequence of six selected dives, colour-coded using the CarHMM-DFT. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 145A.16 The probability that each dive is of a particular type or that the whale is ina particular subdive state, given the fitted CarHMM-DFT. . . . . . . . . . 146A.17 Empirical histogram and pseudoresidual histogram for dive duration gener-ated using the fitted CarHHMM-DFT. . . . . . . . . . . . . . . . . . . . . . 147A.18 Empirical histogram and pseudoresidual histogram for dive duration gener-ated using the fitted HHMM-DFT. . . . . . . . . . . . . . . . . . . . . . . . 147A.19 Empirical histogram and pseudoresidual histogram for dive duration gener-ated using the fitted CarHHMM. . . . . . . . . . . . . . . . . . . . . . . . . 148A.20 Empirical histogram and pseudoresidual histogram for dive duration gener-ated using the fitted CarHMM-DFT. . . . . . . . . . . . . . . . . . . . . . . 148A.21 Empirical histograms and pseudoresiduals for acceleration associated withthe CarHHMM-DFT. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 150A.22 Empirical histograms and pseudoresiduals for acceleration associated withthe HHMM-DFT. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 151A.23 Empirical histograms and pseudoresiduals for acceleration associated withthe CarHHMM. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 152A.24 Empirical histograms and pseudoresiduals for acceleration associated withthe CarHMM-DFT. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 153xivList of FiguresA.25 Empirical histograms and pseudoresiduals for “wiggliness” associated withthe CarHHMM-DFT. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 154A.26 Empirical histograms and pseudoresiduals for “wiggliness” associated withthe HHMM-DFT. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 155A.27 Empirical histograms and pseudoresiduals for “wiggliness” associated withthe CarHMM-DFT. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 156A.28 Acceleration data for five dives of a selected simulated data set. . . . . . . . 157A.29 Kernel density estimate of the distribution of the dive duration parameterestimates associated with the CarHHMM-DFT. . . . . . . . . . . . . . . . . 160A.30 Kernel density estimate of the distribution of the dive duration parameterestimates associated with the HHMM-DFT. . . . . . . . . . . . . . . . . . . 161A.31 Kernel density estimate of the distribution of the dive duration parameterestimates associated with the CarHHMM. . . . . . . . . . . . . . . . . . . . 162A.32 Kernel density estimate of the distribution of the dive duration parameterestimates associated with the CarHMM-DFT. . . . . . . . . . . . . . . . . . 163A.33 Kernel density estimates of the distribution of acceleration parameter esti-mates associated with the CarHHMM-DFT for each subdive state. . . . . . 165A.34 Kernel density estimates of the distribution of acceleration parameter esti-mates associated with the HHMM-DFT for each subdive state. . . . . . . . 166A.35 Kernel density estimates of the distribution of acceleration parameter esti-mates associated with the CarHHMM for each subdive state. . . . . . . . . 167A.36 Kernel density estimates of the distribution of acceleration parameter esti-mates associated with the CarHMM-DFT for each subdive state. . . . . . . 168A.37 Kernel density estimate of the distributions of wiggliness parameter estimatesassociated with the CarHHMM-DFT for each subdive state. . . . . . . . . . 170A.38 Kernel density estimate of the distributions of wiggliness parameter estimatesassociated with the HHMM-DFT for each subdive state. . . . . . . . . . . . 171A.39 Kernel density estimate of the distributions of wiggliness parameter estimatesassociated with the CarHMM-DFT for each subdive state. . . . . . . . . . . 172A.40 Kernel density estimates of the distributions of Γˆ, Γˆ∗()), and Γˆ∗(2) for theCarHHMM-DFT. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 175A.41 Kernel density estimates of the distributions of Γˆ, Γˆ∗()), and Γˆ∗(2) for theHHMM-DFT. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 176A.42 Kernel density estimates of the distributions of Γˆ, Γˆ∗()), and Γˆ∗(2) for theCarHHMM. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 177A.43 Kernel density estimates of the distribution of Γˆ∗()) for the CarHMM-DFT. 178xvList of FiguresB.1 Sensitivity, specificity, and AUC values associated with each dive type fromthe first case study. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 180B.2 Viterbi-decoded dives for a given sub-profile and several different PHMMs. 181C.1 Box plots showing time to terminate, epochs to terminate, and maximumlog-likelihood minus log-likelihood at termination for each optimization algo-rithm in the simulation study. . . . . . . . . . . . . . . . . . . . . . . . . . . 195C.2 Maximum log-likelihood minus the log-likelihood at termination (all over i )versus epochs to termination for all simulation studies with i = 10+. . . . . 196C.3 Maximum log-likelihood minus the log-likelihood at termination (all over i )versus time to termination for all simulation studies with i = 10+. . . . . . 197C.4 Optimally gap between the current log-likelihood and optimal log-likelihoodvs epoch for the first simulated data set associated with experiments i = 10+.199C.5 Optimally gap between the current log-likelihood and optimal log-likelihoodvs time for the first simulated data set associated with experiments i = 10+. 200C.6 Optimally gap between the current log-likelihood and optimal log-likelihoodvs epoch for the second simulated data set associated with experiments i = 10+.201C.7 Optimally gap between the current log-likelihood and optimal log-likelihoodvs time for the second simulated data set associated with experiments i = 10+.202C.8 Optimally gap between the current log-likelihood and optimal log-likelihoodvs epoch for the third simulated data set associated with experiments i = 10+.203C.9 Optimally gap between the current log-likelihood and optimal log-likelihoodvs time for the third simulated data set associated with experiments i = 10+. 204C.10 Optimally gap between the current log-likelihood and optimal log-likelihoodvs epoch for the fourth simulated data set associated with experiments i = 10+.205C.11 Optimally gap between the current log-likelihood and optimal log-likelihoodvs time for the fourth simulated data set associated with experiments i = 10+.206C.12 Optimally gap between the current log-likelihood and optimal log-likelihoodvs epoch for the fifth simulated data set associated with experiments i = 10+.207C.13 Optimally gap between the current log-likelihood and optimal log-likelihoodvs time for the fifth simulated data set associated with experiments i = 10+. 208C.14 Optimally gap between the current log-likelihood and optimal log-likelihoodvs epoch for the first simulated data set associated with experiments i = 105.210C.15 Optimally gap between the current log-likelihood and optimal log-likelihoodvs time for the first simulated data set associated with experiments i = 105. 211C.16 Optimally gap between the current log-likelihood and optimal log-likelihoodvs epoch for the second simulated data set associated with experiments i = 105.212xviList of FiguresC.17 Optimally gap between the current log-likelihood and optimal log-likelihoodvs time for the second simulated data set associated with experiments i = 105.213C.18 Optimally gap between the current log-likelihood and optimal log-likelihoodvs epoch for the third simulated data set associated with experiments i = 105.214C.19 Optimally gap between the current log-likelihood and optimal log-likelihoodvs time for the third simulated data set associated with experiments i = 105. 215C.20 Optimally gap between the current log-likelihood and optimal log-likelihoodvs epoch for the fourth simulated data set associated with experiments i = 105.216C.21 Optimally gap between the current log-likelihood and optimal log-likelihoodvs time for the fourth simulated data set associated with experiments i = 105.217C.22 Optimally gap between the current log-likelihood and optimal log-likelihoodvs epoch for the fifth simulated data set associated with experiments i = 105.218C.23 Optimally gap between the current log-likelihood and optimal log-likelihoodvs time for the fifth simulated data set associated with experiments i = 105. 219C.24 State-dependent density of change in depth for dive types 1, 2, and 3. . . . 221C.25 Dive profiles and change in depth vs time for a selected hour of dive time forkiller whale A100. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 223C.26 Dive profiles and change in depth vs time for a selected hour of dive time forkiller whale A113. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 224C.27 Dive profiles and change in depth vs time for a selected hour of dive time forkiller whale D21. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 225C.28 Dive profiles and change in depth vs time for a selected hour of dive time forkiller whale D26. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 226C.29 Dive profiles and change in depth vs time for a selected hour of dive time forkiller whale I107. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 227C.30 Dive profiles and change in depth vs time for a selected hour of dive time forkiller whale I145. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 228C.31 Dive profiles and change in depth vs time for a selected hour of dive time forkiller whale R48. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 229C.32 Dive profiles and change in depth vs time for a selected hour of dive time forkiller whale R58. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 230xviiAcknowledgementsI would like to thank my supervisors, Dr. Marie Auger-Méthé and Dr. Nancy Heckman, fortheir hard work and support in guiding me through this PhD. I am a better researcher nowbecause of them. I would also like to thank them for their kindness as supervisors.I would also like to thank Dr. Alexandre Bouchard-Côté and Dr. Jim Zidek for their mentor-ship and kindness. Their thoughtful insights greatly enriched my work and my experienceat UBC.I would also like to thank my collaborators, Dr. Andrew Trites, Dr. Sarah Fortune, Dr.Beth Volpov, and Tess McRae, who together provided valuable perspectives to ensure thatthis dissertation was grounded in ecology.I would also like to thank members of UBC’s statistical ecology research group (SERG),the animal movement reading group, and Alex Bouchard-Côté’s reading group for fruitfulconversations that helped form the ideas in this dissertation.I would like to thank my family: Susan Perrigo, Rob Sidrow, Sarah Sidrow, and AaronSidrow, for their continued love and support both during and before the PhD. I would notbe here without them. I would also like to thank Julia Ramos Bujalil for being such aradiant source of love and joy.I also want to thank all of my friends for their support and bringing meaning to my life:Dr. Jonathan Lavington, Gian Carlo Diluvi, Tom Lillis, Dr. Syd Slouka, Dr. Katie Florko,and countless others.This research was enabled in part by support provided by WestGrid (www.westgrid.ca)and Compute Canada (www.computecanada.ca). I also thank reviewers from the CanadianJournal of Statistics and the Journal of Computational and Graphical Statistics for theirconstructive comments.I acknowledge the support of the Natural Sciences and Engineering Research Council ofCanada (NSERC) as well as the support of Fisheries and Oceans Canada (DFO). Thisproject was supported by a financial contribution from the DFO and NSERC (Whale Sci-ence for Tomorrow). This work was also supported by the NSERC Discovery program underxviiiAcknowledgementsgrant RGPIN-2020-04629; the Canadian Research Chairs program for Statistical Ecology;The Canadian Statistical Sciences Institute (CANSSI); the BC Knowledge Developmentfund; the Canada Foundation for Innovation (John R. Evans Leaders Fund) under grant37715; and the University of British Columbia via the Four-Year Doctoral Fellowship pro-gram.Certain data included herein are derived from Clarivate™ (Web of Science™). © Clarivate2024. All rights reserved.xixDedicationTo my mom for teaching me that true wisdom comes from kindergarten.xxChapter 1IntroductionResearchers are often interested in studying latent processes that they cannot directly ob-serve. Estimating these hidden processes is crucial and encountered broadly. For instance,practitioners may want to estimate a person’s intended words from a recording of their voice(Bagos et al., 2003), the regime of a financial market from stock prices (Langrock et al.,2018), or the behaviour of an animal from kinematic data collected from biologging tags(McClintock et al., 2020). Consequently, deducing hidden processes from observed timeseries is a fundamental goal in statistics.Some of the most prevalent models used to infer hidden processes from observed timeseries are hidden Markov models, or HMMs. They are relatively simple and interpretablefor practitioners, but also general enough to effectively model complex data sets. Theirstructure allows researchers to infer model parameters and decipher the underlying hiddenprocess more efficiently than many other latent variable models (Zucchini et al., 2016).Due to these advantages, HMMs have been used in a wide variety of fields. One of thefirst applications was for speech recognition (Baker, 1975), in which a speaker’s intendedmeaning was predicted from a time series of acoustic measurements. Now, HMMs are usedin fields ranging from geology (Bebbington, 2007) and neuroscience (Kottaram et al., 2019)to finance (Mamon and Elliott, 2007) and computer vision (Kundu et al., 1989). Thispopularity is only growing, as the number of papers with the keywords “hidden Markovmodel” or “HMM” has been trending upward steadily from 2000 through 2022 (Figure 1.1).One application of particular interest for this dissertation is ecology and animal movementmodelling. Hidden Markov models for animal movement in particular date back at least 30years, as MacDonald and Raubenheimer (1995) used an HMM to study the effects of feedingon the behaviour of locusts (Locusta migratoria). Later, Franke et al. (2004) studied themovement of Caribou (Rangifer Tarandus) in Alberta, Canada, and Morales et al. (2004)studied relocation data of Elk (Cervus canadensis) in Ontario, Canada.More recently, the creation of R packages such as moveHMM and momentuHMM (Michelotet al., 2016; McClintock and Michelot, 2018; R Core Team, 2023) has also allowed ecologists1Chapter 1. IntroductionFigure 1.1: Number of publications each year with “hidden Markov model” or “HMM” asan author keyword, from the year 2000 to the year 2022. Citation report graphic is derivedfrom Clarivate Web of Science, Copyright Clarivate 2024. All rights reserved.to quickly and easily implement HMMs to model their biologging data. These packageshave been used to study the response of Black Bears (Ursus americanus) to anthropogeniceffects (Zeller et al., 2019), sex differences between foraging behaviours among killer whales(Orcinus orca) (Tennessen et al., 2019a), and the effect of humans on the spatial ecology ofPumas (Felis Concolor) (Nickel et al., 2021).Although HMMs are widely used in ecology, modern biologging sensors now generatecomplex datasets that present challenges for practitioners. For example, large and high-frequency kinematic data sets often violate common HMM assumptions and are compu-tationally difficult to model. Additionally, while modern technology sometimes allows fordirect observation of animal behaviours, incorporating these observations into an HMM canbe challenging. This dissertation aims to address these issues.In this chapter, I review the formal definition of an HMM, highlight common goals andadvantages of HMMs, and detail recent advancements in HMMs with a focus on ecologicalapplications. I also highlight current challenges with modelling time series data using HMMsand outline how this dissertation addresses those challenges.21.1. Definition of Hidden Markov Models1.1 Definition of Hidden Markov ModelsFirst introduced by Baum and Petrie (1966), HMMs are a common model for predictinglatent processes from observed time series data. Namely, HMMs describe an observed timeseries of length i , Y = {nt}it5), by assuming that each observation nt is generated from anunobserved hidden state mt ∈ {1P . . . P N}. The sequence of all hidden states X = {mt}it5)is modelled as a Markov chain. Beal et al. (2001) use a Dirichlet process to account fora countably infinite number of hidden states, but in this dissertation I assume that N isfinite, which is common for ecological applications.To parameterize the hidden Markov chain X, I denote the unconditional distribution ofm) by the row-vector δ =(δ()) · · · δ(c)), where δ(i) = P(m) = i). Further, I denotethe distribution of mt given mt−) for t = 2P . . . P i with the N -by-N transition probabilitymatrixΓt =Γ(),))t · · · Γ(),c)t... . . . ...Γ(c,))t · · · Γ(c,c)t P (1.1)where Γ(i,j)t = P(mt = j | mt−) = i). For simplicity, practitioners often assume that Γt doesnot change over time (i.e. Γt = Γ for all t). In this case, the HMM is time-homogeneous. IfΓ is further aperiodic and irreducible, then the HMM has a unique stationary distributionπ, where π is a row-vector such that πΓ = π. Intuitively, the stationary distribution πgives the long-term proportion of time that the hidden Markov chain spends in each latentstate.Each observation nt is a random variable, where nt given all other observations (Y \ {nt})and hidden states (X) depends only on mt. If mt = i, then the conditional density orprobability mass function of nt is f (i)(·; θ(i)), where θ(i) are the parameters describing thestate-dependent distribution of nt. The collection of all state-dependent parameters isθ = {θ(i)}ci5). Figure 1.2 shows an HMM as a graphical model. For a more completeintroduction to HMMs, see Zucchini et al. (2016).1.2 Goals and Advantages of HMMsPractitioners and researchers are often interested in accomplishing two goals with an HMM:(1) performing inference on the parameters {δPΓPθ}, and (2) predicting, or decoding thehidden states X = {mt}it5) given the observations Y. This section focuses on how HMMsprovide researchers with elegant solutions for both of these goals.31.2. Goals and Advantages of HMMsFigure 1.2: Graphical representation of an HMM. mt corresponds to an unobserved latentstate at time t whose distribution is described by a Markov chain. nt corresponds to anobservation at time t, where nt given all other observations Y \ {nt} and hidden states Xdepends only on mt.1.2.1 Parameter InferenceOne common approach for inferring the parameters of an HMM {δPΓPθ} is to performmaximum likelihood estimation. Denoting a fixed realization of Y as y = {yt}it5) and afixed realization of X as x = {xt}it5), the joint distribution of the observations y and thehidden states x isp(xP y ; δPΓPθ) = δ(x))f (x))(y); θ(x)))i∏t52Γ(xt−),xt)f (xt)(yt; θ(xt)). (1.2)One issue with Equation (1.2) is that the hidden states x are rarely directly observed.Therefore, to obtain maximum likelihood estimates (or MLEs) {δ̂P Γ̂P θ̂}, researchers sumthe joint distribtuion from Equation (1.2) over all possible values of x and maximize theresult with respect to the parameters. However, naively summing Equation (1.2) over allpossible values of x is computationally infeasible because there are Ni possible realizationsof the hidden state sequence x. For example, an HMM with N = 2 and i = 100 has2)(( = 1.27 ∗ 10+( possible realizations of x to sum over. Remarkably, the distribution ofthe observations y alone can be drastically simplified into a matrix product:p(y ; δPΓPθ) = δP (y);θ)i∏t52ΓP (yt;θ)1⊤c P (1.3)where 1c is an N -dimensional row vector of ones and P (yt;θ) is an N × N diagonal ma-trix with entry (iP i) equal to f (i)(yt; θ(i)). Evaluating Equation (1.3) is done using theforward algorithm, which has a time complexity of O(N2i ) (Rabiner, 1989). The forwardalgorithm and computational tools such as automatic differentiation and numerical opti-mization software packages make maximizing the likelihood tractable for a wide range of41.2. Goals and Advantages of HMMsHMMs (Morgenstern, 1985).As an alternative to direct numerical maximization, Baum et al. (1970) also introduced theBaum-Welch algorithm, an iterative method used to estimate the parameters of an HMM.It is a special case of the expectation-maximization (EM) algorithm (Dempster et al., 1977).For a specific class of HMMs, the M step of the EM algorithm has a closed-form solution,further speeding up inference. The class of HMMs with closed-form solutions to the M stepincludes time-homogeneous HMMs with normal or categorical state-dependent observationdistributions. However, many modern HMMs do not have closed-form solutions for the Mstep of their associated EM algorithm. I detail one approach to handle HMMs withoutclosed-form solutions in Chapter 4 of this dissertation.Although this dissertation primarily focuses on maximum likelihood estimation for parame-ter inference, many researchers instead prefer a Bayesian approach and treat the parametersδ, Γ, and θ as random variables. In the Bayesian approach, the joint likelihood of the ob-servations y and hidden states X (see Equation 1.2), or the marginal likelihood of theobservations y alone (see Equation 1.3), is multiplied by a prior distribution over the pa-rameters to obtain a posterior distribution over δ, Γ, θ, and possibly X. Then, the posteriordensity can be maximized to obtain maximum a-posteriori (MAP) estimates {δ̂P Γ̂P θ̂}. Al-ternatively, researchers can approximate the posterior distribution itself using methods suchas sequential Monte Carlo (Douc et al., 2011), variational inference (Foti et al., 2014), andMarkov-chain Monte Carlo (MCMC) (Scott, 2002). Markov chain Monte Carlo methodsfor HMMs have been of particular interest recently. For example, Ma et al. (2017) intro-duced stochastic-gradient MCMC for HMMs, and Sacchi and Swallow (2021) implementeda parallel tempering MCMC algorithm for hierarchical HMMs. For more information aboutthe trade-offs between MAP estimates and MCMC techniques, I direct the reader to Rydén(2008), who compares the advantages and disadvantages of using the EM algorithm versusMarkov-chain Monte Carlo for HMMs. While recent advances in Bayesian inference forHMMs are encouraging, many practitioners still prefer using a frequentist perspective dueto its perceived simplicity and computational advantages.1.2.2 Hidden State DecodingThe other primary goal that practitioners achieve with an HMM is to decode the hid-den states X from the observations Y. When taking a frequentist perspective, prac-titioners usually first obtain parameter estimates {δˆP ΓˆP θˆ} and assume that those esti-mates are the true parameters. Then, one of two algorithms is commonly used to de-code the hidden states X given the observations Y. The first algorithm globally decodesthe single most likely sequence of hidden states X, while the second locally calculates51.3. Advancements in Ecological HMMsthe marginal distribution of each hidden state mt for t = 1P . . . P i . In particular, theViterbi algorithm efficiently calculates the single most likely sequence of hidden statesX̂ = argmaxX P{X | Y = y ; δˆP ΓˆP θˆ} (Viterbi, 1967), while the forward-backward algo-rithm efficiently calculates P{mt = i | Y = y ; δˆP ΓˆP θˆ} for i = 1P . . . P N and t = 1P . . . P i(Baum et al., 1970). Both of these algorithms have a time complexity of O(N2i ).If practitioners take a Bayesian perspective, then both the parameters {δPΓPθ} and thehidden states X are viewed as random variables. As a result, there is no distinction betweeninference and decoding, and the two can be done simultaneously. In particular, practitionerscan obtain a joint posterior distribution over both the parameters and the hidden states bycombining a prior distribution over the parameters with the joint distribution of X and yshown in Equation (1.2). Then, they can simultaneously decode X and infer δ, Γ, and θby sampling from the resulting joint posterior distribution using methods such as MCMCor variational inference.1.3 Advancements in Ecological HMMsThe HMM structure outlined in the previous section is simple and expressive, but it isnot sufficient for many modern animal movement data sets that violate many of the corre-sponding assumptions. To meet this challenge, ecological statisticians have made significantprogress in adapting HMMs to ecological biologging data.Animals exhibit multiple behavioural processes that occur at different scales. For example,harbour porpoises (Phocoena phocoena) in the North Sea exhibit behaviours that occur overentire bouts of dives and last for hours, but the distinct dive types within each bout varyfrom dive to dive. To understand this multi-scale animal movement data, Leos-Barajas et al.(2017) propose using a hierarchical HMM, or HHMM, in which the transition probabilitymatrix Γ of a fine-scale process (i.e. the dive type of the porpoise) changes depending uponthe state of a coarse-scale process (i.e. the long-term, bout-level behaviour of the porpoise).Adam et al. (2019) extend this approach by incorporating an observation process at thecoarse-scale level in addition to the fine-scale observation process.In addition, animals change their behaviour in response to environmental covariates. Forexample, Florida panthers (Puma concolor coryi) change their behaviours depending uponthe hour of the day (Li and Bolker, 2017), and polar bears (Ursus maritimus) change theirtravelling direction depending upon wind direction (Togunov et al., 2021). Standard HMMsdo not incorporate covariates inherently, making them insufficient models for these ecolog-ical processes. However, Chapter 10 of Zucchini et al. (2016) outlines how to incorporatecovariates into both the emission distributions f (i) and probability transition matrices Γ61.3. Advancements in Ecological HMMsof an HMM. For example, Li and Bolker (2017) incorporated hour of day into the tran-sition probability matrix Γ, and Togunov et al. (2021) incorporated wind angle into thestate-dependent distribution f (i). For more information, Florko et al. (2024) described howto include environmental covariates in both the transition probability matrix Γ and thestate-dependent distributions f (i) of an ecological HMM.One additional complication with ecological data is that different individuals within a pop-ulation may have different movement dynamics. For example, a study of 54 subadult greymouse lemurs (Microcebus murinus) found that each lemur had unique feeding behaviourdepending on whether it was hungry or satiated (Schliehe-Diecks et al., 2012). In addition,a study of 37 blue whales (Balaenoptera musculus) found that each individual changed itsbehaviour differently in response to acoustic disturbances (DeRuiter et al., 2017). A naturalway to account for individual differences within time series data is to include random ef-fects into an HMM. For example, Altman (2007) incorporated random effects into both theemission distributions f (i) and the probability transition matrix Γ to model lesion countsfrom multiple sclerosis patients. Further, Schliehe-Diecks et al. (2012) incorporated randomeffects into the state-dependent distributions f (i) of an HMM while DeRuiter et al. (2017)incorporated random effects into the transition probability matrix Γ.Adding random effects within an HMM accounts for individual variation, but it also posessome challenges for researchers. First, adding random effects in the state-dependent dis-tributions can result in individuals having very different characteristics associated with thesame behaviours, which makes it more difficult to interpret those behaviours biologically(Glennie et al., 2023). Random effects also involve calculating either an integral or a sumto evaluate the likelihood function, which can be computationally expensive. To addressthese computational issues, Michelot (2023) recently introduced the hmmTMB package inR, which allows practitioners to implement random effects (and covariates) into HMMs us-ing the TMB package. Another issue is that random effects can be difficult to estimate inthe transition probability matrix. For example, Altman (2007) implemented random effectsin a case study of lesion counts for multiple sclerosis patients and found that there wasrelatively little information to estimate the random effects associated with the probabilitytransition matrix Γ. For a more detailed analysis of the trade-offs associated with incorpo-rating random effects into HMMs for animal movement studies, see McClintock (2021).Many other challenges presented by biologging data have been addressed by ecological statis-ticians. For example, animal movement data can exhibit a large degree of autocorrelation,even after conditioning on the animal’s behavioural state. To address this, Lawler et al.(2019) introduced the CarHMM, which models an observation nt to depend upon both itscorresponding hidden state mt as well as the previous observation nt−). Further, ecologists71.4. Current Challenges for HMMsare often interested in the relative probability that an animal uses different habitat types,which may vary based upon the behaviour of that animal. To address this question, Nicosiaet al. (2017) and Klappstein et al. (2023) incorporated HMMs into another well-known eco-logical statistical model known as a step selection function (Fieberg et al., 2021). Anotherchallenge is that the state-dependent densities in biologging studies often do not match anyparticular parametric density, which can cause researchers to overestimate the true numberof behaviours (Pohle et al., 2017). Thus, Langrock et al. (2018) outlined non-parametricHMMs for animal movement models, flexibly modelling accelerometer data from oceanicwhitetip sharks (Carcharhinus longimanus). In addition, the behavioural process of ananimal may be more complicated than a Markov chain, even after incorporating randomeffects and covariates. As a result, Langrock et al. (2012) applied a hierarchical hiddensemi-Markov model to describe the movement of multiple American Bison (Bison bison)in Saskatchewan, Canada. Ruiz-Suarez et al. (2022) also explored when to use hiddensemi-Markov models for animal movement using a case study of Merino sheep (Ovis aries).Moreover, HMMs are formulated in discrete time with equispaced observations, but realanimals move continuously in time and are often sampled at irregular time intervals thatcan be difficult to model using a discrete-time HMM. To this end, Hanks et al. (2015) used acontinuous-time Markov chain (CTMC) to describe the movement of mountain lions (PumaConcolor), and Michelot and Blackwell (2019) used a continuous-time, state-switching ran-dom walk model to describe the movement of grey seals (Halichoerus grypus). Finally,animal movement studies often exhibit a combination of many of the issues listed above,and practitioners have combined recent methods in novel ways to address those issues. Forexample, Johnson et al. (2021) combined a semi-Markov model with a continuous-timeapproach to model the movements of 15 northern fur seals (Callorhinus ursinus).1.4 Current Challenges for HMMsDespite significant advancements in HMM methodology over the past three decades, re-searchers continue to encounter challenges when modelling modern biologging data setswith HMMs. Recent innovations in tracking technology have enabled ecologists to collectan unprecedented volume of complex movement and kinematic data for a wide variety ofanimals (Patterson et al., 2017). These rich data sets provide researchers with new insightsthat can assist in animal conservation (Nathan et al., 2022). For example, ecologists can de-tect hunting behaviour (Heerah et al., 2017), understand habitat selection (Michelot et al.,2019), and develop activity budgets (Jeanniard du Dot et al., 2016). However, the com-plexity of these data sets also presents significant challenges, as statisticians must developincreasingly sophisticated modelling and inferential techniques to effectively describe these81.4. Current Challenges for HMMsintricate data sets. Glennie et al. (2023) outline several current opportunities and pitfallsfor ecological HMMs, but this dissertation primarily focuses on three key issues.First, modern sensors can collect biologging time series at extremely high frequencies. Tagscan record over 50 observations per second, resulting in time series containing millions ofobservations over the course of several hours. These time series can exhibit processes thatoccur on scales as fine as seconds (e.g., prey capture events) or as coarse as hours (e.g.,travelling between habitats). The fine-scale processes in particular can exhibit intricatedependence structures with a high degree of autocorrelation (e.g., the acceleration of ananimal at a given time is highly correlated with its acceleration 20 milliseconds later). Asmentioned in Section 1.3, HHMMs (Leos-Barajas et al., 2017; Adam et al., 2019) can modelrelated behavioural processes on multiple scales, and CarHMMs (Lawler et al., 2019) canmodel the auto-correlation within fine-scale processes. However, the models described so farcannot easily capture more complicated dependence structures on very short time scales.For example, Adam et al. (2019) fail to capture fine-scale periodic swimming pattern ofhorn sharks (Heterodontus francisci) using a standard HHMM. Thus, large, fine-scale datasets continue to challenge ecologists who aim to model fine-scale animal behaviours.Second, modern technology allows practitioners to directly observe some hidden states mtfor a subset of time indices T ⊆ {1P . . . P i}. For example, some ecological studies use dronefootage or animal-borne cameras to directly observe behaviours of interest (McRae et al.,2024). Using labels within an HMM has been shown to improve its predictive performance(Krogh, 1997), and in some scenarios ignoring unlabelled data altogether can counterintu-itively improve model performance (Singh et al., 2008). However, in many applications, thevast majority of observations are unlabelled due to limitations in battery life or the rangeof tags. Extremely rare labels often have a negligible impact on the parameter estimatesof a statistical model (Chapelle et al., 2006), so it remains an issue in ecology to properlybalance the effect of labelled versus unlabelled data in the likelihood of an HMM.Finally, parameter estimation for HMMs that model complex animal behaviours can becomputationally expensive, especially when the underlying data set is very large. Manypractitioners use either numerical maximization or the EM algorithm to obtain maximumlikelihood estimates, both of which require iterating though the full set of observations toupdate the parameter estimates. This process is computationally infeasible for very largedata sets. As such, there is a need for statisticians to find inference methods that remainefficient even as the number of observations i grows very large.The case study used in this dissertation exhibits all three of the issues listed above. Itinvolves modelling the movement and behaviour of northern and southern resident killer91.5. Dissertation Outlinewhales (Orcinus orca) off the coast of British Columbia, Canada. Biologging data werecollected from eleven killer whales at a rate of 50 observations per second for several hoursfor each whale. The data include fine-scale signals such as individual tail flutters and preycapture events that occur for seconds (Cole et al., 2021; Tennessen et al., 2023), and coarse-scale signals such as foraging and resting behaviours that occur for hours (McRae et al.,2024). In addition, whale ecologists directly labelled behaviours (e.g., foraging and preycapture events) using drone footage and tag-borne video and audio recorders. Finally, thelarge biologging data set contains millions of high-dimensional observations and provides acomputational challenge when fitting traditional HMMs.Extracting the most information from this rich data set has meaningful ecological and con-servation significance, as the northern resident population is designated as threatened andthe southern resident population is classified as endangered (Fisheries and Oceans Canada,2018). Fitting accurate statistical models of killer whale behaviours such as foraging helpsresearchers understand each killer whale’s energy expenditure and caloric intake (Greenet al., 2009; Jeanniard du Dot et al., 2016; Wilson et al., 2019). These models can alsoestimate discrepancies between the two populations and aid in conservation efforts for thenorthern and southern residents (Noren, 2011). Motivated by this case study, this disser-tation addresses the three challenges described above.1.5 Dissertation OutlineThis dissertation is organized into a total of five chapters. As allowed by the guidelinesof the Faculty of Graduate and Postdoctoral Studies at UBC, Chapters 2 and 4 are takenfrom published articles and Chapter 3 is formatted as a journal submission. In Chapter2, I address the first problem from Section 1.4, which relates to modelling multi-scale andhigh-frequency time series. I review HMMs as well as several variations recently introducedin the animal movement literature. I describe a novel approach to synthesize ecologicalHMMs with methods from signal processing to model fine-scale animal movement. Usingthis approach, I construct several movement models and compare their performance onkinematic data from a northern resident killer whale in 2019. In Chapter 3, I address thesecond problem from Section 1.4: incorporating rare labels into HMMs. I highlight theadvantages of using labelled data in an HMM and propose a novel weighting approach tobalance the effect of labelled and unlabelled observations in the likelihood of an HMM. Iuse this approach to fit two HMMs that predict foraging behaviour and success in residentkiller whales. In Chapter 4, I address the final problem from Section 1.4 involving scalableand efficient inference for HMMs. I review several inference methods for HMMs and discusstheir computational complexity. I then take inspiration from the stochastic optimization101.5. Dissertation Outlineliterature to propose a new inference method that allows for more efficient inference ofcomplex HMMs. In Chapter 5, I summarize my results, discuss their broader significance,and propose further work.11Chapter 2Modelling Multi-Scale,State-Switching Biologging DataA version of this chapter appears in the Canadian Journal of Statistics under the title “Mod-elling multi-scale, state-switching functional data with hidden Markov models.” Computercode associated with the chapter can be found at https://github.com/evsi8432/CarHHMM.2.1 IntroductionBiologging technology now provides researchers with kinematic data collected almost con-tinuously in time (Hooten et al., 2017). The collection and analysis of data from devicessuch as accelerometers have brought new insights to research tasks ranging from monitoringmachine health (Getman et al., 2009) to understanding physical activity levels in children(Morris et al., 2006). The study of animal movement in particular has been transformed bytracking devices that record kinematic information in a variety of environments (Jeanniard-du Dot et al., 2016; Börger et al., 2020). Tags can record over 50 observations per second,resulting in time series containing millions of observations over the course of several hours.These data contain a wealth of information about human and animal behaviour, but mod-elling these large data sets poses a challenge for statisticians and biologists. One particulardifficulty is that high-frequency data sets often contain multiple related processes in time.These processes exist on a continuum of time scales as fine as the sampling frequency ofthe biologger or as coarse as the length of the entire data set. Each process may have aparticular physical meaning (e.g., it may correspond to a sequence of individual dives), andeach process exhibits unique and complicated dependence structures. Although the precisetime-scales of the processes are general, I refer to coarse- and fine- scales processes in thischapter to highlight a hierarchical structure.Biologging data are frequently modelled as a set of curves and analyzed using methods forfunctional data analysis, or FDA (e.g., Ramsay and Silverman, 2005). For example, Morriset al. (2006) view a child’s activity level as a set of daily curves where metabolic activity is122.1. Introductiona function of time. Similarly, Fu and Heckman (2019) view the dive profile of a southernelephant seal (Mirounga leonina) as a set of dive curves. Dive amplitude and phase variationare used for classification into “dive types”.FDA was originally developed to process curves assumed to be independent replicates (i.e.,there is no between-curve dependence). Within-curve, fine-scale structure is not usuallyincorporated in FDA models. However, sets of curves often exhibit complex sequentialdependencies, both between and within curves. This is especially the case for biologgingdata (Leos-Barajas et al., 2017). On a coarse scale, the dive profiles of marine animals showa discrete number of distinct dive types, where the sequence of dive types exhibits state-switching behaviour (Tennessen et al., 2019a). On a fine scale, these profiles can displaybouts of short-term periodicity within each dive (Adam et al., 2019). Fine-scale periodicitynested within a coarser state-switching process is also common in fields such as machinehealth (Xin et al., 2018; Lucero et al., 2019) and speech recognition (Juang and Rabiner,1991).Some FDA models account for between-curve dependence that occurs when multiple curvesarise from separate groups of individuals, but these models are inadequate when modellingcertain kinds of time dependencies. For example, previous studies have used multi-levelmodels with random effects to model variation between and within individuals in the dailyactivity levels of children (Morris et al., 2006) or in the menstrual cycles of adults (Brumbackand Rice, 1998). More recent work involving multi-level models includes that of Crainiceanuet al. (2009), Di et al. (2009), and Chen and Müller (2012). However, the first two papersdo not account for temporal between-curve dependence and the third article’s model oftemporal dependence assumes that curves evolve smoothly in time. This is not appropriatein many biologging applications, where behaviours often change suddenly between a discretenumber of types. In addition to multi-level models, FDA researchers have used functionaltime series to model dependence in a sequence of curves. Functional time series extends theideas of classic time series to model the evolution of one curve into the next (Kokoszka andReimherr, 2018), but does not account for sequences of time series curves whose distributionsare determined by a discrete number of well-defined hidden states.Traditional FDA techniques similarly fall short when modelling complicated within-curvedata. In particular, within-curve structure is usually modelled by a generic smooth meanfunction and a covariance function (Yao et al., 2005) or with random regression (Riceand Wu, 2001). However, time series data exhibiting both sharp behavioural changes andperiodic fine-scale structure are difficult to model with these classical FDA techniques.My goal is to identify and describe discrete behavioural states at multiple scales within132.1. Introductionfunctional data (e.g., coarse-scale curve types and fine-scale behavioural states) and tomodel the dependence structure between those states. To accomplish this, I turn to thefield of animal movement modelling (Hooten et al., 2017), where one of the most prevalenttechniques of late is the hidden Markov model, or HMM (Patterson et al., 2017; McClintocket al., 2020). HMMs interpret animal movement data as arising from a Markov chain on adiscrete number of behavioural states, allowing biologists to infer the underlying behaviourof an animal from sequential observations of its position. While ubiquitous in ecologyliterature, HMMs have seen little use in nonparametric functional modelling, with a fewnotable exceptions. In particular, Langrock et al. (2018) take a nonparametric approach tomodel the distributions of HMM observations with B-splines, while de Souza and Heckman(2014) and de Souza et al. (2017) use HMMs to model state-switching behaviour withinfunctional data. However, none of these papers account for temporal correlation takingplace on multiple scales.While useful, HMMs alone are also not sufficient to model high-frequency time series datafor three primary reasons. First, classical HMMs fail to model simultaneous behaviouralprocesses that occur at different time scales (e.g., both between and within curves). Toaddress this issue, statistical ecologists have employed hierarchical HMMs (HHMMs) (Leos-Barajas et al., 2017; Adam et al., 2019), which model both scales with conditionally depen-dent HMMs. Second, HMMs assume that subsequent observations are independent givenan underlying hidden state process, but this is often not the case when observations aretaken at extremely high frequencies. Several solutions have been proposed in the ecologyliterature, including the hidden movement Markov model (Whoriskey et al., 2016) and theconditionally autoregressive HMM, or CarHMM (Lawler et al., 2019). Third, traditionalHMMs, CarHMMs, and HHMMs cannot easily capture complicated dependence structureson short time scales. For example, Adam et al. (2019) fail to capture the fine-scale peri-odic swimming patterns of horn sharks (Heterodontus francisci) using a traditional HHMM.Heerah et al. (2017) successfully use Fourier analysis within an HMM to account for dailybehavioural cycles in marine mammals. Fourier analysis has previously been used withaccelerometer data to explain animal behaviour (Fehlmann et al., 2017; Alex Shorter et al.,2017). Thus, incorporating Fourier analysis into the structure of an HMM appears to be apromising approach to account for fine-scale periodic structures in biologging data.I combine these existing methods from statistical ecology literature in novel ways to classifyand describe state-switching behaviours in functional data while accounting for complextemporal dependence. The resulting methods make up a tool box that can be used to buildarbitrarily complex hierarchical models to explain multi-scale functional and time series datawith intricate dependence structures. I begin in Section 2.2 by describing HMMs as well142.2. Models and Parameter Estimationas two variants, CarHMMs and HHMMs, and discuss how summary statistics over movingwindows can handle fine-scale dependence structures. I also show how these methods canbe combined to analyze increasingly complex data. In Section 2.3 I fit several candidatemodels to data from a killer whale (Orcinus orca) from the threatened northern residentpopulation off the coast of British Columbia, Canada. Section 2.4 details a simulation studybased on these candidate models, and in Section 2.5, I discuss my results.2.2 Models and Parameter EstimationConsider a sequence of i curves indexed in time by t ∈ {1P . . . P i}. Curve t is characterizedby a curve-level (or coarse-scale) observation nt as well as a sequence of i ∗t within-curve(or fine-scale) observations n ∗t . Namely, n ∗t ={n ∗t,)P . . . P n ∗t,i ∗t}is made up of fine-scalequantities derived from curve t and indexed by t∗. Both nt and n ∗t,t∗ can be either vectors orscalars. I call the sequence of coarse-scale observations n ={n)P . . . P ni}and the collectionof all fine-scale observations n ∗ ={n ∗) P . . . P n ∗i}. I assume that the curves n)P . . . P ni areindexed according to the order in which they are observed in time, but the i observationsneed not be equally spaced in time. To develop my model for this data, I detail the structureof a traditional HMM followed by three variations which generalize its base structure. Ithen show how each of these generalized HMMs can be synthesized to form a wide varietyof more-complicated models. I will investigate the trade-offs between simpler and morecomplex models in Section 2.4, but here I focus on defining the models themselves.2.2.1 HMMs as a Base StructureHMMs describe state-switching Markovian processes in discrete time and are the core struc-ture I use to model both n and n ∗. For simplicity I focus on n to introduce the model.An HMM is composed of a sequence of unobserved states m ={m)P . . . P mi}together withan observation sequence n ={n)P . . . P ni}, where mt is associated with the observationnt. The nts are often referred to as “emissions” and t is a discrete index. The mts form aMarkov chain and can take integer values between 1 and N . Their distribution is governedby the distribution of the initial state m) and an N × N transition probability matrix Γ,where Γij = Pr(mt+) = j | mt = i). I consider only time-homogeneous Markov chains,meaning that Γ does not depend on time. I assume that m) follows the chain’s stationarydistribution, which is denoted by an N -dimensional row vector δ, where δi = Pr(m) = i).A Markov chain’s stationary distribution is determined by its transition probability ma-trix via δ = δΓ where ∑ci5) δi = 1. The distribution of an emission nt conditioned onthe corresponding hidden state mt does not depend upon any other observation or hiddenstate. If mt = i, then I denote the conditional density or probability mass function of nt152.2. Models and Parameter Estimationas f (i)(·; θ(i)) or simply f (i)(·), where θ(i) is a state-dependent parameter describing theemission distribution.Suppose that fixed realizations of the observations y = {y)P . . . P yi } and the hidden statesx = {x)P . . . P xi } are known. Then, the complete data likelihood of an HMM is given byLHMM(θPΓ; yP x) = δx)f (x))(y); θ(x)))i∏t52Γxt−)xtf(xt)(yt; θ(xt)). (2.1)However, the hidden states x are rarely directly observed. As such, it is tractable to sumover all possible values of x to obtain the incomplete data likelihoodLHMM(θPΓ; y) = δP (y); θ)i∏t52ΓP (yt; θ)1c P (2.2)where 1c is an N -dimensional column vector of ones and P (yt; θ) is an N × N diagonalmatrix with the (iP i)th entry f (i)(yt; θ(i)). This expression can be evaluated in O(i ) timeusing the well-known forward algorithm (Zucchini et al., 2016).I take a frequentist approach in this chapter, which involves obtaining maximum likeli-hood estimates {θˆP Γˆ} from Equation (2.2). The estimated probability of each hidden statePˆr(mt = i | n = y) can then be calculated using {θˆP Γˆ} and the forward–backward algo-rithm (Zucchini et al., 2016). Alternatively, it is reasonable to use a Bayesian approach,where Equation (2.1) is combined with a prior distribution to obtain a posterior distributionover m, θ, and Γ. This posterior distribution can be sampled from using methods such assequential Monte Carlo (Douc et al., 2011), Markov-chain Monte Carlo (Scott, 2002), orvariational inference (Foti et al., 2014).Following Leos-Barajas et al. (2017), I reparameterize the N × N transition probabilitymatrix Γ such that the entries of the matrix are forced to be non-negative and the rowssum to one:Γij =exp(ηij)∑ck5) exp(ηik)Pwhere iP j = 1P . . . P N and ηii is set to zero for identifiability. This formulation simplifieslikelihood maximization by removing constraints in the optimization problem. I assumethere are no covariate effects, but similar to the work of de Souza et al. (2017) for indepen-dent states and Adam et al. (2019) for Markovian states, one could incorporate covariatesinto Γ by setting ηij(zt) = β⊤ijzt for i ̸= j, where zt is a column vector of known covariates162.2. Models and Parameter Estimationand βij is a column vector of unknown regression coefficients. For simplicity, I will continueto use Γ in my notation, suppressing the reparameterization in terms of η. Figure 2.1 showsthe dependence structure of an HMM.Figure 2.1: Dependence structure of a standard HMM (top), CarHMM (middle-top),HHMM (middle-bottom), and HMM-DFT (bottom). Hidden state sequences are denotedby m on the coarse scale and by m∗ on the fine scale. Observations are denoted by n onthe coarse scale and by n ∗ on the fine scale. The coarse-scale process is not included in theHMM-DFT because moving window transformations often only apply to fine-scale data.The fine-scale observations of the HMM-DFT are transformed using a moving window anddenoted by n˜ ∗ with corresponding fine-scale hidden states m˜∗.2.2.2 Relaxing Conditional Independence with the CarHMMThe CarHMM (Lawler et al., 2019) is a generalization of the HMM, which explicitly modelsautocorrelation in the observation sequence beyond the correlation induced by the hiddenstate process. Like the traditional HMM, the CarHMM is made up of a Markov chain172.2. Models and Parameter Estimationof unobserved states {m)P . . . P mi }, each taking integer values between 1 and N . TheCarHMM also has a transition probability matrix Γ and initial distribution δ equal to thestationary distribution of Γ. Unlike the traditional HMM, the CarHMM assumes that thedistribution of nt conditioned on {m)P . . . P mi } and {n)P . . . P nt−)} depends on both mt andnt−) rather than only mt. The first emission n) is treated as a fixed initial value that doesnot depend upon m). I denote the conditional density or probability mass function of ntgiven nt−) = yt−) and mt = i by f (i)(· | yt−); θ(i)) or simply f (i)(· | yt−)). As a concreteexample, if nt is a scalar, then one may assume that nt given mt = i is normally distributedwith parameters θ(i) = {µ(i)P σ(i)P ϕ(i)}, whereE(nt | nt−) = yt−)P mt = i) = ϕ(i) yt−) + (1− ϕ(i)) µ(i)P (2.3)andV(nt | nt−) = yt−)P mt = i) = (σ(i))2. (2.4)A CarHMM following Equations (2.3) and (2.4) can be viewed as a discrete time version ofa state-switching Ornstein-Uhlenbeck process (Michelot and Blackwell, 2019). This followsin the same way that an AR(1) process is the discrete-time version of a traditional Ornstein-Uhlenbeck process.As above, the likelihood of θ and Γ corresponding to the CarHMM can be calculated usingthe forward algorithm. If y is the sequence of observations, thenLCarHMM(θPΓ; y) = δi∏t52ΓP (yt | yt−); θ)1c Pwhere P (yt | yt−); θ) is an N × N diagonal matrix with the (iP i)th entry equal to f (i)(yt |yt−); θ(i)). Figure 2.1 shows a graphical representation of the dependence structure of theCarHMM.2.2.3 Incorporating Multiple Scales with the HHMMAn HHMM accounts for processes occurring simultaneously at different scales by modellingboth the coarse-scale process and fine-scale process with either HMMs (Leos-Barajas et al.,2017; Adam et al., 2019) or CarHMMs, as defined in Sections 2.2.1 and 2.2.2. Recall that{m)P . . . P mi } is an unobserved Markov chain with N possible states, and {n)P . . . P ni } is theset of corresponding observations with state-dependent parameters θ(i) for i = 1P . . . P N . Inthe hierarchical setting, each state mt also emits a sequence of fine-scale unobserved states,m∗t = {m∗t,)P . . . P mt,i ∗t }. In turn, m∗t emits a sequence of fine-scale observations n ∗t =182.2. Models and Parameter Estimation{n ∗t,)P . . . P nt,i ∗t }. For each curve t, the parameters of the HMM (or CarHMM) describing thefine-scale process {m∗t P n ∗t } depend on the value of mt. If mt = i, then m∗t,t∗ represents one ofN∗(i) possible fine-scale behaviours associated with the coarse-scale hidden state i. Further,the components of m∗t make up a Markov chain with an N∗(i)×N∗(i) transition probabilitymatrix Γ∗(i) and an initial distribution δ∗(i) which I assume is equal to the stationarydistribution of the chain. The distribution of n ∗t,t∗ given n ∗t,t∗−) = y∗t,t∗−), m∗t,t∗ = i∗, andmt = i is governed by the parameter θ∗(i,i∗) and has a density or probability mass functiondenoted by f∗(i,i∗)(· | y∗t,t∗−); θ∗(i,i∗)) or simply f∗(i,i∗)(· | y∗t,t∗−)). I denote the set of fine-scale emission parameters corresponding to mt = i by θ∗(i) ={θ∗(i,))P . . . P θ∗(i,c∗(i))}. Insummary,1. {nPm} follows a (Car)HMM with Γ ∈ Rc×c ,2. (nt | nt−) = yt−)P mt = i) has the density f (i)(· | yt−); θ(i)),3. {n ∗t P m∗t | mt = i} follows a (Car)HMM with Γ∗(i) ∈ Rc∗(i)×c∗(i) , and4. (n ∗t,t∗ | n ∗t,t∗−) = y∗t,t∗−)P m∗t,t∗ = i∗P mt = i) has the density f∗(i,i∗)(· | y∗t,t∗−); θ(i,i∗)).Given the coarse-scale hidden state sequence m, the i +1 sets {m∗) P n ∗) }P . . . P {m∗i P n ∗i }, and{n)P . . . P ni } are assumed to be independent of one another.Forcing certain parameters to be shared can reduce the complexity and increase the inter-pretability of an HHMM. For example, since f∗(i,i∗) depends on both mt = i and m∗t,t∗ = i∗,the distribution of n ∗t,t∗ is defined by the pair (mtP m∗t,t∗). However, in my killer whale casestudy (see Section 2.3), I take N∗(i) = N∗ for all i and share the fine-scale emission pa-rameters across the N coarse-scale hidden states(i.e., θ∗(),i∗) = · · · = θ∗(c,i∗) = θ∗(·,i∗) forall i∗ = 1P . . . P N∗). As a result, the distribution of the fine-scale observation n ∗t,t∗ doesnot depend on mt, but instead is defined exclusively by m∗t,t∗ (or (·P m∗t,t∗) for notationalconsistency).Due to the nested structure of the HHMM, it is straightforward to extend the forwardalgorithm and sum over all possible coarse-scale hidden states m and all possible fine-scalehidden states m∗ in the likelihood. Let y = {y)P . . . P yi } be the sequence of observedcoarse-scale emissions and y∗ = {y∗)P . . . P y∗i } be the collection of i observed fine-scaleemission vectors. In addition, let θ∗ = {θ∗())P . . . P θ∗(c)} denote the collection of all fine-scale emission parameters and Γ∗ = {Γ∗())P . . . PΓ∗(c)} denote the collection of all fine-scale192.2. Models and Parameter Estimationtransition probability matrices. The likelihood of the observed data is thenLHHMM(θP θ∗PΓPΓ∗; yP y∗) = δPm(y)P y∗); θP θ∗PΓ∗)i∏t52ΓPm(ytP y∗t | yt−); θP θ∗PΓ∗)1c P (2.5)where Pm(ytP y∗t | yt−); θP θ∗PΓ∗) is an N × N diagonal matrix whose exact structure de-pends upon the coarse- and fine-scale models. If the coarse-scale model is an HMM,Pm(y)P y∗); θP θ∗PΓ∗) and Pm(ytP y∗t | yt−); θP θ∗PΓ∗), for t ≥ 2, both have (iP i)th entries equalto f (i)(yt)Lfine(θ∗(i)PΓ∗(i); y∗t). If the coarse-scale model is a CarHMM, Pm(y)P y∗); θP θ∗PΓ∗)has its (iP i)th entry equal to Lfine(θ∗(i)PΓ∗(i); y∗))and Pm(ytP y∗t | yt−); θP θ∗PΓ∗), for t ≥ 2,has its (iP i)th entry equal to f (i)(yt | yt−))Lfine(θ∗(i)PΓ∗(i); y∗t). The fine-scale likelihoodLfine corresponds to the likelihood of the fine-scale model, which can be either a CarHMMor an HMM. Figure 2.1 displays the dependence structure of an HHMM.2.2.4 Transforming Fine-scale Observations with the HMM-DFTIn many applications where data are collected at high frequencies, intricate dependencystructures arise within the fine-scale process that cannot be adequately modelled with theHMM variations described thus far. To handle these additional fine-scale structures, Irecommend replacing n ∗t = {n ∗t,)P . . . P n ∗t,i ∗t } with relevant statistics that summarize anynon-Markovian behaviour. To maintain the temporal structure of the fine-scale process,local summary statistics can be calculated from a moving window with the stride lengthh over the elements of n ∗t . Stride length refers to the distance between the first elementof consecutive windows, so a stride length of h implies that the first window starts at n ∗t,),the second at n ∗t,)+h, and so on. Subject matter experts are often required to determinethe specific summary statistics. For example, in Section 3.4.2, I use summary statisticsidenitfied by Tennessen et al. (2019b) to be indicative of foraging behaviour. The optimalwindow size and stride length must also be carefully considered. Larger stride lengths resultin a larger loss of information but also reduce the dimensionality of the fine-scale process,which allows for faster model fitting. In addition, setting the stride length equal to thewindow size avoids artificial residual correlation arising from overlapping windows.I am interested in fine-scale summary statistics that are interpretable to practitioners andcan effectively differentiate between fine-scale behavioural states. As a result, I use thediscrete Fourier transform (DFT) of a moving forward window with a width of h and astride of h across n ∗t :202.2. Models and Parameter EstimationDFT{n ∗t,t∗ P . . . P n ∗t,t∗+h−)}(k) =h−)∑n5(n ∗t,t∗+n exp(− i2.hkn)P (2.6)for t∗ = 1P h + 1P 2h + 1P . . . and k = 0P 1P . . . P h − 1, where i = √−1. All values in theexponential term are unitless. If n ∗t,t∗ is a vector, then the DFT is taken component-wise. Iomit the final window if t∗ + h− 1 exceeds i ∗t , denote the total number of windows by i˜ ∗t ,and index the windows with t˜∗ = 1P . . . P i˜ ∗t . Next, I calculate the transformed observationsn˜ ∗t,t˜∗ = {A˜∗t,t˜∗ P W˜ ∗t,t˜∗} asA˜∗t,t˜∗ =1hh∑n5)n ∗t,h(t˜∗−))+n and W˜ ∗t,t˜∗ =ω˜∑k5)∥∥∥DFT{n ∗t,h(t˜∗−))+)P . . . P n ∗t,ht˜∗}(k)∥∥∥22P (2.7)where ω˜ ≤ h − 1 is a problem-specific tuning parameter corresponding to the maximumrecorded frequency within each window. In words, A˜∗t,t˜∗ is the average value of n ∗t withinwindow t˜∗ and W˜ ∗t,t˜∗ is the squared two-norm of the component of the window that can beattributed to frequencies between one and ω˜ periods per window. More intuitively, W˜ ∗t,t˜∗corresponds to the “wiggliness” of the fine-scale data within curve t and window t˜∗.After performing this transformation, the entire model must be redefined since the fine-scaleHMM (or CarHMM) directly models the distribution of the summary statistics n˜ ∗t ratherthan the discretized curve n ∗t . Because n˜ ∗t exists on a coarser scale than n ∗t , there are onlyi˜ ∗t unobserved states associated with n˜ ∗t ={n˜ ∗t,)P . . . P n˜ ∗t,i˜ ∗t}. I denote these unobservedstates as m˜∗t ={m˜∗t,)P . . . P m˜∗t,i˜ ∗t}. The fine-scale transition probability matrices Γ∗(i) andprobability density functions f∗(i,i∗) correspond to m˜∗t and n˜ ∗t rather than m∗t and n ∗t , asdo all fine-scale model assumptions (e.g., conditional independence, autoregressive structurebetween observations, etc.) Fine-scale model selection and validation should therefore beadjusted accordingly.The likelihood of this model is identical to that of the original HMM or CarHMM definedin Sections 2.2.1 and 2.2.2, but n ∗ is replaced with n˜ ∗ and m∗ is replaced with m˜∗. Toclearly differentiate the models, I refer to an HMM with n˜ ∗ as observations and m˜∗ ashidden states as an HMM-DFT. Figure 2.1 displays the dependence structure of a fine-scaleHMM-DFT.Replacing n ∗t with summary statistics results in a loss of information since it involvessubstituting infinite-dimensional curves with a finite-dimensional representation. While212.2. Models and Parameter EstimationI am primarily interested in classifying and describing coarse- and fine-scale behaviour,some researchers may be less interested in interpretability and more interested in predictingfuture functional data. To this end, Aue et al. (2015) and Gao et al. (2019) both usefunctional principal component analysis (FPCA) and derive bounds on the reconstructionerror caused by dimension reduction. While these papers do not account for state-switchingbehaviour between curves or windows, deriving similar error bounds for FPCA within anHMM framework appears to be a promising direction for future research.2.2.5 Generalized Hierarchical Markov ModelsTraditional HHMMs treat both the coarse-scale and the fine-scale processes as realizationsof an HMM or CarHMM. However, the fine-scale observations of a particular curve n ∗tcan be modelled using a large variety of parametric models that admit easy-to-computelikelihoods or penalized likelihoods. As such, the fine-scale HMM likelihood term Lfinein Equation (2.5) can be replaced by the likelihood of a general fine-scale model whoseparameters depend upon the coarse-scale hidden state. For example, Bebbington (2007)and Borchers et al. (2013) investigate data sets with count onsets as observations, so theyuse variations of a Poisson process as their fine-scale model. If the fine-scale model is a simplePoisson process, then this approach is equivalent to a Markov-modulated Poisson process(Fischer and Meier-Hellstern, 1993). The fine-scale process can also be modelled similarlyto Langrock et al. (2018), who use B-splines to model the emission distribution of an HMM.This nonparametric approach uses a penalized likelihood term that can easily replace theusual fine-scale likelihood term in Equation (2.5). Another class of fine-scale models isthe set of continuous time methods such as the continuous-time HMM (CTHMM) (Liuet al., 2015b) and the state-switching Ornstein-Uhlenbeck process (Michelot and Blackwell,2019). A CTHMM may be appropriate if observations are not equally spaced in time (Liuet al., 2015b). Xu et al. (2020) model high-frequency biologging accelerometer data fromindividuals by incorporating a CTHMM into a hierarchical model similar to mine. However,they assume that individuals are partitioned into subgroups a priori whereas I use an HMMto infer coarse-scale hidden states.These examples include a few of many fine-scale models that can act as initial buildingblocks in a practitioner’s toolbox to construct increasingly complex hierarchical models. Amyriad of possible models can be built using this framework, but these models can quicklybecome complicated and computationally expensive to fit. Therefore, models should beconstructed with care to achieve an adequate fit to the data while avoiding overfitting andhigh computational costs.222.3. Killer Whale Case Study2.3 Killer Whale Case StudyTo illustrate the process of constructing a model using these building blocks, I analyze thedive behaviour of a northern resident killer whale in Queen Charlotte Sound off the coastof British Columbia, Canada, and construct several candidate models to categorize anddescribe its diving behaviour.Understanding animal behaviour is important for conservation efforts, as environmentalchanges caused by anthropogenic activity can directly impact animal behaviour (Sutherland,1998). HMMs have been used to understand how diving behaviours of various species areaffected by disturbances (e.g., DeRuiter et al., 2017; Isojunno et al., 2017). For killer whales,I am interested in categorizing different diving behaviours and identifying potential foragingdives. Northern resident killer whales feed almost exclusively on calorie-rich Chinook salmon(Oncorhynchus tshawytscha) (Ford and Ellis, 2006), which typically occur deeper and areless numerous than smaller types of salmon (Ford et al., 2009). Northern resident killerwhales therefore must expend significant amounts of energy to capture Chinook salmon(Williams and Noren, 2009; Noren, 2011; Wright et al., 2017). Acceleration data can beused to estimate an animal’s energy expenditure (Green et al., 2009; Wilson et al., 2019), butthe animal’s behavioural state must be accounted for in order to obtain accurate estimates(Jeanniard du Dot et al., 2016). Therefore, understanding both the behavioural state of thekiller whale and the distribution of acceleration within each behavioural state is needed todetermine the true energetic requirements of the animal.2.3.1 Data Collection and PreprocessingThe data I use were collected on September 2, 2019 from 12:49 p.m. to 6:06 p.m. PDTand consist of depth and acceleration over time. Observations were collected at a rateof 50 Hz using a CATS biologger (Customizable Animal Tracking Solutions, www.cats.is).Acceleration was measured in three dimensions, which together represent the complete rangeof movement of an animal (forward/backward, upward/downward, and right/left). Tri-axialacceleration readings are common in these types of tags and are often used to infer animalbehaviour such as foraging (Fehlmann et al., 2017; Wright et al., 2017; Cade et al., 2018).The act of attaching and detaching the tag caused anomalous behaviour before 1:20 p.m.and after 6:00 p.m., so observations taken during these time periods were ignored. Therewere also periods of time when the tag failed to record observations, resulting in data gapsbetween 2:25 p.m. and 2:37 p.m. and between 4:07 p.m. and 5:07 p.m. To preprocessthe data, I smooth the depth and acceleration curves by taking a moving average withina window of one-tenth of a second. I then define a killer whale “dive” as any continuousinterval of data that occurs below 0.5 meters in depth and lasts for at least 10 seconds.232.3. Killer Whale Case StudyData are preprocessed in part with the divebomb package in Python (Nunes, 2019). Thepreprocessed data contain a total of 267 dives, all of which are displayed in Figure 2.2. Eachdive is treated as one curve, and the sequence of dives makes up the coarse-scale process.Specifically, the observed coarse-scale observations y ={y)P . . . P y26/}make up a sequenceof dive durations in seconds, and the coarse-scale hidden states x ={x)P . . . P x26/}representthe corresponding dive types. The fine-scale observations for dive t are measured in unitsof m/s2 and are contained in y∗t ={y∗t,)P . . . P y∗t,i ∗t}. The fine-scale hidden states of divet, x∗t ={x∗t,)P . . . P x∗t,i ∗t}, represent the subdive behavioural states of the killer whale. Thecollection of all acceleration data is denoted by y∗ ={y∗)P . . . P y∗26/}and the collection of allunknown subdive behaviours is denoted by x∗ ={x∗)P . . . P x∗26/}.242.3. Killer Whale Case StudyFigure 2.2: Dive depth (top panel) and three-dimensional acceleration (bottom three panels)from a killer whale over approximately 5 hours. An exact physical interpretation of eachcomponent of acceleration is difficult due to variations in tag orientation. There are datagaps occurring from around hours 1.5 to 1.8 and from around hours 3.2 to 4.5. Both datagaps are excluded from analyses.2.3.2 Model Definition and SelectionThe primary goal of this case study is to jointly estimate the dive types and subdive be-haviours of the killer whale, so I only consider models with both a coarse-scale and fine-scalecomponent. Defining a suitable hierarchical model involves selecting an appropriate numberof hidden states, model structure, and set of emission distributions for both the coarse- andfine-scale observations.I do not use information criteria to select the number of dive types N since these metrics252.3. Killer Whale Case Studytend to overestimate the number of behavioural states in biological processes (Pohle et al.,2017). I instead plot the duration of each dive versus the duration of the dive preceding it(yt versus yt−) for t ≥ 2). This type of visualization is known as a lag plot. If the emissiondistributions of the hidden states are well separated, a lag plot should reveal N distinctpatterns, where each pattern corresponds to one dive type (Lawler et al., 2019). This isunfortunately not the case for the killer whale data, as there is one cluster of data centredat approximately yt = yt−) = 30 s. However, longer dives appear to be characterizedby bouts of less-“wiggly” behaviour in the acceleration data compared to shorter dives,so I choose N = 2 to differentiate these dive types. The absence of a more-principledmethod to select N highlights the importance of model validation techniques in lieu ofinformation criteria (see Section 2.3.4). Lag plots reveal no significant autocorrelationbetween dive duration observations (see Figure A.1 of Appendix A) and visual inspectionshows no obvious complicated dependence. Therefore, I select a simple HMM to modelthe coarse-scale process since neither a CarHMM nor a moving-window transformation iscalled for. Given that dive t is of type i, I assume that the dive duration nt follows agamma distribution with unknown parameters µ(i) and σ(i): E(nt | mt = i) = µ(i) andV(nt | mt = i) =(σ(i))2. This is consistent with previous studies, including Leos-Barajaset al. (2017).I then select a model corresponding to the fine-scale observations of acceleration. Similar tothe coarse model, I rely on lag plots and visual inspection to select N∗ = 3 subdive states.Although N∗ is selected heuristically, I test the validity of this model in Section 2.3.4. Incontrast to the coarse-scale observations, the fine-scale acceleration data exhibit significantsinusoidal behaviour. Thus, I transform each fine-scale observation sequence y∗t into y˜∗t usingEquation (2.7) with a window size of h = 100 (2 s) and a maximum frequency of ω˜ = 10(5 Hz). I then have that y˜∗t,t˜∗ = {a˜∗t,t˜∗ P w˜∗t,t˜∗}, where a˜∗t,t˜∗ is a three-dimensional vector ofcomponent-wise average acceleration and w˜∗t,t˜∗ is a scalar describing the “wiggliness” of aparticular window. Even after transforming the raw acceleration data, there is still strongautocorrelation within each component of a˜∗t,t˜∗ (see Figure A.1 of Appendix A). Therefore,I choose a CarHMM as the fine-scale model.I then select the specific emission distribution of n˜ ∗t,t˜∗ for all dive types and subdive states.First, I assume that W˜ ∗t,t˜∗ and all three components of A˜∗t,t˜∗ are independent of one anotherwhen conditioned on the dive types and subdive states. To reduce model complexity, I alsoassume that the three sets of fine-scale emission parameters are shared across the two divetypes(i.e., θ∗(),i∗) = θ∗(2,i∗) = θ∗(·,i∗) for i∗ = 1P 2P 3). This implies that the subdive stateswithin dive type 1 have the same interpretation as those within dive type 2. To specify theemission distribution of A˜∗t,t˜∗ , consider the sequence {A˜∗t,)P . . . P A˜∗t,i ∗t } for a particular dive262.3. Killer Whale Case Studyt. I assume that each of the three components of this sequence are normally distributedas in Equations (2.3) and (2.4), and I assume that all components are independent of oneanother when conditioned on the subdive states m˜∗t = {m˜∗t,)P . . . P m˜∗t,i˜ ∗t }. Each componentis assumed to have its own mean and variance parameters, but all components share thesame autocorrelation parameter. Thus, the distribution of A˜∗t,t˜∗ given m∗t,t˜∗ = i∗ has the pa-rameters µ∗(·,i∗)A ∈ R+, σ∗(·,i∗)A ∈ R+, and ϕ∗(·,i∗)A ∈ [0P 1]. To specify the emission distributionof W˜ ∗t,t˜∗ , I assume that given m˜∗t,t˜∗ = i∗, W˜ ∗t,t˜∗ follows a gamma distribution parameterizedby its mean µ∗(·,i∗)W and standard deviation σ∗(·,i∗)W . In addition, W˜ ∗t,)P . . . P W˜ ∗t,i ∗t are assumedto be independent of one another given the subdive state sequence{m˜t,)P . . . P m˜t,i ∗t}. I donot include W˜ ∗t,t˜∗−) in the distribution of W˜ ∗t,t˜∗ because the autocorrelation evident from thelag plot is not severe and may be explained by subsequent observations occurring withinthe same subdive state.In total, the parameters to be estimated are the transition probability matrices Γ andΓ∗ = {Γ∗())PΓ∗(2)}, the coarse-scale emission parameters θ = {µ())P σ())P µ(2)P σ(2)}, and thefine-scale emission parameters θ∗ = {θ∗(·,))P θ∗(·,2)P θ∗(·,+)}, whereθ∗(·,i∗) = {µ∗(·,i∗)A P σ∗(·,i∗)A P ϕ∗(·,i∗)A P µ∗(·,i∗)W P σ∗(·,i∗)W }.Recall that θ∗(·,i∗) is the set of parameters describing the distribution of n˜ ∗t,t˜∗ conditioned onm˜∗t,t˜∗ = i∗. I refer to this final model as the CarHHMM-DFT since it includes a CarHMM,HHMM, and DFT-based transformation. The likelihood of this model is easily calculatedusing the forward algorithm and can be maximized with respect to the parameters above (seeAppendix A for details). Figure 2.3 shows the dependence structure of the full CarHHMM-DFT.272.3. Killer Whale Case StudyFigure 2.3: Graphical representation of the CarHHMM-DFT used in the simulation andcase studies. The type of dive t is denoted by mt, and nt represents the associated diveduration. The raw acceleration vector associated with dive t and time stamp t∗ is denotedby n ∗t,t∗ . The subdive state of the killer whale during dive t and window t˜∗ is denoted bym˜∗t,t˜∗ , and the corresponding transformed observation is denoted by n˜ ∗t,t˜∗ .In addition to the CarHHMM-DFT, I consider three variations for comparison. As in thefull model, each of the following models assume that all components of n˜ ∗t,t˜∗ are conditionallyindependent of one another given the dive types and subdive states.1. An HHMM-DFT, which models the coarse-scale observations with an HMM and trans-forms the fine-scale observations using Equation (2.7) but models n˜ ∗t,t˜∗ as emissionsfrom a simple HMM rather than a CarHMM.2. A CarHHMM, which models the coarse-scale observations with an HMM, transformsthe fine-scale observations using Equation (2.7), and models A˜∗t,t˜∗ as emissions of aCarHMM.3. A CarHMM-DFT, which models the coarse-scale observations as an independent andidentically distributed sequence of dives, transforms the fine-scale observations usingEquation (2.7), and models n˜ ∗t,t˜∗ as emissions of a CarHMM.Each of the three candidate models above leaves out one important aspect of the fullCarHHMM-DFT: the HHMM-DFT assumes there is no autocorrelation between fine-scaleobservations, the CarHHMM does not incorporate “wiggliness”(W˜ ∗t,t˜∗), and the CarHMM-DFT lacks a hierarchical structure and assumes that there is only one dive type.282.3. Killer Whale Case StudyI estimate the parameters of all four models using the data shown in Figure 2.2 and directlikelihood maximization using the SciPy package in Python (Virtanen et al., 2019). Eachmodel is fit 100 times using random initializations, and I keep the parameter estimatescorresponding to the maximum likelihood of each optimization routine (Zucchini et al.,2016). Likelihood maximization is performed on the Cedar Compute Canada cluster withone CPU and 8 GB of dedicated memory. Most but not all initializations converge to thesame parameter estimates, highlighting the need to perform multiple optimizations witha variety of initial guesses. Plots of the likelihood surface near the maximum likelihoodestimates are shown in Section A.3.2 of Appendix A.2.3.3 Case Study ResultsI first report results from the full CarHHMM-DFT in detail and assess the quality of thefit. I then compare these results with those from the other candidate models.Table 2.1: Estimates and standard errors for the parameters of the distribution of diveduration (nt), acceleration (A˜∗t,t˜∗), and “wiggliness” (W˜ ∗t,t˜∗) of the killer whale kinematicdata using the full CarHHMM-DFT. Estimated means µˆ, standard deviations σˆ, and auto-correlation parameters ϕˆ are defined in Equations (2.3) and (2.4). Figures following ± referto standard errors estimated using the observed information matrix.Feature Dive type / subdive state Parameter estimateµˆ σˆ ϕˆDive Duration (s) – nt 1 27.342± 0.633 10.961± 0.560 —2 127.548± 11.341 63.888± 9.032 —x-Acc. (m/s2) –(A˜∗t,t˜∗)x1 0.449± 0.030 0.039± 0.001 0.968± 0.0022 0.210± 0.012 0.096± 0.002 0.829± 0.0073 0.232± 0.035 0.296± 0.010 0.607± 0.023y-Acc. (m/s2) –(A˜∗t,t˜∗)y1 0.450± 0.038 0.051± 0.001 0.968± 0.0022 0.437± 0.012 0.094± 0.002 0.829± 0.0073 0.366± 0.042 0.365± 0.012 0.607± 0.023z-Acc. (m/s2) –(A˜∗t,t˜∗)z1 −0.691± 0.043 0.058± 0.001 0.968± 0.0022 −0.573± 0.014 0.111± 0.002 0.829± 0.0073 −0.303± 0.041 0.354± 0.012 0.607± 0.023Wiggliness – W˜ ∗t,t˜∗1 34.015± 0.368 22.986± 0.378 —2 490.068± 5.584 502.558± 6.776 —3 9154.156± 220.765 13538.747± 354.281 —The coarse-scale parameter estimates suggest that the killer whale has at least two distinctdive behaviours (see Table 2.1 and Figure 2.4). Dive type 1 corresponds to shorter andshallower dives that likely reflect resting, travelling, and, to a lesser extent, searching forprey. Dive type 2 is longer and deeper and may be associated with behaviours such ashunting (Tennessen et al., 2019a), but it is unclear whether any of the dives in this studyare successful foraging dives. No dive in this data set has a maximum depth greater than30 m, while Wright et al. (2017), in a study of killer whales in Johnstone Strait, found that292.3. Killer Whale Case Studymost prey captures occur at depths deeper than 100 m. However, the killer whale studiedhere was tagged north of Johnstone Strait in Queen Charlotte Sound, and unpublished datacollected by the authors suggest that prey-capture events can occur near the surface. Divetype 2 could also be associated with behaviours such as socializing, which can take placeseveral meters below the surface (Tennessen et al., 2019a).Figure 2.4: Estimated gamma densities of killer whale dive duration (nt) (top), estimatednormal conditional densities of killer whale acceleration (A˜∗t,t˜∗ | A˜∗t,t˜∗−) = µ∗(·,i∗)A ) (middleand bottom left), and estimated gamma densities of “wiggliness” (W˜ ∗t,t˜∗) plotted on a log-log scale (bottom right). The densities of nt correspond to dive types 1 and 2, while thedensities of A˜∗t,t˜∗ and W˜ ∗t,t˜∗ correspond to subdive states 1, 2, and 3. The densities areestimated by fitting the CarHHMM-DFT to the case study data (see Table 2.1).302.3. Killer Whale Case StudyThe means of “wiggliness”(W˜ ∗t,t˜∗)associated with each subdive state are separated byan order of magnitude (see Table 2.1 and Figure 2.4). Subdive state 1 has the smallestmean corresponding to W˜ ∗t,t˜∗ and the smallest standard deviation corresponding to A˜∗t,t˜∗ .It also has the highest autocorrelation in A˜∗t,t˜∗ . This implies less overall activity and moreconsistent acceleration compared to the other subdive states. Subdive state 2 has a mean“wiggliness” one order of magnitude higher than that of subdive state 1 and its accelerationhas about twice the standard deviation compared to subdive state 1. The autocorrelationof acceleration is also slightly lower than in subdive state 1. I therefore hypothesize thatsubdive state 2 corresponds to fluking (active swimming), as strong sinusoidal behaviour inacceleration is characteristic of fluking in marine mammals (Simon et al., 2012). Finally,the mean of W˜ ∗t,t˜∗ and variance of A˜∗t,t˜∗ in subdive state 3 are both much higher than in theother two states, and the autocorrelation of A˜∗t,t˜∗ is also much lower. This corresponds tovigorous swimming activity, especially as the killer whale ends a dive (see Figure 2.5).312.3. Killer Whale Case StudyFigure 2.5: The x-component of acceleration, (y∗t,t∗)x (top two panels), and dive depth(bottom two panels) of a northern resident killer whale for a sequence of six selected dives.Each panel is partitioned into dives by vertical black lines. Curve colour in the first andthird panels corresponds to estimated dive type while curve colour in the second and fourthpanels corresponds to the estimated subdive state. Both dive type and subdive state areestimated by fitting the CarHHMM-DFT to the data and applying the forward–backwardalgorithm to determine the hidden state with the highest probability.The estimated transition probability matrix and associated stationary distribution on thecoarse scale (i.e., between dives) areΓˆ =(0.847 0.1530.914 0.086)(2.8)and δˆ = (0.857P 0.143). The estimated transition probability matrices and stationary distri-butions on the fine scale are322.3. Killer Whale Case StudyΓˆ∗()) =0.745 0.253 0.0020.080 0.869 0.0520.000 0.229 0.771 P Γˆ∗(2) =0.886 0.114 0.0000.150 0.815 0.0360.000 0.225 0.775 Pδˆ∗()) = (0.202P 0.649P 0.149), and δˆ∗(2) = (0.531P 0.405P 0.064) for dive types 1 and 2. Insummary, about 86% of dives are short dives of type 1. The whale performs an averageof 6.54 short type 1 dives before switching to dive type 2 and an average of 1.09 longertype 2 dives before switching back to dive type 1. This finding is consistent with those ofTennessen et al. (2019a) and Williams and Noren (2009), as both describe common boutsof short resting dives before a longer, more-energy-intensive deep dive. Further, this killerwhale is in the less-active subdive state 1 only 20% of the time during a dive of type 1,compared to 53% of the time during a dive of type 2. Less-active swimming behaviour isconsistent with the need for marine mammals to conserve energy when diving to greaterdepths and holding their breath for long periods (Williams et al., 1999; Hastie et al., 2006).Figure 2.5 shows the decoded dive behaviour of six selected dives. Section A.3.4 of AppendixA also shows the probability of each dive type and subdive state given the data and thefitted model.2.3.4 Model ValidationI use two visual tools to evaluate the CarHHMM-DFT: pseudoresidual plots and empiricalhistograms. The pseudoresidual of a coarse-scale observation yt isΦ−)(Pr(nt Q yt | {n)P . . . P ni P n˜ ∗) P . . . P n˜ ∗i } \ {nt}))and the pseudoresidual of a fine-scale observation y˜∗t,t˜∗ isΦ−)(Pr(n˜ ∗t,t∗ Q y˜∗t,t˜∗ | {n)P . . . P ni P n˜ ∗) P . . . P n˜ ∗i } \ {n ∗t,t˜}))Pwhere Φ is the cumulative distribution function of the standard normal distribution. If themodel is correct, then all pseudoresiduals are independent and follow the standard normaldistribution. Histograms of the pseudoresiduals mostly support that the CarHHMM-DFTis well specified. One exception is W˜ ∗t,t˜∗ , whose pseudoresiduals are noticeably right-skewed(see Figure 2.6). This implies that the true distribution of W˜ ∗t,t˜∗ may be heavier-tailed thanthe gamma distribution used in the case study. See Sections A.3.5–A.3.7 of Appendix A forpseudoresidual plots corresponding to all observations and models.332.3. Killer Whale Case StudyFigure 2.6: Pseudoresiduals of “wiggliness” plotted over a standard normal density (left) andweighted empirical distributions of dive duration nt plotted over the corresponding fittedgamma distributions (right). Both plots are generated by fitting the CarHHMM-DFT tothe killer whale case study data and applying the forward–backward algorithm.I also plot histograms of dive duration corresponding to each dive type in Figure 2.6. Eachobservation of dive duration is weighted by the estimated probability that it correspondsto a particular dive type as decoded by the forward–backward algorithm. This procedureresults in two histograms—one corresponding to dive type 1 and another corresponding todive type 2. Each histogram is plotted together with the corresponding emission distributionestimated by the CarHHMM-DFT. Analogous histograms corresponding to the fine-scaleobservations are contained in Sections A.3.5 and A.3.7 of Appendix A. My results mostlyshow that the CarHHMM-DFT explains the data well, but there are some exceptions.In particular, histograms corresponding to subdive state 3 show that A˜∗t,t˜∗ has heavier tailscompared to a normal distribution. This indicates the existence of rare events correspondingto exceptionally sudden changes in the acceleration of the killer whale. These outliers arepotential subjects for future study and may indicate biologically relevant phenomena suchas prey capture (Tennessen et al., 2019b).2.3.5 Comparison with Candidate ModelsThe HHMM-DFT, which ignores autocorrelation in acceleration, decodes dive types andsubdive states similarly to the CarHHMM-DFT, but is less likely to categorize any givenbehaviour as subdive state 3 (see Figures A.11 and A.12 of Appendix A). In addition, for allthree components of σ∗(·,))A , σ∗(·,2)A , and σ∗(·,+)A , the HHMM-DFT produces estimates whichare significantly larger than those of the CarHHMM-DFT. The estimated uncertaintiesof the three components of µˆ∗(·,))A , µˆ∗(·,2)A , and µˆ∗(·,+)A are also less than half of those forthe CarHHMM-DFT (see Tables A.1 and A.2 of Appendix A). This suggests that the342.4. Simulation StudyHHMM-DFT overlooks some autocorrelation in the data, and that the HHMM-DFT maybe overconfident in its parameter estimates compared to the full CarHHMM-DFT.The CarHHMM does not model the “wiggliness” of the acceleration data, so it regularly failsto pick up obvious behavioural changes corresponding to the periodicity shown in Figure2.5 (see Figure A.13 of Appendix A). These results essentially disqualify the CarHHMMas a viable model for this data set. The acceleration pseudoresiduals are also light-tailedrelative to a normal distribution (see Figure A.23 of Appendix A).Finally, the CarHMM-DFT, which lacks a hierarchical structure, produces fine-scale pa-rameter estimates and subdive state estimates similar to those of the CarHHMM-DFT.However, the former’s lack of hierarchical structure means that it fails to differentiate be-tween short and long dives. This model therefore does not infer the dive-level Markov chainor the relationship between the dive types and subdive states. For example, the CarHMM-DFT does not indicate that the whale is more likely to be in subdive state 1 when engagedin longer dives compared to shorter dives.For a more complete set of results for each of the candidate models, see Section A.3 ofAppendix A.2.4 Simulation StudyI perform a simulation study based on data generated from the full CarHHMM-DFT asdefined in Section 2.3.2 to evaluate each candidate model when the ground truth is known.The parameters used to generate the data are based on those estimated in the case study(see Table 2.1), with slight modifications made for simplicity. In particular, I set A˜∗t,t˜∗ toa scalar instead of a three-dimensional vector. I then fit all four models to the simulateddata. Metrics used to evaluate each model include hidden state decoding accuracy, bias inparameter estimates, empirical standard errors of parameter estimates, and fitting times.To assess the accuracy of uncertainty estimates, I also compare the empirical standarderrors of a given model’s parameter estimates with the standard errors estimated using theinverse of the observed Fisher information.2.4.1 Simulation ProcedureI generate 500 independent training data sets using the CarHHMM-DFT as a generativemodel. Each training data set consists of a sequence of 100 curves which I call a sequenceof killer whale dives. Each dive can be one of N = 2 dive types based on a Markov chainwith transition probability matrix352.4. Simulation StudyΓ =(0.847 0.1530.914 0.086).The transition probability matrix above is identical to the one estimated in Equation (2.8)to simulate the case study data as closely as possible. Dive duration is gamma distributedand the coarse-scale emission parameters are µ()) = 27.34 seconds, σ()) = 10.96 seconds,µ(2) = 127.55 seconds, and σ(2) = 63.89 seconds. After generating the dive durations forall 100 dives in a data set, dive t is broken into a sequence of i˜ ∗t = ⌊nt/2⌋ two-secondwindows, where the last nt − 2i˜ ∗t seconds of each simulated dive are ignored. Each two-second segment is assigned one of N∗ = 3 behaviours according to a fine-scale Markov chainm˜∗t ={m˜∗t,)P . . . P m˜∗t,i˜ ∗t}with the transition probability matricesΓ∗()) =0.745 0.253 0.0020.080 0.868 0.0520.000 0.229 0.771 and Γ∗(2) =0.886 0.139 0.0000.150 0.815 0.0350.000 0.225 0.775for dive types 1 and 2, respectively. Instead of generating the raw observations n ∗t,t∗ , Idirectly simulate the fine-scale transformed observations n˜ ∗t,t˜∗ ={A˜∗t,t˜∗ P W˜∗t,t˜∗}. Recall fromSection 2.3.2 that I must specify the mean, standard deviation, and autocorrelation pa-rameters corresponding to{A˜∗t,)P . . . P A˜∗t,i˜ ∗t}as well as the mean and standard deviationparameters corresponding to{W˜ ∗t,)P . . . P W˜ ∗t,i˜ ∗t}. I select the following parameters in linewith the results from the case study:1. µ∗(·,))A = 0.0 s, σ∗(·,))A = 0.05 s, ϕ∗(·,))A = 0.97, µ∗(·,))W = 34.01, and σ∗(·,))W = 22.99.2. µ∗(·,2)A = 0.1 s, σ∗(·,2)A = 0.1 s, ϕ∗(·,2)A = 0.83, µ∗(·,2)W = 490.06, and σ∗(·,2)W = 502.56.3. µ∗(·,+)A = 0.2 s, σ∗(·,+)A = 0.3 s, ϕ∗(·,+)A = 0.61, µ∗(·,+)W = 9154.16, and σ∗(·,+)W = 13538.75.It is not possible to uniquely reconstruct the raw accelerometer data n ∗ from n˜ ∗ alone,but I describe one possible mapping from n˜ ∗ to n ∗ in Appendix A. Figure A.28 of Ap-pendix A shows one realization of n˜ ∗ for six dives of one simulated data set along with thecorresponding reconstructed realizations of n ∗.The two simulated dive types differ in that dives of type 1 are much shorter on average (27s) than dives of type 2 (128 s). The three simulated subdive states differ primarily becauseµ∗W and σ∗W are much higher for subdive state 3 than for subdive state 2, which in turn aremuch higher than for subdive state 1. These larger parameter values correspond to muchmore vigorous and variable periodic behaviour in the acceleration data.I calculate the maximum likelihood estimates {θˆP ΓˆP θˆ∗P Γˆ∗} for all four candidate models for362.4. Simulation Studyeach of the 500 data sets using an optimization procedure similar to that of the case study.For each of the 500 training data sets, I simulate a test data set to assess how well eachmodel predicts the hidden states as follows. Each test data set consists of a sequence of100 dives and is created from the generative model with the true parameters {θPΓP θ∗PΓ∗}.To assess the coarse-scale hidden state prediction, I estimate pt(i | yP y˜∗) = Pr(mt = i |n = yP n˜ ∗ = y˜∗) for i = 1P 2 and t = 1P . . . P 100 using the test set observations (yP y˜∗)and the training set maximum likelihood estimates. These estimates are found using theforward–backward algorithm (Zucchini et al., 2016). I compare these estimated conditionalprobabilities to {x)P . . . P x)((}, the true coarse-scale state realizations in the test data, bycalculating the average dive decoding accuracy for a single training/test data set pair,∑)((t5) pˆt(xt | yP y˜∗)/100. I then report the average of this measure over the 500 training/testdata set pairs. Analogously, to assess fine-scale state prediction, I estimate p∗t,t˜∗(i∗ | yP y˜∗) =Pr(m˜∗t,t˜∗ = i | n = yP n˜ ∗ = y˜∗) for i∗ = 1P 2P 3, t˜∗ = 1P . . . P i˜ ∗t , and t = 1P . . . P 100 usingthe test set observations, the training set maximum likelihood estimates, and the forward–backward algorithm. Denoting the true fine-scale state realizations from the test data setby {x˜∗t,)P . . . P x˜∗t,i˜ ∗t }, I define the overall average subdive decoding accuracy as the averagevalue of pˆ∗t,t˜∗(x˜∗t,t˜∗ | yP y˜∗) across all simulated test data sets, dives, and windows. Theconditional probabilities are estimated according to each of the four models under studyusing the maximum likelihood estimates from the training data set in conjunction with theforward–backward algorithm.372.4. Simulation Study2.4.2 Simulation ResultsTable 2.2: Average decoding accuracies and training times for all models in the simulationstudy. Each of the four models was fit to 500 training data sets composed of 100 simulateddives and tested on test data sets also composed of 100 simulated dives. Reported valuesare averages, and figures following ± refer to sample standard deviation across the 500 datasets. “All” denotes overall average decoding accuracy.Model Training time (min) Dive type Subdive type Dive accuracy Subdive accuracyCarHHMM-DFT 156± 67All All 0.956± 0.028 0.911± 0.0061 10.973± 0.0320.846± 0.0271 2 0.918± 0.0111 3 0.860± 0.0302 10.856± 0.0970.949± 0.0102 2 0.915± 0.0152 3 0.871± 0.057HHMM-DFT 162± 64All All 0.959± 0.025 0.844± 0.0241 10.977± 0.0280.761± 0.0631 2 0.845± 0.0291 3 0.858± 0.0342 10.851± 0.0970.883± 0.0662 2 0.829± 0.0412 3 0.876± 0.061CarHHMM 258± 106All All 0.924± 0.127 0.845± 0.0181 10.933± 0.1530.689± 0.0501 2 0.900± 0.0311 3 0.673± 0.0592 10.864± 0.1340.906± 0.0222 2 0.848± 0.0332 3 0.687± 0.105CarHMM-DFT 45± 22All All — 0.912± 0.0061 1—0.854± 0.0261 2 0.921± 0.0111 3 0.860± 0.0322 1—0.943± 0.0112 2 0.919± 0.0142 3 0.878± 0.053The full CarHHMM-DFT is the best-performing model of the four candidates since it is thegenerating model. Its average dive decoding accuracy is approximately 0.96 and its averagesubdive decoding accuracy is approximately 0.91. All parameter estimates of emissiondistributions (θ and θ∗) and transition probability matrices (Γ and Γ∗) on both the coarsescale and fine scale are either comparable or favourable relative to all other models. Theempirical standard errors of all parameter estimates (θˆ, Γˆ, θˆ∗, and Γˆ∗) are well approximatedby the inverse of the observed Fisher information matrix, although the estimated standarderrors tend to be slightly smaller than the empirical standard errors. This underestimationis especially noticeable for parameters associated with the “wiggliness” W˜ ∗t,t˜∗ , where theempirical standard error can be up to five times as large as the estimated standard error.382.4. Simulation StudySee Tables A.6–A.10 of Appendix A for detailed results.The HHMM-DFT has an average dive decoding accuracy comparable to the CarHHMM-DFT (0.96), but its average subdive decoding accuracy is worse by approximately sevenpercentage points (0.84). The HHMM-DFT’s parameter estimates are comparable to theCarHHMM-DFT with the notable exception that the former greatly overestimates σ∗(·,))Aand σ∗(·,2)A and slightly overestimates σ∗(·,+)A . In addition, the estimated standard errors ofµˆ∗(·,))A , µˆ∗(·,2)A , µˆ∗(·,+)A , σˆ∗(·,))A , σˆ∗(·,2)A , and σˆ∗(·,+)A are much smaller than the associated empiricalstandard errors (see Table A.7 of Appendix A). These results suggest that estimates ofstandard deviation can be too large and estimates of standard errors can be too small whenautocorrelation is ignored. This finding is consistent with the results of the case study,where the HHMM-DFT produce larger estimates of σ∗A and smaller estimates of standarderror compared to the CarHHMM-DFT. When standard errors are underestimated, theassociated confidence intervals are too narrow, so researchers may be overconfident in theirparameter estimates.The CarHHMM is the worst-performing model in terms of its dive decoding accuracy, whichis approximately four percentage points worse than that of the CarHHMM-DFT (0.92).The former’s average subdive decoding accuracy is approximately six percentage pointsworse than the CarHHMM-DFT (0.85). This result is consistent with my expectationsbecause the CarHHMM does not model the “wiggliness” of the fine-scale process, which isthe most distinct difference between the subdive states. In addition to its relatively pooraverage decoding accuracy, the CarHHMM is also the worst of the four candidate models atestimating emission parameters and transition probability matrices. Estimates associatedwith subdive states 2 and 3 (θ∗(·,2) and θ∗(·,+)) are especially poor. See Appendix A for moredetailed results.Finally, the CarHMM-DFT is nearly identical to the CarHHMM-DFT in terms of averagesubdive decoding accuracy, fine-scale parameter biases, and both estimated and empiricalstandard error for the fine-scale parameter estimates. In addition, the time required to fitthe CarHMM-DFT is less than one-third that of the other models (see Table 2.2). However,this model cannot differentiate between dive types as it assumes that there is only one. TheCarHMM-DFT nonetheless fits a (misspecified) single gamma distribution over the diveduration of all dives. The resulting parameter estimates (µˆn and σˆn ) are highly correlated(See Figure A.32 of Appendix A).Figures 2.7 and 2.8 display six dives of one simulated data set as well as the decoded divetypes and subdive states associated with each model. The CarHHMM-DFT and CarHMM-DFT produce similar estimates of subdive state, the HHMM-DFT is slightly more likely to392.4. Simulation Studymisclassify subdive state 1, and the CarHHMM is more likely to misclassify subdive state 3.All models classify dive type with high accuracy, with the exception of the CarHMM-DFT,which does not estimate dive type.Figure 2.7: Estimated probabilities that each dive t is of type xt for six selected dives of asimulated data set of killer whale dive behaviour. Each panel is partitioned into dives byvertical black lines. Curve colour corresponds to true dive type while background colourcorresponds to true subdive state. The CarHMM-DFT is omitted because it assumes thatthere is only one dive type.402.5. DiscussionFigure 2.8: Estimated probabilities that each window (tP t˜∗) corresponds to subdive statex˜∗t,t˜∗ for six selected dives of a simulated data set of killer whale dive behaviour. Each panelis partitioned into dives by vertical black lines. Curve colour corresponds to true subdivestate while background colour corresponds to true dive type.2.5 DiscussionCurrent functional data analysis literature addresses dependence between curves either withmulti-level models (Di et al., 2009; Chen and Müller, 2012), which lack a time component, orwith functional time series, which overlook the possibility that curves have several distinct“types” (Kokoszka and Reimherr, 2018). My work addresses these issues and introduces a412.5. Discussionflexible framework to model functional time series data using HMMs. I suggest handlingtemporal dependence between curves by using either an HMM or a CarHMM to modelthe curve sequence. I then suggest viewing each individual curve as an HMM emissionwhose distribution is described by a fine-scale model. Here I use a CarHMM as the fine-scale model, but there are a wide range of possible fine-scale models, including a Poissonprocess or a continuous time approach similar to that of Michelot and Blackwell (2019).I also incorporate a moving-window transformation at the fine scale to capture intricatedependence structures. Together, the coarse- and fine-scale models make up a hierarchicalstructure that can account for simultaneous processes taking place at different time scales.Provided that the construction is not overly complex, a hierarchical model created usingmy method can be both flexible and easy to fit using maximum likelihood estimation. Mymethod is not intended to minimize prediction error for functional data, but incorporatingHMMs into autoregressive predictive models such as those described by Aue et al. (2015)and Gao et al. (2019) is a promising and natural direction for future study.I demonstrate the usefulness of this framework using a biomechanical/ecological example,where I use HMMs to classify the coarse- and fine-scale diving behaviour of a northernresident killer whale in Queen Charlotte Sound off the coast of British Columbia, Canada.My analysis gives a deeper understanding of the killer whale’s tri-axial movement and thusits behaviour and energy expenditure (Gleiss et al., 2011; Qasem et al., 2012), both of whichare important for understanding the foraging ecology and nutritional status of northernresident killer whales (Noren, 2011). My model is also applicable to many diving animalssuch as sharks (Adam et al., 2019), seals (Jeanniard du Dot et al., 2016), and porpoises(Leos-Barajas et al., 2017). In addition, since complicated state-switching processes withtemporal dependence are common in settings ranging from speech recognition (Juang andRabiner, 1991) and neuroscience (Langrock et al., 2013) to oceanography (Bulla et al., 2012)and ecology (Adam et al., 2019), I believe that researchers can adapt my methodology forthe analysis of a wide range of time series data in a variety of fields.42Chapter 3Incorporating Sparse Labels usingWeighted LikelihoodsThis chapter is written as a manuscript to be submitted in an academic journal. Computercode associated with the chapter can be found at https://github.com/evsi8432/PHMM.3.1 IntroductionUnderstanding the movement and behaviour of animals is crucial for their conservation, andone common statistical model used to do so is the hidden Markov model, or HMM (Suther-land, 1998; Ogburn et al., 2017; McClintock et al., 2020). An HMM is a generalization of amixture model that is used to decode a latent process of interest (e.g., a sequence of animalbehaviours) from an observed time series (e.g., biologging data from tags attached to theanimal). They have been used to uncover a wide variety of animal behaviours, includingforaging activity (Lusseau et al., 2009; Ylitalo et al., 2021) and habitat selection (Klappsteinet al., 2023).Many ecological studies employ unsupervised HMMs, meaning that the true behavioursof the study animals are never directly observed and instead are predicted entirely frombiologging data (Leos-Barajas et al., 2017; Patterson et al., 2017; Pirotta et al., 2018; Adamet al., 2019). However, ecologists are often interested in predicting complicated animalbehaviours (e.g., successful prey captures, Tennessen et al., 2019b) that are difficult toidentify from movement data alone. For these behaviours, the relationship between ananimal’s behaviour and its movement is so complex that it is rarely fully characterized bya statistical model.One solution is to fully observe an animal’s behaviours and incorporate them into theunderlying model, in which case the HMM is fully supervised. Krogh (1997) show thatfully supervised HMMs exhibit better predictive performance than unsupervised HMMsin various settings, and fully supervised HMMs are used in fields ranging from speechrecognition to medicine (Bagos et al., 2003; Tamposis et al., 2018b). To my knowledge fully433.1. Introductionsupervised HMMs are rare in ecology, but some animal behaviour studies use other fullysupervised machine learning techniques (Carroll et al., 2014; Allen et al., 2016). However,these studies often focus on captive animals that are much easier to continuously observecompared to wild animals.While fully observing an animal’s behaviour in the wild can be prohibitively difficult orexpensive, many ecological studies have behavioural information for a small subset of time.Occasional observations of an animal’s behaviour can be incorporated into a semi-supervisedHMM, and some notable ecological studies have used semi-supervised HMMs. For exam-ple, McClintock et al. (2012) label a subset of hidden behavioural states of a grey seal(Halichoerus grypus) using its proximity to known “haul-out” and foraging sites. Pirottaet al. (2018) assume that northern fulmars (Fulmarus glacialis) begin every journey in someknown behavioural state. McRae et al. (2024) use drone footage to directly label the be-haviour of killer whales (Orcinus orca) for a subset of observation times. Saldanha et al.(2023) use a multi-sensor approach to derive behavioural labels for red-billed tropicbirds(Phaethon aethereus). These studies demonstrate that incorporating partial labels into anecological HMM can significantly improve its performance.While behavioural labels can improve an HMM’s prediction accuracy, many ecological stud-ies only have access to labels for a small proportion of observations (e.g., Q 10%, Saldanhaet al., 2023; McRae et al., 2024). In these cases, the labelled data often do not meaning-fully affect the parameter estimates of an HMM because the likelihood is dominated by theunlabelled data (Chapelle et al., 2006; Ren et al., 2020). A study by Ji et al. (2009) uses aweighted likelihood approach to increase the influence of labelled examples, but it assumesthat labels correspond to independent time series. This approach does not apply whenlabels occur within a time series, which is often the case for ecological studies (Saldanhaet al., 2023; McRae et al., 2024).Foraging behaviour can be especially rare and difficult to identify, but it is often of primeinterest in ecology (Stephens et al., 2008; Saldanha et al., 2023). For example, understand-ing foraging behaviour is vital for the conservation of northern and southern resident killerwhales off the coast of British Columbia (Lusseau et al., 2009; Joy et al., 2019; Tennessenet al., 2023). The two sub-populations overlap in their spatial distribution and dietary pref-erences, but the population of the threatened northern residents has been steadily growingfor decades while the population of the endangered southern residents has remained stag-nant (Fisheries and Oceans Canada, 2018; Tennessen et al., 2023). Studies have shownthat various factors contribute to these population trends, including prey availability, an-thropogenic pollutants, and vessel disturbances, but the exact causal mechanisms are notfully understood (Lusseau et al., 2009; Joy et al., 2019; Murray et al., 2021). Each of these443.2. Backgroundfactors affects foraging behaviour and foraging success differently, so understanding howoften and how successfully these sub-populations hunt may help explain differences in theirgrowth (Noren, 2011; Tennessen et al., 2023).In this chapter, I introduce a novel weighted semi-supervised learning approach for hiddenMarkov models that allows practitioners to adjust the influence of sparse labels within atime series. Section 3.2 reviews the definition of an HMM and current semi-supervisedlearning techniques for mixture models that lack time dependence. Section 3.3 formalizesa partially hidden Markov model, or PHMM, which is designed to account for time-seriesthat are partially labelled, and introduces a weighted likelihood approach to balance theinfluence of labelled and unlabelled data within the model. Section 3.4 presents two casestudies that use the weighted likelihood to achieve higher cross-validated accuracy comparedto existing baseline methods. Section 3.5 discusses my results.3.2 Background3.2.1 Hidden Markov ModelsHidden Markov models describe time series that exhibit state-switching behaviour. Theymodel an observed time series of length i , Y = {nt}it5), by assuming that each observationnt is generated from an unobserved hidden state mt ∈ {1P . . . P N}. The sequence of allhidden states X = {mt}it5) is modelled as a Markov chain. The unconditional distributionof m) is denoted by the row-vector δ =(δ()) · · · δ(c)), where δ(i) = P(m) = i). Further,the distribution of mt given mt−) for t = 2P . . . P i is denoted by the N -by-N transitionprobability matrixΓt =Γ(),))t · · · Γ(),c)t... . . . ...Γ(c,))t · · · Γ(c,c)t P (3.1)where Γ(i,j)t = P(mt = j | mt−) = i). For simplicity, I assume that Γt does not change overtime (i.e. Γt = Γ for all t) unless stated otherwise.Each observation nt is a random variable, where nt given all other observations (Y \ {nt})and hidden states (X) depends only on mt. If mt = i, then the conditional density orprobability mass function of nt is f (i)(·; θ(i)), where θ(i) are the parameters describing thestate-dependent distribution of nt. The collection of all state-dependent parameters isθ = {θ(i)}ci5). A fixed realization of Y is denoted as y = {yt}it5) and has probability density453.2. BackgroundFigure 3.1: Graphical representation of an HMM. mt corresponds to an unobserved latentstate at time t whose distribution is described by a Markov chain. nt corresponds to anobservation at time t, where nt given all other observations Y \ {nt} and hidden states Xdepends only on mt.functionp(y ; δPΓPθ) = δP (y);θ)i∏t52ΓP (yt;θ)1⊤c P (3.2)where 1c is an N -dimensional row vector of ones and P (yt;θ) is an N × N diagonal ma-trix with entry (iP i) equal to f (i)(yt; θ(i)). Parameter estimation for HMMs often involvesmaximizing Equation (3.2) with respect to δ, Γ, and θ. Figure 3.1 shows an HMM as agraphical model. For a more complete introduction to HMMs, see Zucchini et al. (2016).3.2.2 Semi-Supervised Mixture ModelsSemi-supervised learning is a paradigm in machine learning that harnesses both labelledand unlabelled data to enhance model performance (Chapelle et al., 2006). There is a largetaxonomy of semi-supervised learning techniques, but here I focus on generative mixturemodels because an HMM is a generalization of a mixture model that includes serial depen-dence between its hidden states (Nigam et al., 2000; van Englan and Hoos, 2020). However,many semi-supervised learning techniques for mixture models do not account for the timedependence of HMMs. As such, I build off of the current approaches for mixture modelsand develop a novel semi-supervised learning technique for HMMs.A mixture model is a simpler version of a hidden Markov model where the hidden statesX = {mt}it5) are modelled as independent categorical random variables instead of a Markovchain. The distribution of mt is denoted by the row-vector π =(.()) · · · .(c))for allt = 1P . . . P N , where .(i) = P(mt = i). A sequence of observations y = {yt}it5) then has the463.2. Backgroundprobability density functionp(y ; πPθ) =i∏t5)(c∑i5).(i)f (i)(yt; θ(i))). (3.3)Now, suppose that a subset of time indices T ⊆ {1P . . . P i} have corresponding labelsZ = {Zt}t∈T . Labels are often observed at random times, but I assume that T is fixed, as iscommon for many semi-supervised learning techniques (Hu and Zidek, 2002; Chapelle et al.,2006). Like nt, each label Zt is a random variable generated from its corresponding hiddenstate mt. The state space of Zt is general, but for simplicity I assume that Zt ∈ {1P . . . P N}.Given all other labels (Z \ {Zt}), observations (Y), and hidden states (X), I assume thatZt depends only on mt for each t ∈ T . If mt = i, then the label Zt has probabilitymass function g(i)(·;β(i)), with parameters β(i). Denote a fixed realization of labels Z asz = {zt}t∈T . Then, the joint probability density of y and z for semi-supervised mixturemodels isp(yP z ; πPθPβ) =∏t∈T(c∑i5).(i)f (i)(yt; θ(i))g(i)(zt;β(i)))∏t/∈T(c∑i5).(i)f (i)(yt; θ(i)))P (3.4)where β = {β(i)}ci5). To write Equation (3.4) in a simpler form, I define zt = ∅ for allunlabelled observations (i.e., for all t /∈ T ) and set g(i)(∅;β(i)) = 1. This abuse of nota-tion results in a relatively simple probability density function for semi-supervised mixturemodels:p(yP z ; πPθPβ) =i∏t5)(c∑i5).(i)f (i)(yt; θ(i))g(i)(zt;β(i))). (3.5)Equation (3.5) can be used to construct a likelihood and maximized with respect to π, θ,and β to perform semi-supervised inference on mixture models (Chapelle et al., 2006). Insome scenarios, subject matter experts can identify the labels z with certainty. In this case,Zt = mt for all t ∈ T , the parameters β do not need to be inferred, and g(i) takes the formg(i)(zt) =1P zt ∈ {iP ∅}P0P zt /∈ {iP ∅}. (3.6)I define g(i) as in Equation (3.6) for the case studies. However, if subject matter experts arenot confident in their labels, I recommend parameterizing g(i) and inferring the parametersβ.473.2. Background3.2.3 Weighted Likelihood for Semi-Supervised Mixture ModelsOne issue in semi-supervised learning occurs when the number of observations i is muchlarger than the number of labels |T |. In this case, the labelled data do not meaningfullyaffect maximum likelihood parameter estimates (Chapelle et al., 2006). As a solution,Chapelle et al. (2006) introduce a parameter λ ∈ [0P 1] which represents the relative weightgiven to unlabelled observations. In particular, they define a weighted likelihood L˜λ basedon Equation (3.5) with weights w˜λ as follows:w˜λ(zt) =(1− λ) i|T | P zt ∈ {1P . . . P N}λ ii−|T | P zt = ∅ P (3.7)L˜λ(πPθPβ ; yP z) =i∏t5)(c∑i5).(i)f (i)(yt; θ(i))g(i)(zt;β(i)))w˜λ(zt). (3.8)Using this formulation, setting λ = 0 throws out all unlabelled data, setting λ = 1 throwsout all labelled data, and setting λ = (i − |T |)/i returns a likelihood that correspondsto the joint density from Equation (3.5). It is unlikely that a practitioner would prefersetting λ S (i − |T |)/i , as this weights unlabelled observations more heavily than labelledobservations. In practice, researchers often select λ by performing cross-validation with anappropriate model evaluation metric (Chapelle et al., 2006).The weighted likelihood L˜λ is a specific instance of a much more general class of relevance-weighted likelihoods that has been studied extensively. Hu and Zidek (2002) provide acomprehensive review of weighted likelihoods. Applying their paradigm to this research,the probability density of the labelled data {ntP Zt}t∈T is given by Equation (3.4), but theprobability density of the unlabelled data {nt}t/∈T is some unknown density that “resembles”the density of the labelled data in some sense. In particular, Hu and Zidek (2002) formallydefine the notion of “resemblance” using Boltzman’s entropy, and the weight λ correspondsto how much the density of the unlabelled data resembles the density of the labelled dataunder this definition. Hu (1997) and Wang (2001) prove the consistency and asymptoticnormality of maximum weighted likelihood estimators under certain regularity conditions.The relevance-weighted likelihood literature thus gives useful theoretical guarantees relatedto the weighted likelihood for mixture models. Unfortunately, these guarantees usuallyassume that the observations y are independent, which is not true for HMMs.483.3. Weighted Likelihood for Semi-Supervised Learning in Hidden Markov Models3.3 Weighted Likelihood for Semi-Supervised Learning inHidden Markov ModelsIn this section, I outline the primary contribution of this chapter and develop a weightedlikelihood approach for semi-supervised learning in HMMs. I begin by writing down theprobability density associated with a partially observed HMM. I use the same notationfrom Section 3.2.2, namely random labels Z = {Zt}t∈T , where Zt is generated from hiddenstate mt and, conditioned on X, Y, and Z \ {Zt}, Zt depends only on mt. As before, afixed realization of Z is denoted as z = {zt}t∈T , and I abuse notation by setting zt = ∅for all unlabelled observations (i.e., t /∈ T ) and g(i)(∅;β(i)) = 1. The joint density of theobservations y and labels z for an HMM is thusp(yP z ; δPΓPθPβ) = δP (y)P z);θPβ)i∏t52ΓP (ytP zt;θPβ)1⊤c P (3.9)where Pt(ytP zt;θPβ) is anN×N diagonal matrix where entry (iP i) is f (i)(yt; θ(i))g(i)(zt;β(i)).I refer to this model as a partially hidden Markov model, or PHMM.Incorporating partial labels in an HMM to define a PHMM is relatively straightforward,but defining a weighted likelihood for PHMMs is more complicated. Recall that each termin Equation (3.8) is a scalar value raised to the power of some weight. However, eachterm in Equation (3.9) is a matrix, so it is not straightforward to raise each term to thepower of a (possibly fractional) weight. While it is possible to calculate fractional powersof matrices, doing so can be computationally expensive and the result can be difficultto interpret (Higham and Lin, 2011). Alternatively, Hu et al. (2000) derive a relevance-weighted likelihood for dependent data using the same paradigm as Hu and Zidek (2002).Although their method is broadly applicable, it does not apply to HMMs. Namely, Hu et al.(2000) adopt a paradigm where each observation nt has a corresponding set of parameters{δtPΓtPθt}, and they assume that nt depends only on its corresponding parameters and theprevious observations {ns}t−)s5). However, this assumption is violated for an HMM becausent depends on mt, which clearly depends upon the previous parameters {δsPΓsPθs}t−)s5). SeeEquation (3) of Hu et al. (2000) for more details.A weighted likelihood for PHMMs should have three desired properties. First, the weightedlikelihood should reduce to Equation (3.9) for some “natural” weight, just as λ = (i−|T |)/idoes for the weighted likelihood in Equation (3.8). Second, some weight should correspondto ignoring all unlabelled data, just as λ = 0 does for the weighted likelihood in Equation(3.8). These two properties allow practitioners to intuitively select a weight that balancesa natural weighting scheme with one that completely ignores all unlabelled data. Finally,493.3. Weighted Likelihood for Semi-Supervised Learning in Hidden Markov Modelsthe weighted likelihood should be relatively simple and intuitive compared to the standardlikelihood from Equation (3.9). I thus propose the weighting parameter α ∈ [0P 1] and thefollowing weighted likelihood for partially hidden Markov models:wα(zt) =1P zt ∈ {1P . . . P N}αP zt = ∅ P (3.10)Lα(δPΓPθPβ ; yP z) = δP (y)P z);θPβ)wα(z))i∏t52ΓP (ytP zt;θPβ)wα(zt)1⊤c P (3.11)This formulation satisfies the three desired properties listed above. First, if α = 0, then theterm corresponding to an unlabelled observation t is ΓP (ytP zt;θPβ)( = Γ. Therefore, thelikelihood of a PHMM with α = 0 is identical to the likelihood of an HMM that treats allunlabelled observations as totally missing (Zucchini et al., 2016). Next, if α = 1, then theterm corresponding to an unlabelled observation t is ΓP (ytP zt;θPβ)) = ΓP (ytP zt;θPβ). Inthis case, the weighted PHMM likelihood in Equation (3.11) corresponds to the standardPHMM density in Equation (3.9). While it is technically possible, I do not recommend con-sidering α S 1 because doing so weights unlabelled observations more heavily than labelledobservations. Finally, I argue that this formulation is intuitive, as it weights unlabelledobservations using some power of α and leaves labelled observations unaltered. Figure 3.2shows graphical representations of PHMMs for several values of α with only Z), Zt−), andZt+) observed for some fixed t.I recommend that practitioners use the following cross-validation procedure to find an op-timal value of α. First, select a range of candidate values of α ∈ [0P 1]. I recommendtesting at least α ∈ {0P |T |/(i − |T |)P 1}, as α = |T |/(i − |T |) approximately balances theinfluence of labelled and unlabelled observations in the likelihood. Second, divide the timeseries data set into several “folds” for cross validation. Third, train the PHMM on the fulldata set while ignoring a fold, and use the forward-backward algorithm on the held-out fold(ignoring its associated labels) to estimate the hidden state probabilities P(mt | Y = y).Fourth, repeat the previous step for all folds to estimate the cross-validated hidden stateprobabilities for the entire data set. Finally, use the estimated probabilities and true labelstogether with standard model validation techniques to evaluate the resulting PHMM andselect the most appropriate value of α. This cross-validation procedure can be computa-tionally expensive, especially for large datasets. Practitioners can train PHMMs on all foldsin parallel to reduce computation time, and I used the Cedar Compute Canada cluster toperform cross-validation in parallel for the case studies in Section 3.4.503.3. Weighted Likelihood for Semi-Supervised Learning in Hidden Markov Models(a) Partially hidden Markov model with α = 1. This PHMM gives equal weight to all observationsin the likelihood function. This is the “natural” formulation of a PHMM with missing labels.(b) Partially hidden Markov model with α ∈ (0P 1). This PHMM weights the influence of nt, nT ,and all other observations that do not have associated observed labels. However, it does not ignorethem altogether.(c) Partially hidden Markov model with α = 0. This PHMM ignores all observations that do nothave associated label information. In particular, nt, nT , and all other observations that do not haveobserved labels are treated as unobserved.Figure 3.2: Graphical representation of various PHMMs with likelihood weightings (a)α = 1, (b) α ∈ (0P 1) and (c) α = 0. The colour (white, light grey, and dark grey) indicateshow much a given observation affects the weighted likelihood. White corresponds to treatingthe random variable as unobserved, dark grey corresponds to treating the variable as fullyobserved, and light grey corresponds to treating the random variable as observed, butweighting it in the likelihood. The latent states {mt′}it′5), observations {nt′}it′5), and labels{Zt′}it′5) are as described in Section 3.3. In this example, Z), Zt−), and Zt+) are observed,while all other labels are unobserved (i.e., Zt′ = ∅ for t′ ̸= 1P t− 1P t+ 1).513.4. Case Studies3.4 Case StudiesI conducted two case studies that use PHMMs to model the behaviour and foraging suc-cess of eleven resident killer whales (nine northern, two southern) off the coast of BritishColumbia, Canada. These case studies were primarily intended to test the predictive per-formance of the PHMM and demonstrate the process of applying it to ecological data.However, I also perform these case studies for their ecological significance. As mentionedearlier, understanding killer whale foraging and successful prey capture has been a researchfocus for years, as differences in foraging success may explain why the southern residentkiller whale population is doing poorly compared to the northern residents (Noren, 2011;Tennessen et al., 2023). Thus, the results from these case studies can help ecologists cor-rectly predict foraging behaviours that are meaningful for conservation.The data for these case studies were collected in August and September 2020 using a CATStime-depth recorder (TDR) from Customizable Animal Tracking Solutions (www.cats.is).All resident killer whales were tagged with suction-cup attached CATS tags off the coast ofBritish Columbia. These tags were deployed using an adjustable 6-8 meter carbon fiber pole.Each tag was programmed to release within 3 to 24 hours of attachment. Post-deployment,the instruments were retrieved utilizing a combination of a Wildlife Computers 363C SPOTtag (providing Argos satellite positions), a goniometer, an ultra high frequency receiver, anda yagi antenna. The tags were equipped with an array of sensors, including 3D kinematicsensors (accelerometer, magnetometer, gyroscope), a time-depth recorder, a hydrophone,and a camera. All sensors were set to sample at a frequency of 50 Hz. Depth readings werecalibrated using a MATLAB package developed by Cade et al. (2021), which allowed forthe extraction of heading, pitch, and roll, as well as three-dimensional dynamic accelerationwithin the reference frame of the killer whale.Using these data, I developed two PHMMs to identify killer whale foraging behavioursat different scales. The first PHMM estimated killer whale dive types using individualdives as observations, with some dives labelled using drone videos. The second PHMMestimated prey capture events using high-frequency biologging data as observations, withsome observations labelled using video and audio recordings of prey capture. For both casestudies, I modelled each killer whale as independent, but with shared parameters for theirassociated PHMMs.In both case studies, subject matter experts could identify labels with high confidence, soI defined g(i) according to Equation (3.6). I also used k-fold cross-validation to calculateevaluation metrics in both case studies. In particular, I divided the data from all killerwhales into k folds, removed a fold from the original dataset, and fit the model using523.4. Case Studies10 random parameter initialization and a custom version of the momentuHMM packagein R (McClintock and Michelot, 2018; R Core Team, 2023). I then used the forward-backward algorithm on the held-out fold (while ignoring its true labels) to obtain estimatedprobabilities of each hidden state (dive phase or dive type). I repeated this process for eachfold to obtain hidden state probability estimates for the entire data set.3.4.1 Case Study 1: Behavioural Classification of Killer Whale DivesFor my first case study, I assigned a latent behaviour to every killer whale dive. To thisend, I modelled the data from each killer whale as a sequence of dives and modelled eachsequence with a PHMM. The hidden Markov chain was a sequence of dive types and theobservations were summary statistics of each dive. I used three well-known killer whalebehaviours as possible dive types: resting, travelling, or foraging (McRae et al., 2024).Data ProcessingI defined a dive as any period in which the killer whale was below a depth of 0.5 metersfor at least 30 seconds, which includes only biologically meaningful dives and excludessurface behaviours. In line with previous studies (Leos-Barajas et al., 2017; McRae et al.,2024), I summarized each dive with its maximum depth and total duration. Formally, theobservation associated with whale s and dive t was denoted as ys,t = (ms,tP ys,t), where ms,tcorresponds to maximum depth in meters and ys,t corresponds to dive duration in seconds.This process resulted in time series for h = 11 killer whales, who together performed a totalof i = 2169 dives.Using drone videos, a subject matter expert visually identified three diving behaviours: rest-ing, travelling, and foraging (classification criteria are given in Table 2 of McRae et al., 2024).Formally, the label associated with whale s and dive t was denoted as zs,t ∈ {∅P 1P 2P 3}, whereeach value of zs,t corresponds to either no label (if zs,t = ∅), resting (if zs,t = 1), travelling(if zs,t = 2), or foraging (if zs,t = 3). This resulted in a total of |T | = 106 labels for divetypes.Model FormulationI used a PHMM with N = 3 dive types to match the drone-identified labels of resting,travelling, and foraging dives. For whale s and dive t, I denoted the hidden dive type asms,t ∈ {1P 2P 3}. Histograms and scatter plots revealed that dive duration and maximumdepth looked to be distributed approximately as mixtures of log-normal distributions, andthe two features were also highly correlated. Therefore, I set the joint, state-dependentdistribution of the observations to be a bivariate log-normal distribution.533.4. Case StudiesRecall that I used thresholds of ms,t ≥ 0.5 and ys,t ≥ 30 to define biologically relevant dives.However, I modelled these observations using a two-dimensional log-normal distributionwhose sample space is R2>(, so my model is misspecified. I could use a truncated multivariatelog-normal distribution instead, but many HMM software packages do not incorporatetruncated log-normal distributions by default (McClintock and Michelot, 2018; Visser andSpeekenbrink, 2010), and several ecological studies make this modelling choice as well (Leos-Barajas et al., 2017; Quick et al., 2017; Tennessen et al., 2019a). I therefore use a non-truncated log-normal distribution for simplicity and reproducibility.Again, I model the killer whale dive profiles as independent from one another, but withshared parameters. Denote the observations from whale s as ys = {ys,t}ist5) and the labelsfrom whale s as zs = {zs,t}ist5), where is is the number of dives associated with whale s.Then, the total likelihood for the model with weight α and parameters δPΓPθ and β is∏))s5) Lα(ysP zs ; δPΓPθPβ), where Lα is defined in Equation (3.11).Model Selection and EvaluationI fit five different candidate PHMMs corresponding to five different values of α. In line withthe cross-validation procedure from Section 3, I tested α = 0, which ignores all unlabelleddata; α = 1, which gives equal weight to all observations; and α = |T |/(i − |T |) = 0.049,which approximately balances the contribution of labelled and unlabelled observations. Forcompleteness, I also tested α = 0.025, which averages α = 0 and α = 0.049; and α = 0.525,which averages α = 0.049 and α = 1.I randomly divided each killer whale dive profile into two “sub-profiles” of contiguous divessuch that both sub-profiles had an equal number of labels. The resulting 22 sub-profileswere used as folds in a cross-validation scheme to estimate the probability of each dive’s typeconditioned on the observations (i.e., P(ms,t = i | Ys = ys) for s = 1P . . . P 11; t = 1P . . . P is;and i = 1P 2P 3). I then calculated the sensitivity, specificity, and area under the receiveroperating characteristic curve (AUC) associated with identifying each dive type. Sensitivityis the proportion of dives that were observed to be a certain type that were correctlyidentified as that type (i.e. the true positive rate). Specificity is the proportion of divesthat were observed to not be of a certain type that were correctly identified as not thattype (i.e. the true negative rate). AUC balances sensitivity and specificity and takes valuesbetween 0 and 1, where higher is better (Bradley, 1997). I also ran the Viterbi algorithmon each fold within the cross validation procedure and plotted the resulting dive profiles asa visual model evaluation tool (Viterbi, 1967).543.4. Case StudiesFigure 3.3: Sensitivity, specificity, and AUC values associated with each dive type. Truevalues are determined via the drone-detected dive types. The PHMMs for each value of αwere fit using the entire dataset with a selected sub-profile held out. Then, the dives forthat sub-profile were estimated by using the forward-backward algorithm on the sub-profilewith the drone-detected labels removed. I repeated this process for every sub-profile to geta full set of estimated dive types. Results for PHMMs with α = 0.025 and α = 0.525 are inAppendix B.ResultsThe PHMMs with α ∈ (0P 1) tended to obtain better results for cross-validated sensitivity,specificity, and AUC, demonstrating the effectiveness of my weighted likelihood approach(Figure 3.3). Compared to the PHMMs with α = 0 and α = 1, the PHMM with α = 0.049had the best sensitivity for foraging dives and travelling dives, and it had the second-bestsensitivity for resting dives. It had the best specificity for resting dives and travelling dives,and its specificity for foraging dives was comparable to the other models. In addition, thePHMM with α = 0.049 had an AUC for resting that was comparable or better than theother PHMMs (0.866), but it had the best AUC for foraging (0.955) and the best AUC fortravelling (0.926). Results for the PHMMs with α = 0.025 and α = 0.525 are similar to thePHMM with α = 0.049 (see Appendix B).553.4. Case Studies(a) Decoded dives for PHMM with α = 1.000(the “natural” weighting).(b) Decoded dives for PHMM with α = 0.049.(c) Decoded dives for PHMM with α = 0.000(treating the unlabelled observations as miss-ing).(d) Dive profile with true drone-detected labels.Figure 3.4: Viterbi-decoded dives for a given sub-profile and several different PHMMs. EachPHMM was fit to the dataset with the sub-profile held out. Then the Viterbi algorithmwas used on the held-out dataset with its labels removed in order to test the predictiveperformance of each PHMM.The PHMMs with α ∈ (0P 1) were also more biologically interpretable compared to thosewith α = 0 and α = 1. Namely, for the PHMMs with α ∈ (0P 1), the time series of decodeddive types contained long sequences of a single dive type (Figure 3.4). This behaviourmatches prior studies that model killer whale behaviours as lasting for tens of minutes tohours (McRae et al., 2024). In comparison, the PHMM with α = 1 resulted in a dive profilethat switched relatively frequently between foraging and resting dives (e.g., Figure 3.4a).The PHMM with α = 0 produced better results than the PHMM with α = 1, but it stillestimated some rapidly switching dive types (e.g., Figure 3.4c).The PHMM that completely ignored unlabelled dives (α = 0) obtained better results thanthe PHMM that fully included the unlabelled dives (α = 1). While one may expect thatmore data should improve accuracy, including unlabelled data is known to degrade theperformance of a classifier in certain situations (Singh et al., 2008). It is thus naturalthat the optimal approach for this case study neither fully included nor totally ignored thelabelled dive types.Finally, I used the PHMM with α = 0.049 to estimate how often the northern and southern563.4. Case Studiesresident killer whales engaged in foraging behaviour. In particular, I fit the PHMM to theentire data set, including all labels, and then ran the forward-backward algorithm on allwhales and dives to calculate P(ms,t = 3 | Ys) for t = 1P . . . P is and s = 1P . . . P 11. Then, Ilabelled dive t of whale s as foraging if its decoded probability of foraging was above 50%.Using this procedure, I estimated that southern resident killer whales foraged for 5.47 hours,or 32.0% of the time, and that northern residents foraged for 17.90 hours, or 26.8% of thetime. This sample size was very small (2 southern residents and 9 northern residents), butthese results are consistent with Tennessen et al. (2023), who found that southern residentsspent less time travelling and resting compared to the northern residents.3.4.2 Case Study 2: Identification of Killer Whale ForagingThe second case study focused on identifying successful foraging events within the killerwhale dives. Therefore, I divided all dives deeper than 30 meters into sequences of two-second windows and modelled each sequence with a PHMM. The hidden Markov chain wasa sequence of unobserved two-second subdive states, and the observations were summarystatistics calculated from each two-second window. I defined six possible subdive statebehaviours: descent, bottom, prey chase, prey capture, ascent with a fish, and ascentwithout a fish.Data ProcessingI defined a killer whale dive identically to the previous case study, but only included divesdeeper than 30 meters because Wright et al. (2017) found that killer whale prey capturesusually occur below that depth. I divided each dive into two-second windows and calculatedsummary statistics that were identified by Tennessen et al. (2019b) to be indicative offoraging behaviour: change in depth (ys,t for dive s and window t), heading total variation(hs,t), and jerk peak (js,t). Change in depth was defined as the last depth reading of thewindow minus the first depth reading of the window in meters. Heading total variationwas determined by calculating the difference between heading readings every 1/50 of asecond (in radians), taking the absolute value of that difference, and summing up all of thedifferences over the course of the two-second window. Jerk peak was calculated by takingthe difference between acceleration vectors every 1/50 of a second (in meters per secondsquared), taking the magnitude of that difference, and then calculating the maximum overeach two-second interval. To adjust for variation between dives and tags, I also divided thejerk peak of each two-second window by the median jerk peak for the bottom 70% of itscorresponding dive (Tennessen et al., 2019b). In summary, the observation associated withdive s and window t was denoted as ys,t = (ys,tP hs,tP js,t). This process resulted in a total573.4. Case Studiesof h = 130 dives and i = 15821 two-second windows.To label windows associated with prey capture, I used a process inspired by previous workon killer whale foraging (Wright et al., 2017; Tennessen et al., 2019b). First, a skilledresearcher identified crunching sounds associated with prey handling from the hydrophone.Next, Wright et al. (2017) found that killer whales catch prey immediately before ascending,so I ignored crunches that occurred more than 30 seconds before a dive’s “ascent” phaseas defined by Tennessen et al. (2019b). Namely, the “ascent” phase began the last momentthe killer whale achieved a depth at least 70% of its maximum dive depth. If the firstnon-ignored crunch occurred before “ascent”, I labelled its window as “prey capture”. If thefirst non-ignored crunch was heard during “ascent”, then the exact moment of prey capturewas ambiguous, so I labelled the final window of the dive as “ascent with a fish”.I was also able to obtain some window labels not associated with prey capture. First, ifthe video recorder was on for the entire dive, but there was no audible crunch or visualindication of foraging (e.g., scales), then I labelled the final window of the dive as “ascentwithout a fish”. Second, I labelled the first window of every dive as “descent”. Formally, thelabel associated with window t of dive s was denoted as zs,t ∈ {∅P 1P 2P 3P 4P 5P 6}, where eachpossible value of zs,t corresponds to no label (zs,t = ∅), descent (zs,t = 1), bottom (zs,t = 2),chase (zs,t = 3), capture (zs,t = 4), ascent without a fish (zs,t = 5), or ascent with a fish(zs,t = 6). In total, I labelled 130 windows as “descent”, five windows as “capture”, twowindows as “ascent with a fish”, and 19 windows as “ascent without a fish”. This resultedin |T | = 156 labels, which make up less than 1% of the i = 15821 windows in total.Model FormulationAs mentioned before, I defined a PHMM with N = 6 subdive states: descent, bottom, chase,capture, ascent without a fish, and ascent with a fish. I assumed that all dives shared aninitial distribution δ =(1 0 0 0 0 0)and transition probability matrix Γ as shown583.4. Case StudiesFigure 3.5: Visualization showing how the Markov chain Xs can evolve within the secondcase study of killer whale foraging. Arrows correspond to non-zero entries in Γ, where arrowspoint from row number to column number. Within a dive, a killer whale can proceed fromdescent, to bottom, to chase, to capture, to ascent with fish. It can also ascend withouta fish at any time before capture, and it can switch between the bottom and chase statesfreely.in Equation (3.12).Γ =descent bottom chase capture ascent w/o fish ascent w/ fishΓ(),)) Γ(),2) 0 0 Γ(),5) 0 descent0 Γ(2,2) Γ(2,+) 0 Γ(),5) 0 bottom0 Γ(+,2) Γ(+,+) Γ(+,4) Γ(+,5) 0 chase0 0 0 Γ(4,4) 0 Γ(4,6) capture0 0 0 0 1 0 asc w/o fish0 0 0 0 0 1 asc w/ fish(3.12)For an intuitive visualization of how the Markov chain could evolve, see Figure 3.5. Inshort, this transition probability matrix reflects that marine mammal dives have distinctdescent, bottom, and ascent phases (Tennessen et al., 2019b), and that Wright et al. (2017)observed that killer whales begin their ascent phase immediately after prey capture. I alsodivided the bottom phase to include a low activity state (bottom) and a high activity state(chase), which is in line with results from Chapter 2.Given that ms,t = i, I modelled change in depth with a normal distribution, heading total593.4. Case Studiesvariation with a gamma distribution, and normalized jerk peak with a gamma distribution.To enforce that the killer whale does not ascend or descend on average during the bottomof its dive, I set the state-dependent distributions of change in depth for the bottom, chase,and capture states to have a mean of zero. Further, I wanted to ensure that the model distin-guished foraging based primarily on the “prey capture” subdive state rather than differencesbetween the “ascent with a fish” and “ascent without a fish” states. As such, I set the twostate-dependent distributions associated with ascent to be identical. Given the hidden stateof a window (ms,t), I also assumed that all summary statistics were independent. Denotethe observations from dive s as ys = {ys,t}ist5) and the labels from dive s as zs = {zs,t}ist5).Then, I modelled all dives as independent, so the total likelihood for the model with weight-ing parameter α and parameters δPΓPθ and β was ∏)+(s5) Lα(ysP zs ; δPΓPθPβ), where Lα isdefined in Equation (3.11).Model EvaluationI fit five different PHMMs corresponding to α ∈ {0.0001P 0.001P 0.01P 0.1P 1}. I did not includeα = 0 in this case study because I did not have labels associated with subdive states 2 and3 (bottom and chase). Thus, if α = 0, there were no observations that could be used toestimate the parameters of the state-dependent distribution parameters θ(2) and θ(+), andthe model would not be identifiable.In addition to the PHMMs, I also implemented the prey-capture identification methodfrom Tennessen et al. (2019b) as a baseline. In particular, Tennessen et al. calculated threesummary statistics for each dive: (1) the maximum of jerk during the bottom 70% of thedive, divided by the median jerk during the same period, (2) the absolute value of the killerwhale’s roll at the moment of jerk peak, and (3) the circular variance of heading during thebottom 70% of the dive. Then, Tennessen et al. took the minimum value of every summarystatistic over all confirmed prey capture dives to obtain thresholds for each of the threedive-level summary statistics. Finally, a dive was labelled as a successful foraging dive ifevery dive-level summary statistic surpassed that threshold.I evaluated each model based on how well it predicted successful foraging dives. Note thatthe probability that dive s is a successful foraging dive is equal to the probability that itends in either the capture or ascent with fish state. In other words, it is the probability thatwindow is of dive s has subdive state 4 (capture) or subdive state 6 (ascent with fish), i.e.P(ms,is ∈ {4P 6} | Ys = ys). To estimate this value, I randomly and equally split the sevenlabelled successful foraging dives and 19 labelled dives without successful foraging into fourfolds and performed cross validation with the forward-backward algorithm. I calculatedAUC values within each fold and reported their averages (see Figure 3.6). Finally, I also603.4. Case StudiesFigure 3.6: Four-fold, cross-validated area under the ROC curve (AUC) values for fivePHMMs as well as the baseline method of Tennessen et al. (2019b). Horizontal jitters havebeen added because dots occasionally fall on top of one another. Averages are shown asstars and connected with a line for the PHMM approaches.fit each PHMM to the entire dataset to visualize their state-dependent distributions (seeFigure 3.7).ResultsMy method outperformed the baseline of Tennessen et al. (2019b), which had an averageAUC of 0.79. In comparison, the PHMM with α = 0.0001 had an average AUC of 0.90, thePHMM with α = 1.0 had an AUC of 0.96, and the PHMM with α = 0.01 had a perfectAUC of 1 across all four folds (Figure 3.6). Thus, not only did using a PHMM producebetter predictive performance over existing baselines, but using α ∈ (0P 1) (namely α = 0.01here) improved predictive performance over α = 0 and α = 1.The biological interpretation of each PHMM heavily depended on the weight α. For exam-ple, the “bottom” and “chase” states looked very similar for the PHMMs with α ≤ 0.01, butthe two states were better separated within the PHMM with α = 1 (Figure 3.7). However,the “bottom” subdive state had higher mean jerk peak and heading total variation than the“chase” subdive state for the PHMM with α = 1. These results are the opposite of whatecologists expect biologically, indicating that a large separation between state-dependentdistributions is not always more biologically interpretable. I conjecture that the “bottom”and “chase” subdive states are particularly unintuitive partially because there are no labelsassociated with either of these subdive states (i.e. zs,t ̸= 2P 3 for any s or t). As a result,there is little information for the model to differentiate the two subdive states.613.4. Case Studies(a) α = 0.0001 (b) α = 0.01(c) α = 1Figure 3.7: State-dependent densities for change in depth (top panels), heading total varia-tion (middle panels), and normalized jerk peak (bottom panels) for PHMMs with α = 0.0001(top left), α = 0.01 (top right), and α = 1 (bottom). Densities are coloured according totheir corresponding subdive state. Parameters were estimated using the entire killer whaledataset (i.e. no cross-validation was performed). For a given observation ns,t, all featureswere assumed to be independent after conditioning on the subdive state ms,t. Note thatthe “ascent with a fish” and “ascent without a fish” states were assumed to have identicaldistributions, so both are listed simply as “ascent”.623.5. DiscussionDifferences between each model’s distribution for the capture state appeared to impactpredictive performance. For example, the PHMM with α = 0.0001 estimated a relativelyhigh mean and low standard deviation for heading total variation (middle panel of Figure3.7a). As such, it failed to identify a prey capture event in which heading total variance washighly variable (Figure 3.8). Alternatively, the PHMM with α = 1 estimated a relativelylow mean for heading variation (middle panel of Figure 3.7c). As a result, it was oftentoo sensitive and decoded some dives that lacked successful foraging as containing a preycapture state (Figure 3.9). The PHMM with α = 0.01 estimated a high mean and a highstandard deviation for heading total variation (middle panel of Figure 3.7b). It correctlyidentified a wide variety of labelled foraging dives, but it was not overly sensitive andcorrectly identified low-activity dives as lacking successful foraging.Finally, I used the the PHMM with α = 0.01 to estimate the total number of successful for-aging dives from southern and northern resident killer whales in this data set. In particular,I fit the full model to the entire dataset, including labels, and then ran the forward-backwardalgorithm on all unlabelled dives to estimate the probability that each unlabelled dive wasa successful foraging dive, P(ms,is ∈ {4P 6} | Ys = ys) for s = 1P . . . P 130. Then, I labelleddive s as a successful foraging dive if the estimate of this probability was above 50%. Thisprocess resulted in 6 estimated successful foraging dives from southern resident killer whalesand 37 estimated successful foraging dives from northern resident killer whales. After com-bining these results with those from the first case study, I found that southern residentkiller whales caught an average of 1.03 fish per hour of foraging effort, while northern resi-dent killer whales caught an average of 2.00 fish per hour of foraging effort. These resultssupport the finding that northern resident killer whales have more foraging success com-pared to southern residents (Tennessen et al., 2023). However, my sample size is small (2southern resident killer whales and 9 northern resident killer whales). Future studies canuse the methods outlined here to further investigate the differences between northern andresident killer whale foraging success.3.5 DiscussionIn this chapter, I incorporated sparse labels into an HMM in a natural way that changes theinfluence of unlabelled observations and demonstrably improves predictive performance. Inparticular, I weighted observations without associated labels within the likelihood of theHMM using a parameter α ∈ [0P 1]. On the extremes, α = 0 corresponds to an HMM whichtotally ignores unlabelled data and α = 1 corresponds to a traditional, unweighted HMM.Practitioners can use cross-validation with their metric of choice to find an optimal valueof α.633.5. DiscussionFigure 3.8: Dive profiles, observations, and the decoded probability that each window isthe “capture” state for a selected successful foraging dive in case study 2. PHMMs withα = 0.0001 (top left), α = 0.01 (top right) and α = 1 (bottom) are shown. Each subplotdisplays change in depth (top panel), raw depth (second panel), heading total variation(third panel), normalized jerk peak (fourth panel), and probability of “capture” (bottompanel). Observations are coloured according to the most-likely sequence of hidden statesas determined by the Viterbi algorithm. The estimated probability of successful foraging,P(ms,is ∈ {4P 6} | Ys = ys), is 0.069 for α = 0.0001, 1 for α = 0.01, and 1 for α = 1.643.5. DiscussionFigure 3.9: Dive profiles, observations, and the decoded probability that each window isthe “capture” state for a selected dive with no foraging in case study 2. PHMMs withα = 0.0001 (top left), α = 0.01 (top right) and α = 1 (bottom) are shown. Each subplotdisplays change in depth (top panel), raw depth (second panel), heading total variation(third panel), normalized jerk peak (fourth panel), and probability of “capture” (bottompanel). Observations are coloured according to the most-likely sequence of hidden statesas determined by the Viterbi algorithm. The estimated probability of successful foraging,P(ms,is ∈ {4P 6} | Ys = ys), is 1 for α = 0.0001, 0.414 for α = 0.01, and 1 for α = 1.653.5. DiscussionIn addition to better performance, my method has two additional benefits compared toprevious methods. First, it estimates the probability that each dive is a successful foragingdive instead of outputting a binary label. This allows practitioners to set a threshold tocontrol the false positive and false negative rates. Second, my method also categorizessubdive states so practitioners can simultaneously categorize the subdive behaviour of thekiller whale while they estimate successful foraging dives.Future work can quantify exactly how the density of labels and level of model misspecifica-tion affect the value of α that optimizes prediction accuracy and model fit. The biologicalinterpretability and predictive accuracy of the model from the first case study were opti-mized for α ≈ 0.05. In the second case study, the model had better predictive performancefor α = 0.01, and each PHMM had distinct biological interpretations for α ≤ 0.01 andα S 0.01. The “bottom” and “chase” subdive states were better separated for the PHMMswith α S 0.01, but not necessarily in a way that was biologically natural. It is intuitivethat subdive states with no labels were better separated within PHMMs that weighted un-labelled observations more heavily. Quantifying the precise relationship between α and thestate-dependent distributions is a promising direction for future research.As is common in movement ecology, my case studies had very few labels, which can neg-atively impact the reliability of cross-validation. Therefore, future work can apply thisweighted likelihood approach to case studies with more labels, potentially from other fields.I also performed case studies where subject matter experts were confident in their labels.Future work can focus on how including labels affects the performance of the PHMM whenpractitioners are less confident in their labels and infer β as a parameter of the PHMM.Although I did not have a large enough sample size to draw definitive conclusions, thefindings from my method are in line with those from Tennessen et al. (2023), who conducteda comprehensive study of foraging behaviour in northern and southern resident killer whales.In particular, my results support their findings that northern resident killer whales tendto spend more time travelling and resting compared to southern residents, and that thenorthern residents tend to catch more fish per unit of effort compared to the southernresidents.I primarily focus on identifying the behaviour of killer whales in this dissertation, butincorporating sparse labels into complex HMMs is a common modelling problem across avariety of use cases and disciplines. In addition, complicated time series data are increasinglycommon as sensing technology continues to improve (Patterson et al., 2017). As such,the modelling approach developed here can help researchers effectively model complicated,sparsely labelled time series to optimize prediction accuracy and model fit.66Chapter 4Efficient Inference usingVariance-Reduced StochasticOptimizationA version of this chapter appears in the Journal of Computational and Graphical Statis-tics under the title “Variance-Reduced Stochastic Optimization for Efficient Inference ofHidden Markov Models.” Computer code associated with the chapter can be found athttps://github.com/evsi8432/sublinear-HMM-inference.4.1 IntroductionHidden Markov models (HMMs) are statistical models for sequential data that are widelyused to model time series in fields such as speech recognition (Gales and Young, 2007),geology (Bebbington, 2007), neuroscience (Kottaram et al., 2019), finance (Mamon andElliott, 2007), and ecology (McClintock et al., 2020). Such models are often used to predicta latent process of interest (e.g., a spoken phrase or an animal’s behavioral state) from anobserved time series (e.g., raw audio or time-depth data). Many practitioners estimate theparameters of an HMM by maximizing the likelihood function using either gradient-basednumerical maximization or the EM algorithm (Baum et al., 1970; Dempster et al., 1977).One serious concern for both numerical maximization and the EM algorithm is that everyparameter update requires iterating though the full set of observations to calculate eitherthe likelihood or its gradient. This concern is likely to only worsen in the future, as time-series data sets are increasingly collected at high frequencies and contain large numbers ofobservations (Patterson et al., 2017; Li et al., 2020). These data sets require progressivelycomplex HMMs which can be computationally expensive to fit (Adam et al., 2019). In addi-tion, many model validation techniques such as cross-validation require repeated parameterestimation which can be prohibitive even for relatively simple HMMs (Pohle et al., 2017).Many inference techniques for independent data sets do not require iterating through the674.1. Introductionentire data set to update a model’s parameters. I henceforth refer to these techniques assub-linear methods. One example of a sub-linear method is stochastic gradient descent(SGD), which estimates the gradient of the full likelihood using a random subset of thedata (Robbins and Monro, 1951). It is ubiquitous in the optimization literature and hasinspired several extensions. For example, Johnson and Zhang (2013), Defazio et al. (2014)and Kingma and Ba (2014) reduce the variance of gradient estimates compared to standardSGD, while Zinkevich et al. (2010) incorporate parallelization into SGD. Similarly, theincremental EM algorithm is a generalization of the EM algorithm that updates only asubset of hidden variables at each E step (Neal and Hinton, 1998; Thiesson et al., 2001;Karimi et al., 2019). Both stochastic gradient descent and incremental EM assume that theunderlying data set is comprised of independent subsets. Such an assumption is sometimesreasonable for HMMs. For example, Gales and Young (2007) infer spoken words from manyrelatively short audio files, and they assume that each audio file is independent from oneanother. However, the assumption of independence is generally violated for HMMs designedfor long, sequentially-dependent time series. Long time series are increasingly common inpractice and are the focus of this chapter.Some work has been done to apply sub-linear inference methods to HMMs for long timeseries. For example, Gotoh et al. (1998) divide a sequence of observations into segmentsand use the incremental EM algorithm to perform inference. This approach assumes thatsegments are independent of one another, which is not true in general. Alternatively, Yeet al. (2024) define a sufficiently large “buffer” before and after segments of data to minimizethe effect of serial dependence. However, the appropriate size of the buffer can be difficultto calculate. More examples of sub-linear inference techniques are given by Khreich et al.(2012), who review on-line and incremental methods for HMM inference. However, most ofthese methods assume either that the M step of the EM algorithm is tractable (see section3.2.1 of Khreich et al. (2012)), or that the emissions of the HMM are discrete (Baldi andChauvin, 1993).In this chapter, I introduce a new inference method for HMMs based on variance-reducedstochastic optimization. This inference method updates the HMM parameters withoutiterating through the entire data set. Critically, it does not require a closed-form solutionfor its M step, does not require any buffer tuning, and does not introduce error into theHMM likelihood.I begin with a formal definition of HMMs, a brief review of standard inference techniquesfor HMMs, and a description of stochastic optimization algorithms before introducing myalgorithm. I then prove that my algorithm converges to a local maximum of the likelihood(under standard regularity assumptions) and note several practical considerations regarding684.2. Backgroundits implementation. Finally, I compare the efficiency of my new algorithm to that of standardoptimization techniques using several simulation studies and a kinematic case study of eightnorthern resident killer whales (Orcinus orca) off the western coast of Canada.4.2 Background4.2.1 Hidden Markov ModelsHMMs are common statistical models used to describe time series that exhibit state-switching behavior. An HMM models an observed sequence of length i , Y = {nt}it5),together with an unobserved (or “hidden”) sequence X = {mt}it5). The hidden sequence Xis a Markov chain, and each observation nt is a random variable, where nt given all other ob-servations (Y \ {nt}) and hidden states (X) depends only on mt. I assume mt ∈ {1P . . . P N}for some finite N . The unconditional distribution of m) is denoted by the row-vectorδ =(δ()) · · · δ(c)), where δ(i) = P(m) = i). Further, the distribution of mt for t S 1conditioned on mt−) is denoted by an N -by-N transition probability matrixΓt =Γ(),))t · · · Γ(),c)t... . . . ...Γ(c,))t · · · Γ(c,c)t P (4.1)where Γ(i,j)t = P(mt = j | mt−) = i). For simplicity, I assume that Γt does not change overtime (i.e. Γt = Γ for all t) unless stated otherwise.To ensure that all entries are positive and all rows sum to one, it is convenient to reparame-terize the transition probability matrix Γ ∈ Rc×c and initial distribution δ ∈ Rc in termsof auxiliary variables η ∈ Rc×c and ν ∈ Rc :Γ(i,j)(η) =exp(η(i,j))∑ck5) exp(η(i,k))P δ(i)(ν) =exp(,(i))∑ck5) exp(,(k))P (4.2)where iP j = 1P . . . P N and η(i,i) and ,()) are set to zero for identifiability. This formulationsimplifies likelihood maximization by removing constraints in the optimization problem.One may also incorporate covariates into Γ by setting η(i,j)t (β) =(β(i,j))⊤zt, where zt is acolumn vector of known covariates at time index t and β(i,j) is a column vector of unknownregression coefficients. While Γ and δ are functions of η and ν, I abuse notation in futuresections and treat Γ and δ as variables since the mappings are bijections.If mt = i, then I denote the conditional density or probability mass function of nt as694.2. BackgroundFigure 4.1: Graphical representation of an HMM. mt corresponds to an unobserved latentstate at time t whose distribution is described by a Markov chain. nt corresponds to anobservation at time t, where nt given all other observations Y \ {nt} and hidden states Xdepends only on mt.f (i)(·; θ(i)), where θ(i) are the parameters describing the state-dependent distribution of nt.The collection of all state-dependent parameters is θ = {θ(i)}ci5). For brevity, I denote thefull set of parameters as ϕ = {θPηPν}. Figure 4.1 shows an HMM as a graphical model.Denote a fixed realization of observations Y and latent states X as y = {yt}it5) and x ={xt}it5). The joint likelihood of an HMM given y and x isp(xP y;ϕ) = δ(x))f (x))(y); θ(x)))i∏t52Γ(xt−),xt)f (xt)(yt; θ(xt)). (4.3)Alternatively, the marginal likelihood of the observed data y alone isp(y;ϕ) = δP (y);θ)i∏t52ΓP (yt;θ)1⊤c P (4.4)where 1c is an N -dimensional row vector of ones and P (yt;θ) is an N ×N diagonal matrixwith entry (iP i) equal to f (i)(yt; θ(i)). For a more complete introduction to HMMs, seeZucchini et al. (2016).4.2.2 State DecodingOne appealing feature of HMMs is that it is simple to determine the distribution of agiven hidden state (mt) conditioned on a set of observations y. Assuming that the HMMparameters ϕ are fixed, define the probability density of the observations between timess and t as p(ys2t;ϕ). Likewise, define forward probabilities α(i)t = p(y)2tP mt = i;ϕ) (fori = 1P . . . P N and t = 1P . . . P i ) and backward probabilities β(i)t = p(y(t+))2i | mt = i;ϕ) (fori = 1P . . . P N and t = 1P . . . P i − 1). By convention, β(i)i = 1 for i = 1P . . . P N . I thus define704.2. Backgroundthe row vectors αt =(α())t · · · α(c)t)and βt =(β())t · · · β(c)t). Both αt and βt canbe calculated using the following recursions:α) = δ P (y);θ)P αt = αt−) Γ P (yt;θ)P t = 2P . . . P iP (4.5)β⊤i = 1⊤c P β⊤t = Γ P (yt+);θ) β⊤t+)P t = 1P . . . P i − 1. (4.6)I define conditional probabilities γ(i)t = P(mt = i | Y = y ; ϕ) and ξ(i,j)t = P(mt−) = iPmt =j | Y = y ; ϕ). I also define the row vector γt =(γ())t · · · γ(c)t)and the matrixξt =ξ(),))t · · · ξ(),c)t... . . . ...ξ(c,))t · · · ξ(c,c)t P t = 2P . . . P i.Both γt and ξt can be calculated from αt−), αt, βt, Γ, and θ. Let diag(·) map a row vectorto the diagonal matrix with that row vector as its diagonal. Then,γ(i)t =α(i)t β(i)tαt β⊤tP γt =αt diag(βt)αt β⊤tP (4.7)ξ(i,j)t =α(i)t−) Γ(i,j) f (j)(yt; θ(j)) β(j)tαt−) Γ P (yt;θ) β⊤tP ξt =diag(αt−)) Γ P (yt;θ) diag(βt)αt−) Γ P (yt;θ) β⊤t. (4.8)For shorthand, I define the sets {αPβPγP ξ} = {αtPβtPγtP ξt}it5) to summarize the con-ditional probabilities for all t. In future sections, when ϕ is not fixed (e.g., during theparameter estimation procedures), I add an argument to the conditional probabilities andwrite {α(ϕ)Pβ(ϕ)Pγ(ϕ)P ξ(ϕ)} = {αt(ϕ)Pβt(ϕ)Pγt(ϕ)P ξt(ϕ)}it5) to highlight the depen-dence on ϕ.4.2.3 The Baum-Welch AlgorithmThe Baum-Welch algorithm is an iterative algorithm used to estimate the parameters of anHMM. It predates the more general expectation-maximization (EM) algorithm (Dempsteret al., 1977), but the two are equivalent when applied to HMMs. At iteration k of theEM algorithm, denote the current parameter estimates as ϕk. One iteration of the EMalgorithm consists of an expectation (or E) step, followed by a maximization (or M) step.For the E step, the function value f(ϕ | ϕk) is defined as the expected value of the jointlog-likelihood log p(XP y;ϕ) taken with respect to X, where X has conditional probability714.2. Backgroundmass function p(X | Y = y;ϕk). For the M step, the next parameter estimate ϕk+) is foundby maximizing f(ϕ | ϕk) with respect to ϕ:f(ϕ | ϕk) = Eϕk [log p(XP y;ϕ) | Y = y] P (4.9)ϕk+) = argmaxϕf(ϕ | ϕk). (4.10)For notational convenience, I will occasionally denote the set of conditional probabilities{αt(ϕk)Pβt(ϕk)Pγt(ϕk)P ξt(ϕk)} as {αk,tPβk,tPγk,tP ξk,t}. Substituting Equation (4.3) intoEquation (4.9) and performing some algebra yields a closed form expression for f:f(ϕ | ϕk) =c∑i5)γ(i)k,) log δ(i)(ν) +i∑t5)c∑i5)γ(i)k,t log f (i)(yt; θ(i)) +i∑t52c∑i5)c∑j5)ξ(i,j)k,t logΓ(i,j)(η).(4.11)The conditional probabilities γ(i)k,t and ξ(i,j)k,t thus act as weights for log δ(i)(ν), log f (i)(yt; θ(i)),and logΓ(i,j)(η) for iP j = 1P . . . P N . I thus refer to γ and ξ as weights in future sections.Detailed pseudocode for the E and the M step are given in Algorithms 1 and 2 below. InAlgorithm 1 E-step(ϕ)Require: Parameter value ϕ = {θPηPν}.1: δ = δ(ν)P Γ = Γ(η)2: α) = δ P (y);θ)3: β⊤i = 1⊤c4: for t = 2P . . . P i do5:αt = αt−) Γ P (yt;θ)P β⊤i−t+) = Γ P (yi−t+2;θ) β⊤i−t+26: end for7: γ) =α) diag(β))α) β⊤)8: for t = 2P . . . P i do9:γt =αt diag(βt)αt β⊤tP ξt =diag(αt−)) Γ P (yt;θ) diag(βt)αt−) Γ P (yt;θ) β⊤t10: end for11: return {αtPβtPγtP ξt}it5)some simple scenarios, the maximization problem in Equation (4.10) above has a closed-form solution. However, this maximization problem is not always straightforward and oftenrequires numerical maximization techniques. I thus review different methods for numericalmaximization via stochastic optimization.724.2. BackgroundAlgorithm 2 Baum-Welch(ϕ(PK)Require: Initial parameter values ϕ(, number of iterations K1: for k = 0P . . . PK − 1 do2: {αk,tPβk,tPγk,tP ξk,t}it5) = E-step(ϕk) C E step3: C M stepϕk+) = argmax{θ,η,ν}N∑i=)γ(i)k,) log δ(i)(ν) +T∑t=)N∑i=)γ(i)k,t log f (i)(yt; θ(i)) +T∑t=2N∑i=)N∑j=)ξ(i,j)k,t logΓ(i,j)(η)4: end for5: return ϕK4.2.4 Stochastic OptimizationStochastic optimization involves a class of optimization methods that use random variablesto maximize or minimize an objective function. I focus on optimization methods to minimizeobjective functions that can be written as a sum of many terms (Robbins and Monro, 1951),namely to solveϕ∗ = argminϕF (ϕ)P where F (ϕ) = 1ii∑t5)Ft(ϕ). (4.12)I denote parameter values at step m of an optimization scheme as ϕ(m) = {θ(m)Pη(m)Pν(m)}to distinguish between the parameter values ϕk at iteration k of the Baum-Welch algorithm.The bold parameters θ(m), η(m), and ν(m) correspond to optimization stepm and are not tobe confused with θ(i), η(i,j), and ,(i) which correspond to hidden states i and j of the HMM.Standard gradient descent at a given step m with step size λ updates the parameter valueϕ(m) by moving in the direction of the negative gradient of F . Formally, the update step isϕ(m+)) = ϕ(m) − λ∇F (ϕ(m)), or ϕ(m+)) = ϕ(m) − (λ/i )∑it5)∇Ft(ϕ(m)) for this problem.The step size λ is user-defined. It should be large enough so ϕ(m+)) moves quickly towardsa minimum of F , but not so large that ϕ(m+)) “overshoots” the minimum. This updaterequires evaluating a gradient for all t = 1P . . . P i , which can be prohibitively expensive ifi is large.In contrast, stochastic gradient descent (SGD) updates the parameters ϕ using ϕ(m+)) =ϕ(m) − λm∇Ftm(ϕ(m))P where tm ∈ {1P . . . P i} is selected uniformly at random at stepm of the algorithm (Robbins and Monro, 1951). Stochastic gradient descent reduces theamount of time between updates by using an unbiased estimate of the gradient to updateϕ(m). However, the gradient estimates can have very high variance, so stochastic gradientdescent requires that the step size λm is smaller than that of full gradient descent. The step734.3. Stochastic Optimization for HMM Inferencesize must also decay to zero as m → ∞ to ensure convergence. Further, SGD has slowerconvergence rates than full gradient descent (Schmidt et al., 2017).Variance-reduced stochastic optimization techniques such as stochastic average gradient(SAG) (Schmidt et al., 2017), stochastic variance reduced gradient (SVRG) (Johnson andZhang, 2013), and stochastic average gradient accelerated (SAGA) (Defazio et al., 2014)enjoy the speed of stochastic gradient descent as well as the convergence rates of full gra-dient descent. These algorithms involve storing gradient approximations at each gradientstep m, ∇̂F (m)t for t = 1P . . . P i , whose average approximates the full gradient ∇F (ϕ(m)).The gradient approximations are updated at various stages in the optimization algorithmand are used to reduce the variance of the full gradient estimate. For example, SVRGand SAGA update ϕ(m) via ϕ(m+)) = ϕ(m) − λm[∇Ftm(ϕ(m))− ∇̂F (m)tm + ∇̂F (m)]P wheretm ∈ {1P . . . P i} is chosen uniformly at random and ∇̂F (m) = (1/i )∑it5) ∇̂F (m)t . LikeSGD, this stochastic update has the same expectation as that of standard gradient descent,but it empirically exhibits lower variance than SGD and it is guaranteed to converge with-out decaying the step size λm (under certain regularity conditions). After updating theparameters at step m, SAGA updates the gradient approximation ∇̂F (m+))tm and recalcu-lates the resulting gradient average ∇̂F (m+)). Algorithm 3 outlines SVRG and SAGA inpseudocode, and I denote it as variance-reduced stochastic optimization, or VRSO.SAGA involves running Algorithm 3 until convergence. However, the gradient approxima-tions ∇̂F (m)t are never updated when using SVRG within Algorithm 3. As such, SVRGrequires repeatedly running Algorithm 3 with b scaling approximately with i so the setof gradient approximations remains up-to-date.4.3 Stochastic Optimization for HMM InferenceBoth the E step and the M step of the Baum-Welch algorithm are expensive when the lengthof the observation sequence (i ) is large. The E step is expensive because γt(ϕk) and ξt(ϕk)must be calculated for t = 1P . . . P i to define f(ϕ | ϕk). If closed-form solutions to (4.10)are not readily available, then the M step is also expensive because evaluating full gradientsof f(ϕ | ϕk) takes O(i ) time. In this section, I introduce an original algorithm that speedsup both the expensive M step as well as the expensive E step of the Baum-Welch algorithm.4.3.1 Variance-Reduced Stochastic M StepTo speed up the expensive M step, I note from Equation (4.11) that f is a large sum andthus implement variance-reduced stochastic optimization. It is straightforward to reframethe M step of iteration k of the Baum Welch algorithm from Equation (4.10) so it looks744.3. Stochastic Optimization for HMM InferenceAlgorithm 3 VRSO(FPϕ(()P λP APb)Require: Loss function F = )i∑it5) Ft, initial value ϕ((), step size λ, algorithm A ∈{SVRG, SAGA}, and number of iterations b .1: for t = 1P . . . P i do C initialize gradients2: ∇̂F (()t = ∇Ft(ϕ(())3: end for4: ∇̂F (() = (1/i )∑it5) ∇̂F (()t5: for m = 0P . . . Pb − 1 do6: Pick tm ∈ {1P . . . P i} uniformly at random.7: C update parametersϕ(m+)) = ϕ(m) − λ[∇Ftm(ϕ(m))− ∇̂F (m)tm + ∇̂F (m)](4.13)8: ∇̂F (m+))t = ∇̂F (m)t for t = 1P . . . P i C update gradient approximations and average9: if A = SAGA then:∇̂F (m+))tm = ∇Ftm(ϕ(m)) (4.14)∇̂F (m+)) = ∇̂F (m) + 1i(∇̂F (m+))tm − ∇̂F(m)tm)(4.15)10: end if11: end for12: return ϕ(M)754.3. Stochastic Optimization for HMM Inferencelike the minimization problem from Equation (4.12). To do so, I define ξ) = ∅ and the lossfunction F (· | γP ξ) as follows:F (ϕ | γP ξ) = 1ii∑t5)Ft(ϕ | γtP ξt)P where (4.16)F)(ϕ | γ)P ξ)) = −c∑i5)γ(i)) log f (i)(yt; θ(i))−c∑i5)γ(i)) log δ(i)(ν)P (4.17)Ft(ϕ | γtP ξt) = −c∑i5)γ(i)t log f (i)(yt; θ(i))−c∑i5)c∑j5)ξ(i,j)t logΓ(i,j)(η)P t = 2P . . . P i. (4.18)The two functions F and f are closely related to one another, as F (ϕ | γ(ϕk)P ξ(ϕk)) =−f(ϕ | ϕk)/i . However, I make a distinction between the two to bridge the gap betweenexisting EM literature (which uses f) and stochastic optimization literature (which usesF ). At any iteration k of the EM algorithm, the loss function F (· | γ(ϕk)P ξ(ϕk)) can beminimized using Algorithm 3.There are additional reasons to use SAGA and SVRG within the Baum-Welch algorithmbeyond the standard benefits of variance-reduced stochastic optimization. Traditionally,SAGA is more memory intensive than SVRG because the gradient at every index must bestored. However, the Baum-Welch algorithm involves storing weights for each time indext to define F (· | γ(ϕk)P ξ(ϕk)), so storing each gradient for SAGA is not considerablymore memory intensive than the Baum-Welch algorithm itself. Alternatively, SVRG canbe more computationally expensive than SAGA partially because it requires periodicallyre-calculating the full gradient approximation ∇̂F ((), and this involves a full pass of theunderlying data set. However, the E step of the Baum-Welch algorithm also involves a fullpass of the data set, so using SVRG is not considerably more computationally expensivethan the Baum-Welch algorithm itself. In this way, using either SAGA or SVRG in the Mstep adds minimal computational and memory complexity to the Baum-Welch algorithm.4.3.2 Partial E Step within the M stepVariance-reduced stochastic optimization reduces the computational cost of the M step, butthe E step itself still has a time complexity of O(i ), which can be prohibitive for large i .To decrease this computational burden, Neal and Hinton (1998) justify a partial E stepwithin the EM algorithm for general latent variable models. However, they assume thatthe optimization of the M step has a closed-form solution. I use their method as inspirationand add a partial E step to the stochastic M step of the Baum-Welch algorithm.764.3. Stochastic Optimization for HMM InferenceConsider running one iteration of my version of the Baum-Welch algorithm with an ini-tial parameter estimate ϕ((). The E step involves calculating the conditional probabilitiesγ(ϕ(()) and ξ(ϕ(()), and the M step involves running variance-reduced stochastic optimiza-tion with loss function F (· | γ(ϕ(())P ξ(ϕ(())) and initial parameter value ϕ((). Now, supposeϕ(m) is to be updated using a gradient estimate using a random observation index tm. Thefunction Ftm(· | γtm(ϕ(())P ξtm(ϕ(())) depends on γtm(ϕ(()) and ξtm(ϕ(()), each of which arevectors of conditional probabilities given ϕ((). However, ϕ(() is an out-of-date parameterestimate since the current parameter estimate is ϕ(m). Therefore, it is natural to updateγtm and ξtm and redefine Ftm(· | γtm P ξtm) before calculating ϕ(m+)).A naive method would be to calculate the new conditional probabilities γtm(ϕ(m)) andξtm(ϕ(m)) and then update Ftm as Ftm(· | γtm(ϕ(m))P ξtm(ϕ(m))). This would ensure thatγtm and ξtm are completely up-to-date, but evaluating γtm(ϕ(m)) and ξtm(ϕ(m)) takesO(iN2) time and requires a full E step. Instead, my goal is to update γtm and ξtm in away that does not scale with i .To this end, I define the mappings α˜t, β˜t, γ˜t, and ξ˜t for t = 1P . . . P i similarly to Equations(4.5–4.8). If a ∈ Rc and b ∈ Rc are generic row vectors, then:α˜)(aPϕ) = δ P (y);θ)P α˜t(aPϕ) = a Γ P (yt;θ)P t = 2P . . . P iP (4.19)β˜⊤i (bPϕ) = 1⊤c P β˜⊤t (bPϕ) = Γ P (yt+);θ) b⊤P t = 1P . . . P i − 1P (4.20)γ˜t(aPb) =a diag(b)a bi P t = 1P . . . P iP (4.21)ξ˜t(aPbPϕ) =diag(a) Γ P (yt;θ) diag(b)a Γ P (yt;θ) b⊤P t = 2P . . . P iP (4.22)all of which takeO(N2) time to compute. I have designed these mappings so that α˜)(aPϕ) =α)(ϕ) , α˜t(αt−)(ϕ)Pϕ) = αt(ϕ) , β˜⊤i (bPϕ) = β⊤i (ϕ) , and β˜⊤t (βt+)(ϕ)Pϕ) = β⊤t (ϕ).At the beginning of the M step I define conditional probability approximations α̂(()t =αt(ϕ(()), β̂(()t = βt(ϕ(()), γ̂(()t = γt(ϕ(()), and ξ̂(()t = ξt(ϕ(()) for t = 1P . . . P i . Thisis simply the E step of the Baum-Welch algorithm and takes O(iN2) time to compute.Then, at any given step m of the stochastic M step, I update Ftm by first updatingα̂(m+))tm = α˜tm(α̂(m)tm−) P ϕ(m))and β̂(m+))tm = β˜tm(β̂(m)tm+) P ϕ(m)), followed by γ̂(m+))tm =γ˜tm(α̂(m+))tm P β̂(m+))tm)and ξ̂(m+))tm = ξ˜tm(α̂(m+))tm−) P β̂(m+))tm P ϕ(m)). Finally, the loss func-774.3. Stochastic Optimization for HMM Inferencetion at index tm can be defined as Ftm(·∣∣∣ γ̂(m+))tm P ξ̂(m+))tm ). Updating Ftm in this way takea total of O(N2) time, which accomplishes a parameter update step that does not scalewith i . Algorithm 4 outlines the M step of the Baum-Welch algorithm with a partial Estep integrated in.The partial E step detailed above is closely related to belief propagation, an algorithmthat calculates conditional probabilities of variables within graphical models (Pearl, 1982).In fact, the E step of the Baum-Welch algorithm is a specific instance of belief propaga-tion, where Equations (4.19 – 4.22) above correspond to “passing messages” within thegraphical model. Belief propagation can only perform exact inference on acyclic graphicalmodels (including HMMs), but a generalization called loopy belief propagation can performapproximate inference on general graphical models (Pearl, 1988). Practitioners balanceapproximation error and computational complexity to decide how long to run loopy beliefpropagation. Likewise, I consider computational complexity when running belief prop-agation and evaluate Equations (4.19 – 4.22) only once to approximate γtm(ϕ(m)) andξtm(ϕ(m)). This forms the basis of the partial E step within the stochastic M step of mymodified Baum-Welch algorithm.4.3.3 Full AlgorithmIt is possible to run Algorithm 4 alone without ever performing a full E step. However, if nopartial E step is used (i.e. P = False) or if SVRG is used as the optimization algorithm,then either the conditional probability approximations{α̂(m)t P β̂(m)t P γ̂(m)t P ξ̂(m)t}it5)or thegradient approximations{∇̂F (m)t}it5)will not be updated and become out-of-date. Toavoid this issue, Algorithm 5 combines the M step defined in Algorithm 4 with a full E stepto complete my new Baum-Welch algorithm for HMMs.There are two versions of EM-VRSO. Version 1 (Algorithm 5), requires the likelihood to notdecrease (i.e. log p(y;ϕk,ℓ) ≥ log p(y;ϕk)) in order to exit the while loop of the M step.Version 2 (Algorithm 6) requires the likelihood to strictly increase by some threshold toexit the while loop of the M step. I use version 2 to prove theoretical results, but the strictthreshold relies on values that are usually not known in practice. Therefore, I use version1 in my simulation and case studies and defer version 2 to Appendix C. My simulation andcase studies show that version 1 of EM-VRSO converges to local maxima of the log-likelihoodfunction in practice.At first, it seems troubling to require log p(y;ϕk,ℓ) ≥ log p(y;ϕk) to exit the while loopof EM-VRSO, since this requirement may cause an infinite loop if it cannot be met. Inpractice, I found that the log-likelihood increased after a pass through the M step in most784.3. Stochastic Optimization for HMM InferenceAlgorithm 4 VRSO-PE({α̂(()t P β̂(()t P γ̂(()t P ξ̂(()t }it5)Pϕ(()P λP AP PPb)Require: Initial conditional probability approximations {α̂(()P β̂(()P γ̂(()P ξ̂(()}, initial pa-rameters ϕ((), step size λ, algorithm A ∈ {SVRG, SAGA}, whether to do a partial-Estep P ∈ {True,False}, and number of iterations b .1: for t = 1P . . . P i do C initialize gradients2: ∇̂F (()t = ∇Ft(ϕ(()∣∣∣ γ̂(()t P ξ̂(()t )3: end for4: ∇̂F (() = (1/i )∑it5) ∇̂F (()t5: for m = 0P . . . Pb − 1 do:6: Pick tm ∈ {1P . . . P i} uniformly at random.7:{α̂(m+))t P β̂(m+))t P γ̂(m+))t P ξ̂(m+))t}={α̂(m)t P β̂(m)t P γ̂(m)t P ξ̂(m)t}for t = 1P . . . P i .8: if P = True then C partial E step9: α̂(m+))tm = α˜tm(α̂(m)tm−)Pϕ(m))P β̂(m+))tm = β˜tm(β̂(m)tm+)Pϕ(m))10: γ̂(m+))tm = γ˜tm(α̂(m+))tm P β̂(m+))tm)P ξ̂(m+))tm = ξ˜tm(α̂(m+))tm−) P β̂(m+))tm Pϕ(m))11: end if12: C update parametersϕ(m+)) = ϕ(m) − λ[∇Ftm(ϕ(m)∣∣∣ γ̂(m+))tm P ξ̂(m+))tm )− ∇̂F (m)tm + ∇̂F (m)] (4.23)13: ∇̂F (m+))t = ∇̂F (m)t for t = 1P . . . P i C update gradients14: if A = SAGA then:∇̂F (m+))tm = ∇Ftm(ϕ(m)∣∣∣ γ̂(m+))tm P ξ̂(m+))tm ) P (4.24)∇̂F (m+)) = ∇̂F (m) + 1i(∇̂F (m+))tm − ∇̂F(m)tm). (4.25)15: end if16: end for17: return ϕ(M)794.3. Stochastic Optimization for HMM InferenceAlgorithm 5 EM-VRSO(ϕ(P λP AP PPbPK) (Version 1)Require: Initial parameters (ϕ(), step size (λ), algorithm A ∈ {SVRG, SAGA}, whetherto do a partial E step P ∈ {True,False}, iterations per update (b), and number ofupdates (K).1: for k = 0P . . . PK − 1 do2: {αk,tPβk,tPγk,tP ξk,t}it5) = E-step(ϕk) C E step3: ℓ← 0 C M step4: while ℓ = 0 or log p(y;ϕk,ℓ) Q log p(y;ϕk) do5: ℓ← ℓ+ 16: ϕk,ℓ = VRSO-PE({αk,tPβk,tPγk,tP ξk,t}it5)PϕkP λP AP PPb)7: end while8: ϕk+) = ϕk,ℓ9: end for10: return ϕKcases unless the step size λ was very large. As such, I detail how to adaptively select λin Section 4.4. For a theoretical justification that the while loop terminates, denote ℓ∗(k)as the (random) number of runs through the inner loop of EM-VRSO for iteration k (i.e.the maximum value obtained by ℓ for a given value of k). I prove in Theorem 4.1 belowthat P(ℓ∗(k) Q ∞) = 1. One final concern is whether the sequence {ϕk}∞k5( generated byEM-VRSO converges to a local maximum of the likelihood as K → ∞. Theorem 4.1 belowguarantees such convergence under standard regularity conditions. I prove Theorem 4.1 inAppendix C.Theorem 4.1. Suppose the following conditions are met in Algorithm 6 with P = Falseand A = SVRG:1. The parameters ϕ lie in Φ = Rr for some dimension r.2. Φϕ( = {ϕ ∈ Φ : log p(y;ϕ) ≥ log p(y;ϕ()} is compact for all ϕ( if log p(y;ϕ() S −∞.3. log p(y;ϕ) is differentiable in ϕ for all ϕ ∈ Φ.4. Ft(ϕ | γt(ϕ′)P ξt(ϕ′)) is convex with respect to ϕ and F (ϕ | γ(ϕ′)P ξ(ϕ′)) is stronglyconvex with respect to ϕ for all ϕ′ with constant X S 0. Namely, for all ϕ, ϕ( andϕ′:Ft(ϕ | γt(ϕ′)P ξt(ϕ′)) ≥ Ft(ϕ( | γt(ϕ′)P ξt(ϕ′)) +∇Ft(ϕ( | γt(ϕ′)P ξt(ϕ′))i (ϕ− ϕ()P (4.26)804.3. Stochastic Optimization for HMM InferenceF (ϕ | γ(ϕ′)P ξ(ϕ′)) ≥ F (ϕ( | γ(ϕ′)P ξ(ϕ′)) +∇F (ϕ( | γ(ϕ′)P ξ(ϕ′))i (ϕ− ϕ() +X2‖ϕ− ϕ(‖22. (4.27)5. Ft(ϕ | γt(ϕ′)P ξt(ϕ′)) is uniformly Lipschitz-smooth with respect to ϕ for all t and ϕ′with constant a ≥ X S 0. Namely, for all t, ϕ, ϕ( and ϕ′:Ft(ϕ | γt(ϕ′)P ξt(ϕ′)) ≤ Ft(ϕ( | γt(ϕ′)P ξt(ϕ′)) +∇Ft(ϕ( | γt(ϕ′)P ξt(ϕ′))i (ϕ− ϕ() +a2‖ϕ− ϕ(‖22. (4.28)6. The step size λ is sufficiently small and b is sufficiently large such thatζ =1Xλ(1− 2aλ)b +2aλ1− 2aλ Q 1. (4.29)7. ∇Ft(ϕ | γt(ϕ′)P ξt(ϕ′)) is uniformly continuous in (ϕPϕ′) for all t.Further, let S = {ℓ∗(k) Q ∞ for all k}. Then, P{S} = 1, and the following holds on S:all limit points of {ϕk}∞k5( are stationary points of log p(y; ·), and log p(y;ϕk) convergesmonotonically to log p∗ = log p(y;ϕ∗) for some stationary point of log p(y; ·), ϕ∗.Conditions 1–3 are from Wu (1983) and are standard assumptions needed to prove the con-vergence of the EM algorithm. Likewise, Conditions 4–6 are from Johnson and Zhang (2013)and are standard assumptions used to prove common properties of stochastic optimizationalgorithms. Condition 7 is needed to prove that SVRG is a continuous mapping.Condition 2 from Wu (1983) and condition 5 of Johnson and Zhang (2013) can be restrictiveand are often violated in common settings. For example, both are violated when estimatingthe variance of normal state-dependent distributions within an HMM. This issue is well-known for maximum likelihood estimation in mixture models (Chen and Li, 2009; Liu et al.,2015a). It can be avoided by setting lower bounds on the variance components (Zucchiniet al., 2016).Condition 4 seems concerning at first because the log-likelihood of a hidden Markov modelis usually multi-modal and non-convex. However, the convexity condition applies to Ftrather than the log-likelihood itself. In addition, Ft is a linear combination of probabilitydensities, which are often convex in θ (Boyd and Vandenberghe, 2004), and log-sum-expfunctions, which are convex in ν and η. Even if Ft is not convex, stochastic gradientmethods can escape local optima within the M step more effectively than standard gradientdescent (Kleinberg et al., 2018).814.4. Practical ConsiderationsTheorem 4.1 guarantees convergence only to a local optimum of the likelihood for version2 of EM-VRSO with P = False and A = SVRG. However, some studies suggest that EMalgorithms like EM-VRSO may escape local optima of the likelihood faster than direct numer-ical optimization of the log-likelihood (Zhang et al., 2020). Unfortunately, the convergenceanalysis for EM-VRSO becomes more complex when P = True, as the E and M steps areintertwined. Nonetheless, Theorem 4.2 below demonstrates that stationary points of thelog-likelihood function serve as fixed points of Algorithm 5 for all values of P and A. Theproof of Theorem 4.2 is provided in Appendix C.Theorem 4.2. If ∇ log p(y;ϕ() = 0, then for all λ ∈ R, A ∈ {SAGAPSVRG}, P ∈{TrueP False}, b ∈ N, and K ∈ N, EM-VRSO(ϕ(P λP AP PPbPK) = ϕ( with probability 1,where EM-VRSO is defined in Algorithm 5.Theorem 4.2 is useful because it guarantees that Algorithm 5 does not change ϕ( when itis a stationary point of the likelihood. However, it makes no guarantees that Algorithm5 will converge if ∇ log p(y;ϕ() ̸= 0 and either P = True or A = SAGA. Nonetheless, Isee in my simulation and case studies that Algorithm 5 approaches local maxima of thelog-likelihood function faster than existing full-batch baselines for all values of P and A.For theoretical guarantees on convergence, practitioners can set P = True or A = SAGA fora predetermined number of iterations, followed by switching to P = False and A = SVRG,or a full-gradient method such as BFGS (Fletcher, 2000).4.4 Practical ConsiderationsAlgorithm 5 outlines a method to perform the Baum-Welch algorithm with a stochasticE and M step. This section outlines implementation details that improve its practicalperformance.4.4.1 Line Search for Step Size SelectionOne drawback of stochastic optimization is that the step size can greatly affect the practicalperformance of the algorithm. Defazio et al. (2014) suggest a step size of λ = 1/(3a) forSAGA, where a is the Lipschitz constant defined in Theorem 1. However, the Lipschitzconstants are rarely known in practice. Therefore, following Schmidt et al. (2017), I initializean estimate of the Lipschitz constant, aˆ, and update it if the following inequality does nothold at any step m of the optimization algorithm:824.4. Practical ConsiderationsF(m+))tm(ϕ(m) − 1aˆ∇F (m+))tm (ϕ(m)))≤ F (m+))tm(ϕ(m))− 12aˆ∥∥∥∇F (m+))tm (ϕ(m))∥∥∥2P whereF(m+))tm = Ftm(·∣∣∣ γ̂(m+))tm P ξ̂(m+))tm ) (4.30)The inequality above is obeyed if aˆ = a, so if it is violated, then I double the Lipschitzconstant estimate aˆ. I also follow Schmidt et al. (2017) and do not check the inequality if‖∇F (m+))tm (ϕ(m))‖2 Q 10−0 due to numerical instability.In addition, the Lipschitz constant a is a global quantity, but the algorithm will likelyremain in a neighborhood around a local optimum of F later in the optimization algorithm.Within this local neighborhood, a smaller value of a may apply (Schmidt et al., 2017),so the Lipschitz constant estimate is decreased by a small amount after each parameterupdate: aˆ← 2−)/i aˆ. Updating aˆ after each parameter update allows the step size of theoptimization algorithm to adapt to the smoothness of the objective function close to theoptimum value.4.4.2 Multiple Step SizesThe optimization problem within the M step of the Baum-Welch algorithm can be written asseparate optimization problems over θ, η, and ν (recall that ϕ = {θPηPν}). In particular,I rewrite Ft as Ft(ϕ | γtP ξt) = Gt(θ | γtP ξt) +Ht(ηPν | γtP ξt), whereGt(θ | γtP ξt) = −c∑i5)γ(i)t log f (i)(yt; θ(i))P t = 1P . . . P iP (4.31)H)(ηPν | γ)P ξ)) = −c∑i5)γ(i)) log δ(i)(ν)P (4.32)Ht(ηPν | γtP ξt) = −c∑i5)c∑j5)ξ(i,j)t logΓ(i,j)(η)P t = 2P . . . P i. (4.33)To this end, let ∇̂G(m)t be equal to the components of ∇̂F (m)t that correspond to θ, and let∇̂H(m)t be equal to the components of ∇̂F (m)t that correspond to η and ν. Likewise, let∇̂G(m) be equal to the components of ∇̂F (m) that correspond to θ, and let ∇̂H(m) be equalto the components of ∇̂F (m) that correspond to η and ν. Then, I rewrite the gradient stepin the stochastic M step of Algorithm 4 as834.4. Practical Considerationsθ(m+)) = θ(m) − λθ[∇Gtm(θ(m)∣∣∣ γ̂(m+))tm P ξ̂(m+))tm )− ∇̂G(m)tm + ∇̂G(m)] P (4.34){ηPν}(m+)) = {ηPν}(m) − λη,ν[∇Htm(η(m)Pν(m)∣∣∣ γ̂(m+))tm P ξ̂(m+))tm )− ∇̂H(m)tm + ∇̂H(m)] .(4.35)where λθ = λη,ν = λ. Note that Gt is a function of only θ and Ht is a function of only ηand ν for given γ̂t and ξ̂t and all t = 1P . . . P i . As such, I allow λθ ̸= λη,ν and have eachdepend upon different Lipschitz constant estimates: λθ = 1/(3aˆG) and λη,ν = 1/(3aˆH).The line search described in Section 4.4.1 can then be used to update the estimates aˆG andaˆH separately.4.4.3 Adaptive Step Size for Fixed Lipschitz ConstantsUnder certain regularity conditions, Defazio et al. (2014) prove that SAGA converges usinga step size of 1/(3a) for a given loss function F with Lipschitz constant a. I thereforeinitialize step sizes of λθ = 1/(3aˆG) and λη,ν = 1/(3aˆH) for all experiments and algorithms.However, for Algorithm 5 with P = True, the objective function F (· | γ̂(m+))tm P ξ̂(m+))tm ) =G(· | γ̂(m+))tm P ξ̂(m+))tm ) + H(· | γ̂(m+))tm P ξ̂(m+))tm ) itself changes over the course of a singleM step as γ̂(m+))tm and ξ̂(m+))tm are updated. As a result, more conservative (i.e. smaller)step sizes λθ and λη,ν may be needed, even if the Lipschitz constant estimates aˆG and aˆHare accurate. I therefore allow Algorithm 5 to change the step size after each attempt ℓthrough the M step of the Baum-Welch algorithm. Namely, if the log-likelihood does notdecrease after iteration k and attempt ℓ through the M step of Algorithm 5 with P = True,I halve the step size (as a function of either aˆG or aˆH) for attempt ℓ + 1. For example,if the step sizes are λθ = 1/(3aˆG) and λη,ν = 1/(3aˆH) for attempt ℓ through a givenM step, and attempt ℓ results in a decrease of the log-likelihood, I define new step-sizesλθ ← 1/(6aˆG) and λη,ν ← 1/(6aˆH) for attempt ℓ+1. I also maintain this halved step sizefor the remainder of Algorithm 5.4.4.4 Sampling for SAGA and SVRG Without ReplacementFinally, I sample each random index t without replacement within Algorithm 4. If b S i ,then I sample without replacement until all time indices are sampled, and then re-samplethe data set without replacement. Sampling without replacement for SGD is often easierto implement and performs better than sampling with replacement (Gürbüzbalaban et al.,2021). Shamir (2016) also gives several convergence results for SVRG when indices aresampled without replacement.844.5. Simulation Study4.5 Simulation Study4.5.1 Simulation ProcedureTo test the performance of Algorithm 5, I ran eight simulation experiments. For a givenexperiment, I simulated i ∈ {10+P 105} observations from an HMM with N ∈ {3P 6} hiddenstates and observations yt ∈ Ry, with y ∈ {3P 6}. All possible combinations of i , N , and ycomprised a total of 2+ = 8 experiments. For each experiment, I simulated five data sets.For every experiment and data set, nt | mt = i followed a normal distribution with meanµ(i) and covariance matrix Σ(i). I defined µ(i) and Σ(i) for every data set as µ(i) ∼ N (0P I)and Σ(i) = diag[exp(−2)] for i ∈ {1P . . . P N} where I is the identity matrix. This simulationprocedure resulted in relatively well separated state-dependent distributions. For example,when N = 3, the Euclidean distance between any pair of means within any of the fivedata sets ranged between 1.0 and 4.0 for y = 3 and between 2.0 and 4.7 for y = 6. I setthe transition probability matrices of the generating process to depend upon i so that theexpected number of total transitions was 100 for all experiments. My purpose in keeping theexpected number of transitions fixed was to induce a high degree of sequential dependencewhile simultaneously encouraging each hidden state to be visited in the simulated timeseries. This decision also simulates sequences of observations that are sampled at either low(i = 10+) or high (i = 105) frequencies. Denote the true transition probability matrix froman experiment with i observations and N hidden states as Γi,c . I defined Γ)(3,+ ∈ R+×+to have diagonal elements of 0.9 and off-diagonal elements of 0.05, Γ)(3,6 ∈ R6×6 to havediagonal elements of 0.9 and off-diagonal elements of 0.02, Γ)(5,+ ∈ R+×+ to have diagonalelements of 0.999 and off-diagonal elements of 5× 10−4, and Γ)(3,6 ∈ R6×6 to have diagonalelements of 0.999 and off-diagonal elements of 2 × 10−4. I randomly defined the initialdistribution as δ ∼ dir(1c ) for every experiment and data set.4.5.2 Optimization ProcedureI estimated the parameters of the generating model for all five data sets and all eight experi-ments using six different versions of Algorithm 5. In particular, I used A ∈ {SVRG, SAGA},and for each value of A, I used the combinations {P = FalseP b = i}, {P = TrueP b =i}, and {P = TrueP b = 10i}. Recall that setting P = True corresponds to integratinga partial E step into the variance-reduced stochastic M step. The variable b correspondsto the number of iterations of SAGA or SVRG that are performed at each M step of thealgorithm. It is natural to set b = i to approximately balance the computational load ofthe E step and the M step. Nonetheless, I ran an experiment with b = 10i and P = Trueto test the algorithm when the majority of the computational load was placed on the com-bined partial E / stochastic M step. As a baseline, I also estimated the HMM parameters854.5. Simulation Studyusing direct likelihood maximization via three gradient-based methods: BFGS (Fletcher,2000), the conjugate gradient method (Fletcher and Reeves, 1964), and full-batch gradientdescent. The model used in my simulation study has closed-form solutions to the M stepof the Baum-Welch algorithm. However, I did not include the standard Baum-Welch algo-rithm as a baseline method because I did not want to rely on closed-form solutions to the Mstep. If such solutions exist in practice, I recommend using either the standard Baum-Welchalgorithm or the Baum-Welch algorithm with a partial E step (Neal and Hinton, 1998).I sampled a total of five random parameter initializations for each data set / experimentpair, and then ran all nine optimization algorithms on every parameter initialization. Eachparameter initialization was re-used for each algorithm to ensure consistency. Throughoutthe optimization procedure, I assumed that Σ(i) was diagonal for all i ∈ {1P . . . P N}, whichis in line with the generating model described above. Further, I reparameterized Σ(i) asΣ(i) = diag[exp(ρ(i))]and performed inference on ρ(i) =(ρ(i)) . . . ρ(i)y)for i = 1P . . . P N ,which is unconstrained. Let y¯ denote the sample mean and Q denote the sample covarianceof the observation sequence {yt}it5). I initialized θ( = {µ(i)( Pρ(i)( }ci5) as µ(i)( ∼ N (y¯P diag(Q))and ρ(i)( ∼ N (log(diag(Q))P 2I) for i = 1P . . . P N . I initialized ν( as ,(i)( ∼ N (0P 1) fori = 2P . . . P N and I initialized η( as η(i,j)( ∼ N (−2P 22) for iP j = 1P . . . P N , where i ̸= j.All six algorithms were initialized with step sizes of λθ = 1/3aˆG and λη,ν = 1/3aˆH .The Lipschitz constants were initialized as aˆG = aˆH = 100/3 and updated during theoptimization routine according to the procedure from section 4.4.1. All algorithms andbaselines were run for a total of 12 hours on the Compute Canada Cedar cluster on nodeswith 16GB of RAM. All baselines were implemented using the Scipy library in Python(Virtanen et al., 2019), and I implemented Algorithm 5 using a custom Python script.I employed several measures to fairly compare the performance of each optimization algo-rithm. To account for differences in speed due to implementation discrepancies, I measuredcomputational complexity in epochs in addition to raw computation time. I defined oneepoch as either i evaluations of Equations (4.19 – 4.22) in the E step of Algorithm 5, istochastic gradient evaluations in the M step of Algorithm 5, or one gradient evaluationin the full-gradient baseline algorithms. I estimated the true maximum likelihood parame-ters ϕ∗ for each data set / experiment pair using the parameters from the best-performingoptimization algorithm and initialization after 12 hours. Convergence was defined as thepoint when the gradient norm of the log-likelihood (divided by i ) was less than 10−2. Thetolerance was set to 10−2 because it was the lowest tolerance that all algorithms regularlyconverged to within 12 hours. Even though each algorithm was run for a full 12 hours, Idenoted the epoch and time when each algorithm either converged or finished running asthe moment of “termination”.864.6. Case Study4.5.3 Simulation ResultsThe results from the simulation study are shown in Figures 4.2–4.4. I was primarily focusedon the big-data setting and thus only present results from experiments with i = 105 in themain text. All figures associated with experiments for i = 10+ can be found in AppendixC.Algorithm 5 with A = SVRG markedly sped up the optimization procedure, as it usuallyconverged in at most half as many epochs compared to the baselines for each experiment.Algorithm 5 with A = SAGA also tended to converge in fewer epochs than the baselines forexperiments with N = 6, particularly when P = True and b = 10i . One epoch of eachbaseline method took less computation time than one epoch of Algorithm 5, but Algorithm5 with A = SVRG consistently converged before the baseline methods in terms of rawcomputation time as well as epoch (Figure 4.3 and Appendix C). All methods were prone toconverge to local minima of the negative log likelihood, especially when N = 6, in which casethe likelihood surface was highly multi-modal. However, the bottom row of Figure 4.3 showsthat in all i = 105 experiments, negative log-likelihood values at termination were lower forall versions of Algorithm 5 compared to baseline methods. In particular, each optimizationalgorithm / experiment pair has a corresponding box plot which summarizes the (re-scaled)negative log-likelihood values at termination from all data sets and initializations. For allexperiments, all medians and minimums corresponding to versions of Algorithm 5 were lowerthan the median and minimum values corresponding to BFGS. The results for experimentswith i = 10+ were more varied (see Figure C.1 of Appendix C), which indicates that myinference method is more efficient when applied to data sets with large i . I present figuresanalogous to Figure 4.2 for all five data sets corresponding to all experiments in AppendixC.4.6 Case StudyI tested the performance of my optimization algorithm by modeling the movement of eightnorthern resident killer whales off the coast of British Columbia, Canada. Biologgers are anessential tool used to understand the behavior of marine mammals. For example, time-depthrecorders allow researchers to estimate behavioral states associated with each dive (e.g.,foraging, resting, and traveling, Tennessen et al., 2023; McRae et al., 2024). Researchersalso use biologging data sets to identify and characterize dive phases, which are importantfor inferring behavior (e.g., prey capture often occurs in the bottom phase of a foraging dive,Wright et al., 2017; Jensen et al., 2023). As such, I developed a model to identify threecommon dive phases (ascent, descent, and bottom) and three dive types that may indicate874.6. Case StudyFigure 4.2: Maximum log-likelihood (across all algorithms and initial values after 12 hours),ℓ(ϕ∗), minus the log-likelihood at the given epoch, ℓ(ϕ), divided by i for a selected run ofeach optimization algorithm in the simulation study. Each algorithm was run for 12 hourson a data set generated with i = 105, N ∈ {3P 6}, or y ∈ {3P 6}. For each experiment andoptimization algorithm, I display the one random initialization (of five) that resulted in thehighest likelihood after 12 hours. FE corresponds to P = False, and PE corresponds toP = True. The y-axis is on a log-scale. Dots correspond to the epoch and likelihood attermination.884.6. Case StudyFigure 4.3: Box plots showing time to terminate (top), epochs to terminate (middle), andmaximum log-likelihood minus log-likelihood at termination (all over i , bottom) for eachoptimization algorithm in the simulation study. The log-likelihood is denoted as ℓ(ϕ),and the maximum log-likelihood across all algorithms and initial values after 12 hours isdenoted as ℓ(ϕ∗). Unlike Figure 4.2, results include all five data sets and all five parameterinitializations per data set. FE corresponds to {P = FalsePb = i}, PE1 corresponds to{P = TruePb = i}, and PE2 corresponds to {P = FalsePb = 10i}. Results are shownfor all simulation studies with i = 105. The y-axis of the bottom row is on a log-scale.894.6. Case StudyFigure 4.4: Maximum log-likelihood minus log-likelihood at termination (all over i ) versusepochs to terminate for each optimization algorithm in the simulation study with i = 105,N ∈ {3P 6}, and y ∈ {3P 6}. The log-likelihood is denoted as ℓ(ϕ), and the maximum log-likelihood across all algorithms and initial values after 12 hours is denoted as ℓ(ϕ∗). UnlikeFigure 4.2, results include all five data sets and all five parameter initializations per dataset. FE corresponds to P = False, and PE corresponds to P = True. The y-axis is on alog-scale.904.6. Case Studydistinct behaviors of the animal, including resting, foraging, and traveling. I performedinference on the resulting model using my optimization algorithm in order to illustrate itscomputational advantages.4.6.1 Data Collection and PreprocessingThe data in this case study were collected in August and September 2020 using a CATS time-depth recorder, or TDR (Customizable Animal Tracking Solutions, www.cats.is). Northernresident killer whales were equipped with suction-cup attached CATs tags in Queen Char-lotte Sound using an adjustable 6-8m carbon fiber pole. The tags were programmed torelease within 3-24 hours of attachment. Instruments were retrieved following each deploy-ment using a Wildlife Computers 363C SPOT tag (providing Argos satellite positions),goniometer, ultra high frequency receiver, and yagi antenna. The tags included 3D kine-matic sensors (accelerometer, magnetometer, gyroscope), time-depth recorder, hydrophoneand camera. All sensors were programmed to sample at 50 hertz. However, for the purposesof this study, I focus on the time-depth recorder data to discern behaviorally distinct dives.Together with a subject matter expert, I calibrated the depth readings using a MATLABpackage developed by Cade et al. (2021), and defined a dive as any sequence of depth read-ings under 0.5 meters that lasted for at least two seconds. I then down-sampled the depthreadings to a frequency of 0.5 hertz. The processed data set contained a total of 5858 divesand 89462 depth readings. Figure 4.5 shows the depth and change in depth for a subset ofdives for one whale in the data set.4.6.2 Model FormulationDive phases may vary depending upon the animal’s behavior. For example, foraging divestend to be deeper and longer than resting dives, so it is natural to model the phases offoraging dives differently compared to those of resting dives (Tennessen et al., 2019a). Assuch, I used a hierarchical model to jointly model dive types and dive phases (Leos-Barajaset al., 2017; McRae et al., 2024). Hierarchical HMMs are specific instances of traditionalHMMs, so the machinery developed here is applicable to perform inference. I assumed thereto be three dive types, which is consistent with other studies of cetaceans (e.g., resting,foraging and traveling, Leos-Barajas et al., 2017). I also assumed that there were threedive phases per dive type (descent, bottom, and ascent), which is also consistent with otherstudies of diving birds and mammals (e.g., Viviant et al., 2014). This resulted in a total ofN = 9 hidden states, each corresponding to a different dive phase / dive type combination.Rather than modeling raw depth every two seconds as the observation sequence, I encodedeach two-second window of depth data with summary statistics. Namely, I denoted an914.6. Case StudyFigure 4.5: Depth profile and change in depth vs time of day for a selected killer whale(I107, male, born 2004) off the coast of British Columbia, Canada. The data in panels oneand three are color-coded according to the most likely hidden coarse-scale state (i.e. divetype) for each dive. The data in panels two and four are color-coded according to the mostlikely hidden fine-scale state (i.e. dive phase) for each two-second window.924.6. Case Studyobservation as nt = {YtP Zt}, where Yt ∈ R is the change in depth in meters and Zt = 1if a dive ended at index t and Zt = 0 otherwise. Within dive type i and dive phase j, Iassumed Yt followed a normal distribution with mean µ(i,j) and standard deviation σ(i,j),and I assumed that Zt followed a Bernoulli distribution with probability p(i,j). I assumedthat dives must end on the ascent phase, so I set p(i,)) = p(i,2) = 0 for dive types i = 1P 2P 3.Conditioned on the dive type and dive phase, Yt and Zt were assumed to be independentof one another.Since each dive must begin with the descent phase, I set the initial distribution δ to have theform δ =(δ()) 0 0 δ(2) 0 0 δ(+) 0 0), where δ(i) represents the probability that akiller whale begins its dive profile with a dive of type i. I defined the transition probabilitymatrix at time t to depend upon the previous observation Zt−) because the transition matrixwill naturally depend upon when a dive begins and ends. For example, the killer whalecannot change dive type mid-dive, it must begin each dive in the descent phase, and it mustend each dive in the ascent phase. The structure of this transition probability matrix meansthat my model is technically a generalization of a standard HMM (namely an HMMSDO,Li, 2005). Nonetheless, Li (2005) and Tamposis et al. (2018a) show that the likelihood ofthis model is similar to a traditional HMM, and that the standard Baum-Welch algorithmis still valid to perform inference. To this end, I defined a coarse-scale, inter-dive transitionprobability matrix Γ(x) ∈ R+×+ as well as a fine-scale, intra-dive probability transition matrixΓ(f,i) ∈ R+×+ for each dive type i. Γ(f,i) was upper-triangular for i ∈ {1P 2P 3} because thedescent and bottom phases of a dive cannot occur after ascent, and the descent phase of adive cannot occur after the bottom phase. Formally, the transition matrix was a functionof Zt−) and defined asΓt =Γ(f,)) 0 00 Γ(f,2) 00 0 Γ(f,+) P Zt−) = 0P (4.36)Γt = Γ(x) ⊗1 0 01 0 01 0 0 P Zt−) = 1P (4.37)where ⊗ denotes the Kronecker product. See Appendix C for an expanded version of Γt.934.6. Case Study4.6.3 Optimization ProcedureI used a procedure similar to the simulation study to initialize the case study parameters.Let Y¯ denote the sample mean and s denote sample standard deviation of {Yt}it5). Iinitialized the initial estimates for the mean(µ(i,j)()and the log of the standard deviation(log(σ(i,j)())of the state-dependent density of Yt as µ(i,j)( ∼ N (Y¯P s2) and log(σ(i,j)()∼N (log(s)P 1) for iP j = 1P 2P 3, where i refers to the dive type and j to the dive phase.Further, let Z¯ represent the mean of {Zt}it5). I initialized the state-dependent probabilityof observing a dive end as p(i,))( = 0, p(i,2)( = 0 and logit(p(i,+)( ) ∼ N (logit(Z¯)P 1) for i = 1P 2P 3,where p(i,j)( is the initial estimate corresponding to the Bernoulli distribution of Zt duringdive type i and dive phase j. Dive phase 3 is ascent.Let νk ∈ R+ denote the parameters associated with δ at iteration k of a given optimizationalgorithm. I initialized the first element of ν( as zero for identifiability and the second andthird elements with a standard normal distribution, N (0P 1).Let η(x)k ∈ R+×+ denote the parameters associated with the coarse-scale probability transi-tion matrix at iteration k of a given optimization algorithm. The reparameterization fromη(x)k to Γ(x)k is given in Equation (4.2). I initialized the diagonal elements of η(x)( as zeros,and I initialized the off-diagonal elements of η(x)( with a normal distribution with mean −3and unit variance, N (−3P 1).Let η(f,i)k ∈ R+×+ denote the parameters associated with fine-scale transition probabilitymatrix Γ(f,i)k . The reparameterization from η(f,i)k to Γ(f,i)k is given in Equation (4.2). Iinitialized all diagonal elements of η(f,i)( as zeros and all off-diagonal elements of η(f,i)( witha normal distribution with mean -1 and unit variance, N (−1P 1).Similarly to the simulation study, I estimated the parameters of the model using six dif-ferent inference algorithms (I set A ∈ {SVRG, SAGA} in addition to setting (PPb) ∈{(FalseP i )P (TrueP i )P (TrueP 10i )}) and direct likelihood maximization via three baselinealgorithms (BFGS, conjugate gradient, and gradient descent). All algorithms were run us-ing 50 random initializations for a total of 12 hours each on Compute Canada Cedar nodeswith 16GB of RAM.I employed similar measures as those in the simulation study to fairly compare the optimiza-tion algorithms. In particular, I measured computational complexity in epochs in additionto raw computation time and defined an epoch using the definition from the simulationstudy. I also estimated the maximum likelihood parameters ϕ∗ for each data set / exper-iment pair using the same method as the simulation study. Finally, I recorded the epochand likelihood at termination for each optimization algorithm using the same definition of944.6. Case StudyFigure 4.6: Maximum log-likelihood minus log-likelihood (all over i ) vs epoch and com-putation time for the model from the killer whale case study. All optimization algorithmswere run for a total of 12 hours. For each optimization algorithm, I display the one randominitialization (of 50) that resulted in the highest likelihood after 12 hours. FE correspondsto P = False, and PE corresponds to P = True. The log-likelihood of ϕ is denoted asℓ(ϕ) on the y-axis, which is on a log-scale. Dots correspond to the epoch and likelihood attermination for each algorithm.termination as the simulation study.4.6.4 Case Study ResultsMy model predicted dive phases and dive types that are in line with previous studiesof marine mammal diving behavior. For example, dive types are separated by shallow,medium, and deep depths, which is similar to results from Leos-Barajas et al. (2017).Further, each dive has a well-characterized bottom phase that occurs at approximately70% of the maximum depth for all dive types. This finding is similar to the results fromTennessen et al. (2019b). See Figure 4.5 and Appendix C for further detail.Most importantly, this case study demonstrates that all of my stochastic algorithms con-verged in fewer epochs and to regions of higher likelihood compared to the full-batch base-lines; see Figures 4.6–4.7.All algorithms occasionally converged to sub-optimal local minima, but Algorithm 5 withA = SVRG, P = True, andb = i tended to converge with the highest likelihood relatively954.7. DiscussionFigure 4.7: Maximum log-likelihood minus log-likelihood at termination (all over i ) versusepochs to terminate for the killer whale case study. Unlike Figure 4.6, results include all 50parameter initializations. FE corresponds to P = False, and PE corresponds to P = True.The log-likelihood of ϕ is denoted as ℓ(ϕ) on the y-axis, which is on a log-scale.quickly in terms of both epoch number and raw computation time (see Figure 4.7). As withthe simulation study, Algorithm 5 with A = SVRG tended to converge in fewer epochscompared with A = SAGA (see Figure 4.6). Setting P = True appears to be of particularuse early in the optimization procedure (i.e the first ≈ 5 epochs, see Figure 4.6). Thisbehavior is intuitive because the proper weights(γ(ϕ(m)k ) and ξ(ϕ(m)k ))change rapidlyearly in the optimization procedure.4.7 DiscussionThe advent of high-frequency sensing technology has allowed researchers to model exception-ally long, high-frequency stochastic processes with increasingly complex HMMs (Pattersonet al., 2017). However, these complex models can be computationally expensive to fit (Glen-nie et al., 2023). I introduce an inference algorithm that speeds up maximum likelihoodestimation for HMMs compared to existing batch-gradient methods. I do so without ap-proximating the likelihood, which is required for many existing stochastic inference methods(Gotoh et al., 1998; Ye et al., 2024).My method does not require a closed-form solution for the M step, which enables quickinference for a diverse range of HMM models. Such a method is useful in practice because964.7. Discussionmany HMMs lack closed-form solutions for the M step. In finance, Langrock et al. (2018)modeled the relationship between the price of energy and price of the oil in Spain usingnon-parametric HMMs. In ecology, Lawler et al. (2019) used autoregression in the state-dependent distributions of an HMM to model the movement of grey seals (Halichoerusgrypus). Likewise, Pirotta et al. (2018) used covariates in the transition probability matrixof an HMM to determine the effect of fishing boats on the behavior of Northern Fulmars(Fulmarus glacialis). My method will be especially relevant as advances in biologging andtracking technology allow practitioners to collect increasingly large and high-frequency datasets (Patterson et al., 2017).My new algorithm is particularly effective when performing inference over large data sets.For example, when applied to simulations with i = 105 observations, my algorithm con-verged in approximately half as many epochs and tended to converge to regions of higherlikelihood compared to existing baselines. The case study of killer whale kinematic datashowed similar improvements, demonstrating how my optimization procedure makes com-plex hierarchical HMMs less computationally expensive to fit on large biologging data sets.The partial E step variant of my algorithm (i.e. with P = True) outperforms baselinesparticularly well early in the optimization procedure (in the first ≈ 5 epochs). As such,using a partial E step may be particularly advantageous when researchers are modelinglarge data sets with relatively low convergence thresholds.One method that is particularly aligned with my algorithm is that of Zhu et al. (2017),who implement variance-reduced stochastic optimization to perform the M step of the EMalgorithm on high-dimensional latent-variable models. Their method obtains a sub-linearcomputational complexity in the length of the observation sequence as well as a linearconvergence rate. However, they focus primarily on mixture models rather than HMMs,and they do not combine the variance-reduced stochastic M step with a partial E step,which is an extension that I implement here. Further, their theoretical results assumeindependence between observations, which I do not rely on here.Future work can explore the performance of my new algorithm when applied to increas-ingly complex HMMs. Parameter inference for these complicated models may add diffi-culties beyond those presented in my simulation and case studies. For example, HMMswith covariates in the transition matrix are less stable than time-homogeneous HMMs, andoptimization algorithms can easily get stuck in local optima of the likelihood surface. Myoptimization algorithm was less prone to getting stuck in local optima than the baselinemethods in this work, so the same may be true for more complicated HMMs.While I use SVRG and SAGA in my analysis, there are other variance-reduced stochastic974.7. Discussionoptimization algorithms that could be applied within my framework. For example, SARAHrecursively updates the control variate in the inner loop of the optimization algorithm(Nguyen et al., 2017), SPIDER uses a path-integrated differential estimator of the gradient(Fang et al., 2018), and LiSSA is a second-order variance-reduced stochastic optimizationscheme (Agarwal et al., 2017). Future work can integrate these algorithms within myframework and evaluate the resulting performance. While I focus on an ecological casestudy here, the inference procedures I developed can unlock more complicated HMMs withlarger latent spaces and bigger data sets for practitioners across a variety of disciplines.98Chapter 5Conclusions5.1 Contributions to Statistical MethodologyHidden Markov models are an important and growing statistical method to describe timeseries data (Figure 1.1). They are simple and interpretable, yet general and flexible, allowingresearchers to infer parameters and decode hidden processes more easily than many otherlatent variable models. However, as tracking and sensing technology improves, time seriesgrow larger and more complex, which can cause several modelling difficulties (Nathan et al.,2022). While researchers have developed HMMmethodology to extract as much informationas possible from these time series (Leos-Barajas et al., 2017; Langrock et al., 2018; Lawleret al., 2019), many challenges still remain. For example, studies involving HMMs can fail tocapture intricate dependence structures associated with fine-scale behaviours (Adam et al.,2019), they can perform sub-optimally when incorporating rare labels of hidden behaviours(Saldanha et al., 2023), and they can encounter computational difficulties when fittingmodels to large data sets (Sacchi and Swallow, 2021). This dissertation addressed all threeof these problems, thus advancing the utility of HMMs to model complex time series in arange of applications.5.1.1 Modelling Fine-Scale DependenceHigh-frequency time series often display intricate dependence structures that violate tra-ditional assumptions of HMMs at very fine scales. To address this issue, in Section 2.2I proposed a moving-window approach where each window is summarized with biologi-cally relevant summary statistics. I modelled each summary statistic as a random variablegenerated from an unobserved hidden state, and I modelled the sequence of window-levelhidden states as a Markov chain. The size of the moving window should be small enoughto capture fine-scale state-switching behaviours, but large enough to fully contain complexdependence structures. I used this moving window approach throughout this dissertationto extract meaningful information from high-frequency killer whale kinematic data. Forexample, in Section 2.3 I used signal processing techniques within each window to extractfine-scale swimming behaviours. In Section 3.4.2, I used foraging-related summary statistics995.1. Contributions to Statistical Methodologyof each window to identify fine-scale foraging behaviours such as prey capture. In Section4.6, I extracted change in depth over a window to identify individual killer whale divephases, including descent, bottom, and ascent.Researchers have developed several extensions of traditional HMMs to model complex eco-logical processes, and I applied some of these HMMs extensions to model the moving win-dows. For example, I employed a CarHMM (Lawler et al., 2019) in Section 2.3 to modelautocorrelation between windows that was not explained by state-switching behaviour alone.In addition, I used hierarchical HMMs (Leos-Barajas et al., 2017; Adam et al., 2019) in Sec-tions 2.3 and 4.6 to simultaneously model killer whale dive types and, within each dive,window-level hidden states.5.1.2 Incorporating Rare LabelsIn addition to collecting high-frequency time series, modern technology also allows practi-tioners to directly observe and label previously hidden processes. Incorporating these labelsinto a semi-supervised HMM can improve its predictive performance (Krogh, 1997), but theeffect can be negligible when the labels are exceedingly rare (Chapelle et al., 2006). There-fore, in Section 3.3 I proposed a weighted likelihood approach for semi-supervised HMMsthat balances the influence of labelled and unlabelled observations. This weighted approachis simple and intuitive for researchers to implement. It can reduce to an unweighted like-lihood approach (if the weight α = 1) or correspond to a model that completely ignoresunlabelled examples (if the weight α = 0). Practitioners can use cross-validation withtheir preferred accuracy metric to select the optimal weight α. In two case studies of killerwhale foraging behaviour, I used cross-validated accuracy metrics to demonstrate that myweighted likelihood approach improves predictive performance compared to current baselinemethods.5.1.3 Performing Efficient InferenceIf researchers apply a complicated HMM to a very large time series, then parameter infer-ence can be computationally prohibitive. One way that this dissertation made parameterinference more efficient is through the moving window approach proposed in Section 2.2and reviewed in Section 5.1.1. Namely, the moving window approach summarizes windowscontaining long sequences of data by a relatively small number of summary statistics. Forexample, in Chapter 2 I summarized two-second windows of 50 Hertz, three-axis accelera-tion data with only four summary statistics, reducing the size of the data set by a factor ofalmost 100. Similar gains were achieved using the moving window approach in Chapters 3and 4.1005.2. Impact Beyond StatisticsEven though my moving window approach drastically compresses ecological biologging data,the resulting time series of summary statistics can still be large. To speed up computation,in Section 4.3 I developed a new inference algorithm that uses variance-reduced stochasticoptimization in the M step of the EM algorithm for HMMs. I also integrated a partialE step within the variance-reduced stochastic M step. The resulting algorithm does notiterate through the entire dataset for every parameter update, and it does not requireclosed-form solutions to the M step. In Theorem 4.1, I proved that my algorithm convergesmonotonically to a local optimum of the likelihood under standard regularity conditions.Conversely, in Theorem 4.2 I proved that all local optima of the HMM log-likelihood arefixed points of my algorithm under even less strict conditions. I conducted eight simulationstudies and a case study of killer whale kinematic data to compare the performance of myalgorithm with direct numerical optimization in scenarios with varying numbers of hiddenstates N , observation dimensions y, and time series lengths i . My method outperformeddirect numerical optimization in all of these scenarios, but it was particularly effective whenN , y, and i were large.5.2 Impact Beyond StatisticsThis dissertation provides researchers across many disciplines with methods to model state-switching behaviour within high-frequency time series at multiple scales. These methods areparticularly applicable to high-frequency data from diving animals, including seals (Jean-niard du Dot et al., 2016), porpoises (Leos-Barajas et al., 2017), and sharks (Adam et al.,2019). However, the methods outlined here are straightforward to adapt to data from manyother species. Outside of ecology, high-frequency sensor data is common in many applica-tions, including neuroscience (Forbes, 2024), engineering (Kamarianakis et al., 2024), andweather modelling (Kashlak et al., 2023).New technology allows researchers to label previously hidden processes, but labels of latentprocesses are still rare in many fields. In ecology, Saldanha et al. (2023) found that labelsof foraging in red-billed tropic-birds (Phaethon aethereus) were more difficult to obtainthan labels of other behaviours. Outside of ecology, semi-supervised HMMs are seen inapplications including computational genetics (Tamposis et al., 2018b), human activityidentification (Ghazvininejad et al., 2011), and credit-score modelling (El Annas et al.,2023). The weighted likelihood method described in Chapter 3 demonstrably improved thepredictive performance of semi-supervised HMMs, so it is a promising technique for studiesacross disciplines that have access to rare labels of otherwise hidden processes.Next, this dissertation outlines an inference method that enables practitioners to fit HMMs1015.2. Impact Beyond Statisticsto large and complicated time series, even if the HMM does not have a closed-form solutionto the M step of its associated EM algorithm. For example, my method could be usedto fit models such as those in Langrock et al. (2018), who used non-parametric HMMs tomodel the price of oil in Spain; Lawler et al. (2019), who used auto-correlation in the state-dependent distribution of an HMM to model grey seal (Halichoerus grypus) movement; orPirotta et al. (2018), who used covariates in the transition probability matrix to model thebehaviour of northern fulmars (Fulmarus glacialis) in response to fishing vessels. Vesselnoise in particular is a major concern for southern resident killer whales, so future studiesof the southern residents may involve modelling large time series with HMMs similar tothose in Pirotta et al. (2018).The inference methods from Chapter 4 also allow researchers to perform otherwise com-putationally prohibitive simulation studies. For example, my computations in Chapter 2would have been prohibitive for researchers without access to a large computing cluster.Namely, I re-fit four candidate HMMs to 500 simulated data sets, each of which was onthe order of thousands of observations long. One of these models took an average of 4.3hours to fit, and I was able to estimate each model in parallel using a large computingcluster. However, if researchers cannot parallelize their computation and instead need toestimate the parameters of each HMM sequentially, this simulation study would requireapproximately 90 days of continuous computation. Thus, large simulation studies can beextremely computationally expensive, and my inference approach from Chapter 4 makesthose simulation studies more accessible to researchers.This dissertation also has important implications for the conservation of resident killerwhales off the coast of British Columbia, Canada. The threatened northern resident sub-population has been growing steadily over the past several decades, while the endangeredsouthern resident sub-population has remained stagnant (Fisheries and Oceans Canada,2018; Tennessen et al., 2023). Many factors may explain this disparity, including preyavailability, vessel disturbances, and pollution, but the exact causal mechanism is still notunderstood (Murray et al., 2021; Tennessen et al., 2023). Each of these factors affects forag-ing differently: killer whales have trouble finding prey if it is scarce, and vessel disturbancesoften interrupt killer whale foraging behaviour. Therefore, understanding the frequencyand success of northern and southern resident foraging has important implications for un-derstanding the causes of their divergent populations. To this end, I used the tools fromChapters 2 and 3 to estimate the amount of time spent foraging and the foraging successfor northern and southern resident killer whales. Although the number of whales in mycase study is small, my analysis supports the findings of Tennessen et al. (2023), who findthat southern residents spend less time travelling and resting and catch fewer fish per hour1025.3. Future Workforaging than the northern residents. I focus on killer whales here, but ecologists can usethe modelling techniques from this dissertation to decode the behaviours of other animalsand assist in their conservation as well.The methods from this dissertation have already been used and cited in studies both withinand outside of ecology. Within ecology, I co-authored a study which derived behaviour-specific respiration rates for killer whales (McRae et al., 2024). These findings are importantto understand a killer whale’s energetic needs (Jeanniard-du Dot et al., 2016). This disser-tation has also been cited independently by Togunov et al. (2021), who noted that samplingfrequency (e.g., the size of the moving window described in Chapter 2) can affect the per-formance of statistical methods for biologging data. Outside of ecology, this dissertationhas been cited independently by Kamarianakis et al. (2024), who used autoregression anda sliding window to perform renewable energy forecasting; Kashlak et al. (2023), who usedfunctional data analysis within a topological HMM to model snowfall in Edmonton, Canada;and Forbes (2024), who used functional data analysis to model brain activity for the mentalhealth care of adolescents.5.3 Future WorkThe moving window approach from Chapter 2 allows researchers to extract biologicallyrelevant information from high-frequency time series, but researchers must specify relevantsummary statistics and the window size itself prior to fitting an HMM. In reality, practi-tioners may not know the scale, structure, or complexity of biologically relevant behavioursbefore fitting their model. To this end, future work can focus on methods that automat-ically determine the size of the moving windows or the summary statistics within thosewindows. For learning the summary statistics in particular, deep learning models such asauto-encoders can learn a low-dimensional latent representation of the data within eachmoving window.Next, replacing the raw data within each window with summary statistics makes it difficultto reconstruct the original time series. This dissertation is primarily interested in decodingkiller whale behaviour, but some researchers may be interested in reconstruction as well. Tothis end, Aue et al. (2015) and Gao et al. (2019) describe methods that minimize predictionerror in functional time series. Alternatively, auto-encoders can reconstruct time seriesfrom low-dimensional latent representations. Incorporating these methods into the moving-window HMM approach outlined in Section 2.2 is a promising direction for future research.In addition, there are several possibilities to build on the weighted likelihood approachfor partially labelled HMMs presented in Chapter 3. First, the weighted likelihood from1035.3. Future WorkEquation (3.11) does not correspond to a valid probability density when α ∈ (0P 1), soseveral common techniques in maximum likelihood estimation do not apply. For example,researchers cannot reliably estimate uncertainty using the observed Fisher information,and standard expectation-maximization (EM) inference algorithms can not be used forparameter inference. Some theoretical guarantees for independent data sets are outlinedby Hu and Zidek (2002), so extending those results to PHMMs is a promising direction forfuture work. Second, I selected an optimal weight α using cross-validated accuracy metricssuch as AUC. However, the optimal value of α likely depends upon the scarcity of labels andthe level of model misspecification. Future work can investigate how the optimal value of αdepends upon properties of the underlying model. A smaller project may involve runningan extensive simulation study to investigate this relationship, while a more robust projectwould also involve proving theoretical results about approximation error when learning aPHMM under model misspecification. Third, the stochastic optimization scheme presentedin Chapter 4 does not explicitly involve any parallelization, but Zinkevich et al. (2010)proposed a method that involves parallelization within an SGD optimization algorithm.Future work can focus on incorporating parallelization into the methods from Chapter 4 toleverage modern computing clusters and GPU architectures.Next, the computational methods developed in Chapter 4 have many promising directionsfor future research. First, my variance-reduced stochastic EM algorithm involves a full Estep (and thus, a full pass of the data set) to initialize conditional probability estimatesand gradient estimates. However, Khreich et al. (2012) describes several online learningalgorithms that begin parameter estimation before a full E step is performed. Future workcan incorporate these online learning algorithms into the variance-reduced EM algorithmfrom this dissertation. Second, I tested my variance-reduced stochastic EM algorithm usingrelatively simple simulations and case studies, but my algorithm may have even greatercomputational advantages for more complicated HMMs. Such models include HMMs withcovariates in the probability transition matrix (DeRuiter et al., 2017), HMMs with a largenumber of possible hidden states N (Glennie et al., 2023), and non-parametric HMMs(Langrock et al., 2018). Future work can focus on adapting and applying the computationalmethods from this dissertation to these complicated models.Further, this dissertation focused on frequentist methods for parameter estimation, butBayesian methods for HMM inference are also an area of active research. For example,Foti et al. (2014) applied a variational inference method for HMM inference, and Ma et al.(2017) outlined a stochastic gradient MCMC approach for HMMs. More recently, Sacchiand Swallow (2021) implemented a parallel tempering MCMC scheme on a hierarchicalHMM with applications to animal movement. This promising direction of study could1045.3. Future Workbenefit from recent advancements in the non-reversible parallel tempering literature (Syedet al., 2021a,b; Surjanovic et al., 2022). In addition, Kou et al. (2012) proposed a multi-resolution approach that could be combined with a non-reversible parallel tempering schemeto perform Bayesian inference on extremely fine-scale HMMs.HMMs have become incredibly important tools to describe time series in a variety of fields.However, as sensing technology improves, traditional HMMs alone are too simplistic andthe standard Baum-Welch algorithm is too slow for modern high-frequency data sets. Thisdissertation addressed these challenges and has already been cited in fields ranging fromweather forecasting (Kamarianakis et al., 2024) to health care (Forbes, 2024). It is clearthat inferring latent processes from large, high-frequency, and partially-labelled time seriesis broadly of interest for researchers across disciplines. Thus, future work can continue toimplement the ideas of this dissertation to take advantage of HMMs in the future of complexbig data.105BibliographyAdam, T., Griffiths, C., Leos Barajas, V., Meese, E., Lowe, C., Blackwell, P., Righton, D.,and Langrock, R. (2019). Joint modelling of multi-scale animal movement data usinghierarchical hidden Markov models. Methods in Ecology and Evolution, 10(9):1536–1550.Agarwal, N., Bullins, B., and Hazan, E. (2017). Second-order stochastic optimization formachine learning in linear time. The Journal of Machine Learning Research, 18(1):4148–4187.Alex Shorter, K., Shao, Y., Ojeda, L., Barton, K., Rocho-Levine, J., van der Hoop, J., andMoore, M. (2017). A day in the life of a dolphin: Using bio-logging tags for improvedanimal health and well-being. Marine Mammal Science, 33(3):785–802.Allen, A. N., Goldbogen, J. A., Friedlaender, A. S., and Calambokidis, J. (2016). Devel-opment of an automated method of detecting stereotyped feeding events in multisensordata from tagged rorqual whales. Ecology and Evolution, 6(20):7522–7535.Altman, R. M. (2007). Mixed hidden Markov models. Journal of the American StatisticalAssociation, 102(477):201–210.Aue, A., Norinho, D. D., and Hörmann, S. (2015). On the prediction of stationary functionaltime series. Journal of the American Statistical Association, 110(509):378–392.Bagos, P. G., Liakopoulos, T. D., and Hamodrakas, S. J. (2003). Maximum likelihoodand conditional maximum likelihood learning algorithms for hidden Markov models withlabeled data: Application to transmembrane protein topology prediction. In Proceedingsof the International Conference on Computational Methods in Sciences and Engineering,ICCMSE ’03, page 47–55. World Scientific Publishing Co., Inc.Baker, J. (1975). The DRAGON system–an overview. IEEE Transactions on Acoustics,Speech, and Signal Processing, 23(1):24–29.Baldi, P. and Chauvin, Y. (1993). Smooth on-line learning algorithms for hidden Markovmodels. Neural Comput., 6.106BibliographyBaum, L. E. and Petrie, T. (1966). Statistical inference for probabilistic functions of finitestate Markov chains. The Annals of Mathematical Statistics, 37(6):1554 – 1563.Baum, L. E., Petrie, T., Soules, G., and Weiss, N. (1970). A maximization techniqueoccurring in the statistical analysis of probabilistic functions of Markov chains. TheAnnals of Mathematical Statistics, 41(1):164–171.Beal, M., Ghahramani, Z., and Rasmussen, C. (2001). The infinite hidden markov model. InDietterich, T., Becker, S., and Ghahramani, Z., editors, Advances in Neural InformationProcessing Systems, volume 14. MIT Press.Bebbington, M. S. (2007). Identifying volcanic regimes using hidden Markov models. Geo-physical Journal International, 171(2):921–942.Borchers, D. L., Zucchini, W., Heide-Jørgensen, M. P., Cañadas, A., and Langrock, R.(2013). Using hidden Markov models to deal with availability bias on line transect surveys.Biometrics, 69(3):703–713.Boyd, S. and Vandenberghe, L. (2004). Convex Optimization. Cambridge University Press.Bradley, A. P. (1997). The use of the area under the ROC curve in the evaluation of machinelearning algorithms. Pattern Recognition, 30(7):1145–1159.Brumback, B. A. and Rice, J. A. (1998). Smoothing spline models for the analysis ofnested and crossed samples of curves. Journal of the American Statistical Association,93(443):961–976.Bulla, J., Lagona, F., Maruotti, A., and Picone, M. (2012). A multivariate hidden Markovmodel for the identification of sea regimes from incomplete skewed and circular timeseries. Journal of Agricultural, Biological and Environmental Statistics, 17:544–567.Börger, L., Bijleveld, A., Fayet, A., Machovsky-Capuska, G., Patrick, S., Street, G., andWal, E. (2020). Biologging special feature. Journal of Animal Ecology, 89(1):6–15.Cade, D. E., Barr, K. R., Calambokidis, J., Friedlaender, A. S., and Goldbogen, J. A.(2018). Determining forward speed from accelerometer jiggle in aquatic environments.Journal of Experimental Biology, 221(2):jeb170449.Cade, D. E., Gough, W. T., Czapanskiy, M. F., Fahlbusch, J. A., Kahane-Rapport, S. R.,Linsky, J. M. J., et al. (2021). Tools for integrating inertial sensor data with videobio-loggers, including estimation of animal orientation, motion, and position. AnimalBiotelemetry, 9.107BibliographyCarroll, G., Slip, D., Jonsen, I., and Harcourt, R. (2014). Supervised accelerometry analysiscan identify prey capture by penguins at sea. Journal of Experimental Biology, 217:4295–4302.Chapelle, O., Schölkopf, B., and Zien, A., editors (2006). Semi-Supervised Learning. TheMIT Press.Chen, J. and Li, P. (2009). Hypothesis test for normal mixture models: The EM approach.The Annals of Statistics, 37(5A):2523 – 2542.Chen, K. and Müller, H.-G. (2012). Modeling repeated functional observations. Journal ofthe American Statistical Association, 107(500):1599–1609.Cole, M. R., Zeligs, J. A., Skrovan, S., and McDonald, B. I. (2021). Head-mounted ac-celerometry accurately detects prey capture in California sea lions. Animal Biotelemetry,9(1):44.Crainiceanu, C. M., Staicu, A.-M., and Di, C.-Z. (2009). Generalized multilevel functionalregression. Journal of the American Statistical Association, 104(488):1550–1561.de Souza, C. P. E. and Heckman, N. E. (2014). Switching nonparametric regression models.Journal of Nonparametric Statistics, 26(4):617–637.de Souza, C. P. E., Heckman, N. E., and Xu, F. (2017). Switching nonparametric regressionmodels for multi-curve data. Canadian Journal of Statistics, 45(4):442–460.Defazio, A., Bach, F., and Lacoste-Julien, S. (2014). SAGA: A fast incremental gradientmethod with support for non-strongly convex composite objectives. In Advances in NeuralInformation Processing Systems, NIPS ’14, page 1646–1654.Dempster, A. P., Laird, N. M., and Rubin, D. B. (1977). Maximum likelihood from in-complete data via the EM algorithm. Journal of the Royal Statistical Society. Series B(Methodological), 39(1):1–38.DeRuiter, S. L., Langrock, R., Skirbutas, T., Goldbogen, J. A., Calambokidis, J., Friedlaen-der, A. S., and Southall, B. L. (2017). A multivariate mixed hidden Markov model forblue whale behaviour and responses to sound exposure. The Annals of Applied Statistics,11(1):362–392.Di, C.-Z., Crainiceanu, C. M., Caffo, B. S., and Punjabi, N. M. (2009). Multilevel functionalprincipal component analysis. The Annals of Applied Statistics, 3(1):458–488.108BibliographyDouc, R., Garivier, A., Moulines, E., and Olsson, J. (2011). Sequential Monte Carlo smooth-ing for general state space hidden Markov models. The Annals of Applied Probability,21(6):2109–2145.El Annas, M., Benyacoub, B., and Ouzineb, M. (2023). Semi-supervised adapted HMMs forp2p credit scoring systems with reject inference. Computational Statistics, 38(1):149–169.Fang, C., Li, C. J., Lin, Z., and Zhang, T. (2018). SPIDER: Near-optimal non-convexoptimization via stochastic path-integrated differential estimator. Advances in NeuralInformation Processing Systems, 31.Fehlmann, G., O’Riain, J., Hopkins, P. W., O’Sullivan, J., Holton, M. D., Shepard, E. L. C.,and King, A. J. (2017). Identification of behaviours from accelerometer data in a wildsocial primate. Animal Biotelemetry, 5:6.Fieberg, J., Signer, J., Smith, B., and Avgar, T. (2021). A ‘how to’ guide for interpretingparameters in habitat-selection analyses. Journal of Animal Ecology, 90(5):1027–1043.Fischer, W. and Meier-Hellstern, K. S. (1993). The Markov-modulated Poisson process(MMPP) cookbook. Perform. Evaluation, 18(2):149–171.Fisheries and Oceans Canada (2018). Amended recovery strategy for the northern andsouthern resident killer whales (Orcinus orca) in Canada. Species at Risk Act RecoveryStrategy Series.Fletcher, R. (2000). Newton-Like Methods. John Wiley & Sons, Ltd.Fletcher, R. and Reeves, C. M. (1964). Function Minimization by Conjugate Gradients.The Computer Journal, 7(2):149–154.Florko, K. R. N., Togunov, R. R., Gryba, R., Sidrow, E., Ferguson, S. H., Yurkowski,D. J., and Auger-Méthé, M. (2024). A review of statistical models used to characterizespecies-habitat associations with animal movement data. arXiv preprint.Forbes, O. (2024). Statistical machine learning methods for neuroscientific data: Analysingbrain activity, psychopathology and cognitive function in adolescents. PhD thesis, Queens-land University of Technology.Ford, J. K. B. and Ellis, G. M. (2006). Selective foraging by fish-eating killer whales Orcinusorca in British Columbia. Marine Ecology-Progress Series, 316:185–199.Ford, J. K. B., Ellis, G. M., Olesiuk, P. F., and Balcomb, K. C. (2009). Linking killer whalesurvival and prey abundance: Food limitation in the oceans’ apex predator? BiologyLetters, 6(1):139–42.109BibliographyFoti, N., Xu, J., Laird, D., and Fox, E. (2014). Stochastic variational inference for hiddenMarkov models. In Advances in Neural Information Processing Systems 27, volume 27,pages 3599–3607.Franke, A., Caelli, T., and Hudson, R. J. (2004). Analysis of movements and behav-ior of caribou (Rangifer tarandus) using hidden Markov models. Ecological Modelling,173(2):259–270.Fu, E. and Heckman, N. (2019). Model-based curve registration via stochastic approxima-tion EM algorithm. Computational Statistics and Data Analysis, 131:159–175.Gales, M. and Young, S. (2007). The application of hidden Markov models in speechrecognition. Found. Trends Signal Process., 1(3):195–304.Gao, Y., Shang, H. L., and Yang, Y. (2019). High-dimensional functional time series fore-casting: An application to age-specific mortality rates. Journal of Multivariate Analysis,170:232–243. Special Issue on Functional Data Analysis and Related Topics.Getman, A., Cooper, C. D., Key, G., Zhou, H., and Frankle, N. (2009). Detection of mobilemachine damage using accelerometer data and prognostic health monitoring techniques.In 2009 IEEEWorkshop on Computational Intelligence in Vehicles and Vehicular Systems,pages 101–104.Ghazvininejad, M., Rabiee, H. R., Pourdamghani, N., and Khanipour, P. (2011). HMMbased semi-supervised learning for activity recognition. In Proceedings of the 2011 Inter-national Workshop on Situation Activity & Goal Awareness, SAGAware ’11, page 95–100,New York, NY, USA. Association for Computing Machinery.Gleiss, A. C., Wilson, R. P., and Shepard, E. L. C. (2011). Making overall dynamic bodyacceleration work: On the theory of acceleration as a proxy for energy expenditure.Methods in Ecology and Evolution, 2(1):23–33.Glennie, R., Adam, T., Leos-Barajas, V., Michelot, T., Photopoulou, T., and McClintock,B. T. (2023). Hidden Markov models: Pitfalls and opportunities in ecology. Methods inEcology and Evolution, 14(1):43–56.Gotoh, Y., Hochberg, M., and Silverman, H. (1998). Efficient training algorithms forHMMs using incremental estimation. IEEE Transactions on Speech and Audio Pro-cessing, 6(6):539–548.Green, J. A., Halsey, L. G., Wilson, R. P., and Frappell, P. B. (2009). Estimating energyexpenditure of animals using the accelerometry technique: Activity, inactivity and com-parison with the heart-rate technique. Journal of Experimental Biology, 212(5):745–746.110BibliographyGürbüzbalaban, M., Ozdaglar, A., and Parrilo, P. (2021). Why random reshuffling beatsstochastic gradient descent. Mathematical Programming, 186:49––84.Hanks, E. M., Hooten, M. B., and Alldredge, M. W. (2015). Continuous-time discrete-spacemodels for animal movement. The Annals of Applied Statistics, 9(1):145 – 165.Hastie, G. D., Rosen, D. A. S., and Trites, A. W. (2006). The influence of depth on a breath-hold diver: Predicting the diving metabolism of Steller sea lions (Eumetopias jubatus).Journal of Experimental Marine Biology and Ecology, 3:163–170.Heerah, K., Woillez, M., Fablet, R., Garren, F., Martin, S., and De Pontual, H. (2017).Coupling spectral analysis and hidden Markov models for the segmentation of behaviouralpatterns. Movement Ecology, 5:20.Higham, N. J. and Lin, L. (2011). A Schur–Padé algorithm for fractional powers of a matrix.SIAM Journal on Matrix Analysis and Applications, 32(3):1056–1078.Hooten, M. B., King, R., and Langrock, R. (2017). Guest editor’s introduction to thespecial issue on “animal movement modeling”. Journal of Agricultural, Biological andEnvironmental Statistics, 22(3):224–231.Hu, F. (1997). The asymptotic properties of the maximum-relevance weighted likelihoodestimators. The Canadian Journal of Statistics, 25(1):45–59.Hu, F., Rosenberger, W. F., and Zidek, J. V. (2000). Relevance weighted likelihood fordependent data. Metrika, 51(3):223–243.Hu, F. and Zidek, J. V. (2002). The weighted likelihood. Canadian Journal of Statistics,30(3):347–371.Isojunno, S., Sadykova, D., DeRuiter, S., Curé, C., Visser, F., Thomas, L., Miller, P. J. O.,and Harris, C. M. (2017). Individual, ecological, and anthropogenic influences on activitybudgets of long‐finned pilot whales. Ecosphere, 8(12):e02044.Jeanniard-du Dot, T., Guinet, C., Arnould, J. P., Speakman, J. R., and Trites, A. W.(2016). Accelerometers can measure total and activity-specific energy expenditures infree-ranging marine mammals only if linked to time-activity budgets. Functional Ecology,31(2):377–386.Jeanniard du Dot, T., Trites, A. W., Arnould, J. P. Y., Speakman, J. R., and Guinet, C.(2016). Activity-specific metabolic rates for diving, transiting, and resting at sea canbe estimated from time-activity budgets in free-ranging marine mammals. Ecology andEvolution, 7(9):2969–2976.111BibliographyJensen, F. H., Tervo, O. M., Heide-Jørgensen, M. P., and Ditlevsen, S. (2023). Detect-ing narwhal foraging behaviour from accelerometer and depth data using mixed-effectslogistic regression. Animal Biotelemetry, 11(14).Ji, S., Watson, L. T., and Carin, L. (2009). Semisupervised learning of hidden Markovmodels via a homotopy method. IEEE Transactions on Pattern Analysis and MachineIntelligence, 31(2):275–287.Johnson, D., Pelland, N., and Sterling, J. (2021). A continuous-time semi-Markov model foranimal movement in a dynamic environment. The Annals of Applied Statistics, 15(2):797– 812.Johnson, R. and Zhang, T. (2013). Accelerating stochastic gradient descent using predictivevariance reduction. In Proceedings of the 26th International Conference on Neural Infor-mation Processing Systems - Volume 1, NuerIPS, page 315–323, Red Hook, NY, USA.Curran Associates Inc.Joy, R., Tollit, D., Wood, J., MacGillivray, A., Li, Z., Trounce, K., and Robinson, O.(2019). Potential benefits of vessel slowdowns on endangered southern resident killerwhales. Frontiers in Marine Science, 6.Juang, B. H. and Rabiner, L. R. (1991). Hidden Markov models for speech recognition.Technometrics, 33(3):251–272.Kamarianakis, Y., Pantazis, Y., Kalligiannaki, E., Katsaounis, T. D., Kotsovos, K., Gereige,I., Abdullah, M., Jamal, A., and Tzavaras, A. (2024). Robust day-ahead solar forecastingwith endogenous data and sliding windows. Journal of Renewable and Sustainable Energy,16(2):026103.Karimi, B., Wai, H.-T., Moulines, E., and Lavielle, M. (2019). On the global convergenceof (fast) incremental expectation maximization methods. In Wallach, H., Larochelle, H.,Beygelzimer, A., d'Alché-Buc, F., Fox, E., and Garnett, R., editors, Advances in NeuralInformation Processing Systems, volume 32. Curran Associates, Inc.Kashlak, A. B., Loliencar, P., and Heo, G. (2023). Topological hidden Markov models.Journal of Machine Learning Research, 24(340):1–49.Khreich, W., Granger, E., Miri, A., and Sabourin, R. (2012). A survey of techniques forincremental learning of HMM parameters. Information Sciences, 197:105–130.Kingma, D. and Ba, J. (2014). Adam: A method for stochastic optimization. InternationalConference on Learning Representations.112BibliographyKlappstein, N. J., Thomas, L., and Michelot, T. (2023). Flexible hidden Markov modelsfor behaviour-dependent habitat selection. Movement Ecology, 11(1):30.Kleinberg, B., Li, Y., and Yuan, Y. (2018). An alternative view: When does SGD escapelocal minima? In Dy, J. and Krause, A., editors, Proceedings of the 35th InternationalConference on Machine Learning, volume 80 of Proceedings of Machine Learning Research,pages 2698–2707. PMLR.Kokoszka, P. and Reimherr, M. (2018). Introduction to Functional Data Analysis, chapter 8.Chapman and Hall/CRC.Kottaram, A., Johnston, L., Cocchi, L., Ganella, E., Everall, I., Pantelis, C., et al. (2019).Brain network dynamics in schizophrenia: Reduced dynamism of the default mode net-work. Human Brain Mapping, 40:2212–2228.Kou, S., Olding, B. P., Lysy, M., and Liu, J. S. (2012). A multiresolution method for pa-rameter estimation of diffusion processes. Journal of the American Statistical Association,107(500):1558–1574.Krogh, A. (1997). Two methods for improving performance of a HMM and their applica-tion for gene finding. Proceedings. International Conference on Intelligent Systems forMolecular Biology, 5:179–86.Kundu, A., He, Y., and Bahl, P. (1989). Recognition of handwritten word: First and secondorder hidden Markov model based approach. Pattern Recognition, 22(3):283–297.Langrock, R., Adam, T., Leos-Barajas, V., Mews, S., Miller, D. L., and Papastamatiou,Y. P. (2018). Spline‐based nonparametric inference in general state‐switching models.Statistica Neerlandica, 72(3):179–200.Langrock, R., King, R., Matthiopoulos, J., Thomas, L., Fortin, D., and Morales, J. M.(2012). Flexible and practical modeling of animal telemetry data: Hidden Markov modelsand extensions. Ecology, 93:2336–42.Langrock, R., Swihart, B. J., Caffo, B. S., Punjabi, N. M., and Crainiceanu, C. M. (2013).Combining hidden Markov models for comparing the dynamics of multiple sleep elec-troencephalograms. Statistics in Medicine, 32(19):3342–3356.Lawler, E., Whoriskey, K., Aeberhard, W. H., Field, C., and Mills Flemming, J. (2019).The conditionally autoregressive hidden Markov model (CarHMM): Inferring behaviouralstates from animal tracking data exhibiting conditional autocorrelation. Journal of Agri-cultural, Biological and Environmental Statistics, 24(4):651–668.113BibliographyLeos-Barajas, V., Gangloff, E. J., Adam, T., Langrock, R., van Beest, F. M., Nabe-Nielsen,J., and Morales, J. M. (2017). Multi-scale modeling of animal movement and generalbehavior data using hidden Markov models with hierarchical structures. Journal of Agri-cultural, Biological and Environmental Statistics, 22(3):232–248.Li, M. and Bolker, B. M. (2017). Incorporating periodic variability in hidden Markov modelsfor animal movement. Movement Ecology, 5(1):1.Li, Y. (2005). Hidden Markov models with states depending on observations. PatternRecognition Letters, 26(7):977–984.Li, Z., Han, J., and Song, Y. (2020). On the forecasting of high-frequency financial timeseries based on ARIMA model improved by deep learning. Journal of Forecasting,39(7):1081–1097.Liu, S., Wu, H., and Meeker, W. Q. (2015a). Understanding and addressing the unbounded‘likelihood’ problem. The American Statistician, 69(3):191–200.Liu, Y.-Y., Li, S., Li, F., Song, L., and Rehg, J. M. (2015b). Efficient learning of continuous-time hidden Markov models for disease progression. Advances in Neural InformationProcessing Systems, 28:3599–3607.Lucero, P., Sánchez, R., Macancela, J., Cabrera, D., Cerrada, M., Li, C., and Alonso,H. R. (2019). Accelerometer placement comparison for crack detection in railway axlesusing vibration signals and machine learning. In 2019 Prognostics and System HealthManagement Conference (PHM-Paris), pages 291–296.Lusseau, D., Bain, D., Williams, R., and Smith, J. (2009). Vessel traffic disrupts the foragingbehavior of southern resident killer whales Orcinus orca. Endangered Species Research,6(3):211–221.Ma, Y.-A., Foti, N. J., and Fox, E. B. (2017). Stochastic gradient MCMC methods forhidden Markov models. In Precup, D. and Teh, Y. W., editors, Proceedings of the 34thInternational Conference on Machine Learning, volume 70 of Proceedings of MachineLearning Research, pages 2265–2274. PMLR.MacDonald, I. L. and Raubenheimer, D. (1995). Hidden Markov models and animal be-haviour. Biometrical Journal, 37(6):701–712.Mamon, R. S. and Elliott, R. J. (2007). Hidden Markov Models in Finance. Springer.114BibliographyMcClintock, B. T. (2021). Worth the effort? A practical examination of random effectsin hidden Markov models for animal telemetry data. Methods in Ecology and Evolution,12(8):1475–1497.McClintock, B. T., King, R., Thomas, L., Matthiopoulos, J., McConnell, B. J., and Morales,J. M. (2012). A general discrete-time modeling framework for animal movement usingmultistate random walks. Ecological Monographs, 82(3):335–349.McClintock, B. T., Langrock, R., Gimenez, O., Cam, E., Borchers, D. L., Glennie, R.,and Patterson, T. A. (2020). Uncovering ecological state dynamics with hidden Markovmodels. Ecology Letters, 23(12):1878–1903.McClintock, B. T. and Michelot, T. (2018). momentuHMM: R package for generalized hid-den Markov models of animal movement. Methods in Ecology and Evolution, 9(6):1518–1530.McRae, T. M., Volpov, B. L., Sidrow, E., Fortune, S. M. E., Auger-Méthé, M., Heckman,N., and Trites, A. W. (2024). Killer whale respiration rates. PLOS ONE, 19(5):1–26.Michelot, T. (2023). HmmTMB: Hidden Markov models with flexible covariate effects inR. arXiv Preprint.Michelot, T. and Blackwell, P. G. (2019). State-switching continuous-time correlated ran-dom walks. Methods in Ecology and Evolution, 10(5):637–649.Michelot, T., Gloaguen, P., Blackwell, P. G., and Étienne, M.-P. (2019). The Langevindiffusion as a continuous-time model of animal movement and habitat selection. Methodsin Ecology and Evolution, 10(11):1894–1907.Michelot, T., Langrock, R., and Patterson, T. A. (2016). moveHMM: an R package for thestatistical modelling of animal movement data using hidden Markov models. Methods inEcology and Evolution, 7(11):1308–1315.Morales, J. M., Haydon, D. T., Frair, J., Holsinger, K. E., and Fryxell, J. M. (2004).Extracting more out of relocation data: Building movement models as mixtures of randomwalks. EEB Articles, 85.Morgenstern, J. (1985). How to compute fast a function and all its derivatives: a variationon the theorem of baur-strassen. SIGACT News, 16(4):60–62.Morris, J. S., Arroyo, C., Coull, B. A., Ryan, L. M., Herrick, R., and Gortmaker, S. L.(2006). Using wavelet-based functional mixed models to characterize population het-115Bibliographyerogeneity in accelerometer profiles. Journal of the American Statistical Association,101(476):1352–1364.Murray, C. C., Hannah, L. C., Doniol-Valcroze, T., Wright, B. M., Stredulinsky, E. H.,Nelson, J. C., Locke, A., and Lacy, R. C. (2021). A cumulative effects model for populationtrajectories of resident killer whales in the northeast pacific. Biological Conservation,257:109124.Nathan, R., Monk, C. T., Arlinghaus, R., Adam, T., Alós, J., Assaf, M., Baktoft, H.,Beardsworth, C. E., Bertram, M. G., Bijleveld, A. I., Brodin, T., Brooks, J. L., Campos-Candela, A., Cooke, S. J., Gjelland, K. Ø., Gupte, P. R., Harel, R., Hellström, G.,Jeltsch, F., Killen, S. S., Klefoth, T., Langrock, R., Lennox, R. J., Lourie, E., Madden,J. R., Orchan, Y., Pauwels, I. S., Říha, M., Roeleke, M., Schlägel, U. E., Shohami, D.,Signer, J., Toledo, S., Vilk, O., Westrelin, S., Whiteside, M. A., and Jarić, I. (2022). Big-data approaches lead to an increased understanding of the ecology of animal movement.Science, 375(6582):eabg1780.Neal, R. M. and Hinton, G. E. (1998). A View of the EM Algorithm that Justifies Incre-mental, Sparse, and other Variants, pages 355–368. Springer Netherlands, Dordrecht.Nguyen, L. M., Liu, J., Scheinberg, K., and Takáč, M. (2017). SARAH: A novel methodfor machine learning problems using stochastic recursive gradient. In International Con-ference on Machine Learning, pages 2613–2621. PMLR.Nickel, B. A., Suraci, J. P., Nisi, A. C., and Wilmers, C. C. (2021). Energetics and fear ofhumans constrain the spatial ecology of pumas. Proceedings of the National Academy ofSciences, 118(5):e2004592118.Nicosia, A., Duchesne, T., Rivest, L.-P., and Fortin, D. (2017). A multi-state conditionallogistic regression model for the analysis of animal movement. The Annals of AppliedStatistics, 11(3):1537 – 1560.Nigam, K., Mccallum, A. K., Thrun, S., and Mitchell, T. (2000). Text classification fromlabeled and unlabeled documents using EM. Machine Learning, 39:103–134.Noren, D. P. (2011). Estimated field metabolic rates and prey requirements of resident killerwhales. Marine Mammal Science, 27(1):60 – 77.Nunes, A. (2019). Divebomb version 1.0.7. Ocean Tracking Network, gitlab.oceantrack.org/anunes/divebomb.116BibliographyOgburn, M. B., Harrison, A.-L., Whoriskey, F. G., Cooke, S. J., Mills Flemming, J. E.,and Torres, L. G. (2017). Addressing challenges in the application of animal movementecology to aquatic conservation and management. Frontiers in Marine Science, 4.Patterson, T. A., Parton, A., Langrock, R., Blackwell, P. G., Thomas, L., and King, R.(2017). Statistical modelling of individual animal movement: an overview of key methodsand a discussion of practical challenges. Advances in Statistical Analysis, 101:399–438.Pearl, J. (1982). Reverend Bayes on inference engines: A distributed hierarchical approach.In Proceedings of the Second AAAI Conference on Artificial Intelligence, AAAI’82, page133–136. AAAI Press.Pearl, J. (1988). Chapter 4: Belief updating by network propagation. In ProbabilisticReasoning in Intelligent Systems, pages 143–237. Morgan Kaufmann, San Francisco (CA).Pirotta, E., Edwards, E. W. J., New, L., and Thompson, P. M. (2018). Central placeforagers and moving stimuli: A hidden-state model to discriminate the processes affectingmovement. Journal of Animal Ecology, 87(4):1116–1125.Pohle, J., Langrock, R., van Beest, M., and Schmidt, N. M. (2017). Selecting the number ofstates in hidden Markov models: Pragmatic solutions illustrated using animal movement.Journal of Agricultural, Biological and Environmental Statistics, 22:1–24.Qasem, L., Cardew, A., Wilson, A., Griffiths, I., Halsey, L. G., Shepard, E. L. C., Gleiss,A. C., and Wilson, R. (2012). Tri-axial dynamic acceleration as a proxy for animalenergy expenditure; should we be summing values or calculating the vector? PloS one,7(2):e31187.Quick, N. J., Isojunno, S., Sadykova, D., Bowers, M., Nowacek, D. P., and Read, A. J.(2017). Hidden Markov models reveal complexity in the diving behaviour of short-finnedpilot whales. Scientific Reports, 7(1):45765.R Core Team (2023). R: A Language and Environment for Statistical Computing. RFoundation for Statistical Computing, Vienna, Austria.Rabiner, L. (1989). A tutorial on hidden markov models and selected applications in speechrecognition. Proceedings of the IEEE, 77(2):257–286.Ramsay, J. O. and Silverman, B. W. (2005). Functional Data Analysis. Springer.Ren, Z., Yeh, R., and Schwing, A. (2020). Not all unlabeled data are equal: Learning toweight data in semi-supervised learning. In Larochelle, H., Ranzato, M., Hadsell, R.,117BibliographyBalcan, M. F., and Lin, H., editors, Advances in Neural Information Processing Systems,volume 33, pages 21786–21797. Curran Associates, Inc.Rice, J. A. and Wu, C. O. (2001). Nonparametric mixed effects models for unequallysampled noisy curves. Biometrics, 57(1):253–9.Robbins, H. and Monro, S. (1951). A stochastic approximation method. The Annals ofMathematical Statistics, 22(3):400 – 407.Ruiz-Suarez, S., Leos-Barajas, V., and Morales, J. M. (2022). Hidden Markov and semi-Markov models when and why are these models useful for classifying states in time seriesdata? Journal of Agricultural, Biological and Environmental Statistics, 27(2):339–363.Rydén, T. (2008). EM versus Markov Chain Monte Carlo for estimation of hidden Markovmodels: a computational perspective. Bayesian Analysis, 3(4):659 – 688.Sacchi, G. and Swallow, B. (2021). Toward efficient Bayesian approaches to inference inhierarchical hidden Markov models for inferring animal behavior. Frontiers in Ecologyand Evolution, 9:249.Saldanha, S., Cox, S. L., Militão, T., and González-Solís, J. (2023). Animal behaviour onthe move: the use of auxiliary information and semi-supervision to improve behaviouralinferences from hidden Markov models applied to GPS tracking datasets. MovementEcology, 11:41.Schliehe-Diecks, S., Kappeler, P. M., and Langrock, R. (2012). On the application of mixedhidden Markov models to multiple behavioural time series. Interface Focus, 2(2):180–189.Schmidt, M., Le Roux, N., and Bach, F. (2017). Minimizing finite sums with the stochasticaverage gradient. Mathematical Programming, 162(1):83–112.Scott, S. L. (2002). Bayesian methods for hidden Markov models. Journal of the AmericanStatistical Association, 97(457):337–351.Shamir, O. (2016). Without-replacement sampling for stochastic gradient methods. InAdvances in Neural Information Processing Systems, volume 29. Curran Associates, Inc.Simon, M., Johnson, M., and Madsen, P. T. (2012). Keeping momentum with a mouth-ful of water: Behavior and kinematics of humpback whale lunge feeding. Journal ofExperimental Biology, 215(21):3786–3798.Singh, A., Nowak, R., and Zhu, J. (2008). Unlabeled data: Now it helps, now it doesn’t.In Koller, D., Schuurmans, D., Bengio, Y., and Bottou, L., editors, Advances in NeuralInformation Processing Systems, volume 21. Curran Associates, Inc.118BibliographyStephens, D. W., Brown, J. S., and Ydenberg, R. C. (2008). Foraging: behavior and ecology.University of Chicago Press.Surjanovic, N., Syed, S., Bouchard-Côté, A., and Campbell, T. (2022). Parallel temperingwith a variational reference. In Koyejo, S., Mohamed, S., Agarwal, A., Belgrave, D., Cho,K., and Oh, A., editors, Advances in Neural Information Processing Systems, volume 35,pages 565–577. Curran Associates, Inc.Sutherland, W. J. (1998). The importance of behavioural studies in conservation biology.Animal Behaviour, 56(4):801–809.Syed, S., Bouchard-Côté, A., Deligiannidis, G., and Doucet, A. (2021a). Non-reversibleparallel tempering: A scalable highly parallel MCMC scheme. Journal of the RoyalStatistical Society Series B: Statistical Methodology, 84(2):321–350.Syed, S., Romaniello, V., Campbell, T., and Bouchard-Côté, A. (2021b). Parallel temperingon optimized paths. ICML. arXiv:2102.07720.Tamposis, I. A., Theodoropoulou, M. C., Tsirigos, K. D., and Bagos, P. G. (2018a). Ex-tending hidden Markov models to allow conditioning on previous observations. Journalof Bioinformatics and Computational Biology, 16(05):1850019.Tamposis, I. A., Tsirigos, K. D., Theodoropoulou, M. C., Kontou, P. I., and Bagos, P. G.(2018b). Semi-supervised learning of hidden Markov models for biological sequence anal-ysis. Bioinformatics, 35(13):2208–2215.Tennessen, J., Holt, M. M., Ward, E. J., Hanson, M. B., Emmons, C. K., Giles, D. A., andHogan, J. T. (2019a). Hidden Markov models reveal temporal patterns and sex differencesin killer whale behavior. Scientific Reports, 9:14951.Tennessen, J. B., Holt, M. M., Hanson, M. B., Emmons, C. K., Giles, D. A., and Hogan, J. T.(2019b). Kinematic signatures of prey capture from archival tags reveal sex differencesin killer whale foraging activity. Journal of Experimental Biology, 222(3).Tennessen, J. B., Holt, M. M., Wright, B. M., Hanson, M. B., Emmons, C. K., Giles, D. A.,Hogan, J. T., Thornton, S. J., and Deecke, V. B. (2023). Divergent Foraging Strate-gies Between Populations of Sympatric Matrilineal Killer Whales. Behavioral Ecology,34(3):373–386.Thiesson, B., Meek, C., and Heckerman, D. (2001). Accelerating EM for large databases.Machine Learning, 45:279–299. Also appears as Technical Report MSR-TR-99-31, Mi-crosoft Research, May, 1999 (Revised February, 2001).119BibliographyTogunov, R. R., Derocher, A. E., Lunn, N. J., and Auger-Méthé, M. (2021). Characteris-ing menotactic behaviours in movement data using hidden Markov models. Methods inEcology and Evolution, 12(10):1984–1998.van Englan, J. E. and Hoos, H. H. (2020). A survey on semi-supervised learning. MachineLearning, 109:373–440.Virtanen, P., Gommers, R., Oliphant, T. E., Haberland, M., Reddy, T., Cournapeau, D.,Burovski, E., Peterson, P., Weckesser, W., Bright, J., van der Walt, S. J., Brett, M.,Wilson, J., Jarrod Millman, K., Mayorov, N., Nelson, A. R. J., Jones, E., Kern, R.,Larson, E., Carey, C., Polat, İ., Feng, Y., Moore, E. W., Vand erPlas, J., Laxalde, D.,Perktold, J., Cimrman, R., Henriksen, I., Quintero, E. A., Harris, C. R., Archibald,A. M., Ribeiro, A. H., Pedregosa, F., van Mulbregt, P., and Contributors, S. . . (2019).SciPy 1.0–Fundamental Algorithms for Scientific Computing in Python. arXiv e-prints,page arXiv:1907.10121.Visser, I. and Speekenbrink, M. (2010). depmixs4: an R package for hidden Markov models.Journal of Statistical Software, 36(7):1––21.Viterbi, A. (1967). Error bounds for convolutional codes and an asymptotically optimumdecoding algorithm. IEEE Transactions on Information Theory, pages 260–269.Viviant, M., Monestiez, P., and Guinet, C. (2014). Can we predict foraging success in amarine predator from dive patterns only? Validation with prey capture attempt data.PLOS ONE, 9(3):1–10.Wang, S. X. (2001). Maximum weighted likelihood estimation. PhD thesis, University ofBritish Columbia.Whoriskey, K., Auger-Méthé, M., Albertsen, C. M., Whoriskey, F. G., Binder, T. R.,Krueger, C. C., and Mills Flemming, J. (2016). A hidden Markov movement modelfor rapidly identifying behavioral states from animal tracks. Ecology and Evolution,7(7):2112–2121.Williams, R. and Noren, D. P. (2009). Swimming speed, respiration rate, and estimatedcost of transport in adult killer whales. Marine Mammal Science, 25(2):327–350.Williams, T. M., Haun, J. E., and Friedl, W. A. (1999). The diving physiology of bot-tlenose dolphins (Tursiops truncatus): I. balancing the demands of exercise for energyconservation at depth. Journal of Experimental Biology, 202(20):2739–2748.120BibliographyWilson, R. P., Börger, L., Holton, M. D., Scantlebury, D. M., Gómez-Laich, A., Quintana,F., Rosell, F., Graf, P. M., Williams, H., Gunner, R., Hopkins, L., Marks, N., Geraldi,N. R., Duarte, C. M., Scott, R., Strano, M. S., Robotka, H., Eizaguirre, C., Fahlman,A., and Shepard, E. L. C. (2019). Estimates for energy expenditure in free-living animalsusing acceleration proxies: A reappraisal. Journal of Animal Ecology, 89(1):161–172.Wright, B. M., Ford, J. K. B., Ellis, G. M., Deecke, V. B., Shapiro, A. D., Battaile, B. C.,and Trites, A. W. (2017). Fine-scale foraging movements by fish-eating killer whales(Orcinus orca) relate to the vertical distributions and escape responses of salmonid prey(Oncorhynchus spp.). Movement Ecology, 5:3.Wu, C. F. J. (1983). On the convergence properties of the EM algorithm. The Annals ofStatistics, 11(1):95–103.Xin, G., Hamzaoui, N., and Antoni, J. (2018). Semi-automated diagnosis of bearing faultsbased on a hidden Markov model of the vibration signals. Measurement, 127:141–166.Xu, Z., Laber, E. B., and Staicu, A.-M. (2020). Hierarchical Continuous Time HiddenMarkov Model, with Application in Zero-Inflated Accelerometer Data, pages 125–142.Springer International Publishing, Cham.Yao, F., Müller, H.-G., and Wang, J.-L. (2005). Functional data analysis for sparse longi-tudinal data. Journal of the American Statistical Association, 100(470):577–590.Ye, F. X.-F., an Ma, Y., and Qian, H. (2024). Estimate exponential memory decay inhidden Markov model and its applications to inference. Physica D: Nonlinear Phenomena,460:134053.Ylitalo, A.-K., Heikkinen, J., and Kojola, I. (2021). Analysis of central place foragingbehaviour of wolves using hidden Markov models. Ethology, 127(2):145–157.Zeller, K. A., Wattles, D. W., Conlee, L., and DeStefano, S. (2019). Black bears altermovements in response to anthropogenic features with time of day and season. MovementEcology, 7(1):19.Zhang, G., Poupart, P., and Trimponias, G. (2020). Comparing EM with GD in mixturemodels of two components. In Adams, R. P. and Gogate, V., editors, Proceedings ofThe 35th Uncertainty in Artificial Intelligence Conference, volume 115 of Proceedings ofMachine Learning Research, pages 164–174. PMLR.Zhu, R., Wang, L., Zhai, C., and Gu, Q. (2017). High-dimensional variance-reduced stochas-tic gradient expectation-maximization algorithm. In Proceedings of the 34th International121Conference on Machine Learning, volume 70 of Proceedings of Machine Learning Re-search, pages 4180–4188. PMLR.Zinkevich, M., Weimer, M., Li, L., and Smola, A. (2010). Parallelized stochastic gradientdescent. In Lafferty, J., Williams, C., Shawe-Taylor, J., Zemel, R., and Culotta, A., edi-tors, Advances in Neural Information Processing Systems, volume 23. Curran Associates,Inc.Zucchini, W., MacDonald, I. L., and Langrock, R. (2016). Hidden Markov Models for TimeSeries - An Introduction Using R. CRC Press.122Appendix AAppendix for Chapter 2123A.1. Detailed Description of the Data Simulation in Section 2.4.1A.1 Detailed Description of the Data Simulation in Section2.4.1I can easily simulate realizations of the coarse-scale HMM (m and n ) given the parametersΓ and θ. For each dive t, I can also easily generate the fine-scale hidden Markov chainm˜∗t ={m˜∗t,)P . . . P m˜∗t,i˜ ∗t}according to one of the transition probability matrices Γ∗()) or Γ∗(2)depending upon the value of mt. This determines the sequence of fine-scale hidden statescorresponding to each window. Recall that the fine-scale model is based on a sequence ofi˜ ∗t two-second windows, each containing 100 observations, and that my model is formulatedin terms of quantities derived from the raw data within each window (namely, the averageacceleration, A˜∗t,t˜∗ , and wiggliness, W˜ ∗t,t˜∗). Generating the raw acceleration data from A˜∗t,t˜∗and W˜ ∗t,t˜∗ is not straightforward. In my simulation study, I generate raw acceleration dataso that I can visualize my results in terms of the underlying data curves for each dive.Here, I explain how I generate the acceleration curves so that A˜∗t,t˜∗ and W˜ ∗t,t˜∗ both followthe specified model. A key component is the DFT of the 100 raw acceleration values inwindow t˜∗ of dive t,nˆ∗(k)t,t˜∗ = DFT{n ∗t,)(((t˜∗−))+)P . . . P n∗t,)((t˜∗}(k)for k ≥ 0, as defined in Equation (2.6).I simulate the raw acceleration data for dive t in three steps: (1) simulate the averageacceleration within each window(nˆ∗(()t,t˜∗), (2) simulate all other Fourier coefficients withineach window(nˆ∗(k)t,t˜∗ P k = 1P . . . P 99), and (3) take the inverse DFT of nˆ ∗t,t˜∗ ,{n ∗t,)(((t˜∗−))+)P . . . P n∗t,)((t˜∗} = IDFT{nˆ∗(()t,t˜∗ P . . . P nˆ∗(11)t,t˜∗}for t˜∗ = 1P . . . P i˜ ∗t . The details of steps (1) and (2) are below.For step (1), I generate nˆ ∗(()t,) P . . . P nˆ∗(()t,i˜ ∗tas a CarHMM with the underlying Markov statesequence m˜∗t,)P . . . P m˜∗t,i˜ ∗t and a random first emission. Specifically, I letnˆ∗(()t,) | m˜∗t,) = i∗ ∼ N(100µ∗(·,i∗)A P(100σ∗(·,i∗)A)2)124A.1. Detailed Description of the Data Simulation in Section 2.4.1andnˆ∗(()t,t˜∗ | m˜∗t,t˜∗ = i∗P nˆ∗(()t,t˜∗−) ∼ N(ϕ∗(·,i∗)A nˆ∗(()t,t˜∗−) + 100(1− ϕ∗(·,i∗)A )µ∗(·,i∗)A P(100σ∗(·,i∗)A)2)P(A.1)for t˜∗ = 2P . . . P i˜ ∗t , where µ∗(·,i∗)A , σ∗(·,i∗)A , and ϕ∗(·,i∗)A are as defined in the simulation studyin Section 2.4.For step (2), I first construct nˆ ∗(k)t,t˜∗ for k = 1P . . . P 49, asnˆ∗(k)t,t˜∗ = a(k)t,t˜∗i√w(k)t,t˜∗ P (A.2)where the a(k)t,t˜∗s are independent and equal to either 1 or -1 each with probability 1/2 and iis the imaginary unit. I include i in Equation (A.2) to force all variation within a window totake the form of a sine wave, which reduces the variation between the endpoints of windowsrelative to a cosine wave. Given the fine scale states, the w(k)t,t˜∗s are independent of oneanother and independent of the a(k)t,t˜∗s. The distribution of w(k)t,t˜∗ isw(k)t,t˜∗ | m˜∗t,t˜∗ = 1 ∼ Gamma(11.03/(k + 1)+P 15.54)w(k)t,t˜∗ | m˜∗t,t˜∗ = 2 ∼ Gamma(4.80/(k + 1)+P 515.38)w(k)t,t˜∗ | m˜∗t,t˜∗ = 3 ∼ Gamma(2.31/(k + 1)+P 20023.44).(A.3)The first argument of Gamma(·P ·) is the shape parameter and the second is the scaleparameter. The squared magnitude of the kth Fourier coefficient is equal to w(k)t,t˜∗ , whichdecays on the order of 1/k+ to “smooth out” the raw acceleration data.I then define the remaining 50 Fourier coefficients as nˆ ∗(5()t,t˜∗ = 0 and nˆ∗(k)t,t˜∗ = −nˆ∗()((−k)t,t˜∗ fork = 51P . . . P 99. This guarantees that the inverse DFT is real-valued.I now show that this construction of the raw acceleration data results in the distributionslisted in Section 2.4.1. It suffices to show that the construction of the DFTs, the nˆ (k)t,t˜∗ s,yields the desired distributions.First, nˆ (()t,t˜∗ =∑)((n5) n∗t,)(((t˜∗−))+n = 100A˜∗t,t˜∗ , so Equation (A.1) implies that A˜∗t,)P . . . P A˜∗t,i˜ ∗follows a CarHMM with normal emission distributions and parameters as defined in thesimulation study in Section 2.4.125A.2. Likelihood of the CarHHMM-DFTFrom Equations (2.7) and (A.2), the “wiggliness” within window t˜∗ of dive t isW˜ ∗t,t˜∗ =ω˜∑k5)∣∣∣∣nˆ (k)t,t˜∗∣∣∣∣2 = ω˜∑k5)w(k)t,t˜∗ .If ω˜ Q 50, then W˜ ∗t,t˜∗ is the sum of independent gamma distributed random variables withidentical scale parameters, so the distribution of W˜ ∗t,t˜∗ is also gamma distributed. Thus, byEquation (A.3),W˜ ∗t,t˜∗ | m˜∗t,t˜∗ = 1 ∼ Gamma(ω˜∑k5)11.03/(k + 1)+P 15.54)PW˜ ∗t,t˜∗ | m˜∗t,t˜∗ = 2 ∼ Gamma(ω˜∑k5)4.80/(k + 1)+P 515.38)PandW˜ ∗t,t˜∗ | m˜∗t,t˜∗ = 3 ∼ Gamma(ω˜∑k5)2.31/(k + 1)+P 20023.44).Setting ω˜ to 10 and carrying out simple calculation of the mean and variance of a gammadistribution gives the desired values for µ∗(·,i∗)W and σ∗(·,i∗)W .A.2 Likelihood of the CarHHMM-DFTThe overall likelihood of the CarHHMM-DFT isLCarHHMM-DFT(θP θ∗PΓPΓ∗; yP y˜∗) = δP (y)P y˜∗); θP θ∗PΓ∗)i∏t52ΓP (ytP y˜∗t ; θP θ∗PΓ∗)1cwhereP (ytP y˜∗t ; θP θ∗PΓ∗) = diag[f ())(yt; θ()))Lfine(θ∗PΓ∗()); y˜∗t)P . . . Pf (c)(yt; θ(c))Lfine(θ∗PΓ∗(c); y˜∗t) ]and f (i)(yt; θ(i)) is the emission distribution of dive duration given that mt = i. Thelikelihood corresponding to the fine-scale model is126A.2. Likelihood of the CarHHMM-DFTLfine(θ∗PΓ∗(i); y˜∗t)= δ∗(i)i˜ ∗t∏t˜∗52Γ∗(i)P (y˜∗t,t˜∗ | y˜∗t,t˜∗−); θ∗)1c∗ Pwhere P (y˜∗t,t˜∗ | y˜∗t,t˜∗−); θ∗) is anN∗×N∗ diagonal matrix with the (i∗P i∗)th entry f∗(·,i∗)(y˜∗t,t˜∗ |y˜∗t,t˜∗−); θ∗(·,i∗)). Recall that f∗(·,i∗)(· | y˜∗t,t∗−); θ∗(·,i∗)) is the probability density function ofn˜ ∗t,t˜∗ when m˜∗t,t˜∗ = i∗ and n˜ ∗t,t˜∗−) = y˜∗t,t˜∗−).127A.3. Additional Results from Chapter 2 Case StudyA.3 Additional Results from Chapter 2 Case StudyA.3.1 Lag PlotsFigure A.1: Collection of five lag plots, where a given observation is on the x-axis and thesubsequent observation is on the y-axis. Each plot corresponds to a feature in the killerwhale dive data. Lag plots are used to decide whether to include autocorrelation in theHMMs for modelling the killer whale’s kinematic data. These plots can also be used todetermine the number of hidden states if multiple distinct patterns are visible.128A.3. Additional Results from Chapter 2 Case StudyA.3.2 Likelihood Surface of the CarHHMM-DFTFigure A.2: Log-likelihood of the CarHHMM-DFT as a function of the dive duration param-eters µ(i)n and σ(i)n for dive types i = 1P 2. All other parameters and transition probabilitymatrices are set to be equal to the maximum likelihood estimates shown in Table 1 ofthe main text. The log-likelihood is added to the negative log-likelihood of the maximumlikelihood estimate (µˆn P σˆn ), which is denoted by red stars.Figure A.3: Log-likelihood of the CarHHMM-DFT as a function of the coarse-scale tran-sition probability matrix parameters Γ),2 and Γ2,). All other parameters and transitionprobability matrices are set to be equal to the maximum likelihood estimates shown inTable 1 of the main text. The log-likelihood is added to the negative log-likelihood of themaximum likelihood estimate (Γˆ),2P Γˆ2,)), which is denoted by a red star.129A.3. Additional Results from Chapter 2 Case StudyFigure A.4: Log-likelihood of the CarHHMM-DFT as a function of the “wiggliness” param-eters µ∗(·,i∗)W and σ∗(·,i∗)W for subdive types i∗ = 1P 2P 3. All other parameters and transitionprobability matrices are set to be equal to the maximum likelihood estimates shown inTable 1 of the main text. The log-likelihood is added to the negative log-likelihood of themaximum likelihood estimate (µˆ∗(·,i∗)W P σˆ∗(·,i∗)W ), which is denoted by red stars.130A.3. Additional Results from Chapter 2 Case StudyFigure A.5: Log-likelihood of the CarHHMM-DFT as a function of the x-acceleration pa-rameters µ∗(·,i∗)Ax P σ∗(·,i∗)Axand the logit of ϕ∗(·,i∗)A for subdive types i∗ = 1P 2P 3. All otherparameters and transition probability matrices are set to be equal to the maximum like-lihood estimates shown in Table 1 of the main text. The log-likelihood is added to thenegative log-likelihood of the maximum likelihood estimate (µˆ∗(·,i∗)Ax P σˆ∗(·,i∗)AxP ϕˆ∗(·,i∗)A ), whichis denoted by red stars.131A.3. Additional Results from Chapter 2 Case StudyFigure A.6: Log-likelihood of the CarHHMM-DFT as a function of the y-acceleration pa-rameters µ∗(·,i∗)Ay P σ∗(·,i∗)Ayand the logit of ϕ∗(·,i∗)A for subdive types i∗ = 1P 2P 3. All otherparameters and transition probability matrices are set to be equal to the maximum like-lihood estimates shown in Table 1 of the main text. The log-likelihood is added to thenegative log-likelihood of the maximum likelihood estimate (µˆ∗(·,i∗)Ay P σˆ∗(·,i∗)AyP ϕˆ∗(·,i∗)A ), whichis denoted by red stars.132A.3. Additional Results from Chapter 2 Case StudyFigure A.7: Log-likelihood of the CarHHMM-DFT as a function of the z-acceleration pa-rameters µ∗(·,i∗)Az P σ∗(·,i∗)Azand the logit of ϕ∗(·,i∗)A for subdive types i∗ = 1P 2P 3. All otherparameters and transition probability matrices are set to be equal to the maximum like-lihood estimates shown in Table 1 of the main text. The log-likelihood is added to thenegative log-likelihood of the maximum likelihood estimate (µˆ∗(·,i∗)Az P σˆ∗(·,i∗)AzP ϕˆ∗(·,i∗)A ), whichis denoted by red stars.133A.3. Additional Results from Chapter 2 Case StudyFigure A.8: Log-likelihood of the CarHHMM-DFT as a function of the fine-scale transitionprobability matrix parameters associated with both dive types (Γ∗(i)i∗,j∗ for dive types i = 1P 2and subdive states i∗P j∗ = 1P 2P 3). All other parameters and transition probability matricesare set to be equal to the maximum likelihood estimates shown in Table 1 of the maintext. The log-likelihood is added to the negative log-likelihood of the maximum likelihoodestimate Γˆ∗(i), which is denoted by red stars. Note that Γˆ∗(i)),+ ≈ Γˆ∗(i)+,) ≈ 0: T=this indicatesthat the whale rarely switches directly between subdive states 1 and 3.134A.3. Additional Results from Chapter 2 Case StudyA.3.3 Parameter EstimatesCarHHMM-DFTFeature Dive type / subdive stateParameter estimateµˆ σˆ ϕˆDive Duration (s) – nt1 27.342± 0.633 10.961± 0.560 —2 127.548± 11.341 63.888± 9.032 —x-Acc. (m/s2) –(A˜∗t,t˜∗)x1 0.449± 0.030 0.039± 0.001 0.968± 0.0022 0.210± 0.012 0.096± 0.002 0.829± 0.0073 0.232± 0.035 0.296± 0.010 0.607± 0.023y-Acc. (m/s2) –(A˜∗t,t˜∗)y1 0.450± 0.038 0.051± 0.001 0.968± 0.0022 0.437± 0.012 0.094± 0.002 0.829± 0.0073 0.366± 0.042 0.365± 0.012 0.607± 0.023z-Acc. (m/s2) –(A˜∗t,t˜∗)z1 −0.691± 0.043 0.058± 0.001 0.968± 0.0022 −0.573± 0.014 0.111± 0.002 0.829± 0.0073 −0.303± 0.041 0.354± 0.012 0.607± 0.023“Wiggliness” – W˜ ∗t,t˜∗1 34.015± 0.368 22.986± 0.378 —2 490.068± 5.584 502.558± 6.776 —3 9154.156± 220.765 13538.747± 354.281 —Table A.1: Estimates and predicted standard errors of emission parameters for the distribu-tions of dive duration (nt), acceleration (A˜∗t,t˜∗), and “wiggliness” (W˜ ∗t,t˜∗) of the killer whalecase study data under the full CarHHMM-DFT. Figures following ± refer to estimatedstandard errors derived from the inverse of the observed information matrix.135A.3. Additional Results from Chapter 2 Case StudyHHMM-DFTFeature Dive / Subdive TypeParameter Estimateµˆ σˆ ϕˆDive Duration (s) – nt1 26.613± 0.608 10.161± 0.519 —2 116.785± 10.626 64.712± 8.574 —x-Acc. (m/s2) –(A˜∗t,t˜∗)x1 0.000± 0.002 0.065± 0.001 —2 0.085± 0.004 0.175± 0.003 —3 0.291± 0.020 0.484± 0.014 —y-Acc. (m/s2) –(A˜∗t,t˜∗)y1 0.365± 0.002 0.068± 0.001 —2 0.408± 0.002 0.108± 0.002 —3 0.275± 0.017 0.407± 0.012 —z-Acc. (m/s2) –(A˜∗t,t˜∗)z1 −0.442± 0.003 0.101± 0.002 —2 −0.471± 0.003 0.158± 0.002 —3 −0.298± 0.017 0.415± 0.012 —“Wiggliness” – W˜ ∗t,t˜∗1 43.652± 0.521 33.662± 0.576 —2 608.019± 6.764 717.285± 9.194 —3 7101.915± 155.145 13617.196± 312.209 —Table A.2: Estimates and predicted standard errors of emission parameters for the distribu-tions of dive duration (nt), acceleration (A˜∗t,t˜∗), and “wiggliness” (W˜ ∗t,t˜∗) of the killer whalecase study data under the HHMM-DFT. Figures following ± refer to estimated standarderrors derived from the inverse of the observed information matrix.136A.3. Additional Results from Chapter 2 Case StudyCarHHMMFeature Dive / Subdive TypeParameter Estimateµˆ σˆ ϕˆDive Duration (s) – nt1 27.163± 0.627 10.780± 0.549 —2 124.235± 10.965 64.590± 8.816 —x-Acc. (m/s2) –(A˜∗t,t˜∗)x1 0.064± 0.007 0.035± 0.001 0.874± 0.0082 0.342± 0.017 0.095± 0.002 0.879± 0.0053 0.257± 0.037 0.305± 0.010 0.605± 0.023y-Acc. (m/s2) –(A˜∗t,t˜∗)y1 0.376± 0.009 0.044± 0.001 0.874± 0.0082 0.470± 0.017 0.097± 0.002 0.879± 0.0053 0.350± 0.045 0.377± 0.013 0.605± 0.023z-Acc. (m/s2) –(A˜∗t,t˜∗)z1 −0.462± 0.011 0.052± 0.001 0.874± 0.0082 −0.660± 0.020 0.113± 0.002 0.879± 0.0053 −0.306± 0.045 0.369± 0.013 0.605± 0.023Table A.3: Estimates and predicted standard errors of emission parameters for the distri-butions of dive duration (nt) and acceleration (A˜∗t,t˜∗) of the killer whale case study dataunder the CarHHMM. Figures following ± refer to estimated standard errors derived fromthe inverse of the observed information matrix.137A.3. Additional Results from Chapter 2 Case StudyCarHMM-DFTFeature Dive / Subdive TypeParameter Estimateµˆ σˆ ϕˆDive Duration (s) – nt 1 41.695± 1.244 30.499± 1.220 —x-Acc. (m/s2) –(A˜∗t,t˜∗)x1 0.460± 0.030 0.039± 0.001 0.968± 0.0022 0.209± 0.012 0.096± 0.002 0.828± 0.0073 0.232± 0.035 0.296± 0.010 0.607± 0.023y-Acc. (m/s2) –(A˜∗t,t˜∗)y1 0.457± 0.038 0.052± 0.001 0.968± 0.0022 0.436± 0.012 0.094± 0.002 0.828± 0.0073 0.365± 0.042 0.366± 0.012 0.607± 0.023z-Acc. (m/s2) –(A˜∗t,t˜∗)z1 −0.704± 0.044 0.059± 0.001 0.968± 0.0022 −0.571± 0.014 0.111± 0.002 0.828± 0.0073 −0.301± 0.042 0.355± 0.012 0.607± 0.023“Wiggliness” – W˜ ∗t,t˜∗1 34.403± 0.371 23.331± 0.381 —2 495.033± 5.691 505.423± 6.879 —3 9189.109± 223.965 13588.427± 360.082 —Table A.4: Estimates and predicted standard errors of emission parameters for the distribu-tions of dive duration (nt), acceleration (A˜∗t,t˜∗), and “wiggliness” (W˜ ∗t,t˜∗) of the killer whalecase study data under the CarHMM-DFT. Figures following ± refer to estimated standarderrors derived from the inverse of the observed information matrix.138A.3. Additional Results from Chapter 2 Case StudyA.3.4 Decoded Dive StatesCarHHMM-DFTFigure A.9: The x-component of acceleration, (y∗t,t∗)x (top two panels), and dive depth(bottom two panels) of a northern resident killer whale for a sequence of six selected dives.Each panel is partitioned into dives by vertical black lines. Curve colour in the first and thirdpanels corresponds to estimated dive type while curve colour in the second and fourth panelscorresponds to estimated subdive state. Both dive type and subdive state are estimated byfitting the CarHHMM-DFT to the data and applying the forward–backward algorithm todetermine the hidden state with the highest probability.139A.3. Additional Results from Chapter 2 Case StudyFigure A.10: The probability that each dive is of a particular type or that the whale is ina particular subdive state, given the fitted model. Each panel is partitioned into dives byvertical black lines. Curve colour in the first and third panels corresponds to estimated divetype while curve colour in the second and fourth panels corresponds to estimated subdivestate. Both dive type and subdive state are estimated by fitting the CarHHMM-DFT tothe data and applying the forward–backward algorithm.140A.3. Additional Results from Chapter 2 Case StudyHHMM-DFTFigure A.11: The x-component of acceleration, (y∗t,t∗)x (top two panels), and dive depth(bottom two panels) of a northern resident killer whale for a sequence of six selected dives.Each panel is partitioned into dives by vertical black lines. Curve colour in the first and thirdpanels corresponds to estimated dive type while curve colour in the second and fourth panelscorresponds to estimated subdive state. Both dive type and subdive state are estimatedby fitting the HHMM-DFT to the data and applying the forward–backward algorithm todetermine the hidden state with the highest probability.141A.3. Additional Results from Chapter 2 Case StudyFigure A.12: The probability that each dive is of a particular type or that the whale is ina particular subdive state, given the fitted model. Each panel is partitioned into dives byvertical black lines. Curve colour in the first and third panels corresponds to estimated divetype while curve colour in the second and fourth panels corresponds to estimated subdivestate. Both dive type and subdive state are estimated by fitting the HHMM-DFT to thedata and applying the forward–backward algorithm.142A.3. Additional Results from Chapter 2 Case StudyCarHHMMFigure A.13: The x-component of acceleration, (y∗t,t∗)x (top two panels), and dive depth(bottom two panels) of a northern resident killer whale for a sequence of six selected dives.Each panel is partitioned into dives by vertical black lines. Curve colour in the first and thirdpanels corresponds to estimated dive type while curve colour in the second and fourth panelscorresponds to estimated subdive state. Both dive type and subdive state are estimatedby fitting the CarHHMM to the data and applying the forward–backward algorithm todetermine the hidden state with the highest probability.143A.3. Additional Results from Chapter 2 Case StudyFigure A.14: The probability that each dive is of a particular type or that the whale is ina particular subdive state, given the fitted model. Each panel is partitioned into dives byvertical black lines. Curve colour in the first and third panels corresponds to estimated divetype while curve colour in the second and fourth panels corresponds to estimated subdivestate. Both dive type and subdive state are estimated by fitting the CarHHMM to the dataand applying the forward–backward algorithm.144A.3. Additional Results from Chapter 2 Case StudyCarHMM-DFTFigure A.15: The x-component of acceleration, (y∗t,t∗)x (top two panels), and dive depth(bottom two panels) of a northern resident killer whale for a sequence of six selected dives.Each panel is partitioned into dives by vertical black lines. Curve colour in the first and thirdpanels corresponds to estimated dive type while curve colour in the second and fourth panelscorresponds to estimated subdive state. Both dive type and subdive state are estimatedby fitting the CarHMM-DFT to the data and applying the forward–backward algorithm todetermine the hidden state with the highest probability.145A.3. Additional Results from Chapter 2 Case StudyFigure A.16: The probability that each dive is of a particular type or that the whale is ina particular subdive state, given the fitted model. Each panel is partitioned into dives byvertical black lines. Curve colour in the first and third panels corresponds to estimated divetype while curve colour in the second and fourth panels corresponds to estimated subdivestate. Both dive type and subdive state are estimated by fitting the CarHMM-DFT to thedata and applying the forward–backward algorithm.146A.3. Additional Results from Chapter 2 Case StudyA.3.5 Model Checking: Dive Duration (Yt)CarHHMM-DFTFigure A.17: Empirical histogram (left) and pseudoresidual histogram (right) for dive du-ration (nt) plotted over the estimated emission distribution and a standard normal density,respectively. Both plots are generated using the fitted CarHHMM-DFT and the killer whalecase study data.HHMM-DFTFigure A.18: Empirical histogram (left) and pseudoresidual histogram (right) for dive du-ration (nt) plotted over the estimated emission distribution and a standard normal density,respectively. Both plots are generated using the fitted HHMM-DFT and the killer whalecase study data.147A.3. Additional Results from Chapter 2 Case StudyCarHHMMFigure A.19: Empirical histogram (left) and pseudoresidual histogram (right) for dive du-ration (nt) plotted over the estimated emission distribution and a standard normal density,respectively. Both plots are generated using the fitted CarHHMM and the killer whale casestudy data.CarHMM-DFTFigure A.20: Empirical histogram (left) and pseudoresidual histogram (right) for dive du-ration (nt) plotted over the estimated emission distribution (left) and a standard normaldensity (right), respectively. Both plots are generated using the fitted CarHMM-DFT andthe killer whale case study data.148A.3. Additional Results from Chapter 2 Case Study149A.3. Additional Results from Chapter 2 Case StudyA.3.6 Model Checking: Acceleration (A˜∗t,t˜∗)CarHHMM-DFTFigure A.21: Empirical histograms (top three rows) and pseudoresiduals (bottom row)for acceleration (A˜∗t,t˜∗) plotted over estimated emission distribution and standard normaldensity, respectively. Mean acceleration at time t˜∗ depends on acceleration at time t˜∗ − 1,so only the deviation from the conditional mean at a time step is plotted. All plots aregenerated using the fitted CarHHMM-DFT and the killer whale case study data.150A.3. Additional Results from Chapter 2 Case StudyHHMM-DFTFigure A.22: Empirical histograms (top three rows) and pseudoresidual histograms (bottomrow) for acceleration (A˜∗t,t˜∗) plotted over the estimated emission distribution and a standardnormal density, respectively. All plots are generated using the fitted HHMM-DFT and thekiller whale case study data.151A.3. Additional Results from Chapter 2 Case StudyCarHHMMFigure A.23: Empirical histograms (top three rows) and pseudoresidual histograms (bot-tom row) for acceleration (A˜∗t,t˜∗) plotted over the estimated emission distribution and astandard normal density, respectively. Note that the mean acceleration at time t˜∗ dependson acceleration at time t˜∗ − 1, so only the deviation from the conditional mean at eachparticular time step is plotted. All plots are generated using the fitted CarHHMM and thekiller whale case study data.152A.3. Additional Results from Chapter 2 Case StudyCarHMM-DFTFigure A.24: Empirical histograms (top three rows) and pseudoresidual histograms (bottomrow) for acceleration (A˜∗t,t˜∗) plotted over the estimated emission distribution and a standardnormal density, respectively. Note that the mean acceleration at time t˜∗ depends on accel-eration at time t˜∗ − 1, so only the deviation from the conditional mean at each particulartime step is plotted. All plots are generated using the fitted CarHMM-DFT and the killerwhale case study data.153A.3. Additional Results from Chapter 2 Case StudyA.3.7 Model Checking: “Wiggliness” (W˜ ∗t,t˜∗)CarHHMM-DFTFigure A.25: Empirical histograms (top row) and pseudoresidual histograms (bottom row)for “wiggliness” (W˜ ∗t,t˜∗) plotted over the estimated emission distribution and a standardnormal density, respectively. All plots are generated using the fitted CarHHMM-DFT andthe killer whale case study data.154A.3. Additional Results from Chapter 2 Case StudyHHMM-DFTFigure A.26: Empirical histograms (top row) and pseudoresidual histograms (bottom row)for “wiggliness” (W˜ ∗t,t˜∗) plotted over the estimated emission distribution and a standardnormal density, respectively. All plots are generated using the fitted HHMM-DFT and thekiller whale case study data.CarHHMMThe CarHHMM does not model “wiggliness”.155A.3. Additional Results from Chapter 2 Case StudyCarHMM-DFTFigure A.27: Empirical histograms (top row) and pseudoresidual histograms (bottom row)for “wiggliness” (W˜ ∗t,t˜∗) plotted over the estimated emission distribution and a standardnormal density, respectively. All plots are generated using the fitted CarHMM-DFT andthe killer whale case study data.156A.4. Additional Results from Chapter 2 Simulation StudyA.4 Additional Results from Chapter 2 Simulation StudyA.4.1 Simulated DataFigure A.28: Acceleration data for five dives of a selected simulated data set. The rawaccelerometer data is denoted by n ∗t,t˜∗ and recorded at 50 Hz while A˜∗t,t˜∗ and W˜ ∗t,t˜∗ arerecorded at 0.5Hz. Dives are separated by vertical black lines. Curve colour corresponds tothe true subdive state, while background colour corresponds to true dive type. Subdive state1 can plausibly be interpreted as “gliding”, subdive state 2 can plausibly be interpreted asactive swimming, and subdive state 3 can be interpreted as sudden or aggressive movement.157A.4. Additional Results from Chapter 2 Simulation StudyA.4.2 Accuracy ComparisonsTable A.5: Average decoding accuracies and training times for all models used to categorizedive type and subdive state in the simulation study. Each of the four models was fit to500 training data sets composed of 100 simulated dives and tested on test data sets alsocomposed of 100 simulated dives. Reported values are averages, and figures following ±refer to sample standard deviation across the 500 data sets. “All” denotes overall averagedecoding accuracy.Model Training time (min) Dive type Subdive state Dive accuracy Subdive accuracyCarHHMM-DFT 156± 67All All 0.956± 0.028 0.911± 0.0061 10.973± 0.0320.846± 0.0271 2 0.918± 0.0111 3 0.860± 0.0302 10.856± 0.0970.949± 0.0102 2 0.915± 0.0152 3 0.871± 0.057HHMM-DFT 162± 64All All 0.959± 0.025 0.844± 0.0241 10.977± 0.0280.761± 0.0631 2 0.845± 0.0291 3 0.858± 0.0342 10.851± 0.0970.883± 0.0662 2 0.829± 0.0412 3 0.876± 0.061CarHHMM 258± 106All All 0.924± 0.127 0.845± 0.0181 10.933± 0.1530.689± 0.0501 2 0.900± 0.0311 3 0.673± 0.0592 10.864± 0.1340.906± 0.0222 2 0.848± 0.0332 3 0.687± 0.105CarHMM-DFT 45± 22All All — 0.912± 0.0061 1—0.854± 0.0261 2 0.921± 0.0111 3 0.860± 0.0322 1—0.943± 0.0112 2 0.919± 0.0142 3 0.878± 0.053158A.4. Additional Results from Chapter 2 Simulation StudyA.4.3 Empirical Joint Distributions: Dive Duration (Yt)Table A.6: Estimates and standard errors for the parameters of the distribution of diveduration across all four models in the simulation study. The “Estimate” column givesaverages across 500 simulations, and the “Observed Fisher SE” column gives medians across500 simulations. Figures following ± refer to inter quartile range. “SE” stands for standarderror. Reported P -values correspond to one-sample t-tests of whether bias is statisticallysignificant.Model Parameter Dive type Estimate Bias Empirical SE Observed Fisher SECarHHMM-DFTµn1 27.33 0.07 (P = 0.287) 1.39 1.03± 0.122 127.14 0.07 (P = 0.945) 22.56 15.71± 6.73σn1 10.80 −0.11 (P = 0.035) 1.15 0.84± 0.112 59.52 −4.54 (P = 0.000) 17.68 13.04± 5.53HHMM-DFTµn1 27.37 0.10 (P = 0.084) 1.33 1.03± 0.112 126.84 −0.23 (P = 0.818) 22.00 15.37± 6.11σn1 10.86 −0.05 (P = 0.351) 1.19 0.85± 0.112 57.14 −6.92 (P = 0.000) 16.12 12.62± 5.17CarHHMMµn1 27.25 −0.02 (P = 0.826) 2.20 1.04± 0.142 117.62 −9.45 (P = 0.000) 34.34 15.13± 7.10σn1 10.52 −0.39 (P = 0.001) 2.59 0.86± 0.152 56.98 −7.08 (P = 0.000) 16.33 12.53± 5.95CarHMM-DFTµn1 41.69 — 4.27 2.00± 0.35— — — — —σn1 30.30 — 5.44 1.95± 0.44— — — — —159A.4. Additional Results from Chapter 2 Simulation StudyCarHHMM-DFTFigure A.29: Kernel density estimate of the distribution of the dive duration parameterestimators (µˆn and σˆn ). These estimates are associated with the CarHHMM-DFT andcorrespond to both dive types. White stars represent the mean of all maximum likelihoodestimates and red stars represent the true values of µn and σn for both dive types.160A.4. Additional Results from Chapter 2 Simulation StudyHHMM-DFTFigure A.30: Kernel density estimate of the distribution of the dive duration parameterestimators (µˆn and σˆn ). These estimates are associated with the HHMM-DFT and cor-respond to both dive types. White stars represent the mean of all maximum likelihoodestimates and red stars represent the true values of µn and σn for both dive types.161A.4. Additional Results from Chapter 2 Simulation StudyCarHHMMFigure A.31: Kernel density estimate of the distribution of the dive duration parameterestimators (µˆn and σˆn ). These estimates are associated with the CarHHMM and correspondto both dive types. White stars represent the mean of all maximum likelihood estimatesand red stars represent the true values of µn and σn for both dive types.162A.4. Additional Results from Chapter 2 Simulation StudyCarHMM-DFTFigure A.32: Kernel density estimate of the distribution of the dive duration parameterestimators (µˆn and σˆn ). This estimate is associated with the CarHMM-DFT. The Whitestar represents the mean of all maximum likelihood estimates. Unlike in other figures, thereis no red star representing true values because the CarHMM-DFT assumes that there isonly one dive type.163A.4. Additional Results from Chapter 2 Simulation StudyA.4.4 Empirical Joint Distributions: Acceleration (A˜∗t,t˜∗)Table A.7: Estimates and standard errors for the parameters of the distribution of acceler-ation across all four models in the simulation study. The “Estimate” column gives averagesacross 500 simulations, and the “Observed Fisher SE” column gives medians across 500simulations. Figures following ± refer to inter quartile range. “SE” stands for standarderror. Note that standard error cannot be estimated when ϕ∗A ≈ 0 or ϕ∗A ≈ 1. ReportedP -values correspond to one-sample t-tests of whether bias is statistically significant.Model Parameter Subdive state Estimate Bias Empirical SE Observed Fisher SECarHHMM-DFTµ∗A1 −0.012 −0.012 (P = 0.004) 0.093 0.066± 0.0382 0.099 −0.001 (P = 0.436) 0.019 0.019± 0.0033 0.200 0.000 (P = 0.965) 0.060 0.053± 0.013σ∗A1 0.050 0.000 (P = 0.376) 0.002 0.002± 0.0002 0.100 −0.000 (P = 0.672) 0.002 0.002± 0.0003 0.298 −0.002 (P = 0.024) 0.016 0.015± 0.002ϕ∗A1 0.968 −0.002 (P = 0.001) 0.012 0.010± 0.0022 0.828 −0.002 (P = 0.007) 0.017 0.017± 0.0023 0.600 −0.010 (P = 0.001) 0.064 0.065± 0.011HHMM-DFTµ∗A1 0.048 0.048 (P = 0.000) 0.033 0.007± 0.0012 0.087 −0.013 (P = 0.000) 0.018 0.006± 0.0013 0.215 0.015 (P = 0.000) 0.065 0.022± 0.004σ∗A1 0.160 0.110 (P = 0.000) 0.023 0.005± 0.0012 0.158 0.058 (P = 0.000) 0.010 0.004± 0.0003 0.356 0.056 (P = 0.000) 0.035 0.015± 0.002ϕ∗A1 —— —— —— ——2 —— —— —— ——3 —— —— —— ——CarHHMMµ∗A1 0.060 0.060 (P = 0.000) 0.034 0.019± 0.0062 0.097 −0.003 (P = 0.000) 0.016 0.007± 0.0013 0.201 0.001 (P = 0.634) 0.058 0.062± 0.011σ∗A1 0.111 0.061 (P = 0.000) 0.004 0.003± 0.0002 0.285 0.185 (P = 0.000) 0.009 0.005± 0.0003 1.150 0.850 (P = 0.000) 0.080 0.048± 0.009ϕ∗A1 0.830 −0.140 (P = 0.000) 0.048 0.016± 0.0032 0.004 −0.826 (P = 0.000) 0.012 0.001± 0.0233 0.000 −0.610 (P = 0.000) 0.000 0.000± 0.000CarHMM-DFTµ∗A1 −0.008 −0.008 (P = 0.142) 0.114 0.064± 0.0372 0.101 0.001 (P = 0.438) 0.019 0.019± 0.0033 0.200 0.000 (P = 0.926) 0.057 0.054± 0.014σ∗A1 0.050 −0.000 (P = 0.005) 0.001 0.002± 0.0002 0.100 −0.000 (P = 0.000) 0.002 0.002± 0.0003 0.299 −0.001 (P = 0.117) 0.015 0.015± 0.002ϕ∗A1 0.968 −0.002 (P = 0.001) 0.012 0.010± 0.0022 0.830 −0.000 (P = 0.552) 0.016 0.017± 0.0023 0.595 −0.015 (P = 0.000) 0.071 0.065± 0.010164A.4. Additional Results from Chapter 2 Simulation StudyCarHHMM-DFTFigure A.33: Kernel density estimates of the distribution of acceleration parameter esti-mators (µˆ∗A, σˆ∗A, and ϕˆ∗A) for each subdive state. White stars represent the mean of allmaximum likelihood estimates and red stars represent the true values of µ∗A, σ∗A, and ϕ∗Afor each subdive state. These estimates are for the CarHHMM-DFT.165A.4. Additional Results from Chapter 2 Simulation StudyHHMM-DFTFigure A.34: Kernel density estimate of the distribution of acceleration parameter estima-tors (µˆ∗A and σˆ∗A) for each subdive state. White stars represent the mean of all maximumlikelihood estimates and red stars represent the true values of µ∗A and σ∗A for each subdivestate. These estimates are for the HHMM-DFT. Note that the HHMM-DFT assumes noautocorrelation in A˜∗t,t˜∗ , so ϕˆ∗(·,i∗)A is absent from these plots.166A.4. Additional Results from Chapter 2 Simulation StudyCarHHMMFigure A.35: Kernel density estimates of the distribution of acceleration parameter esti-mators (µˆ∗A, σˆ∗A, and ϕˆ∗A) for each subdive state. White stars represent the mean of allmaximum likelihood estimates and red stars represent the true values of µ∗A, σ∗A, and ϕ∗Afor each subdive state. These estimates are for the CarHHMM.167A.4. Additional Results from Chapter 2 Simulation StudyCarHMM-DFTFigure A.36: Kernel density estimates of the distribution of acceleration parameter esti-mators (µˆ∗A, σˆ∗A, and ϕˆ∗A) for each subdive state. White stars represent the mean of allmaximum likelihood estimates and red stars represent the true values of µ∗A, σ∗A, and ϕ∗Afor each subdive state. These estimates are for the CarHMM-DFT.168A.4. Additional Results from Chapter 2 Simulation StudyA.4.5 Empirical Joint Distributions: “Wiggliness” (W˜ ∗t,t˜∗)Table A.8: Estimates and standard errors for the parameters of the distribution of wigglinessacross all four models in the simulation study. The “Estimate” column gives averages across500 simulations, and the “Observed Fisher SE” column gives medians across 500 simulations.Figures following ± refer to inter quartile range. “SE” stands for standard error. ReportedP -values correspond to one-sample t-tests of whether bias is statistically significant.Model Parameter Subdive state Estimate Bias Empirical SE Observed Fisher SECarHHMM-DFTµ∗W1 34.981 0.001 (P = 0.978) 1.026 0.658± 0.0882 505.104 −0.686 (P = 0.394) 17.936 9.740± 0.8143 9787.146 17.166 (P = 0.718) 1060.450 262.144± 97.120σ∗W1 23.849 −0.001 (P = 0.991) 0.989 0.652± 0.0942 514.207 −2.473 (P = 0.010) 21.427 11.410± 0.9473 14429.732 −32.818 (P = 0.647) 1596.020 302.698± 165.935HHMM-DFTµ∗W1 35.067 0.087 (P = 0.097) 1.170 0.679± 0.1002 500.928 −4.862 (P = 0.000) 24.024 10.754± 1.2053 6840.995 −2928.985 (P = 0.000) 1164.288 198.162± 77.050σ∗W1 23.800 −0.050 (P = 0.351) 1.192 0.676± 0.1042 513.263 −3.417 (P = 0.001) 22.596 12.497± 1.3013 11458.795 −3003.755 (P = 0.000) 1738.793 280.244± 136.259CarHHMMµ∗W1 —— —— —— ——2 —— —— —— ——3 —— —— —— ——σ∗W1 —— —— —— ——2 —— —— —— ——3 —— —— —— ——CarHMM-DFTµ∗W1 35.034 0.054 (P = 0.264) 1.076 0.661± 0.0932 503.762 −2.028 (P = 0.013) 18.258 9.641± 0.9163 9817.785 47.805 (P = 0.328) 1089.110 280.244± 136.259σ∗W1 23.890 0.040 (P = 0.401) 1.058 0.656± 0.1012 515.228 −1.452 (P = 0.125) 21.087 11.371± 0.9903 14437.867 −24.683 (P = 0.740) 1660.764 302.698± 165.935169A.4. Additional Results from Chapter 2 Simulation StudyCarHHMM-DFTFigure A.37: Kernel density estimate of the distributions of wiggliness parameter estimates(µˆ∗W and σˆ∗W ) for each subdive state. White stars represent the mean of all maximumlikelihood estimates and red stars represent the true values of µ∗W and σ∗W for each subdivestate. These estimates are for the CarHHMM-DFT.170A.4. Additional Results from Chapter 2 Simulation StudyHHMM-DFTFigure A.38: Kernel density estimate of the distributions of wiggliness parameter estimates(µˆ∗W and σˆ∗W ) for each subdive state. White stars represent the mean of all maximumlikelihood estimates and red stars represent the true values of µ∗W and σ∗W for each subdivestate. These estimates are for the HHMM-DFT.CarHHMMThe CarHHMM did not use wiggliness to model the whale’s behaviour.171A.4. Additional Results from Chapter 2 Simulation StudyCarHMM-DFTFigure A.39: Kernel density estimate of the distributions of wiggliness parameter estimates(µˆ∗W and σˆ∗W ) for each subdive state. White stars represent the mean of all maximumlikelihood estimates and red stars represent the true values of µ∗W and σ∗W for each subdivestate. These estimates are for the CarHMM-DFT.172A.4. Additional Results from Chapter 2 Simulation StudyA.4.6 Empirical Joint Distributions - Transition Probability Matrices (Γand Γ∗)Table A.9: Estimates and standard errors for the coarse-scale transition probability matricesacross all four models in the simulation study. The “Estimate” column gives averages across500 simulations, and the “Observed Fisher SE” column gives medians across 500 simulations.Figures following ± refer to inter quartile range. “SE” stands for standard error. Note thatstandard error cannot be estimated when Γi,j ≈ 0. Reported P -values correspond to one-sample t-tests of whether bias is statistically significant.Model Parameter Estimate Bias Empirical SE Observed Fisher SECarHHMM-DFTΓ)2 0.160 0.005 (P = 0.030) 0.056 0.042± 0.012Γ2) 0.907 −0.006 (P = 0.214) 0.102 0.094± 0.032HHMM-DFTΓ)2 0.161 0.007 (P = 0.007) 0.055 0.041± 0.011Γ2) 0.916 0.004 (P = 0.372) 0.088 0.091± 0.028CarHHMMΓ)2 0.198 0.044 (P = 0.000) 0.152 0.043± 0.013Γ2) 0.832 −0.081 (P = 0.000) 0.195 0.093± 0.034CarHMM-DFTΓ)2 —— —— —— ——Γ2) —— —— —— ——173A.4. Additional Results from Chapter 2 Simulation StudyTable A.10: Estimates and standard errors for fine-scale transition probability matricesacross all four models in the simulation study. The “Estimate” column gives averagesacross 500 simulations, and the “Observed Fisher SE” column gives medians across 500simulations. Figures following ± refer to inter quartile range. “SE” stands for standarderror. Note that standard error cannot be estimated when Γ∗(i)i∗,j∗ ≈ 0. Reported P -valuescorrespond to one-sample t-tests of whether bias is statistically significant.Model Parameter Estimate Bias Empirical SE Observed Fisher SECarHHMM-DFTΓ∗()))2 0.258 0.006 (P = 0.004) 0.043 0.032± 0.006Γ∗()))+ 0.002 0.002 (P = 0.000) 0.004 0.009± 0.004Γ∗())2) 0.083 0.001 (P = 0.243) 0.015 0.012± 0.002Γ∗())2+ 0.052 −0.000 (P = 0.984) 0.010 0.009± 0.001Γ∗())+) 0.003 0.003 (P = 0.000) 0.008 0.014± 0.005Γ∗())+2 0.234 0.002 (P = 0.322) 0.042 0.036± 0.008Γ∗(2))2 0.111 −0.001 (P = 0.349) 0.017 0.015± 0.004Γ∗(2))+ 0.001 0.000 (P = 0.000) 0.002 0.003± 0.002Γ∗(2)2) 0.151 0.001 (P = 0.584) 0.023 0.021± 0.005Γ∗(2)2+ 0.036 −0.000 (P = 0.944) 0.013 0.010± 0.003Γ∗(2)+) 0.007 0.007 (P = 0.000) 0.024 0.025± 0.015Γ∗(2)+2 0.252 0.026 (P = 0.000) 0.096 0.063± 0.028HHMM-DFTΓ∗()))2 0.240 −0.012 (P = 0.000) 0.042 0.032± 0.006Γ∗()))+ 0.003 0.003 (P = 0.000) 0.006 0.009± 0.004Γ∗())2) 0.077 −0.005 (P = 0.000) 0.019 0.012± 0.002Γ∗())2+ 0.047 −0.004 (P = 0.000) 0.009 0.008± 0.001Γ∗())+) 0.005 0.005 (P = 0.000) 0.009 0.014± 0.004Γ∗())+2 0.153 −0.079 (P = 0.000) 0.029 0.026± 0.005Γ∗(2))2 0.109 −0.004 (P = 0.000) 0.018 0.016± 0.004Γ∗(2))+ 0.002 0.002 (P = 0.000) 0.004 0.005± 0.002Γ∗(2)2) 0.151 0.001 (P = 0.712) 0.031 0.023± 0.006Γ∗(2)2+ 0.037 0.002 (P = 0.009) 0.013 0.011± 0.004Γ∗(2)+) 0.021 0.021 (P = 0.000) 0.026 0.023± 0.012Γ∗(2)+2 0.135 −0.091 (P = 0.000) 0.057 0.038± 0.018CarHHMMΓ∗()))2 0.138 −0.114 (P = 0.000) 0.131 0.018± 0.005Γ∗()))+ 0.023 0.022 (P = 0.000) 0.107 0.007± 0.004Γ∗())2) 0.037 −0.044 (P = 0.000) 0.054 0.007± 0.001Γ∗())2+ 0.027 −0.025 (P = 0.000) 0.050 0.004± 0.001Γ∗())+) 0.013 0.013 (P = 0.000) 0.022 0.011± 0.005Γ∗())+2 0.134 −0.099 (P = 0.000) 0.043 0.023± 0.006Γ∗(2))2 0.056 −0.056 (P = 0.000) 0.035 0.008± 0.002Γ∗(2))+ 0.002 0.002 (P = 0.004) 0.017 0.002± 0.001Γ∗(2)2) 0.069 −0.081 (P = 0.000) 0.032 0.011± 0.003Γ∗(2)2+ 0.017 −0.019 (P = 0.000) 0.012 0.005± 0.002Γ∗(2)+) 0.011 0.011 (P = 0.000) 0.016 0.018± 0.009Γ∗(2)+2 0.138 −0.088 (P = 0.000) 0.079 0.037± 0.016CarHMM-DFTΓ∗()))2 0.158 —— 0.019 0.014± 0.002Γ∗()))+ 0.001 —— 0.002 0.003± 0.001Γ∗())2) 0.096 —— 0.012 0.010± 0.001Γ∗())2+ 0.046 —— 0.007 0.007± 0.001Γ∗())+) 0.002 —— 0.004 0.008± 0.003Γ∗())+2 0.227 —— 0.035 0.030± 0.006174A.4. Additional Results from Chapter 2 Simulation StudyCarHHMM-DFTFigure A.40: Kernel density estimates of the distributions of Γˆ, Γˆ∗()), and Γˆ∗(2) for theCarHHMM-DFT. There are two fine-scale transition probability matrices—one correspond-ing to each dive type. I plot only the off-diagonal elements of the transition probabilitymatrices since these elements uniquely define a stochastic matrix. Red stars represent thetrue values and white stars represent the mean of all maximum likelihood estimates.175A.4. Additional Results from Chapter 2 Simulation StudyHHMM-DFTFigure A.41: Kernel density estimates of the distributions of Γˆ, Γˆ∗()), and Γˆ∗(2) for theHHMM-DFT. There are two fine-scale transition probability matrices—one correspondingto each dive type. I plot only the off-diagonal elements of the transition probability matricessince these elements uniquely define a stochastic matrix. Red stars represent the true valuesand white stars represent the mean of all maximum likelihood estimates.176A.4. Additional Results from Chapter 2 Simulation StudyCarHHMMFigure A.42: Kernel density estimates of the distributions of Γˆ, Γˆ∗()), and Γˆ∗(2) for theCarHHMM. There are two fine-scale transition probability matrices—one corresponding toeach dive type. I plot only the off-diagonal elements of the transition probability matricessince these elements uniquely define a stochastic matrix. Red stars represent the true valuesand white stars represent the mean of all maximum likelihood estimates.177A.4. Additional Results from Chapter 2 Simulation StudyCarHMM-DFTFigure A.43: Kernel density estimates of the distribution of Γˆ∗()) for the CarHMM-DFT.There is only one fine-scale transition probability matrix and no coarse-scale transitionprobability matrix since the CarHMM-DFT assumes that there is only one dive type. Iplot only the off-diagonal elements of the transition probability matrix since these elementsuniquely define a stochastic matrix. White stars represent the mean of all maximum like-lihood estimates. Unlike other figures, there are no red stars representing the true valuesbecause the CarHMM-DFT assumes that there is only one dive type.178Appendix BAppendix for Chapter 3179B.1. Additional Results from Chapter 3 Case Study 1B.1 Additional Results from Chapter 3 Case Study 1This appendix recreates Figures 3.3 and 3.4 from the first case study of Chapter 3 forα = 0P 0.025P 0.049P 0.525P 1. The results for α = 0.025 and α = 0.525 are very similar tothose for α = 0.049, but I include them here for completeness.Figure B.1: Sensitivity, specificity, and AUC values associated with each dive type. Truevalues are determined via the drone-detected dive types. The PHMMs for each value of αwere fit using the entire dataset with a selected sub-profile held out. Then, the dives forthat sub-profile were estimated by using the forward-backward algorithm on the sub-profilewith the drone-detected labels removed. I repeated this process for every sub-profile to geta full set of estimated dive types.180B.1. Additional Results from Chapter 3 Case Study 1(a) Decoded dives for PHMM with α = 1.000(the “natural” weighting).(b) Decoded dives for PHMM with α =0.525.(c) Decoded dives for PHMM with α = 0.049. (d) Decoded dives for PHMM with α =0.025.(e) Decoded dives for PHMM with α = 0.000(treating the unlabelled observations as miss-ing).(f) Dive profile with true drone-detected la-bels.Figure B.2: Viterbi-decoded dives for a given sub-profile and several different PHMMs. EachPHMM was fit to the dataset with the sub-profile held out. Then the Viterbi algorithmwas used on the held-out dataset with its labels removed in order to test the predictiveperformance of each PHMM.181Appendix CAppendix for Chapter 4182C.1. Version 2 of EM-VRSOC.1 Version 2 of EM-VRSOFor the purposes of readability, I abuse notation and use the shorthand F (ϕ | γ(ϕ′)P ξ(ϕ′)) ≡F (ϕ | ϕ′) and Ft(ϕ | γt(ϕ′)P ξt(ϕ′)) ≡ Ft(ϕ | ϕ′) in this appendix. Further, I denoteF ∗(ϕ′) ≡ infϕ∈Φ F (ϕ | ϕ′).Algorithm 6 EM-VRSO(ϕ(P λP APbPK) (Version 2)Require: Initial parameters (ϕ(), step size (λ), algorithm A ∈ {SVRG, SAGA}, whetherto do a partial E step P ∈ {True,False}, number of iterations per update (b), andnumber of updates (K). Assume known Lipschitz constant a and known strong con-vexity constant X.1: ζ = (Xλ(1− 2aλ)b)−) + (2aλ)/(1− 2aλ) Q 1 C ζ Q 12: for k = 0P . . . PK − 1 do3: {αk,tPβk,tPγk,tP ξk,t}it5) = E-step(ϕk) C E step4: K(ϕk) = F ∗(ϕk) + [(1 + ζ)/2] (F (ϕk | ϕk)− F ∗(ϕk))5: ℓ← 0 C M step6: while ℓ = 0 or F (ϕk,ℓ | ϕk) S K(ϕk) do7: ℓ← ℓ+ 18: ϕk,ℓ = VRSO-PE({αk,tPβk,tPγk,tP ξk,t}it5)PϕkP λP AP PPb)9: end while10: ϕk+) = ϕk,ℓ11: end for12: return ϕKOne potential concern with Algorithm 6 is that it requires quantities that are not knownin practice, namely ζ and F ∗(ϕk). However, these quantities are only used in the whileloop to ensure a strict increase in the likelihood for every iteration of VRSO-PE. In addition,version 1 of the algorithm is intuitively preferable since it will accept new parameters thatincrease the likelihood by any amount. I only use version 2 above to prove Theorem 4.1.Another concern with Algorithm 6 is that F ∗(ϕk) is defined using an infimum, which canbe problematic in some settings. However, I prove in Lemma 1 that if the conditions ofTheorem 4.1 hold, then F ∗(ϕ′) is finite, has a unique minimizer, and is uniformly continuous.Lemma 1. If the conditions of Theorem 4.1 hold, then for all ϕk ∈ Φ, F ∗(ϕk) =infϕ∈Φ F (ϕ | ϕk) is finite and has a unique minimizer, so F ∗(ϕk) = minϕ∈Φ F (ϕ | ϕk) =F (ϕ∗k+) | ϕk) for a unique ϕ∗k+) ∈ Φ. Further, F ∗(ϕk) is uniformly continuous for allϕk ∈ Φ.Proof. For any fixed ϕk, if the conditions of Theorem 4.1 hold, then there exists a uniqueminimizer of F (· | ϕk) over Φϕk because Φϕk is compact and F is strongly convex and183C.2. Proof of Theorem 4.1continuous. Denote this minimizer as ϕ∗k+) = argminϕ∈Φϕk F (ϕ | ϕk). I show that ϕ∗k+)minimizes F (· | ϕk) over the entirety of Φ using two cases:1. For any ϕ ∈ Φϕk with ϕ ̸= ϕ∗k+), we have that F (ϕ | ϕk) S F (ϕ∗k+) | ϕk) becauseϕ∗k+) is the unique minimizer over Φϕk .2. For any ϕ /∈ Φϕk , log p(y;ϕ) Q log p(y;ϕk) ≤ log p(y;ϕ∗k+)) by the definition of Φϕk .This implies that F (ϕ | ϕk) S F (ϕk | ϕk) ≥ F (ϕ∗k+) | ϕk) by basic properties of theEM algorithm (Dempster et al., 1977).Therefore, ϕ∗k+) is a global minimizer of F (· | ϕk), and F ∗(ϕk) = F (ϕ∗k+) | ϕk).I now prove that F ∗(ϕk) is uniformly continuous. Let ϵ S 0 be fixed. Then, becauseF (y | x) is uniformly continuous in (xP y) ∈ Φ × Φ, there exists some δ such that for all(yP x) and (yP x() with |x − x(| Q δ we have that |F (y | x) − F (y | x()| Q ϵ. Now, Ifix arbitrary x and x( with |x − x(| Q δ and show that |F ∗(x) − F ∗(x()| Q ϵ. To thisend, pick y∗ such that F (y∗ | x) = miny∈Φ F (y | x) = F ∗(x) and pick y∗( such thatF (y∗( | x() = miny∈Φ F (y | x() = F ∗(x(). Then:F ∗(x() = F (y∗( | x() ≤ F (y∗ | x() Q F (y∗ | x) + ϵ = F ∗(x) + ϵPF ∗(x() = F (y∗( | x() S F (y∗( | x)− ϵ ≥ F (y∗ | x)− ϵ = F ∗(x)− ϵPso |F ∗(x)− F ∗(x()| Q ϵ.C.2 Proof of Theorem 4.1I begin by introducing notation needed for the proof. Let I ∈ {1P . . . P i}M be an b -dimensional vector of indices. Let VRSO-PE∗(IP λPϕ′) correspond to the mapping that resultsfrom performing b steps of SVRG on objective function F (· | ϕ′) starting at ϕ′ with stepsize λ and random realization I. That is,VRSO-PE∗(IP λPϕ′) = VRSO-PE({αt(ϕ′)Pβt(ϕ′)Pγt(ϕ′)P ξt(ϕ′)}it5)Pϕ′P λP SVRGP FalsePb)when the random vector of indices(t( · · · tM−))is equal to I. Below, I prove thatVRSO-PE∗(IP λPϕ′) is uniformly continuous in ϕ′ ∈ Φ for fixed I and λ in Lemma 2.Lemma 2. Suppose all conditions from Theorem 4.1 hold. Then VRSO-PE∗(IP λPϕ′) isuniformly continuous in ϕ′ ∈ Φ for fixed I and λ.Proof. Define ϕ′(() = ϕ′. The function VRSO-PE∗(IP λPϕ′) involves calculating the parame-184C.2. Proof of Theorem 4.1Table C.1: Legend connecting my notation to that of Johnson and Zhang (2013).Johnson and Zhang (2013) My Notationα ζη λa aγ Xm bw˜( ϕkw˜) ϕk,ℓw∗ ϕ∗k+)P F (· | ϕk)ψi Ft(· | ϕk)ters ϕ′(()Pϕ′())P . . . Pϕ′(m)P . . . Pϕ′(M−)) by iteratively applying the following mapping (withtm = I[m+ 1]) and returning ϕ′(M) ≡ ϕ:ϕ′(m+))(ϕ′(m)Pϕ′(()) = λ[∇Ftm(ϕ′(m) | ϕ′(())−∇Ftm(ϕ′(() | ϕ′(()) +1iT∑t=)∇Ft(ϕ′(() | ϕ′(())].Note that ∇Ft(ϕ | ϕ′) is uniformly continuous in (ϕPϕ′) for each t by condition (7) ofTheorem 4.1. Therefore, the mapping above is also uniformly continuous in (ϕ′(m)Pϕ′(()),and so VRSO-PE∗(IP λPϕ′) is uniformly continuous in ϕ′ for fixed I and λ.If the conditions of Theorem 4.1 are met, iteration k of Algorithm 6 is equivalent to drawinga sequence of index vectors Ik,(P Ik,)P . . . and calculating ϕk,ℓ = VRSO-PE∗(Ik,ℓP λPϕk) untilthe condition to exit the while loop is satisfied, which happens in finite time by Lemma 3below. Lemma 3 depends upon Theorem 4.1 of Johnson and Zhang (2013), so Table (C.1)identifies how my notation corresponds to theirs.Lemma 3. Suppose all conditions from Theorem 4.1 hold. Then, the while loop withinAlgorithm 6 almost surely terminates in finite time, i.e. P{ℓ∗(k) Q ∞} = 1 for all k ≥ 0.Furthermore, P{∀k : ℓ∗(k) Q∞} = 1.Proof. By Lemma 1, F ∗(ϕk) = infϕ∈Φ F (ϕ | ϕk) is finite and uniformly continuous in ϕk.Further, Theorem 4.1 of Johnson and Zhang (2013) applies for one iteration through bsteps of SVRG. In particular, for all ℓ ≥ 0:E[F (ϕk,ℓ | ϕk)− F ∗(ϕk)∣∣∣ ϕk] ≤ ζ(F (ϕk | ϕk)− F ∗(ϕk))P (C.1)185C.2. Proof of Theorem 4.1where ζ is defined in condition (6) of Theorem 4.1. Using (C.1) and Markov’s inequalitygivesP[F (ϕk,ℓ | ϕk)− F ∗(ϕk) ≥1 + ζ2(F (ϕk | ϕk)− F ∗(ϕk))∣∣∣ ϕk] ≤ ζ (F (ϕk | ϕk)− F ∗(ϕk)))+ζ2(F (ϕk | ϕk)− F ∗(ϕk))(C.2)=21 + 1/ζQ 1. (C.3)Taking the complement of the above expression and rearranging terms givesP[F (ϕk,ℓ | ϕk) ≤ F ∗(ϕk) +1 + ζ2(F (ϕk | ϕk)− F ∗(ϕk)) ∣∣∣ ϕk] ≥ 1− ζ1 + ζ S 0P (C.4)so with some probability greater than or equal to (1− ζ)/(1 + ζ) S 0, the condition to exitthe while loop at iteration ℓ of Algorithm 6 is satisfied.Further, repeatedly calling VRSO-PE within the inner while loop of Algorithm 6 correspondsto drawing independent samples of ϕk,ℓ conditioned on ϕk. The draws are independentbecause VRSO-PE∗ is only random due to Ik,λ after conditioning on ϕk, and each iterationof the inner while loop of Algorithm 6 involves drawing independent samples of Ik,λ. As aresult, ℓ∗(k) follows a geometric distribution with some positive success probability pk ≥(1− ζ)/(1 + ζ). Therefore, P{ℓ∗(k) Q∞}=∑ℓ≥) pk(1− pk)ℓ−) = 1, andP{∀k : ℓ∗(k) Q∞}= 1− P{∃k : ℓ∗(k) =∞}= 1.Finally, I introduce gM,λ as a point-to-set map corresponding to one iteration of Algorithm6 with b steps of step size λ. First, recall thatK(ϕ′) = F ∗(ϕ′) + [(1 + ζ)/2](F (ϕ′ | ϕ′)− F ∗(ϕ′)). (C.5)186C.2. Proof of Theorem 4.1Then, define gM,λ(ϕ′) asgM,λ(ϕ′) ={ϕ : ϕ = VRSO-PE∗(IP λPϕ′) for some I ∈ {1P . . . P i}M and F (ϕ | ϕ′) ≤ K(ϕ′)}.(C.6)Iteration k of Algorithm 6 corresponds to randomly sampling ϕk+) from gM,λ(ϕk). If theconditions of Theorem 4.1 are met, then P(ℓ∗(k) Q ∞) = 1, so gM,λ(ϕk) must not be theempty set. I prove two more useful Lemmas regarding gM,λ before finally proving Theorem4.1.Lemma 4. Suppose all conditions from Theorem 4.1 hold. Then gM,λ is a closed point-to-set map for all non-stationary points ϕ′ ∈ Φ. Note that gM,λ is closed at a point ϕ′ ifϕ′n → ϕ′ and ϕn → ϕ with ϕn ∈ gM,λ(ϕ′n), implies that ϕ ∈ gM,λ(ϕ′).Proof. Given a point ϕ′, suppose there exists some sequence ϕ′n → ϕ′ as well as anothersequence ϕn → ϕ with ϕn ∈ gM,λ(ϕ′n). To prove the Lemma, I prove that ϕ ∈ gM,λ(ϕ′),i.e. that1. F (ϕ | ϕ′) ≤ K(ϕ′), and2. ϕ = VRSO-PE∗(IP λPϕ′) for some I ∈ {1P . . . P i}MI start with the first condition. Since ϕn → ϕ and ϕ′n → ϕ′, we have that F (ϕn | ϕ′n) →F (ϕ | ϕ′) because F is continuous by condition (7) of Theorem 4.1. Likewise, K(ϕ′n) →K(ϕ′) because K is continuous by Lemma 1. Finally, F (ϕ | ϕ′) ≤ K(ϕ′) because for all n,ϕn ∈ gM,λ(ϕ′n), which implies that F (ϕn | ϕ′n) ≤ K(ϕ′n). Taking the limit of both sides asn→∞ gives the result.I now show the second condition. By way of contradiction, assume ϕ ̸= VRSO-PE∗(IP λPϕ′)for any value of I ∈ {1P . . . P i}M . Then, minI ||ϕ− VRSO-PE∗(IP λPϕ′)|| is strictly positive.By assumption, for each n, ϕn ∈ gM,λ(ϕ′n), so ϕn = VRSO-PE∗(InP λPϕ′n) for some In ∈{1P . . . P i}M . Using the triangle inequality, we have:0 Q minI||ϕ− VRSO-PE∗(IP λPϕ′)|| (C.7)≤ ||ϕn − ϕ||+ ||ϕn − VRSO-PE∗(InP λPϕ′)|| (C.8)= ||ϕn − ϕ||+ ||VRSO-PE∗(InP λPϕ′n)− VRSO-PE∗(InP λPϕ′)|| (C.9)≤ ||ϕn − ϕ||+maxI||VRSO-PE∗(IP λPϕ′n)− VRSO-PE∗(IP λPϕ′)||. (C.10)187C.2. Proof of Theorem 4.1However, the right-hand side of the equation converges to zero since ϕn → ϕ, ϕ′n → ϕ′,and VRSO-PE∗(IP λPϕ′) is continuous for fixed I and λ. This is a contradiction, so it mustbe that ϕ = VRSO-PE∗(IP λPϕ′) for some I.Lemma 5. Suppose all conditions from Theorem 4.1 hold, and consider the sequence{ϕk}Kk5( generated from running Algorithm 6 with ϕ( ∈ Φ. For k = 0P . . . PK − 1, we havethat log p(y;ϕk+)) ≥ log p(y;ϕk). Further, if ϕk is not a stationary point of log p, thenlog p(y;ϕk+)) S log p(y;ϕk).Proof. First, note that ϕk is a stationary point of log p if and only if it is a stationary pointof F (· | ϕk) because ∇ log p(ϕk) = ∇f(ϕk | ϕk) = −∇F (ϕk | ϕk)/i .I begin with the case where ϕk is a stationary point of log p (and therefore a stationarypoint of F (· | ϕk)). Then, F (ϕk | ϕk) = F ∗(ϕk) since F is continuously differentiable andstrongly convex. To exit the while loop at iteration k of Algorithm 6, it must be the casethat F (ϕk+) | ϕk) ≤ F ∗(ϕk)+ [(1+ ζ)/2](F (ϕk | ϕk)−F ∗(ϕk))= F (ϕk | ϕk). Finally, weknow from Dempster et al. (1977) that log p(y;ϕk+))− log p(y;ϕk) ≥ F (ϕk | ϕk)−F (ϕk+) |ϕk) ≥ 0. This implies that log p(y;ϕk+)) ≥ log p(y;ϕk).Next, I focus on the case where ϕk is not a stationary point of log p (and therefore not astationary point F (· | ϕk)). Then, F (ϕk | ϕk) S F ∗(ϕk) since F is continuously differen-tiable and strongly convex. To exit the while loop at iteration k of Algorithm 6, it must bethe case that F (ϕk+) | ϕk) ≤ F ∗(ϕk) + [(1 + ζ)/2](F (ϕk | ϕk) − F ∗(ϕk))Q F (ϕk | ϕk).Finally, we know from Dempster et al. (1977) that log p(y;ϕk+)) − log p(y;ϕk) ≥ F (ϕk |ϕk)− F (ϕk+) | ϕk) S 0. This implies that log p(y;ϕk+)) S log p(y;ϕk).I now prove Theorem 4.1.Proof. The first part of Theorem 4.1 states that P(ℓ∗(k) Q ∞) = 1 for all k ≥ 0 and istrue by Lemma 3. The second part of Theorem 4.1 states that all limit points of {ϕk}∞k5(are stationary points of log p(y;ϕ), and log p(y;ϕk) converges monotonically to log p∗ =log p(y;ϕ∗) for some stationary point of log p, ϕ∗. This is a direct application of Theorem4.1 of Wu (1983), which requires the following conditions:(a) gM,λ is a closed point-to-set map for all non-stationary points.(b) log p(y;ϕk+)) ≥ log p(y;ϕk) for all ϕk ∈ Φ, and log p(y;ϕk+)) S log p(y;ϕk) for allϕk ∈ Φ that are not stationary points of log p.Condition (a) is satisfied by Lemma 4, and condition (b) is satisfied by Lemma 5.188C.3. Proof of Theorem 4.2C.3 Proof of Theorem 4.2I first give a basic outline of the proof. If ∇ log p(y;ϕ() = 0, then ∇F (ϕ( | ϕ() = 0,so every gradient step of SAGA and SVRG within VRSO-PE does not change ϕ(m)( , so= ϕ(m+))( = ϕ(m)( = ϕ(()( (we prove this in step 3 of lemma 6). As a result, the gradientapproximations do not change either, so ∇̂F (m+)) = ∇̂F (m) and ∇̂F (m+))t = ∇̂F (m)t for t =1P . . . P i (we prove this in steps 4 and 5 of lemma 6). Finally, since the parameter estimatesdo not change, the conditional probability approximations{α̂(m)t P β̂(m)t P γ̂(m)t P ξ̂(m)t}it5)={αt(ϕ()Pβt(ϕ()Pγt(ϕ()P ξt(ϕ()}it5) do not change either (we prove this in steps 1 and 2 oflemma 6). Because VRSO-PE returns ϕ) = ϕ(M)( = ϕ(, each iteration of EM-VRSO must dothe same, so by induction ϕK = ϕ( (we prove this in the body of theorem 2).To prove the Theorem more formally, I first prove a useful Lemma.Lemma 6. If:1. α̂(()t = αt(ϕ(()) and β̂(()t = βt(ϕ(()) for all t = 1P . . . P i ,2. γ̂(()t = γt(ϕ(()) and ξ̂(()t = ξt(ϕ(()) for all t = 1P . . . P i ,3. ∇F (ϕ(() | γ(ϕ(())P ξ(ϕ(())) = 0,then with probability 1,VRSO-PE({α̂(()t P β̂(()t P γ̂(()t P ξ̂(()t}it5)Pϕ(()P λP AP PPb)= ϕ(() (C.11)for all λ ∈ R, A ∈ {SAGAPSVRG}, P ∈ {TrueP False}, and b ∈ N.Proof. I use induction to show that the following conditions hold for all m ≥ 0:1. α̂(m)t = αt(ϕ(()) and β̂(m)t = βt(ϕ(()) for all t = 1P . . . P i .2. γ̂(m)t = γt(ϕ(()) and ξ̂(m)t = ξt(ϕ(()) for all t = 1P . . . P i .3. ϕ(m) = ϕ(()4. ∇̂F (m)t = ∇Ft(ϕ(() | γt(ϕ(())P ξt(ϕ(()))for all t = 1P . . . P i .5. ∇̂F (m) = 0Algorithm 4 returns ϕ(M) for some finite b , so proving the above will prove the lemma.The base case (m = 0) is trivial either from the assumptions of the Lemma or from lines1–4 of Algorithm 4, so I focus on the inductive case. I assume that all of the conditions(1–5) above hold for m, and prove each condition holds for m+ 1 sequentially.189C.3. Proof of Theorem 4.21. First note that if P = False or if P = True and t ̸= tm, then the conditionalprobability approximations are not updated, so α̂(m+))t = α̂(m)t = αt(ϕ(()) andβ̂(m+))t = β̂(m)t = βt(ϕ(()) by inductive hypothesis (1).If P = True and t = tm, then both α̂(m+))tm and β̂(m+))tm are updated in line 9 ofAlgorithm 5. The approximation α̂(m+))tm is updated using the mappingα̂(m+))tm = α˜tm(α̂(m)tm−)Pϕ(m))= α˜tm(αtm−)(ϕ(())Pϕ(())=δ(ν(()) P (y);θ(())P for tm = 1αtm−)(ϕ(()) Γ(η(()) P (ytm ;θ(())P for tm = 2P . . . P i= αtm(ϕ(()) (C.12)The second line is true by inductive hypotheses (1) and (3), the third line is thedefinition of α˜tm , and the final line is the definition of αtm(ϕ(()). Similar logic canbe used to show thatβ̂(m+))tm = βtm(ϕ(()). (C.13)2. First note that if P = False or if P = True and t ̸= tm, then the conditional prob-ability approximations are not updated, so γ̂(m+))t = γ̂(m)t = γt(ϕ(()) and ξ̂(m+))t =ξ̂(m)t = ξt(ϕ(()) by inductive hypothesis (2).If P = True and t = tm, both γ̂(m+))tm and ξ̂(m+))tm are calculated in line 10 of Algorithm5. ξ̂(m+))tm is updated using the mappingξ̂(m+))tm = ξ˜tm(α̂(m+))tm−) P β̂(m+))tm Pϕ(m))= ξ˜tm(αtm−)(ϕ(())Pβtm(ϕ(())Pϕ(())=diag(αt−)(ϕ(())) Γ(η(()) P (yt;θ(()) diag(βt(ϕ(()))αt−)(ϕ(()) Γ(η(()) P (yt;θ(()) βt(ϕ(())⊤= ξtm(ϕ(()). (C.14)The second line is true by Equations (C.12 – C.13) and inductive hypothesis (3), thethird line is the definition of ξ˜tm , and the final line is the definition of ξtm(ϕ(()).190C.3. Proof of Theorem 4.2Similar logic can be used to show thatγ̂(m+))tm = γtm(ϕ(()). (C.15)3. The parameter ϕ(m+)) is calculated in line 12 of VRSO-PE as follows:ϕ(m+)) = ϕ(m) − λ[∇Ftm(ϕ(m) | γ̂(m+))tm P ξ̂(m+))tm)− ∇̂F (m)tm + ∇̂F (m)]= ϕ(() − λ[∇Ftm(ϕ(() | γ̂(m+))tm P ξ̂(m+))tm)−∇Ftm(ϕ(() | γtm(ϕ(())P ξtm(ϕ(()))+ 0]= ϕ(() − λ[∇Ftm(ϕ(() | γtm(ϕ(())P ξtm(ϕ(()))−∇Ftm(ϕ(() | γtm(ϕ(())P ξtm(ϕ(())) ]= ϕ(().The second line is true by the inductive hypotheses (3), (4), and (5), and the thirdline is true by Equations (C.14 – C.15).4. Note that if A = SVRG or if A = SAGA and t ̸= tm, then the gradient approximationat index t is not updated, so ∇̂F (m+))t = ∇̂F (m)t = ∇Ft(ϕ(() | γt(ϕ(())P ξt(ϕ(()))byinductive hypothesis (4).If A = SAGA and t = tm, then ∇̂F (m+))tm is calculated in line 14 of VRSO-PE as follows:∇̂F (m+))tm = ∇Ftm(ϕ(m) | γ̂(m+))tm P ξ̂(m+))tm)= ∇Ftm(ϕ(() | γ̂(m+))tm P ξ̂(m+))tm)= ∇Ftm(ϕ(() | γtm(ϕ(())P ξtm(ϕ(())). (C.16)The second line is true by inductive hypothesis (3), and the third line is true byEquations (C.14 – C.15).5. If A = SVRG, then the full gradient approximation is not updated, so ∇̂F (m+)) =∇̂F (m) = 0 by inductive hypothesis (5).If A = SAGA, then ∇̂F (m+)) is calculated in line 14 of VRSO-PE as follows:191C.3. Proof of Theorem 4.2∇̂F (m+)) = ∇̂F (m) + 1i(∇̂F (m+))tm − ∇̂F(m)tm)= 0 +1i[∇̂F (m+))tm −∇Ftm(ϕ(() | γtm(ϕ(())P ξtm(ϕ(()))]=1i[∇Ftm(ϕ(() | γtm(ϕ(())P ξtm(ϕ(()))−∇Ftm(ϕ(() | γtm(ϕ(())P ξtm(ϕ(())) ]= 0.The second line is true by inductive hypotheses (4) and (5), and the third line is trueby Equation (C.16).This completes the inductive step, and proves the lemma.I now prove Theorem 4.2.Proof. I use induction and show that if ∇ log p(y;ϕ() = 0, then ϕk = ϕ( for all iterationsk of Algorithm 5. The base case, k = 0, is trivial. For the inductive case, I assume thatϕk = ϕ( and prove that ϕk+) = ϕk. First, note that∇F (ϕ( | γ(ϕ()P ξ(ϕ()) =1ii∑t5)∇Ft(ϕ( | γt(ϕ()P ξt(ϕ())= − 1i∇f(ϕ( | ϕ()= − 1i∇ log p(y ; ϕ()= 0. (C.17)Equation (C.17) shows that the conditions for Lemma 6 are satisfied with ϕ(() = ϕ(.Further, step 2 of Algorithm 5 is the E step of the EM algorithm, so it is clearly the casethat {αk,tPβk,tPγk,tP ξk,t}it5) = {αt(ϕk)Pβt(ϕk)Pγt(ϕk)P ξt(ϕk)}it5). So:192C.3. Proof of Theorem 4.2ϕk,) = VRSO-PE({αk,tPβk,tPγk,tP ξk,t}it5)PϕkP λP AP PPb)= VRSO-PE({αt(ϕk)Pβt(ϕk)Pγt(ϕk)P ξt(ϕk)}it5) PϕkP λP AP PPb)= VRSO-PE({αt(ϕ()Pβt(ϕ()Pγt(ϕ()P ξt(ϕ()}it5) Pϕ(P λP AP PPb)= ϕ(.The second line is true by the definition of the E step in the EM algorithm, the thirdline is true by the inductive hypothesis, and the final line is true by Lemma 6. Sincelog p(y;ϕk,)) = log p(y;ϕk) = log p(y;ϕ(), ϕk,) satisfies the condition to exit the while loopof Algorithm 5, so ϕk+) = ϕk,) = ϕ(.This completes the inductive step, so ϕk = ϕ( for all k ≥ 0. To complete the proof, notethat Algorithm 5 returns ϕK , so EM-VRSO(ϕ(P λP APbPK) = ϕK = ϕ(.193C.4. Additional Results from Chapter 4 Simulation StudyC.4 Additional Results from Chapter 4 Simulation StudyC.4.1 Simulation Study Results, T = 10+The pages that follow present results for the experiments with i = 10+. They are analogousto Figure 4.3 and Figure 4.4 in the main text.194C.4. Additional Results from Chapter 4 Simulation StudyFigure C.1: Box plots showing time to terminate (top), epochs to terminate (middle), andmaximum log-likelihood minus log-likelihood at termination (all over i , bottom) for eachoptimization algorithm in the simulation study. The log-likelihood is denoted as ℓ(ϕ), andthe maximum log-likelihood across all algorithms and initial values after 12 hours is denotedas ℓ(ϕ∗). Each box plot summarizes results from all five data sets and all five parameterinitializations per data set. FE corresponds to {P = FalsePb = i}, PE1 corresponds to{P = TruePb = i}, and PE2 corresponds to {P = FalsePb = 10i}. Results are shownfor all simulation studies with i = 10+. The y-axis of the bottom row is on a log-scale.195C.4. Additional Results from Chapter 4 Simulation StudyFigure C.2: Maximum log-likelihood minus the log-likelihood at termination (all over i )versus epochs to termination for all simulation studies with i = 10+. One epoch representseither one full E step, i iterations within the M step, or one gradient step for full-gradientalgorithms. Termination is defined as either convergence or 12 hours of computation time,whichever came first. The log-likelihood is denoted as ℓ(ϕ), and the maximum log-likelihoodacross all algorithms and initial values after 12 hours is denoted as ℓ(ϕ∗). The y-axis is ona log-scale.196C.4. Additional Results from Chapter 4 Simulation StudyFigure C.3: Maximum log-likelihood minus the log-likelihood at termination (all over i )versus time to termination for all simulation studies with i = 10+. Termination is definedas either convergence or 12 hours of computation time, whichever came first. The log-likelihood is denoted as ℓ(ϕ), and the maximum log-likelihood across all algorithms andinitial values after 12 hours is denoted as ℓ(ϕ∗). The y-axis is on a log-scale.The pages that follow present figures that show the optimality gap vs epoch or time foreach data set of every experiment with i = 10+ for all nine tested algorithms. They aresimilar to Figure 4.2 from the main text. EM-VRSO tended to converge more quickly than197C.4. Additional Results from Chapter 4 Simulation StudyBFGS for experiments with N = 3. BFGS often out-performed EM-VRSO in the long-run(after approximately 100 epochs), but BFGS is a second-order method and this behavioris expected in the long run on small data sets. My method is particularly well-suited tofitting HMMs on large and complex data sets.198C.4. Additional Results from Chapter 4 Simulation StudyFigure C.4: Optimally gap between the current log-likelihood and optimal log-likelihood forthe first simulated data set associated with experiments i = 10+, N ∈ {3P 6} and y ∈ {3P 6}.One epoch represents either one full E step, i iterations within the M step, or one gradientstep for full-gradient algorithms. The log-likelihood of ϕ is denoted as ℓ(ϕ) on the y-axis,which is on a log-scale. Dots correspond to the point of termination for each algorithm.199C.4. Additional Results from Chapter 4 Simulation StudyFigure C.5: Optimally gap between the current log-likelihood and optimal log-likelihood forthe first simulated data set associated with experiments i = 10+, N ∈ {3P 6} and y ∈ {3P 6}.The log-likelihood is denoted as ℓ(ϕ), and the maximum log-likelihood across all algorithmsand initial values after 12 hours is denoted as ℓ(ϕ∗). The y-axis is on a log-scale. Dotscorrespond to the point of termination for each algorithm.200C.4. Additional Results from Chapter 4 Simulation StudyFigure C.6: Optimally gap between the current log-likelihood and optimal log-likelihoodfor the second simulated data set associated with experiments i = 10+, N ∈ {3P 6} andy ∈ {3P 6}. One epoch represents either one full E step, i iterations within the M step, orone gradient step for full-gradient algorithms. The log-likelihood is denoted as ℓ(ϕ), andthe maximum log-likelihood across all algorithms and initial values after 12 hours is denotedas ℓ(ϕ∗). The y-axis is on a log-scale. Dots correspond to the point of termination for eachalgorithm.201C.4. Additional Results from Chapter 4 Simulation StudyFigure C.7: Optimally gap between the current log-likelihood and optimal log-likelihoodfor the second simulated data set associated with experiments i = 10+, N ∈ {3P 6} andy ∈ {3P 6}. The log-likelihood is denoted as ℓ(ϕ), and the maximum log-likelihood across allalgorithms and initial values after 12 hours is denoted as ℓ(ϕ∗). The y-axis is on a log-scale.Dots correspond to the point of termination for each algorithm.202C.4. Additional Results from Chapter 4 Simulation StudyFigure C.8: Optimally gap between the current log-likelihood and optimal log-likelihoodfor the third simulated data set associated with experiments i = 10+, N ∈ {3P 6} andy ∈ {3P 6}. One epoch represents either one full E step, i iterations within the M step, orone gradient step for full-gradient algorithms. The log-likelihood is denoted as ℓ(ϕ), andthe maximum log-likelihood across all algorithms and initial values after 12 hours is denotedas ℓ(ϕ∗). The y-axis is on a log-scale. Dots correspond to the point of termination for eachalgorithm.203C.4. Additional Results from Chapter 4 Simulation StudyFigure C.9: Optimally gap between the current log-likelihood and optimal log-likelihoodfor the third simulated data set associated with experiments i = 10+, N ∈ {3P 6} andy ∈ {3P 6}. The log-likelihood is denoted as ℓ(ϕ), and the maximum log-likelihood across allalgorithms and initial values after 12 hours is denoted as ℓ(ϕ∗). The y-axis is on a log-scale.Dots correspond to the point of termination for each algorithm.204C.4. Additional Results from Chapter 4 Simulation StudyFigure C.10: Optimally gap between the current log-likelihood and optimal log-likelihoodfor the fourth simulated data set associated with experiments i = 10+, N ∈ {3P 6} andy ∈ {3P 6}. One epoch represents either one full E step, i iterations within the M step, orone gradient step for full-gradient algorithms. The log-likelihood is denoted as ℓ(ϕ), andthe maximum log-likelihood across all algorithms and initial values after 12 hours is denotedas ℓ(ϕ∗). The y-axis is on a log-scale. Dots correspond to the point of termination for eachalgorithm.205C.4. Additional Results from Chapter 4 Simulation StudyFigure C.11: Optimally gap between the current log-likelihood and optimal log-likelihoodfor the fourth simulated data set associated with experiments i = 10+, N ∈ {3P 6} andy ∈ {3P 6}. The log-likelihood is denoted as ℓ(ϕ), and the maximum log-likelihood across allalgorithms and initial values after 12 hours is denoted as ℓ(ϕ∗). The y-axis is on a log-scale.Dots correspond to the point of termination for each algorithm.206C.4. Additional Results from Chapter 4 Simulation StudyFigure C.12: Optimally gap between the current log-likelihood and optimal log-likelihoodfor the fifth simulated data set associated with experiments i = 10+, N ∈ {3P 6} andy ∈ {3P 6}. One epoch represents either one full E step, i iterations within the M step, orone gradient step for full-gradient algorithms. The log-likelihood is denoted as ℓ(ϕ), andthe maximum log-likelihood across all algorithms and initial values after 12 hours is denotedas ℓ(ϕ∗). The y-axis is on a log-scale. Dots correspond to the point of termination for eachalgorithm.207C.4. Additional Results from Chapter 4 Simulation StudyFigure C.13: Optimally gap between the current log-likelihood and optimal log-likelihoodfor the fifth simulated data set associated with experiments i = 10+, N ∈ {3P 6} andy ∈ {3P 6}. The log-likelihood is denoted as ℓ(ϕ), and the maximum log-likelihood across allalgorithms and initial values after 12 hours is denoted as ℓ(ϕ∗). The y-axis is on a log-scale.Dots correspond to the point of termination for each algorithm.208C.4. Additional Results from Chapter 4 Simulation StudyC.4.2 Simulation Study Results, T = 105The pages that follow present figures that are similar to Figure 4.2 from the simulationstudy in the main text. They show the optimality gap vs epoch for each experiment andeach of the nine tested algorithms. However, here I include all simulated data sets and plotthe optimality gap vs time.209C.4. Additional Results from Chapter 4 Simulation StudyFigure C.14: Optimally gap between the current log-likelihood and optimal log-likelihoodfor the first simulated data set associated with experiments i = 105, N ∈ {3P 6} andy ∈ {3P 6}. One epoch represents either one full E step, i iterations within the M step, orone gradient step for full-gradient algorithms. The log-likelihood is denoted as ℓ(ϕ), andthe maximum log-likelihood across all algorithms and initial values after 12 hours is denotedas ℓ(ϕ∗). The y-axis is on a log-scale. Dots correspond to the point of termination for eachalgorithm.210C.4. Additional Results from Chapter 4 Simulation StudyFigure C.15: Optimally gap between the current log-likelihood and optimal log-likelihoodfor the first simulated data set associated with experiments i = 105, N ∈ {3P 6} andy ∈ {3P 6}. The log-likelihood is denoted as ℓ(ϕ), and the maximum log-likelihood across allalgorithms and initial values after 12 hours is denoted as ℓ(ϕ∗). The y-axis is on a log-scale.Dots correspond to the point of termination for each algorithm.211C.4. Additional Results from Chapter 4 Simulation StudyFigure C.16: Optimally gap between the current log-likelihood and optimal log-likelihoodfor the second simulated data set associated with experiments i = 105, N ∈ {3P 6} andy ∈ {3P 6}. One epoch represents either one full E step, i iterations within the M step, orone gradient step for full-gradient algorithms. The log-likelihood is denoted as ℓ(ϕ), andthe maximum log-likelihood across all algorithms and initial values after 12 hours is denotedas ℓ(ϕ∗). The y-axis is on a log-scale. Dots correspond to the point of termination for eachalgorithm.212C.4. Additional Results from Chapter 4 Simulation StudyFigure C.17: Optimally gap between the current log-likelihood and optimal log-likelihoodfor the second simulated data set associated with experiments i = 105, N ∈ {3P 6} andy ∈ {3P 6}. The log-likelihood is denoted as ℓ(ϕ), and the maximum log-likelihood across allalgorithms and initial values after 12 hours is denoted as ℓ(ϕ∗). The y-axis is on a log-scale.Dots correspond to the point of termination for each algorithm.213C.4. Additional Results from Chapter 4 Simulation StudyFigure C.18: Optimally gap between the current log-likelihood and optimal log-likelihoodfor the third simulated data set associated with experiments i = 105, N ∈ {3P 6} andy ∈ {3P 6}. One epoch represents either one full E step, i iterations within the M step, orone gradient step for full-gradient algorithms. The log-likelihood is denoted as ℓ(ϕ), andthe maximum log-likelihood across all algorithms and initial values after 12 hours is denotedas ℓ(ϕ∗). The y-axis is on a log-scale. Dots correspond to the point of termination for eachalgorithm.214C.4. Additional Results from Chapter 4 Simulation StudyFigure C.19: Optimally gap between the current log-likelihood and optimal log-likelihoodfor the third simulated data set associated with experiments i = 105, N ∈ {3P 6} andy ∈ {3P 6}. The log-likelihood is denoted as ℓ(ϕ), and the maximum log-likelihood across allalgorithms and initial values after 12 hours is denoted as ℓ(ϕ∗). The y-axis is on a log-scale.Dots correspond to the point of termination for each algorithm.215C.4. Additional Results from Chapter 4 Simulation StudyFigure C.20: Optimally gap between the current log-likelihood and optimal log-likelihoodfor the fourth simulated data set associated with experiments i = 105, N ∈ {3P 6} andy ∈ {3P 6}. One epoch represents either one full E step, i iterations within the M step, orone gradient step for full-gradient algorithms. The log-likelihood is denoted as ℓ(ϕ), andthe maximum log-likelihood across all algorithms and initial values after 12 hours is denotedas ℓ(ϕ∗). The y-axis is on a log-scale. Dots correspond to the point of termination for eachalgorithm.216C.4. Additional Results from Chapter 4 Simulation StudyFigure C.21: Optimally gap between the current log-likelihood and optimal log-likelihoodfor the fourth simulated data set associated with experiments i = 105, N ∈ {3P 6} andy ∈ {3P 6}. The log-likelihood is denoted as ℓ(ϕ), and the maximum log-likelihood across allalgorithms and initial values after 12 hours is denoted as ℓ(ϕ∗). The y-axis is on a log-scale.Dots correspond to the point of termination for each algorithm.217C.4. Additional Results from Chapter 4 Simulation StudyFigure C.22: Optimally gap between the current log-likelihood and optimal log-likelihoodfor the fifth simulated data set associated with experiments i = 105, N ∈ {3P 6} andy ∈ {3P 6}. One epoch represents either one full E step, i iterations within the M step, orone gradient step for full-gradient algorithms. The log-likelihood is denoted as ℓ(ϕ), andthe maximum log-likelihood across all algorithms and initial values after 12 hours is denotedas ℓ(ϕ∗). The y-axis is on a log-scale. Dots correspond to the point of termination for eachalgorithm.218C.4. Additional Results from Chapter 4 Simulation StudyFigure C.23: Optimally gap between the current log-likelihood and optimal log-likelihoodfor the fifth simulated data set associated with experiments i = 105, N ∈ {3P 6} andy ∈ {3P 6}. The log-likelihood is denoted as ℓ(ϕ), and the maximum log-likelihood across allalgorithms and initial values after 12 hours is denoted as ℓ(ϕ∗). The y-axis is on a log-scale.Dots correspond to the point of termination for each algorithm.219C.5. Additional Results from Chapter 4 Case StudyC.5 Additional Results from Chapter 4 Case StudyC.5.1 Expanded Form of Transition Probability MatricesThe transition matrix at time t, Γt, depended on Zt−). The main text represented Γt ineither block-diagonal form or as a Kronecker product. The full, expanded representation ofthe Γt isΓt =Γ(f,))),) Γ(f,))),2 Γ(f,))),+ 0 0 0 0 0 00 Γ(f,))2,2 Γ(f,))2,+ 0 0 0 0 0 00 0 1 0 0 0 0 0 00 0 0 Γ(f,2)),) Γ(f,2)),2 Γ(f,2)),+ 0 0 00 0 0 0 Γ(f,2)2,2 Γ(f,2)2,+ 0 0 00 0 0 0 0 1 0 0 00 0 0 0 0 0 Γ(f,+)),) Γ(f,+)),2 Γ(f,+)),+0 0 0 0 0 0 0 Γ(f,+)2,2 Γ(f,+)2,+0 0 0 0 0 0 0 0 1for Zt−) = 0PΓt =Γ(x)),) 0 0 Γ(x)),2 0 0 Γ(x)),+ 0 0Γ(x)),) 0 0 Γ(x)),2 0 0 Γ(x)),+ 0 0Γ(x)),) 0 0 Γ(x)),2 0 0 Γ(x)),+ 0 0Γ(x)2,) 0 0 Γ(x)2,2 0 0 Γ(x)2,+ 0 0Γ(x)2,) 0 0 Γ(x)2,2 0 0 Γ(x)2,+ 0 0Γ(x)2,) 0 0 Γ(x)2,2 0 0 Γ(x)2,+ 0 0Γ(x)+,) 0 0 Γ(x)+,2 0 0 Γ(x)+,+ 0 0Γ(x)+,) 0 0 Γ(x)+,2 0 0 Γ(x)+,+ 0 0Γ(x)+,) 0 0 Γ(x)+,2 0 0 Γ(x)+,+ 0 0for Zt−) = 1.C.5.2 Parameter EstimatesThe parameters presented here come from a run of EM-VRSO with A = SAGA, P = True,and b = 10i . They represent the parameters with the highest log-likelihood among allrandom initializations and optimization algorithms in the case study from the main text.220C.5. Additional Results from Chapter 4 Case StudyTable C.2: Maximum likelihood parameter estimates from the killer whale case study de-scribed in the main text. Recall that Yt represents the change in depth in meters at timeindex t and Zt ∈ {0P 1} encodes whether a dive ends at time index t. The parameter µ cor-responds to the state-dependent mean of Yt, σ corresponds to the state-dependent standarddeviation of Yt, and p corresponds to the state-dependent probability that Zt = 1.Dive Type Parameter Estimate Descent Bottom Ascent1µˆ 0.70 0.02 -0.67σˆ 0.45 0.23 0.40pˆ 0 0 0.122µˆ 0.33 0.00 -0.44σˆ 0.15 0.15 0.21pˆ 0 0 0.473µˆ 2.71 0.01 -2.50σˆ 1.85 0.64 1.58pˆ 0 0 0.04Figure C.24 shows the estimated densities of change in depth Yt for all dive types and divedurations.Figure C.24: State-dependent density of change in depth for dive types 1, 2, and 3 (left toright). Fine-scale state 1 corresponds to descent, fine-scale state 2 corresponds to bottom,and fine-scale state 3 corresponds to ascent. Dive type 1 corresponds to medium-depthdives, dive type 2 corresponds to shallow dives, and dive type 3 corresponds to deep dives.The coarse-scale, inter-dive transition probability matrix and initial distribution estimateswere221C.5. Additional Results from Chapter 4 Case StudyΓˆ(x)=0.223 0.758 0.0170.123 0.853 0.0270.074 0.857 0.068 P (C.18)δˆ(x)=(0.258 0.002 0.740). (C.19)The long-run proportion of each dive type is found by solving for the stationary distributionof Γˆ(x), i.e. solving the equation πˆ(x)Γˆ(x) = πˆ(x). Doing so gives the following:πˆ(x) =(0.136 0.840 0.024). (C.20)In words, approximately 13.6% of all dives are of type 1 (medium-depth), 84.0% of all divesare of type 2 (shallow), and 2.4% of all dives are of type 3 (deep).The fine-scale, intra-dive transition probability matrices and initial distributions were esti-mated asΓˆ(f,))=0.871 0.123 0.0060 0.962 0.0380 0 1 P Γˆ(f,2) =0.668 0.304 0.0280 0.788 0.2120 0 1 P (C.21)Γˆ(f,+)=0.958 0.040 0.0020 0.982 0.0180 0 1 P (C.22)δˆ(f,))=(1 0 0)P δˆ(f,2)=(1 0 0)P δˆ(f,+)=(1 0 0)(C.23)Recall that each dive must begin with descent, so all initial distributions are identical. Inaddition, note that the Markov chains defined by Γˆ(f,i) above are not ergodic, so stationarydistributions do not exist for the fine-scale, intra-dive Markov chains.222C.5. Additional Results from Chapter 4 Case StudyC.5.3 Decoded Dive ProfilesThe figures below show approximately one hour of dive profiles associated with each of theeight whales used to train the hierarchical HMM in the case study.Figure C.25: Dive profiles and change in depth vs time for a selected hour of dive time forkiller whale A100. All data is color-coded by most likely dive type (rows one and three) aswell as most likely dive phase (rows two and four).223C.5. Additional Results from Chapter 4 Case StudyFigure C.26: Dive profiles and change in depth vs time for a selected hour of dive time forkiller whale A113. All data is color-coded by most likely dive type (rows one and three) aswell as most likely dive phase (rows two and four).224C.5. Additional Results from Chapter 4 Case StudyFigure C.27: Dive profiles and change in depth vs time for a selected hour of dive time forkiller whale D21. All data is color-coded by most likely dive type (rows one and three) aswell as most likely dive phase (rows two and four).225C.5. Additional Results from Chapter 4 Case StudyFigure C.28: Dive profiles and change in depth vs time for a selected hour of dive time forkiller whale D26. All data is color-coded by most likely dive type (rows one and three) aswell as most likely dive phase (rows two and four).226C.5. Additional Results from Chapter 4 Case StudyFigure C.29: Dive profiles and change in depth vs time for a selected hour of dive time forkiller whale I107. All data is color-coded by most likely dive type (rows one and three) aswell as most likely dive phase (rows two and four).227C.5. Additional Results from Chapter 4 Case StudyFigure C.30: Dive profiles and change in depth vs time for a selected hour of dive time forkiller whale I145. All data is color-coded by most likely dive type (rows one and three) aswell as most likely dive phase (rows two and four).228C.5. Additional Results from Chapter 4 Case StudyFigure C.31: Dive profiles and change in depth vs time for a selected hour of dive time forkiller whale R48. All data is color-coded by most likely dive type (rows one and three) aswell as most likely dive phase (rows two and four).229C.5. Additional Results from Chapter 4 Case StudyFigure C.32: Dive profiles and change in depth vs time for a selected hour of dive time forkiller whale R58. All data is color-coded by most likely dive type (rows one and three) aswell as most likely dive phase (rows two and four).230
Thesis/Dissertation
eng
Statistics
Vancouver : University of British Columbia Library
Attribution-NonCommercial-NoDerivatives 4.0 International
Graduate
Heckman, Nancy
Auger-Méthé, Marie
Modelling complex biologging data with hidden Markov models
Text