Open Collections

Abstract

In serverless computing, a developer deploys their code without the burden of infrastructure provisioning. The cloud provider scales compute units dynamically to ensure the demand for the deployed application is met. A customer can then execute the deployed code. The latency incurred while a request waits to be assigned the necessary resources before execution is called a cold start. The cloud provider initializes resources before or beyond code execution to avert cold starts, leading to resource wastage. Lifetime management policies seek to balance cold starts with efficient resource management. To make these policies adaptive, we need a mechanism to predict the exact time when the policy needs to kick in. A forecaster makes predictions to create adaptive lifetime management policies. Forecasting involves utilizing historical application traffic and predicting future traffic. This ensures lifetime management policies kick in based on predicted application traffic. However, there is no consensus on which forecaster is the best regarding cold start aversion and resource efficiency. Moreover, no single forecaster is customized per application nor adapts to traffic pattern changes. To ensure customer satisfaction and cloud provider savings, it is important to maintain a low forecasting overhead and high accuracy. We propose a Markov Chain forecaster that combines features from serverless application data to improve accuracy. Features are statistical properties that enable forecasting. We use lightweight features to improve the accuracy of our forecaster. We further create an ensemble Markov Chain forecaster with a subset of features and explore a tradeoff between accuracy and overhead. We create multiple ensemble method forecasters using a lifetime management framework called FeMux, which multiplexes between different Markov Chains based on statistical property changes in the application traffic. Our forecaster's prediction accuracy is measured using simulation based on Azure serverless traces and a custom metric based on cold start time and memory wastage called Representative Unified Metric (RUM). We prototype our forecaster on Knative, a Kubernetes-based serverless platform, and reduce RUM by 45% compared to vanilla Knative, translating to a 37% cold start reduction with less than 2% additional memory. Our forecaster reduces RUM by 30% compared to prior work.