Machine learning model research requires running expensive, long-running experiments where even a slight mis-calibration can cost millions of dollars in underutilized compute resources. Once trained, model deployment, production monitoring, and observability requirements all present unique operational challenges. Chris Van Pelt is the Chief Information Officer of Weights and Biases, which is the industry standard in