Kayvon Fatahalian - From Ideas to Video Analysis Models in Hours, Not Weeks

My students and I often find ourselves as "subject matter experts" needing to create video understanding models that serve computer graphics and video analysis applications.  Unfortunately, like many, we are frustrated by how a smart grad student, armed with a large *unlabeled* video collection, a palette of pre-trained models, and an idea of what novel object or activity they want to detect/segment/classify, requires days-to-weeks to create and validate a model for their task.  In this talk I will discuss challenges we've faced in the iterative process of curating data, training models, and validating models for the specific case of rare events and categories in image and video collections.  In this regime we've found that conventional wisdom about training on imbalance data sets, and data acquisition via active learning does not lead to the most efficient solutions.  I'll discuss these challenges in the context of image and video analysis applications, and elaborate on our ongoing vision of how a grad student, armed with massive amounts of unlabeled video data, pretrained models, and available-in-seconds-supercomputing-scale elastic compute should be able to interactively iterate on cycles of acquiring training data, training models, and validating models.