MSEE and VTT Presentation

Mathematics of Sensing, Exploitation and Execution (MSEE) is a DARPA program that started in 2011. The project aims to develop a unified mathematical foundation of representation, inference and learning for sensing, exploitation and execution. The presentation was given by IAI Program Manager, Dr. Mun Wei Lee. He discussed the development of an end-to-end system for understanding scenes and events from multi-modal inputs. The system computes 3D scene-based probabilistic spatial-temporal-causal parse graph representation for the activities of humans and vehicles; and answers user queries in storylines about “who and what, where, when, and how.” The MSEE program evaluated the end-to-end system through a Visual Turing Test (VTT) by a third method, asking 1000+ queries on 30+ hours of recorded videos.  The presentation gave an overview of the project and presented its findings for future work.