The Artificial Intelligence system observes the world through live video and automatically interprets the unfolding visual scene

Deep Learning pioneer Twenty Billion Neurons (TwentyBN) debuts an AI technology that is the world’s first to show an awareness of its environment and of the actions occurring within it. The system observes the world through live video and automatically interprets the unfolding visual scene. The interpretation provides a comprehensive understanding of actions and human behaviors, as well as of concepts relating to the physical world that are beyond what still-image analysis can reveal.

At REWORK Montreal, a gathering of AI researchers and practitioners, the company will give a public demonstration of its technology that takes in a real-time camera stream and describes the actions and behaviors it observes. This recent breakthrough is based on a completely new approach to video understanding. It leverages deep neural networks, a form of machine learning, applied to a vast amount of video data recorded specifically for the purpose of teaching machines “common sense” about the physical world.

With the proliferation of connected camera devices, there is an urgent need for better ways to analyze video. The vast majority of commercial use cases in video understanding are out of reach for current technological capabilities. “Machines can identify objects in images, but they are unable to fully decipher the most important aspect: what’s really going on in front of you. Our technology lays the foundation for use cases that require an understanding of the context and actions that occur in the world,” CEO and scientific lead, TwentyBN, Roland Memisevic said.

“Roland and his team are definitely onto something. Their approach of using video to teach neural networks common sense about the world, by working their way up from visual primitives towards more complex concepts, could have a very strong impact,” Yoshua Bengio, professor at the Universite de Montréal and advisor, TwentyBN said.

TwentyBN is working with clients to deploy enterprise-grade systems in a range of industries, including smart homes, automotive, retail, robotics and healthcare. By providing situational awareness through vision, the company’s novel technology fills a significant gap in the market for intelligent systems.