Will Facebook’s Gesture Recognition Tool Make the Cut in AR VR Market?

By Sudipto Ghosh On Oct 1, 2019

Virtual tools would rule the future. For all those who are high on sci-fi intersecting with the world of commercial gaming, here’s a big announcement for you. Facebook is bringing alive its Virtual Reality-focused Gesture Recognition tool. Last week from Oculus Connect conference, the leading technology enabler confirmed building the future of Hand Gesture simulation. This includes mixing powerful computing, AI and VR techniques into their Oculus device. Oculus Link will now bring a truly VR-gaming experience to Quest and a gaming PC from the Rift Platform – a direct competition to XBOX and Play Station.

Google’s Project Soli is the closest competitor in this advanced VR-integrated hand gesture technology, at least theoretically.

The announcement comes within a week of Facebook acquiring wearable band maker, CTRL-labs for a speculated $1 Billion.

Let’s take a quick look into how Facebook’s new Gesture Recognition Tool makes an impact in the AR VR market.

But First, What’s Gesture Recognition?

Gesture Recognition is an advanced computing simulation technique for replicating human-hand gestures. It involves a play of live motion sensor for Gesture Recognition, Computer Vision algorithms, and VR output mechanism.

*image courtesy- Oculus for Business official blog*

How Does It Work?

Gesture Simulation projects are mostly built from gesture identification, IoT and ML codes. A webcam detects gestures made by the human to perform basic live motion operations simulating a particular gesture. For example, a handshake, high-fives, or in advanced mechanics, pulling out depleted radioactive fuel-rods from the nuclear reactors.

Adding VR component in the whole mix of Gesture Recognition technologies enable users an unmatched experience that allows them “to connect to the people, places, and opportunities that matter most.”

How is Facebook’s Gesture Recognition Project Different?

In May 2019, Facebook launched the all-in-one VR device- Oculus Quest. Oculus Quest, created by the VR department of Facebook boasts of being one of the most powerful VR-gaming headsets ever launched for commercial gamers.

Facebook’s VR labs would be releasing a fully-compatible software in beta this November, and it will work with most high-quality USB-C cables. Later this year, it will also release a premium cable with maximum throughput to run Rift content and a longer cord so you can move easily in VR.

In an official blog, Facebook announced that the company has identified a sustainable marketplace for its newly launched product. It stated –

In order for VR to truly transform the way we live, work and connect with each other, we need to build a sustainable ecosystem — from action-packed games and artistic experiences to cutting-edge training apps and beyond. We announced that people have spent $100M USD on the Oculus Store and 20% of that is from Quest alone, which is a testament to the health of the ecosystem, as well as the passion and commitment of the developers and content creators designing and building for VR today.

In 2016, Oculus released their advanced VR sensors ‘Touch controllers, ushering into a new era of gesture recognition revolution. Oculus Touch mimics the neuroscience mechanism to feel the sensation that your own hands are actually there with you in a virtual environment. The latest announcement of hand tracking on Oculus Quest, enabling natural interaction in VR using your own hands is actually part of Oculus Touch neuroscience that renders external hardware obsolete.

In addition, Facebook Horizon makes an entry into an “ever-expanding VR world” with amazing software experiences. Facebook Horizon is a new social experience in VR where you can build your own worlds with easy-to-use tools (no coding skills required).

Oculus for Business

In November, Oculus for the Business platform will be available for VR deployment platforms and developer communities. It would enable VR adoption across a variety of industries, transforming the workplace. That’s where Oculus Quest and Rift are expected to play a huge part in gamification of commercial operations and human simulation market. It is expected to allow room for a large-scale broad-spectrum VR deployment while expanding market opportunities for a growing community of VR developers.

These would be fully compatible with current enterprise Cloud-based management tools with easy Device Setup app and Device Manager control center for better management of Oculus devices across various departments and locations. Oculus for Business would also support provisioning services, like MobileIron and VMWare Airwatch.

Oculus for Developers: Cutting-Edge Tool Provisioning

Keeping a view of increasing open-source dependence and technology enablement in the VR market, Facebook is adding new features to the Developer Tools gallery. Here is a quick overview of Oculus Quest developer tools that were updated recently:

RenderDoc Updates

The open-source frame debugger would now support all of our extensions and uses less RAM for smoother operations on Quest. Performance Data reporting is also added.

OVRMetricsTool

New enhancements to OVRMetricsTool expected in the next few months. These would enable direct analysis of VR headset run-time, performance and other critical information to improve the experience across devices and locations.

Vulkan News

Oculus has added support for Vulkan Multiview and Fixed Foveated Rendering and in Unreal Engine. The idea is to boost performance significantly, in addition, to further expanding the current Vulkan support on Quest. It will allow for quicker debugging.

Unity GPU

Unity’s GPU Profiler on Quest and Go enabled to give you more detailed information to increase accuracy in pinpointing and resolving performance issues.

New Purchase Funnel Analytics, Coming This Fall

In Fall, metrics for Quest in the Developer Analytics tool, expected to arrive. This would help understand how your app is performing in the store. Other metrics visible to customers include data related to browsing, searching, and engagement with your product detail page.

Mixed Reality Capture for Quest

With support from Unity and Unreal Integrations, Oculus is also native support to make it just as easy for content creators to produce mixed-reality videos. The most exciting part of the announcement comes in relation to Oculus Link and availability of Remote Rendering for developers.

The blog informed –

“When Oculus Link software ships, developers will be able to iterate more quickly by running on a PC, while streaming the output to Quest. Just hit the play button for the Unity or Unreal Oculus plugin, and you’ll be able to preview right on the device—no need to compile and transfer a ew APK first. It’s development at PC speed, for a mobile device.”

Beating Vr Fatigue: Will Oculus Quest Put an End to It All

With multiple challenges and side-effects attributed to the use of hand gesture control sensors and VR wearable headsets, things were beginning to a little weary in the market for retailers and technology providers. However, now, Facebook’s VR department has brought a new lease of life in the AR VR and Mixed Reality scenario keeping in mind the social and gamification behaviors of users of all age-group.

We expect a new wave of VR makers coming to the party and enable customers with neuro-science enablers and gesture control tools in the coming months. 2020-2030 era would be all about IoT and Machine-based human simulation devices.