AI just came to VR in a big way

By Alan Truly Published July 23, 2024

Meta just announced a huge update to its AI and it’s coming soon to the Meta Quest platform, home to some of the best VR headsets you can buy. In the next few weeks, you’ll soon have access to a smart voice assistant anytime you wear your Quest 3, Quest Pro, or Quest 2.

You can see some examples of how Meta AI works on the Quest 3 in the video below. Note that Meta AI can use the Quest 3’s and Quest Pro’s mixed reality mode to see real-world objects and answer your questions. The Quest 2’s black-and-white passthrough camera isn’t supported for visual input.

Meta Quest VR headsets in the U.S. and Canada will start getting these new multimodal AI capabilities as early as August. It will be a gradual rollout, so you might not see the update until September.

I asked Meta about generating AI images in Quest headsets and whether the AI would be able to see my Meta avatar and virtual environment. Those features might come at a later date, but won’t arrive in the next update.

Meta AI has been in Facebook, Messenger, and WhatsApp since September 2023, helping you with customized AIs for cooking, jokes, and games. It can even generate images when you use the @MetaAI /imagine command while chatting with a friend.

Meta AI also supercharges Ray-Ban Meta Smart Glasses, giving you cutting-edge AI assistance wherever you go. The upcoming Meta Quest update will bring a version of AI that’s similar to what’s available in Meta smart glasses. You can ask general questions like what to wear in the typically hot weather of Palm Springs and even hold up a pair of shorts and ask Meta’s AI to suggest a matching top.

This announcement came at the same time Meta announced its 405B parameter Llama 3.1, the largest AI ever released as open source. Meta also posted AI benchmarks that indicate Llama 3.1 is competitive with the latest OpenAI and Anthropic models.

You can try Meta AI 3.1 right now at meta.ai and on WhatsApp in the U.S. Keep your Meta Quest VR headset plugged in and in sleep mode to get the update as soon as its available in your area.