In a major upgrade to ChatGPT’s Advanced Voice Mode, OpenAI now introduces vision capabilities. With this, ChatGPT can now analyze and respond to live video input and share screens during conversations. Although the feature was introduced with GPT-4o, only the audio feature was live until now. However, now users can chat with ChatGPT using their smartphone camera; Essentially, a chatbot can see what you see.
With this update, ChatGPT can understand and discuss visual context in real time. The feature works through a new video icon in the mobile app, where screen sharing is available through a separate menu option. The new updates are available for ChatGPT Plus, Pro, and Team members, with Enterprise and Edu users getting access in January. The AI powerhouse has also introduced a festive voice option, enabling users to chat with Santa, a limited-time seasonal addition that will essentially begin in early January.
OpenAI CPO Kevin Weil, along with other team members at the AI startup, showed a demo of ChatGPT that helps on how to make pour-over coffee. During the demo, members pointed the camera at the action, and the bot showed that it understood the principle of the coffee maker and explained brewing to the team. The team also showed how ChatGPT now supports screen sharing by understanding an open message on a smartphone where it recognized Weil wearing a Santa beard.
The latest update comes a day after Google unveiled its latest Gemini 2.0 model, capable of processing visual and audio inputs. Gemini 2.0 is capable of multi-stage operations through three research projects, such as Project Astra (Universal Assistant), Project Mariner, and Project Jules, which are essentially developer tools. On the other hand, OpenAI’s latest demo shows ChatGPT’s vision modality, which so far is capable of recognizing objects and seamless interactions.
Why should you buy our membership?
You want to be the smartest in the room.
You want access to our award-winning journalism.
You don’t want to be confused and misinformed.
Choose your subscription package