OpenAI just held an event a few hours back and made some interesting announcements, including its latest flagship AI model called, GPT-4o. Aside from the newer capabilities of the GPT-4o, the event also saw other new reveals, such as a ChatGPT App for MacOS and a Windows app that’s also in the works.
Welcome GPT-4o, The Latest From OpenAI
One of the most significant announcements at the OpenAI event was GPT-4o, an iterative upgrade to GPT-4. It will power ChatGPT going forward, with updated capabilities such as its much faster response rate and the improvements to capabilities across text, vision, and audio.
GPT-4o is natively multimodal, meaning it can support input and output through various types of content, such as text, audio, or images. The audio part, which is the voice mode, is getting a significant upgrade where GPT-4o will be able to respond in real-time and will also have the ability to observe the world around you through the camera. The current voice mode, in comparison, can respond to only one prompt at a time and works with only what it can hear, and not what it can see.
GPT-4o can respond to audio inputs in as little as 232 milliseconds, with an average of 320 milliseconds, which is similar to human response time in a conversation, as per OpenAI. It matches GPT-4 Turbo performance on text in English and code, with significant improvement on text in non-English languages. It is also much faster and 50% cheaper in the API. GPT-4o is especially better at vision and audio understanding compared to existing models.
“The new voice (and video) mode is the best computer interface I’ve ever used. It feels like AI from the movies, and it’s still a bit surprising to me that it’s real. Getting to human-level response times and expressiveness turns out to be a big change. The original ChatGPT showed a hint of what was possible with language interfaces; this new thing feels viscerally different. It is fast, smart, fun, natural, and helpful”, said OpenAI CEO Sam Altman in a blog.
GPT-4o will be free for all users, and paid users will continue to have up to five times the capacity limits of free users. GPT-4o’s capabilities will be rolled out iteratively (with extended red team access starting today). Its text and image capabilities are starting to roll out today in ChatGPT. A new version of Voice Mode with GPT-4o will be rolled out in alpha within ChatGPT Plus in the coming weeks.
Read More: OpenAI Expands Media Partnerships, Partners With Dotdash Meredith
New ChatGPT Features For Free Users (That Were Paid Earlier)
Free users now have access to these new features that were earlier limited to paid users:
- GPT-4 level intelligence
- Get responses from both the model and the web
- Analyze data and create charts
- Chat about photos you take
- Upload files for assistance summarizing, writing or analyzing
- Discover and use GPTs and the GPT Store
- Build a more helpful experience with Memory
A New Mac App (and an upcoming Windows one)
For both free and paid users, OpenAI is also launching a new ChatGPT desktop app for macOS With a simple keyboard shortcut (Option + Space), you can instantly ask ChatGPT a question. You can also take and discuss screenshots directly in the app. It will also support the Voice Mode that has been available in ChatGPT at launch, with GPT-4o’s new audio and video capabilities coming in the future. The macOS app will first be made available to Plus users starting today, and will be made more broadly available in the coming weeks. A Windows app is also in development and should be released later in the year.
A New Look For ChatGPT
OpenAI is also introducing a new look and feel for ChatGPT that’s designed to be “friendlier and more conversational”. Users will begin to notice a new home screen, message layout and more.