ChatGPT is taking conversations beyond the chat box

OpenAI is giving ChatGPT a more interactive makeover with GPT-6, allowing the chatbot to generate visual interfaces, tools and interactive experiences directly within conversations.

At the centre of the rollout is "Intelligent UI", a new GPT-6 capability that lets ChatGPT decide how best to present an answer using a combination of text, visuals and interactive elements. Instead of relying on a conventional text response, answers can now include graphics, tappable buttons, forms, charts and other elements users can interact with on the spot.

The format changes depending on what a user is trying to do. A comparison, for example, could appear side by side, while a complex explanation could be presented as an interactive diagram that lets users explore different components or change inputs to see how the outcome changes.

Don’t miss: OpenAI wants ChatGPT ads to start conversations, not just clicks

Everyday planning tasks can also become more visual. OpenAI demonstrated how ChatGPT could turn a Sunday roast plan into an adjustable experience where users change the number of guests and see ingredient quantities update accordingly, alongside a cooking timeline and checklist.

Similarly, a road trip could be presented with stops plotted on a map and recommendations for potential detours, while wardrobe planning could incorporate visual elements alongside suggestions.

ChatGPT can also build simple interactive tools directly within a conversation when asked. These could include a bill splitter for a dinner with friends, a retirement savings calculator or even a playable game, without users needing to leave the chat.

The capability is designed to make more complicated subjects easier to explore too. Rather than only describing a concept through paragraphs of text, GPT-6 can create visual and interactive explanations that allow users to work through a topic step by step.

Behind the feature is a library of native, streamable components that GPT-6 can use to assemble an interface according to the question being asked. OpenAI has also developed a compiler that processes the interface as the model generates it, allowing elements to progressively appear rather than making users wait for the entire response.

The model has been trained to make decisions around content, layout, visuals and interaction, including how information should be organised and when interactivity would actually be useful. When a straightforward text answer makes more sense, ChatGPT can still stick to text.

According to OpenAI, Intelligent UI represents a shift away from giving every question essentially the same conversational treatment. Instead, the company wants ChatGPT to increasingly shape its interface around what a user is trying to accomplish.

The feature arrives as OpenAI expands GPT-6 beyond the paid customers that received the first models last month. GPT-6 with Intelligent UI is rolling out globally to ChatGPT Plus, Pro, Business and Enterprise users in the Chat tab, before expanding to Free and Go users from 8 October.

OpenAI said its longer-term ambition is for users to be able to describe what they want to accomplish and have ChatGPT build an experience around that task, with Intelligent UI marking an early step towards interfaces that are generated according to users' needs.

The rollout also comes as OpenAI increasingly experiments with how visual experiences can extend beyond ChatGPT's core responses and into its advertising business. Recently, OpenAI said it is adding a visual ad format to ChatGPT Ads, alongside expanded measurement tools and brand suitability pilots.

The format will first be tested during image generation, using images to showcase product inspiration, usage or experiences associated with a product. The ads will be clearly labelled and kept separate from generated images, with OpenAI reiterating that advertising does not influence ChatGPT's answers.

Related articles:
OpenAI gives teens their own ChatGPT experience
OpenAI, Anthropic and Google reportedly explore joint AI standards body
‘Run as fast as you can’: Nvidia boss rejects the idea AI safety means slowing down