By Ritendra Datta and Karthik Gopalratnam, Members of Technical Staff
Share How should ads work in agentic chat? How should ads work in agentic chat? Conversation memory, without influencing ChatGPT answers An LLM interprets conversations and calls tools How the orchestrator calls ad tools in an agentic loop Preparing retrieval queries that capture intent Retrieving possible ads Filtering ineligible candidates Evaluating relevance and quality Running the ads auction Final eligibility checks The final step: showing the ads to the user What happens after the ad is shown High-power models evaluate and improve the entire stack Human feedback steers tasteful direction What's next How should ads work in agentic chat? Conversation memory, without influencing ChatGPT answers An LLM interprets conversations and calls tools How the orchestrator calls ad tools in an agentic loop Preparing retrieval queries that capture intent Retrieving possible ads Filtering ineligible candidates Evaluating relevance and quality Running the ads auction Final eligibility checks The final step: showing the ads to the user What happens after the ad is shown High-power models evaluate and improve the entire stack Human feedback steers tasteful direction What's next Traditional advertising systems are built to respond to discrete units of user intent. A user searches for or views something online, the ads system receives those queries and page views as events, and the system looks for matching ads based on those events. Building an ads stack for ChatGPT was different: user intent develops across conversations and shifts over multiple turns of a conversation, sometimes without a fitting advertising moment. To support this new paradigm, we built a bespoke ads delivery system. This new type of ad stack is based on a custom, post-trained frontier LLM that makes tool calls for retrieval (identifying the top options), ranking (predicting user engagement), relevance (evaluating the usefulness of the ad), and running auctions for final ad selection and pricing
Our ads system supports the nuance of evolving user goals by making three key design decisions: 1) implementing stateful ads sessions, meaning information is remembered across turns of a conversation and across multiple conversations, 2) relying on a custom, post-trained LLM that uses a tool call architecture to interpret a conversation's context and underlying intent, and then effectively select, rank, and deliver ads, and 3) running the ads selection process separately from ChatGPT's answer generation, in its own service, to keep ads from influencing answers and maintain ChatGPT's neutral voice. We use our post-trained LLM to orchestrate the entire ads flow, determining the right ad to surface at the right time based on both the user's current turn and conversation history. While bootstrapping a self-improving ads system like this used to take years, ChatGPT Ads reached $1 billion in annualized revenue run rate less than 200 days after launch ; now, more than 100,000 advertisers have run ads on ChatGPT since launch
