The Science of Scaling · 16 Aug 2023
Why OpenAI Had to Un-do Sales w/ Aliisa Rosenthal (Head of Sales, OpenAI)
These are notes on the conversation, checked against its transcript. The episode itself has the full discussion.
In brief
Aliisa Rosenthal, Head of Sales at OpenAI, joins host Mark Roberge to describe joining OpenAI in early 2022, when it was a small research lab with a two-person sales team that was selling its own time through innovation licenses. She covers how she unwound that model, found repeatable API use cases through customer interviews, and introduced a committed-consumption commercial model, then how the ChatGPT launch flooded inbound and changed how she qualifies customers. The central point is that a go-to-market leader for a product the market does not yet understand should stay close to product and customers, and qualify buyers by whether they will succeed with the product.
For founders
- OpenAI set a $100K committed-consumption minimum for API customers, which Rosenthal said qualified serious buyers, though she said the minimum became negligible once ChatGPT created demand.
- Early on, OpenAI sold consulting-style innovation licenses, which Mark Roberge noted got the company paid to run customer interviews while it worked out what to build.
- Rosenthal said OpenAI tested candidate use cases against three criteria: actual API consumption, feasibility, and a real return on investment for the customer.
- Mark Roberge argued, drawing on Rosenthal's practice of declining customers without a consumption plan, that qualification should rest on whether a customer will succeed with the product, not on how much money they have or how easily they close.
- ChatGPT launched as a low-key research preview with no emails, no marketing team and a buried call to action in a blog post, yet it spread quickly and overwhelmed the company.
For revenue leaders
- Rosenthal said she makes reps show a forecast model for how a customer will reach its committed spend, and she declines to move forward when the answer is that it is not yet known.
- Rosenthal said each customer on the committed-consumption model was weighed against the cost of contract review, security questionnaires and DPAs, which she called a big deal for the company.
- Under the committed-consumption model, Rosenthal said a rep could handle 10 to 25 accounts because each customer no longer needed a full-time team member.
- Rosenthal said she spends at least one meeting a day in product stand-ups and that anyone on her team could join most company meetings and understand what was going on.
- Rosenthal said her candidate interview asked applicants to build something with the API, and that the process also asks what the API's limitations are and what would be a hard use case right now.
What was said 22, most useful first
A $100K committed-consumption minimum acted as a qualifier that showed which API customers were serious. Listen
Rosenthal said OpenAI moved its innovation-license model to committed consumption, with a $100K minimum, when the company had about 200 people and a go-to-market team of roughly 15 including support. She then built spreadsheets forecasting token use by use case, volume and model, which she called not a perfect science but a forcing mechanism for conversations about pricing and volumes. The minimum was also used to judge whether a customer was worth the security, legal and compliance work that each enterprise customer required.
“The reason I liked it is because, first of all, it provided some sort of qualifier. It said, okay, if you're willing to commit to 100K in consumption with us, then you must be serious.”
Listen to the episode Pricing & packaging Link to this Report a problem
Committed consumption stopped filtering buyers once ChatGPT created demand, because customers signed the minimum without a plan to use it. Listen
Rosenthal said that after ChatGPT, many organisations came to OpenAI saying they had been told to work with it, sometimes because they needed the name in a press release or for a board meeting. Those buyers agreed to the $100K minimum without a real plan to consume it, so the minimum became negligible. She said she did not yet have a fix and called the model far from perfect.
“It worked up until the until Chachi BT blew everything up where companies were just wanting to work with us so much that the 100k was negligible and it was worth just saying sure sure sure will commit to 100k without any real plan to consume it.”
Listen to the episode Pricing & packaging Link to this Report a problem
Reps had to show a forecast of how a customer would reach its commitment, and if the answer was unknown, OpenAI did not move forward with that customer. Listen
Rosenthal said that when reps brought in customers willing to sign for 250K or 500K, she required them to show the forecast model and map how the customer would reach that figure. If the answer was that the use cases were not yet defined, she said the company was not ready to move forward with that customer. She described this as the current process at OpenAI, after the ChatGPT surge.
“I make them show me the forecast model. How are we going to get to that 100K? And let's map it out so I know it's real. And if the answer is, well, we don't know yet, we don't really have our use cases defined. Then I say, okay, we're not ready and we're not ready to move forward with this customer.”
Listen to the episode Sales process & deals Link to this Report a problem
Customer selection should follow whether a customer will succeed with the product, with ICP and MQL aligned to LTV rather than minimal CAC. Listen
Mark Roberge said that some customers were handing over money and still said no at OpenAI, despite high demand. He argued that the ideal customer profile and marketing qualified lead should not be based on how much money a business has or how easily it closes. He said they should be aligned to lifetime value rather than minimal customer acquisition cost.
“The types of businesses that we want to pursue should not be based on how much money they have and how easy they are to close. It should be based on whether they're gonna be successful with your business, with your product.”
Listen to the episode Pipeline & demand generation Link to this Report a problem
When she joined, OpenAI had sold more innovation licenses than it could deliver, so she could not sell anything else. Listen
Rosenthal said the innovation license was essentially selling OpenAI's time, with a weekly meeting about AI and help implementing it, and that it was not scalable. When she joined, she was told the company had oversold itself and was out of service capacity, so she could not sell anything else. Mark Roberge described this as her first duty being to undo sales. She spent her first months working out how to go to market with what the company had rather than making more sales calls.
“We've sold too many innovation licenses. What do we do?”
Listen to the episode Strategy & market Link to this Report a problem
OpenAI judged prospective customers by consumption of the API, low handholding and feedback that pushed its mission forward. Listen
Rosenthal said that when she changed the business model, the goals were first actual consumption, with customers using the API. The second was to achieve this with as little handholding as possible, because the company was thinly stretched on resources. The third was feedback or something that pushed the mission forward, and the company used these criteria to decide which customers to pursue.
“our goal was A, to get actual consumption, to get customers using our API. Ideally to do it with as little handholding as possible because we were so thinly stretched on resources.”
Listen to the episode Strategy & market Link to this Report a problem
She interviewed every customer-facing person, grouped customers by use case, and tested each use case for consumption, feasibility and customer ROI. Listen
Rosenthal said that to find the right use cases, she interviewed everyone in customer-facing roles, including support, the solutions team that delivered innovation licenses, and the two sales reps. She organised customers by use case and then interviewed the customers to test three things: whether a use case yielded actual API consumption, whether it was feasible and produced useful output, and whether it would generate a real ROI for the customer. She said the ROI test was the hardest and was still a challenge.
“What were the use cases that were a, yielding actual consumption of our API? B, were feasible? They worked. You could implement them and the output was useful. And then C, and this is the hardest and still a challenge today, but would generate a real ROI to the customer.”
Listen to the episode Strategy & market Link to this Report a problem
OpenAI's sales candidates were asked to build something with the API, so they could experience sending a call and getting a completion back. Listen
Rosenthal said that when she interviewed at OpenAI, she had to build an app with the API in Python as a final project, and that she kept this part of the candidate process until recently. She wanted candidates to experience sending an API call to GPT and getting a completion back, which she said is mind blowing. She said candidates could get help, but she wanted them to explain what was happening and the limits of the API.
“I want my candidates to experience of sending an API call to GPT and getting a completion back because it is, it's just mind blowing.”
Listen to the episode Hiring & team building Link to this Report a problem
Under the committed-consumption model, a rep could handle 10 to 25 accounts because each customer no longer needed a full-time team member. Listen
Rosenthal said that once customers were signed to a $100K committed-consumption plan, OpenAI could provide enough resources to make them successful. She said the company did not need to assign a full-time team member whose only job was making that customer successful. She said the company could start to handle 10, 15, 20 or 25 accounts per rep.
“I don't need to sign a full-time team member. This is their only job as making this customer successful. And we can start to handle 10, 15, 20, 25 accounts per rep.”
Listen to the episode Sales team, hiring & comp Link to this Report a problem
Inbound went from about 30 companies a week before ChatGPT to 100, 200, 300 and then 1,000 a day after launch, with two reps and no BDRs. Listen
Rosenthal said that before ChatGPT launched she received about 30 inbounds a week, which she and her two sales reps could manage. After launch, inbound rose to 100 a day, then 200, 300 and 1,000 a day, and she still had two sales reps and no BDRs. She said people were surprised that a 200-person company did not have the resources to take every call.
“Before Chatchuby T launch, I was getting about 30 and bounds a week. 30 companies a week reaching out, super manageable.”
Listen to the episode Pipeline & demand generation Link to this Report a problem
ChatGPT launched with no emails, no marketing and a tiny buried call to action in a blog post, and it still went viral. Listen
Rosenthal said the team thought the audience would be researchers and that they would get a few thousand users. She said the launch was a blog post with a small buried call to action, with no emails and no marketing team, and that it was not a big launch. She said the free, no-waitlist access in a simple UI let the world experience what GPT could do, and it took on a life of its own.
“there were no emails. There was no marketing. We still don't have a marketing team. It was a blog post with a tiny buried call to action in the bottom of the blog post to get to this thing.”
Listen to the episode Positioning & marketing Link to this Report a problem
Rosenthal predicts AI assistants will take over the mundane administrative work of sales, leaving human-to-human selling for buyers. Listen
Rosenthal described a hypothetical future in which a rep's AI assistant prepares briefings on a company and its people, listens in on calls with hints, and updates the CRM with next steps, follow-up emails and decks. She said the assistant would also handle contracts, red-lines, RFPs, questionnaires and CRM entry. She said she thinks it is unlikely that people will still make big purchases without looking somebody in the eye and having a conversation.
“all of the minutia of sales, all of the mundane administrative tasks, all of the managing contracts. and tracking down legal documents, getting them red -lined answering RFPs, answering questionnaires, filling out your sales where CRM, all of that stuff will be handled by your assistant, your AI assistant.”
Audio intelligence, which means categorising, summarising and labelling call transcripts with embeddings, was the use case that worked best in her customer interviews. Listen
Rosenthal said she built playbooks around two or three use cases that were scalable. The best one she found was taking call transcripts and categorising, summarising and labelling them, which she called audio intelligence. She said it used OpenAI's embedding tool to store and cluster information and search across it to recommend similar content, and that a second use case was embeddings for recommendations in search.
“taking the transcript and then categorizing it, summarizing it, labeling it, using our embedding tool which is basically a way to store information and cluster it in search across it to recommend similar content.”
Support chatbots, which people kept calling the best use case, were not working for customers yet because of hallucinations and inaccurate answers. Listen
Rosenthal said she kept hearing that support bots and chatbots were the best use case for the API, but when she spoke to five customers they said the chatbots were not really working. She said the models hallucinated too much, responses were not accurate enough, and some customers had turned the bots off. She said this made it hard to find what she called the Holy Grail use cases.
“the chat bots aren't really working yet. You know, the models hallucinate too much, or the responses aren't accurate enough, or yeah, we had to turn it off, we're not really using it.”
OpenAI reduced its model costs by nearly 99% across the board since Rosenthal joined. Listen
Rosenthal said that when she joined, the models were still very expensive, and that the companies using them were mostly experimental innovation labs, beta products and small native startups built on GPT. She said that since she joined, the company had reduced model costs across the board by nearly 99%. She said that before this it was hard for most companies to adopt GPT.
“We've reduced the models by across the board nearly 99 % since I joined.”
Listen to the episode Pricing & packaging Link to this Report a problem
She said she spent more time on product than she thinks 99% of sales leaders do, and she enjoyed it. Listen
Rosenthal said that in her first few months at OpenAI she spent more time on product than anything else. She said she still spends more time on product than she thinks 99% of sales leaders do, which she described as a joy. She said being in the room for product launches and being on the front lines of user feedback were the reasons she enjoyed it.
“I still spent more time on product than I think 99 % of sales leaders do, which frankly I find a joy.”
Listen to the episode Leadership & culture Link to this Report a problem
Rosenthal goes to at least one product stand-up almost every day, and her team can join most company meetings and understand what is happening. Listen
Rosenthal said she attends at least one product stand-up pretty much every day, and that OpenAI has daily product meetups. She said she was proud that anyone on her sales team could join most meetings in the company and have some sense of what was going on. She said the habit set the spirit that the rest of the company saw sales in a healthy light.
“I go to a product stand -up, at least one, pretty much every day.”
Listen to the episode Leadership & culture Link to this Report a problem
OpenAI asks sales candidates about the API's limitations and which use case would be hard right now. Listen
Rosenthal said the interview process asks candidates what the limitations of the API are and what they think would be a really hard use case right now. She said she wanted candidates to show they understood that generative AI is its own type of AI with its own limits, and not to jump to every possible machine learning exercise such as predictive analytics.
“What are the limitations of our API? What do you think would be a really hard use case right now?”
Listen to the episode Hiring & team building Link to this Report a problem
OpenAI weighed the internal cost of contract review, security questionnaires and DPAs before taking on a customer. Listen
Rosenthal said that at the time, the company had about 200 people and a go-to-market team of roughly 15 including support, so each customer had a real cost. She said the team asked what it costs to review a contract, to answer a security questionnaire, and to handle a DPA, which she called a big deal for the company. She said this is why the team had to be thoughtful about who it worked with.
“what is the cost of reviewing a contract? What is the cost of doing a security questionnaire? If somebody wants a DPA, that's a big deal for us.”
Listen to the episode Sales process & deals Link to this Report a problem
OpenAI is still early in building an ecosystem of delivery partners for customers who want to work with it but do not know how to use the product. Listen
Rosenthal said that OpenAI was trying to build up an ecosystem of delivery partners. She said the idea was for customers who said they really wanted to work with OpenAI but did not yet know how to use it, so that third-party partners could help them ideate. She described the work as still early.
“The other thing that I'm trying to do, and this is still early, but is to build up an ecosystem of delivery partners so that, you know, for these customers that say, I really, really just want to work with you.”
Listen to the episode Strategy & market Link to this Report a problem
Early-stage founders can sell themselves as consultants and get paid to do the customer interviews needed to decide what product to build. Listen
Mark Roberge said that not enough entrepreneurs sell themselves as consultants early on, and called it a powerful strategy while figuring out what product to build and staying close to customers. He said that at OpenAI, the early innovation licenses essentially got the company paid to do customer interviews. He pointed to OpenAI's experience as the example.
“I don't think enough entrepreneurs sell themselves as consultants and consultant engagements early on. And that's a powerful strategy as you're trying to figure out what product to build as you're building your product and you want to stay close to customers.”
Listen to the episode Strategy & market Link to this Report a problem
At OpenAI, revenue is not always the goal, because every decision is driven toward the company's mission of AGI. Listen
Rosenthal said that at most organisations the goal of the go-to-market team is straightforwardly revenue, but at OpenAI the company is mission driven, and its mission is to get to AGI. She said every decision the company makes is ultimately driving toward that mission, so it is not always about revenue. She called this a strange thing for a sales leader to get their head around.
“It's not always just about revenue, which is a really strange thing to get your head around as a sales leader.”
Listen to the episode Leadership & culture Link to this Report a problem