OpenAI want to build the best developer product ever

Logan Kilpatrick is a member of OpenAI’s developer advocacy team. Logan has sometimes been described as OpenAI’s first DevRel. This is a summary of Logan's interview on Scaling DevTools

What growth challenges do you have when OpenAI already has so much attention?

It’s a great problem to have. My previous experiences have been more like screaming into the void telling developers why they should care about the the thing I’m excited about.

When I was originally being hired, OpenAI had GPT 3.5, but developers didn’t really understand what they could build. There were a few customers like Jasper but nowhere near widespread market adoption.

But it just so happened that it was a long hiring process and my first day was ChatGPT hitting a million users. All of a sudden, we didn’t really have this problem with top-of-funnel awareness with developers anymore. Everyone was really excited to build stuff.

At the time, we had, I think, a seven-person team that did all of the API stuff - the entire developer platform team was seven people. And so my job became, how do we make the best developer product ever?

We had one product manager and a few engineers and me and it was a lot of scrambling and a lot of work.

Even after we had the success of ChatGPT, it still takes a while to fire up the hiring engine. So, even for the GPT-4 release and the ChatGPT API release, we didn’t have a huge team.

So, a lot of my focus was developer experience - how do we make things better for developers?

So to actually answer your question about what is the hard part for us. It’s really: how do we help people scale reliably on our platform? There are so many challenges with that because of how compute intensive these tools are and how much demand there is.

We have to make a lot of trade-offs. No one wants to be making these trade-offs, but it’s just the position we’re in today. We’re trying to optimise the crappy trade-offs to have the most developer-friendly thing we can do even though there’s so much more that we have to do.

On another podcast, you gave an example of a trade-off being not having billing by API keys because even though it’s relatively easy to implement, you would need to think about scaling it.

Yeah, I do remember saying that. The good news is we released API key based usage now!

But it is a great example. There are all these core platform things that we need to do to make life easier for enterprises, companies and individual developers to use our platform.

We trade those off against, well, new models that we need to release and new APIs that we need to build - like the Assistant API for example. And how do you make the trade-offs between core features that people want and new modalities and new APIs?

Historically we’ve skewed towards shipping new models and APIs and the actual platform has fallen to the wayside. And I think that that was the right trade off. Now, we have more resources - an actual enterprise platform team that’s focused on shipping core platform improvements. So you'll see a lot coming from that team that makes our products better.

But yeah, historically, bad trade-offs. We wanted to make the platform better. But we also wanted to ship new API modalities. I'm excited for the future, where we'll be able to do all of the things at once and hopefully do them all really well.

Speaking of doing all the things at once, how do you decide what to spend your time on?

There's an interesting tension point between where I think I'm going to spend my time and - because of how quickly the pace of OpenAI is moving - where I end up spending my time.

if you had asked me two weeks ago, how would I be spending this week? I would have given you a different answer than what I'll actually spend this week doing.

That's actually the most challenging part - it's really difficult to plan ahead and be like: I want to do these five things. Because so much is going to change. I don't have a great answer.

In general, the big buckets of things that I spend my time doing are probably:

• Core documentation
• Product - spend a ton of time being the DRI (Directly Responsible Individual) for a lot of the improvements on a granular low level on a on a very macro level as well. (roughly 20% of time)
• Talking directly to developers - hearing feedback from them & seeing problems.(roughly 20% of time)
• Internal coordination - everybody has questions, and somebody needs to answer them. So spend a lot of time doing that. (roughly 20% of time)
• Launches - we launched so much stuff that I feel like we're always in launch mode. I feel at least a quarter of my time is spent on new launches for the last couple of weeks it's been the GPT store.

So it's it's all over the place. And there's an infinite to-do list that's never going to stop growing. And I think that's the really fun part. If there was less work to do, and there was less stuff happening, it just wouldn't be as exciting to me. So I'm happy in a lot of ways that there's so much craziness going on.

It sounds a bit like you have the same job as a founder

100%. The company has changed in many ways for the better, and we now have so many more people. We used to be so resource-constrained that you would just have to say no to everything.

And now I feel like we can say yes to a lot more.

I get to say yes to a lot more because I know that we have five other people who are doing some really important work behind the scenes, so that part is refreshing. Because we've scaled so much.

What do you think is the most impactful thing that your team spends time on?

It's definitely the documentation.

At the end of the day, if I had to say no to everything else, and focus on a single thing, it would be our docs.

Our API is brought to developers through our documentation. If there's something wrong with our documentation, or it doesn't make sense, or people can't find what they're looking for, then we've failed. it doesn’t matter that GPT-4 is incredible if people can't find it, and learn how to use it.

That’s always priority number one. I can say no to everything else, knowing that the docs are really what matters.

Romain, who's my manager, has joined and we're just now getting to the point where we'll be able to scale and build a team because, historically, it's been me doing the docs in collaboration with random other teams, like engineering teams, or product teams, or whoever. Our go-to-market team has also done a bunch of things related to docs, but, at the end of the day, those are all one-off contributions, and no one else is really thinking super deeply about it.

So, having other folks to help will be super nice, and hopefully, developers will be able to see the change in quality once we have more people who are waking up every day and living and breathing. developer docs.

Your docs get used very heavily and probably scrutinised too. What kind of things have you learned?

I’m really a sucker for a beautiful design that invokes deep emotion as you're using developer documentation. Stripe is a great example of this.

That manifests itself for a lot of other developer tooling companies around like landing pages. I'm not a designer by trade so I have a really hard time picturing how to build what I want.

We just throw you into the technical details but I think what actually makes sense is for us to have some well designed signposted landing pages that take you along different journeys for different personas.

We don't do any of that today. We pretty minimally give you the information you need and assume that everybody's a developer who's reading it.

But that's not the reality. There are a lot of other personas who are showing up today to use OpenAI’s tools.

So the lesson has been that we need to build those things for people. I don't personally have the right skill set to be the one to do that. I can think about what makes sense for developers, but there's all these other things that we need to consider for other personas. And it ends up being a huge endeavour to take on, which is why we haven't solved the problem yet.

How are you figuring out the gaps in the docs?

It's chatting with people and also looking through other great documentation examples.

For example, our documentation is very heavy on text and historically has lacked diagrams and videos.

We’ve just been moving so quickly. Essentially, all of our docs from a year ago are gone. Not because we rewrote everything, it’s just that we have completely different surface areas.

A year ago, the API was very different than it is today. So it's been hard to make the longer term investment into video content and stuff when you know that it's potentially not going to stand the test of time.

I think things are slightly different now. I think GPT-4 is going to be around for a while. But it's still an interesting trade off.

I always think with OpenAI, you do some things so well that it doesn’t really matter if you don’t have all the other things; people will still come.

I think that is true. But I think OpenAI is only as good as its last model. If that mindset is true, with all the competition there is, we really need to have this holistically incredible developer experience across our surface areas.

Selfishly, I think we will always have the best models because I see the order of magnitude of the investments that our company is making and the people we have doing research, but I think it's the wrong thing to do to assume that that's going to always be the case and not try to make the investments into incredible content and developer experience and all these other things.

How do you feel you can build a better developer experience than other model providers?

People associate OpenAI with ChatGPT, but really, for OpenAI to be successful, our API has to be successful. We have to make it the best thing for developers to build with.

Our mission is to make artificial general intelligence that benefits everybody. I think ChatGPT is going to be immensely successful. It could be of the magnitude of Google Search. But I still don’t think ChatGPT is going to be the mechanism we achieve our mission.

There are many orders of magnitudes of more distribution that you could have through an API product.

There are so many people who use ChatGPT but there are probably hundreds of millions of end users who don’t even know that OpenAI is powering their experience.

For us to be successful, we need developers as part of that. And I don't think that's necessarily true for a lot of companies. For instance, I like Google. I think Google is a great company, but Google’s mission is not reliant on building the best product for developers.

I think that's probably the biggest differentiator. It's just a core piece of our mission.

How do you keep the noise out? Especially when OpenAI is getting a lot of attention.

It’s a tough question. I think it's especially challenging for me, as somebody who has inadvertently become a public face for OpenAI developers. I don't think that was very intentional, I was just kind of doing the thing that needed to be done.

Sometimes I’m close to tweeting about taking some time off Twitter and it would be really nice. But it’s a trade-off because at the end of the day, it's part of my job. There's a huge need. And I continue to get extremely high signal via DMs from people telling me “this thing is broken” or “this thing's not working to me”. And I get to go and share that with our teams.

If I take time away and decide not to do that, then I'm doing a disservice to our developers and our customers.

It's a tricky trade-off.

On deep work, the thing that matters for me is making docs improvements. So docs improvements alongside helping support our launches. are like the two things that I will always have to focus on and make really difficult trade-offs elsewhere.

For example, I’ve historically had to step away from the developer community forum for a month or two at a time because I just did not have enough bandwidth to manage a community of millions of people and do all the other things that needed to happen.

So it’s about focusing on your priorities and accepting that you have to be on Twitter to some extent…

…And being apologetic!

I spend a lot of my time apologizing to people in our developer community that we can't do more.

Part of it is just like being humble about the reality that we're not always giving people what they want. And there are gaps. It’s easy to buy into the idea that OpenAI has an incredible developer platform: “we've had all this success and therefore, we don't need to apologize to people.”

But I think apologizing actually goes a long way. I do, in many cases, feel apologetic about the fact that we've dropped the ball for some, you know, sliver or portion of people who are trying to build stuff with our platform.

Finally, some rapid fire questions from the community

What do you want people to make with OpenAI that they aren’t currently? (Lu from tlDraw)

I think the text-first chat assistant is what is going to bring AI to the next 3 billion people. Everybody who has technology already knows what texting is, and they don't need to learn some new interface. I think there's a huge opportunity to build a multiplayer, multimodal text-first AI assistant, and I'm really excited about that. Fingers crossed, somebody builds something cool.

Are you planning events in London and other cities? (via Louis from Bloop)

Yes, we’re hiring aggressively in London, and I think that will be a kickstarter.

I live in Chicago. I love Chicago. Everyone does not live in San Francisco. Our mission is to build AI that benefits everybody. For me, it's super important that we show up in places that are not just the most exciting tech hubs in the world. I'm talking to someone right now about doing an event in Atlanta.

If folks have events and things like that, that they're excited about, please feel free to send them to me.

It's a difficult trade-off because we have a small team. I'm hopeful as the team grows, we'll be able to do many, many more of these types of events.

Your number 1 DevRel strategy for 2024? (via John from Joyfill)

Making great docs. That’s the theme of this conversation. Everything comes after that.

What is something that OpenAI developers are currently unhappy with that you want to improve in 2024? (via Edo, software engineer from London)

Freedom to scale is a big one. I saw this thread from somebody a few months ago, talking about how challenging and how upset they were with the experience as they've grown their company and their business using the OpenAI API: how hard it was to get higher rate limits and quota and stuff like that. I think that's historically been the biggest challenge.

A lot of it is rooted in the capacity constraints - we just don't have enough capacity for everyone to do the things that they want to do, which is just so mind blowing to me.

For the first year, I was like, “How can we not just get more computers?I don't understand. We have money! We can give people money! And there's nothing for us to buy with our money!”

It's really challenging. If we solve that, it’s going to be a huge unlock if they can scale to whatever level they want by default.

I’m hopeful we’ll solve this by the end of the year.

Have you taken any vacation since you joined?

I haven’t really, but I’m planning some between Christmas and New Year 2024 when we have a code freeze. Every time I’ve planned a vacation, something big comes up, e.g., a launch. But I’m excited for Christmas 2024!

Best deep dish in Chicago?

Lou Malnati's is great. I ate it a lot as a kid, but I love Paradise Park. It’s really high quality, perfectly cooked and consistent and they have a great side salad.