How Index Ventures jumped to the front of the AI GPU line

Earlier this week, the New York Times shone a light on some of the desperation that founders are experiencing as they try and fail to secure compute power for their nascent artificial intelligence startups, thanks to the big companies (and even rich nations) racing to snatch them up. One founder reportedly said of the graphics processing units, or GPUs, that he needs for his company: “I think about [them] as a rare earth metal at this point.”

According to that Times piece, founders are trying numerous measures to amass the chips, including calling in favors from friends at large equipment vendors that might have GPUs to spare, and navigating an obscure U.S. government program called Access.

At least one firm, the global investor Index Ventures, happened on an additional idea, it told the outlet. To help ensure its portfolio companies aren’t hamstrung by the shortage, it struck a deal with Oracle to provide its founders with some of these sought-after chips (specifically Nvidia’s H100 chips and Nvidia’s A100 chips).

To learn more about the arrangement — one that other venture firms are undoubtedly trying to replicate —  we talked earlier today with Erin Price-Wright, a Bay Area-based partner with Index who focuses on enterprise software and AI and who, before joining the venture firm in 2019, was the head of product for Palantir’s data analytics and machine learning platform. Excerpts from our chat have been lightly edited for length and clarity below; you can hear our longer conversation here.

TechCrunch: Tell us about this partnership with Oracle. 

Erin Price-Wright: Access to compute is one of the biggest challenges that AI companies face, and it’s especially hard for an early-stage company to get their hands on GPUs. It’s less about the cost in particular but the fact that something like more than 95% of GPU capacity is already allocated to large players in this space [because] they make these pretty big pre-commitments with cloud vendors. So if you’re an early-stage company, and you’re just trying to get started training, or fine tuning the model, there’s usually a really long lead time between when GPUs are even available. It can be three months to a year in some cases and it’s really hard to just get started.

If you’re an early-stage company that’s still figuring out what your product is, you don’t even know how many GPUs you need. So even that process of discovery of understanding what your workloads are going to look like can be super challenging for early-stage companies. So we’re partnering with Oracle to provide GPUs to our earliest-stage portfolio companies, because we want to help remove that barrier of access so that they can really focus on what matters from day zero. Ultimately, the goal is to help all of these companies graduate to their own cluster. We’re not in the business of providing these massive GPU clusters to our companies. . .but we really want to give them a head start, so that they can start building faster as a way to help level the playing field.

How did the deal come together?

We wanted to make sure that people who are building against very tangible business problems didn’t feel like they had to change their business model or change the way they were representing themselves or change the way they were fundraising in order to just get access to GPUs. So it was really born out of seeing this pattern again and again with early stage companies where we were like, ‘This is where Index as a fund actually has real leverage. And we can use our position in the market, our relationships, and the fact that we can kind of aggregate this demand across multiple companies to really provide value-additive services’ [to our founders].

Did Index put a down payment together or has it purchased chips outright from Oracle? Are you giving Oracle a stake in these startups?

We’re not purchasing any chips outright. So the partnership with Oracle is that Index makes the pre commitment on the behalf of our startups and pays the cloud bill. Oracle manages the cluster — they’ve been a fantastic partner — and then our companies get access to that GPU cluster for free.

So you’re paying [this cloud bill] in advance. Did you have to talk with your own investors about that? That’s not typical of what [a venture firm] would do historically.

In terms of the actual structure of how the agreement works, I’ll probably hold off on sharing too many of those details.

Is this an exclusive relationship? Is there anything to prevent other venture firms from doing the same thing?

Yeah, of course [they could do the same], there certainly isn’t [an exclusive relationship with Index].

One benefit that Oracle gets out of it is to meet the next generation of fantastic companies as early as possible. In the process of using our GPU cluster, we’re actively helping our companies navigate the process of signing their own dedicated cloud deal. So the idea is not for them to [do] this in perpetuity; it’s for them to develop relationships with Oracle and AWS and the other large cloud providers and sign their own dedicated contract.

One of your portfolio companies, Cohere, counts Oracle as one of its backers along with Nvidia, which are two of the companies you most want to have involved with your portfolio companies right now.

One of the ways we really can help our portfolio companies is making sure they’re connected to the right people at the right time, so that they get the resources they need.

Index has at least 20 portfolio companies that fall into the AI/ML bucket, including Cohere [which has already raised $445 million] and another company that recently raised a huge seed round, Mistral AI in France. Is too much money being invested broadly in generative AI or are we still in the ‘early innings,’ as VCs like to say?

We are in the early innings. I do think we’re rapidly entering a cooling off period in terms of sentiment, especially for some of these very large rounds and especially from traditional VCs. There’s still a really big gap between the promise and power of the core models of technology and what it’s going to take for them to be actually used and useful across many use cases in the enterprise. There’s just a huge infrastructure gap missing that needs to be filled, and it’s not going to be filled overnight; it’s going to take some time.

Over the coming 12 months, while I’m still very excited about the power of the core technology and how transformational it’s going to be for the world, I think we’re going to see a little bit of a backing off as companies really grapple with it, figure out the ROI, kind of prioritize use cases and start actually building real things beyond maybe the one or two prototype demo apps that they’ve been working on for the last six months. That’s when we’re going to start seeing the infrastructure emerge that’s going to start supporting these use cases at scale.

How do you as an investor ensure that your AI companies don’t overlap? And is that any harder or more difficult than when it comes to traditional startups?

I don’t think it’s massively different than how we think about competition elsewhere. Everyone paints AI as this standalone category. But if I look forward even two years, let alone five or 10, every single piece of software that we use will have AI as its beating heart. There will be no piece of code, no software, no application, no website that you visit, that doesn’t have AI as a core component of it. I almost think about it like SaaS. Is every single SaaS company the same? No. Every single SaaS company has a database, every single SaaS company has a front end, every single SaaS company has some interaction between the two. AI is kind of similar to a database in that respect. It’s just kind of a core building block in how you build software.

We’re very early in the market, so there’s going to be some movement and some change as companies figure out how to use these tools and what specific problems to go after. But it’s not different than how we think about traditional SaaS investing from my perspective.

source https://techcrunch.com/2023/08/19/how-index-ventures-jumped-to-the-front-of-the-ai-gpu-line/