AI Startup Quietly Develops Its Own Procurement Chips

A group of rapidly rising startups has a business model of renting computing power from cloud providers and reselling it to customers, previously investing nothing in their own AI chips. Now they find themselves at a crossroads: it is becoming increasingly difficult to secure enough computing power to serve customers, and some companies have begun to procure chips on their own, which could fundamentally alter their cost structure.

The tight supply of computing power has also forced these startups to turn to dozens of smaller new cloud providers (neocloud) for resources. Compared to large-scale cloud providers like Microsoft Azure, AWS, and Google Cloud, new cloud providers typically have more surplus computing power. The recent hacking incident involving the well-known inference service provider Fireworks also highlights the risks hidden in this route. {#quote_GOOG}

According to insiders, the hacking incident stemmed from the weak security of a small cloud provider used by Fireworks. After the attack, some of Fireworks' clients began considering switching service providers.

Companies like Fireworks are known as inference service providers: they provide services to clients that need chips to run AI models, a process known as inference. They originally did not own chips or operate data centers, focusing instead on developing software to help clients optimize the efficiency of AI model operations.

These companies are growing rapidly. Fireworks reported that its annualized revenue surpassed $1 billion in July, a fivefold increase compared to the previous year. The Information reported in September that the revenue growth of peers like Fal had doubled in just a few months. Fireworks' latest valuation is $17.5 billion, the highest among this group of inference service providers.

However, the increasing difficulty in obtaining computing power may hinder the rapid growth of the entire sector.

Some cloud providers are becoming less willing to rent computing power to competitors. Insiders revealed that CoreWeave decided earlier this year to stop selling computing power to Baseten. Reports indicate that CoreWeave launched its own inference service in March and is unwilling to supply computing resources to competitors. However, a person close to Baseten denied that CoreWeave had reduced its supply of computing power to Baseten.

A CoreWeave spokesperson stated, "We maintain good cooperation with various inference service providers, many of whom are our partners. As long as they adapt to our business model, they will procure CoreWeave's services, and we are happy to support them."

The CEO of cloud provider DigitalOcean, Paddy Srinivasan, said he prioritizes serving clients with real end-user demand rather than companies that resell computing power: "Honestly, there are a lot of clients developing software based on our platform, and the demand is strong." DigitalOcean operates its own data centers.

Some inference service providers' response is to widely connect with numerous small cloud providers. Erik Bernhardsson, CEO of Modal, shared in an interview that in the first four years after the company's establishment, its partners were mainly large-scale cloud providers and a few new clouds; later, unable to obtain the desired quantity and price of NVIDIA Blackwell chips from the big players, it added 30 new cloud partners in the past year. Modal continues to collaborate with large-scale cloud providers but typically procures other models of chips from them.

Some executives in the cloud computing industry believe that if the demand for computing power declines or if large providers cut off supply, inference service providers without their own hardware will find themselves in a passive position.

Kent Draper, Chief Business Officer of Iren (which has its own data center), commented: "Inference service providers are like WeWork in the GPU world. WeWork signs long-term leases for office space and then subleases it short-term. Everything runs smoothly when the market is good, but once demand dries up or financing runs out, the model becomes unsustainable. Everyone is now seeing the value of vertical integration." {#quote_WEWKQ}

No Choice

Some inference service providers believe they have no choice but to invest in building their own hardware. Raghu Rao, who joined Fal in July to oversee data center strategy, stated that Fal has recently begun procuring NVIDIA chips to deploy in rented data centers. Earlier this month, Fal completed a $1.65 billion financing round, which will be used to expand computing power, including purchasing GPUs and renting data center resources from AWS. Rao mentioned that Fal is also expanding its data center-related teams. The hardware procurement plan of Fal had not been reported previously.

"As our business scales dramatically, we need both to rent computing power and to own our own hardware," Rao said.

Insiders revealed that Fal has also indicated to investors its intention to support hardware procurement through debt financing. Rao declined to comment on the possibility of debt financing.

In recent months, several inference service providers have been negotiating loans with lending institutions and intermediaries to sign computing power procurement contracts with new cloud providers. Insiders say these companies hope to use debt funds rather than consume venture capital equity financing to secure computing resources.

On the other hand, Together has been procuring its own hardware since last year. The Information reported in March that it plans to have 250 megawatts of its own data center capacity by the end of 2026. Insiders say Baseten also plans to procure NVIDIA GPUs to deploy in data centers it operates and manages.

CoreWeave itself has also undergone a similar transformation: previously, it mostly operated data centers owned by others, but last year it announced the establishment of its own campus, believing this is the best way to control hardware resources.

However, not all inference service providers believe it is necessary to own hardware. Insiders say Fireworks' management has indicated to investors that procuring physical chips and building and operating data centers is a commoditized business; Fireworks' moat is developing software that can run models at high speed, which is harder to replicate. Another insider stated that Fireworks currently has no plans to procure hardware.

Insiders also indicated that Modal has expressed to investors its intention to procure more of its own hardware for data centers. Bernhardsson said in an interview that the company will do everything possible to obtain the required computing power, but he does not believe that differentiation comes from owning hardware—this aligns with the criticism from hardware-building companies like Iren towards inference service providers like Modal.

"The hardware layer is not where a lot of value is created. Ultimately, you are just buying chips and reselling computing power," Bernhardsson said. "So we focus on the software layer, which is where we continue to create the most value."

This content is provided for general informational purposes only and doesn't constitute financial, investment, legal, or tax advice. Any events, rewards, online promotions, or related information mentioned herein should not be considered a recommendation, solicitation, or invitation to purchase, sell, trade, or otherwise deal in any crypto assets. Crypto assets are highly volatile and may result in loss. The availability of WEEX services, products, and related events may vary by region. You are responsible for ensuring that your participation is in accordance with applicable local laws and regulations.