Neo-Cloud Primer: Business Models, Tech Stack, and the Chaos in Between

This is a multi-part series where I take a deeper look at the evolving world of AI Neo-Clouds, covering business models, infrastructure decisions, and the underlying technology stack.

The whole AI boom and bust cycle is incomplete without mentioning or understanding Neo Cloud economics, which often get overshadowed by hyperscaler billion dollar investment news cycles and discussions around gigawatt scale data center facilities.

In this blog, I will take a look at Neo Clouds, first derivative picks and shovels play for AI, from a technical standpoint and also at how the business model has evolved over the years. Even though I have been working in the Neo Cloud space for over five years now, I did not fully understand it until I listened to the famous Odd Lots podcast, where the CoreWeave CEO breaks down the bigger picture of AI Neo Clouds and how they transitioned from a crypto miner to an AI cloud provider.

For those outside the core technology space, transitioning from a crypto miner to a GPU cloud might seem natural. But under the hood, the technology stack is very different, to the point where you almost have to redesign everything, from data center architecture to software tooling.

Before I dig into the Neo Cloud tech stack, let me first break down the different classes of Neo Clouds in the market and their business models. I will be focusing only on GPU focused clouds that primarily target AI and LLM development shops.

This space is constantly evolving, and many players frequently change their business models by moving up or down the technology stack.

What Neo-Clouds actually are

• Neo-Clouds are a new class of infrastructure providers built primarily around GPUs and AI workloads.
• They sit between hyperscalers and traditional hosting providers, both in terms of scale and complexity.
• They are not trying to build a general purpose cloud with hundreds of managed services.
• Their core value is reliable access to GPUs, fast networking, and just enough software to run AI workloads.
• Most Neo-Clouds exist because it’s hard to certain kind of GPU’s with Hyperscaler as they have mostly pre-allocated them to their premium clients or for their internal applications.
• Neo-Clouds focus on utilization and efficiency rather than feature breadth.
• Neo-Cloud pricing are very volatile with cut throat competition and survival of the fittest.

What Neo-Clouds are not

• They are not hyperscalers and do not compete directly with AWS, GCP, or Azure.
• They are not model companies and do not own or train foundation models as their primary business.
• They are not API-only AI platforms that abstract away infrastructure completely.
• They are not trying to hide the infrastructure from users; in most cases, the infrastructure is the product.
• They do not promise magic or simplicity at all costs. They sell compute, and everything else in the stack exists to make that compute usable and rentable.

Neo-Cloud Categories

Neo-Cloud — Marketplace & Hybrid Resellers

Neo-Cloud — Pure Play

You might notice that I have used Mid-Market as a broad blanket category for many providers. The simple reason is that the Neo Cloud business model can be very complicated. In many cases, these providers are selling each other’s excess supply and frequently switching roles between being a market maker and simply reselling someone else’s capacity to make a margin.

In the table below, I have provided some basic criteria for grouping and categorizing these players.

Categories

Marketplace

These are players who do not own any infrastructure but instead provide a platform where other Neo Cloud providers can sell their excess capacity. Shadeform is a market leader in many ways, considering its broad market access, snappy developer friendly UI, and fast onboarding with quick launch capabilities.

Hybrid

This category gets a bit complicated, as many players frequently change their business models. A good example here is Hydra Host, where they may operate their own colocation facilities while also collaborating with other data center providers to sell cloud GPU instances as if they were their own. The key difference between a marketplace and a reseller is that, in the case of a reseller, the end customer may not know whose infrastructure their workload is actually running on. In many cases, these hybrid cloud providers also offer L1 customer support to help customers directly.

Own and Operate

These are classic Neo Cloud providers who actually own and operate their own fleet of data centers and GPUs. This is a capital intensive endeavor, and not many companies can sustain or survive at this level. In this category, the clear market leader is CoreWeave, with customers like Meta, Microsoft, Google, and OpenAI. Due to their market position and operational expertise, they are able to command premium pricing and the best margins in the market.

Core Market

Developers

This represents a true developer first and developer friendly environment, where customer onboarding should be almost one click, with minimal friction. Most Neo Cloud providers in this category offer developer friendly consoles, notebooks, and prebuilt applications.

Mid-Market

Mid-Market is a very broad term used for mixed use cases, where the ideal customer profile is not clearly defined and cloud providers are primarily trying to meet demand as it appears. Mid-market not only includes the end customer, but often Neo-Clouds trade GPUs with one another to manage their GPU utilization.

Enterprise

Its well know fact by now that most enterprises, money making business, won’t go with any of Neo-Cloud with their services. Essentially they will be using AI services provided by their Hyperscaler partner, unlike startups, they are not price sensitive. Right now most neo clouds are focused on getting as many as startup customers into their books to keep their fleet utilization high.

Not Neo Cloud

AWS, GCP, Azure, Oracle, xAI

You may notice that I have not included hyperscalers in this discussion. That is intentional. Hyperscalers are not Neo Clouds. They operate in a different market altogether and function with near monopolistic pricing power across developers, enterprises, and government customers.

MyAICompany.ai

This category can be especially confusing, as many AI companies present themselves as offering everything under the sun on their landing pages. For this blog, I have excluded companies that primarily provide API endpoints running on someone else’s hardware, such as Together AI and other Bedrock style offerings. This segment is a large market on its own and is mostly operating expense heavy rather than capital expense heavy.

In case I have misrepresented or missed anything feel free to comment.

Neo-Cloud Product Offering: Picks and Shovels

The core business model for Neo-Clouds remains simple.

They provide the picks and shovels for the AI gold rush. No matter who wins, if anyone does, the demand for their core products is expected to remain strong.

At the same time, many Neo-Clouds aim to grow fast enough to become acquisition targets for larger players or hyperscalers.

Core Product Offerings

• Bare Metal — Bare metal is by far the most popular product offering for many AI startup customers. Most of these startups are either training their own custom models or using off the shelf models from Hugging Face and offering AWS Bedrock style LLM API endpoints. These customers tend to be well funded and usually have access to a healthy end customer pipeline.
• Virtual Machines — Similar to bare metal, but typically offered at a much smaller scale.
• Jupyter Notebooks — Primarily targeted at individual developers and small startups who are experimenting with GPU based AI models.