Nvidia’s Blackwell Ignites the Generative AI Supernova! Expert Breakdown here.

Before the keynote speech, Nvidia’s CEO, Jensen Huang, said, “I hope you realize this is not a concert.” He sure left the audience at the SAP Center with an experience they will never forget, as they may have just witnessed a monumental event in the field of AI (or the starting point of Skynet).

Nvidia’s CEO Jensen Huang presenting the new AI chip, Blackwell

The keynote presentation was one of the critical highlights of GTC 2024. GTC, or GPU Technology Conference, is Nvidia’s global stage, where it presents its innovative and industry-disrupting products. Every year, top AI voices and establishments gather in San Jose, California, to witness what Nvidia has in store for them.

Everyone, whether developers, researchers, engineers, inventors, or just plain enthusiasts, can participate in workshops, AI conferences, expos, etc., which allows them to network with like-minded people and form connections.

GTC 2024 will be a five-day event, with workshops from March 17–21, the main star of the show, a keynote on the 18th, and Conferences from March 18–21. Many topics will be discussed, from AR/VR to cybersecurity to Gen AI.

Why are these AI Chips important for Gen AI?

AI chips like the old Nvidia GH100 or AMD’s MI300 are crucial for generative AI because these applications require tremendous computational power.

Photo by Jonathan Kemper on Unsplash

How artificial intelligence works is itself a mind-boggling concept, and the next step is Generative AI models, such as those used for text generation, image synthesis, or 3D creation, which involve processing vast amounts of data and performing complex calculations.

These specialized AI chips offer massively parallel processing capabilities optimized for the matrix operations and data flow patterns inherent in generative AI workloads.

With their high performance and efficiency, AI chips enable generative models to run faster, scale to larger sizes, and produce higher-quality outputs, unlocking the full potential of cutting-edge generative AI applications.

What is Nvidia Blackwell?

Named after David Blackwell, the first black scholar inducted into the National Academy of Science, Blackwell (GB200, which stands for Grace-Blackwell 200) follows the success of its predecessor, Hopper(GH100, which stands for Grace-Hopper 100), which boosted Nvidia’s sales.

For anybody who is not familiar with the naming conventions of Nvidia’s chip world, G or Grace is the CPU used in Nvidia’s AI computing modules called DGX, and B or Blackwell is a GPU, which is an upgrade from the previous H or Hopper.

Nvidia Blackwell and the whole DGXGB200 SuperPOD AI computer

Apart from the apparent benchmark scores that Nvidia proudly wears on its chest, the most assuming part is that Nvidia has taken two dies or chips and just merged them without any loss of computational capabilities, giving birth to GB200, which is insane from an engineering perspective.

One of the direct competitors, AMD MI300, will indeed be shivering when they hear these numbers.

• The new Blackwell ship is 4–5 times faster than its predecessor, Hopper. But when integrated into its environment of DGX Module, it becomes a beast and outputs almost 30x the inference performance, yes, not 30% but 30 times, and wait, wait, there is more, all while consuming 25X less power again that is not 25% its 25 times.
• This allows LLM’s models to train on over 27 trillion parameters. For comparison, the famous ChatGPT 4.0 was trained on 1.7 trillion parameters.
• Blackwell will be made in partnership with Nvidia’s best friend, TSMC, during a fantastic 3 nm fabrication process. It has 208 billion transistors, allowing data speeds up to 10Tb/sec. Amazing right?

How is Blackwell(GB200) better than Hopper GPU(GH100)?

The previous generation of AI-optimized GPUs was called Hopper. Blackwell is between 2 and 30 times faster, depending on how you measure it. Huang explained that it took 8,000 GPUs, 15 megawatts, and 90 days to create the GPT-MoE-1.8T model. With the new system, you could use just 2,000 GPUs and 25% of the power.

Blackwell is genuinely a very successful replacement for the Hopper GPU because it builds upon GH100’s strengths and fills in the gaps or limitations, like improved communication bottlenecks for a larger number of racks. It has no Memory locality issues or cache storage issues.

This essentially means there are much lower bottlenecks when sharing data between servers, dramatically reducing the idle time for each GPU on each server. Thus, efficiency is boosted since more stuff gets done in the same clock cycles.

GB200 offers improved connectivity and data processing for AI tasks and is part of Nvidia’s “super chip” lineup, complementing its central processing unit, Grace.

What is the DGXGB200?

So, Blackwell wasn’t the only new product to be launched; Nvidia launched a new connecting chip named the NVlink Switch, which, using its 50 billion transistors, connects all the brains of the supercomputer.

Blackwell has a form-fit function capability, so you can just swap out the hopper from the existing DGX modules and replace it with a black one.

The main purpose of this chip is to give data centers the capability to talk to each other at full speed without any hiccups. TSMC also builds it on a 4nm fabrication process. Each module has 4 NVlinks, which transfer at 1.8Tb/s and have computation.

To understand how a data center or, as Jensen puts it, AI Factories are made, take a look here:

• Combining one grace CPU, 2 Blackwell GPU, and NVlinks makes up 1 GB200 Superchip module.
• DGX GB200 module. The system features 36 NVIDIA GB200 Superchips — which include 36 NVIDIA Grace CPUs and 72 NVIDIA Blackwell GPUs — connected as one supercomputer using NVlinks
• The Grace Blackwell-powered DGX SuperPOD features eight or more DGX GB200 systems and can scale to tens of thousands of GB200 Superchips connected via NVIDIA Quantum InfiniBand ( Used to connect modules)

DGX Module is an exo flop ai system, which means it is capable of 1 quintillion operations in a second; only 2–3 other machines in the world are capable of reaching that number

In an interview, the computing VP Ian Buck, Nvidia, gave the following statements to the press.

“The fabric of NVLink, the spine, is connecting all those 72 GPUs to deliver an overall performance of 720 petaflops of training, 1.4 exa flops of inference,”

“Overall, the NVLink domain can support a model of 27 trillion parameters and 130 terabytes of bandwidth.” The system has two miles of NVLink cabling across 5,000 cables. “To get all this compute to run that fast, this is a fully liquid-cooled design,”

Pricing of the new Blackwell Chip?

According to an interview with Jensua Huang, a single Blackwell GPU will cost between $30K and $40k, and the R&D budget to make this new chip was almost $10 Billion.

In comparison, AMD’s answer to the GH100 costs just $10–15k, which is a huge price gap. Considering you need a large number of these GPUs, that cost gap is too high when comparing both.

Top companies incorporating Nvidia GB200

All the major companies at the forefront of the AI race jumped on board to adopt the new DGXGB SuperPODs in their ecosystems. Take a look:

All the top companies using Nvidia’s blackwell

• Nvidia AI Foundry partners with Dell to build AI factories for chatbots and generative AI, utilizing the virtual world of the Omniverse.
• Amazon is partnering with Nvidia to build GPU with secure AI and a mammoth 222 exa flop system. Also, Amazon Robotics will be taking advantage of the new Isaac sim and the omniverse.
• Google and GCP are already near completion of integrating it into their infrastructure.
• Oracle, Microsoft, BYD, etc, are a few of the giants that announced their new partnership with Nvidia’s partnership.
• Omniverse is now available on the Vision Pro.

What else has Nvidia announced at GTC 2024?

I will cover every other significant development in different blogs, but some of the key announcements are:

• Project Gr00t is a general-purpose AI foundation model for humanoid robot learning.
• OSMO is a new compute orchestration service that coordinates workflows across DGX systems for training and OVX for simulation.
• A new chip to power the robots, Jetson Thor.
• Isaac Sim is an AI robot training world hosted virtually on Omniverse for training robots that represent the world digitally and makes a sorta gym for robots where robots can train to become robots if that makes sense. It runs on the Omniverse engine, OVX, and is hosted on Microsoft Azure.

What did top AI voices have to say about GTC2024

Elon Musk, CEO of Tesla and xAI

“There is currently nothing better than NVIDIA hardware for AI.”

Sundar Pichai, CEO of Alphabet and Google

“Scaling services like Search and Gmail to billions of users has taught us much about managing compute infrastructure. As we enter the AI platform shift, we continue to invest deeply in infrastructure for our products and services and our Cloud customers. We are fortunate to have a longstanding partnership with NVIDIA, and look forward to bringing the breakthrough capabilities of the Blackwell GPU to our Cloud customers and teams across Google, including Google DeepMind, to accelerate future discoveries.”

Andy Jassy, president and CEO of Amazon

“Our deep collaboration with NVIDIA goes back more than 13 years, to when we launched the world’s first GPU cloud instance on AWS. Today, we offer the widest range of GPU solutions available anywhere in the cloud, supporting the world’s most technologically advanced accelerated workloads.

It’s why the new NVIDIA Blackwell GPU will run so well on AWS and the reason that NVIDIA chose AWS to co-develop Project Ceiba, combining NVIDIA’s next-generation Grace Blackwell Superchips with the AWS Nitro System’s advanced virtualization and ultra-fast Elastic Fabric Adapter networking for NVIDIA’s own AI research and development. Through this joint effort between AWS and NVIDIA engineers, we’re continuing to innovate to make AWS the best place for anyone to run NVIDIA GPUs in the cloud.”

Demis Hassabis, cofounder and CEO of Google DeepMind

“The transformative potential of AI is incredible, and it will help us solve some of the world’s most important scientific problems. Blackwell’s breakthrough technological capabilities will provide the critical computing needed to help the world’s brightest minds chart new scientific discoveries.”

Michael Dell, founder and CEO of Dell Technologies

“Generative AI is critical to creating smarter, more reliable, and more efficient systems. Dell Technologies and NVIDIA are working together to shape the future of technology. With the launch of Blackwell, we will continue to deliver the next generation of accelerated products and services to our customers, providing them with the tools they need to drive innovation across industries.”

Mark Zuckerberg, founder and CEO of Meta

“AI already powers everything from our large language models to our content recommendations, ads, and safety systems, and it will only get more important in the future. We look forward to using NVIDIA’s Blackwell to help train our open-source Llama models and build the next generation of Meta AI and consumer products.”

Satya Nadella, executive chairman and CEO of Microsoft:

“We are committed to offering our customers the most advanced infrastructure to power their AI workloads. By bringing the GB200 Grace Blackwell processor to our data centers globally, we are building on our long-standing history of optimizing NVIDIA GPUs for our cloud as we make the promise of AI real for organizations everywhere.”

Sam Altman, CEO of OpenAI:

“Blackwell offers massive performance leaps and will accelerate our ability to deliver leading-edge models. We’re excited to continue working with NVIDIA to enhance AI computing.”

Larry Ellison, chairman and CTO of Oracle:

“Oracle’s close collaboration with NVIDIA will enable qualitative and quantitative AI, machine learning, and data analytics breakthroughs. For customers to uncover more actionable insights, an even more powerful engine like Blackwell, purpose-built for accelerated computing and generative AI, is needed.”

Why aren’t we using GPU-centric PCs, then??

GPUs have many cores, which is excellent for parallel tasks such as neural networks or image rendering, whereas CPUs have only a few but more powerful cores for tasks that require a lot of computing power on a single or few threads.

Photo by Andrey Matveev on Unsplash

GPUs are what’s traditionally called SIMD (Single Instruction Multiple Data), which is essentially being able to perform the same operation on many data items. For example, when you apply a filter to an image on your phone, it needs to use multiplication with each pixel; the multiplication part is the operation, and each pixel is a data item. GPUs can effectively process 100s to 1000s of such pixels in one cycle.

CPUs are usually known as SISD (Single Instruction, Single Data), primarily serial operators. Even if they can do stuff in parallel, that capacity is limited (a much lower number of cores compared to GPUs), and the kind of stuff they can do in parallel on an instruction level is also different (difference in architecture instruction set).

Photo by Luis Gonzalez on Unsplash

They usually have higher clock speeds and are more optimized for serial operations, so in the above example, it might process that same image much more slowly. Still, it’ll be able to find that image on your storage much faster and load it into the GPU buffer, for instance. I hope that helps!

CPUs are better at operating system tasks, like moving files around and dealing with all I/O. GPUs are highly customized to do specific tasks very well, aligning with the needs of training AI models. While you could train AI models using CPUs only, it would be slow to the point of being completely intractable.

The current LLMs from OpenAI and Google require trillions of parameters that would take traditional CPUs literally years or even decades to complete. A GPU cluster could do the same work in a few weeks or months.

What is the outcome of GTC 2024?

In conclusion, Nvidia’s unveiling of the Blackwell GPU and DGX GB200 SuperPOD at GTC 2024 marks a significant milestone in advancing generative AI and high-performance computing. These new offerings’ staggering computational power and efficiency promise to propel AI development to unprecedented heights, enabling more sophisticated models to be trained and deployed with remarkable speed and accuracy.

The groundbreaking capabilities of Blackwell, combined with the seamless integration of NVLink and Grace CPUs, create a formidable AI factory that empowers researchers, developers, and enterprises to tackle the most complex challenges in natural language processing, computer vision, and robotics.

As industry giants like Google, Amazon, and Microsoft embrace these technologies, we can expect innovative applications and services to reshape various sectors. Moreover, Nvidia’s commitment to fostering an ecosystem through platforms like Omniverse and collaborations with partners further underscores the company’s vision of democratizing AI and fostering a vibrant community of creators and innovators.

As we stand on the cusp of a new era in AI, Nvidia’s relentless pursuit of technological advancement positions it as a driving force in shaping the future of intelligent systems and their profound impact on society.

This story is published on Generative AI. Connect with us on LinkedIn and follow Zeniteq to stay in the loop with the latest AI stories. Let’s shape the future of AI together!