Articles

SPT MDA AI

MK M K Pavan Kumar

Compressing Gemma 4 12B with LLM Compressor and Serving It on vLLM

This is the full journey of taking Gemma 4 12B from a 23.9GB BF16 model to a 15GB FP8 one that serves on a single GPU — start to finish…

Jun 13, 2026 · 14 min read · 3 claps #llm-compression #vllm #large-language-models

AI SCI TCH SPT

SN Sneha Rangole

Hugging Face for Absolute Beginners: Models, Datasets, Spaces, and Every Confusing Number on a…

If you’ve ever opened Hugging Face, seen “4B params · BF16 · 78,925 downloads · Quantizations: 4 models” and quietly closed the tab — this…

Jun 03, 2026 · 19 min read · 3 claps #hugging-face #hugging-face-tutorials #data-science

AI SPT

MA Manikandan Thangaraj

72B Parameters, Zero Quantization, One GPU: Benchmarking Qwen2-VL on AMD MI300X

I loaded Qwen2-VL-72B-Instruct at full BF16 precision on a single GPU, served 64 concurrent DocVQA streams, and kept the system stable at…

May 13, 2026 · 13 min read · 0 claps #vllm #amd-mi300x #rocm

HUM AI SPT

OX Oxotall

Tensor and sequence parallelism — explained with pictures

A modern LLM does not fit on a single GPU. A 70B-parameter model in bf16 needs ~140 GB just for the weights, and during training the…

May 08, 2026 · 8 min read · 7 claps #machine-learning #tensor-parallelism #sequence-parallelism

AI SPT

RO Roya

TPU 101 — Part 4: Inside the TPU, and a Head-to-Head With a GPU

A beginner-friendly series on Google TPUs and JAX. This part: what @jit actually produces (we'll read the compiled program), why bf16…

Apr 30, 2026 · 13 min read · 0 claps #tpu #gpu #jax

AI SPT

JA Jamsheed K

LoRA fine-tuning of Qwen3.5 on NVIDIA DGX Spark

This article covers the complete environment setup for bf16 LoRA fine-tuning of Qwen3.5–35B-A3B on a single NVIDIA DGX Spark (GB10…

Apr 14, 2026 · 8 min read · 1 claps #fine-tuning-llm #lora #nvidia-dgx-spark

AI SCI SPT

SA Sarankannan

Bleeding Edge or Bleeding Out? The Quest for vLLM on NVIDIA Blackwell

Why my 57GB BF16 experiment failed, and what it reveals about the gap between hardware delivery and software reality.

Feb 12, 2026 · 7 min read · 0 claps #dgx-spark #vllm #hugging-face

AI SPT

NE Neurobyte

5 PyTorch Memory Tactics for Bigger, Faster Models

Practical moves — KV cache reuse, gradient checkpointing, BF16, selective offload, and FlashAttention — that squeeze more sequence length…

Oct 03, 2025 · 5 min read · 14 claps paywalled #pytorch #memory-optimization #bf16

HUM AI SPT

ST StackGpu

FP8, BF16, and INT8: How Low-Precision Formats Are Revolutionizing Deep Learning Throughput

Explore how FP8, BF16, and INT8 are transforming deep learning performance by optimizing throughput, reducing energy consumption, and…

Jun 20, 2025 · 7 min read · 6 claps #fp8 #bf16 #int8

HUM AI SPT

VE VectorWorks Academy

Ace AI Interview Series 18 — Understanding Mixed Precision Training and bfloat16 (BF16) in Deep…

Deep learning has evolved at an extraordinary pace, and along with it, new techniques and optimizations are constantly being developed to…

Feb 24, 2025 · 7 min read · 0 claps paywalled #mixed-precision-training #bf16 #bfloat16

AI SPT

PY PyTorch · PyTorch

Accelerate PyTorch with IPEX and oneDNN using Intel BF16 Technology

Pytorch extended with Intel IPEX using oneDNN delivers high performance of Intel DL Boost BF16 Tech on the 3rd Gen Intel Xeon Processors

Feb 03, 2021 · 8 min read · 79 claps #bf16 #intel #pytorch