TheSeriousProgrammer

Binary Magic: Building BitNet 1.58bit Using PyTorch from Scratch

Spoiler Alert:

Are Transformers secretly just linear layers?

This article is a rewrite of a interesting analysis from group of researchers who discovered that a good chunk of transformer blocks in…

Is Kompact AI-IIT Madras’s LLMs in CPU Breakthrough Overstated?

JPEG Compression: The Unsung Algorithm That Powers the Web (AAP-1)

An appreciation post of the genius algorithm which allows us to power the web

The Golden Trojan: Exposing Industrial-Scale Telegram Scams

Context:

LLM powered Automatic Text Data Janitor from Microsoft, named Rho-1

Recently a team from microsoft research published a paper titled “Not all tokens are what you need”. Wherein they discuss a approach to…

RNNs have something which Transformers don’t

It also explains why context length in LLMs is limited even in theory

WTF is a Energy Based Transformer

Explained for both noobs and experts. Also has my review

Disasterous examples of Label Leaking in LLMs

Explores on what is Label Leaking and how its effects can be bad in llms

Can KV Cache be chopped by half by removing V cache

Made a World Action Model at home

Just trained a world action model on Lunar Lander, not RL.