#floating-points

Debugging NaN Semantics Between Triton’s Interpreter and JIT

A small but important fix for consistent reduction behavior when NaN values are present

Starting Go #005: Data Types in Go — Float & Boolean

Let Go run the lesson.

Shrinking Giants: A Word on Floating-Point Precision in LLM Domain for Faster, Cheaper Models

Ever wondered how floating-point decision can have an impact on LLM’s output?

Why Hackers Love SSE & AVX

Floating Point & SIMD in Modern Malware

The Computer Is a Terrible Calculator

Why even the smartest machines still stumble over simple math.

The Great Number Hide-and-Seek: Why Computers Sometimes Lose Track

Imagine a game of hide-and-seek. The players? Numbers. The seeker? Your computer. Most of the time, everyone gets found. But sometimes, a…

IEEE 754 부동소수점과 Java에서의 표현

부동소수점 오차 문제, 한 번쯤 겪어본 적 있을 것이다. 필자는 대학교 1학년 때 기초프로그래밍 중간고사 서술형 문제로 처음 마주했고, 최근에는 기업 필기시험에서도 다시 등장해 이번 기회에 개념부터 실전까지 확실히 정리해보려고 한다.

dtypes of tensors: bfloat16 vs float32 vs float16

You may have come across the term bfloat16 in the context of machine learning and artificial intelligence. This article explains what…

[SOLVED] Does GCC's __builtin_isnan do something other than isnan?

Generic FIR Filter Using Floating-Point IP in Vivado

In this post, I will be showing you my FIR filter design using floating-point IP in Vivado.

FP64, FP32, FP16, BFLOAT16, TF32, and other members of the ZOO

There are many floating point formats you can hear about in the context of deep learning. Here is a summary of what are they about and…