#stream-processing

KA Kazimirov Oleksandr

Kafka record headers in Stateful Functions

Apache Stateful Functions never let a function see Kafka record headers, and never let it set them. Five years, no headers. StateFun Actors…

Jul 24, 2026 · 1 min read · 0 claps #apache-flink #kafka #stream-processing

ART TCH MDA MKT

BR Bran Kop, Engineer @Conformal, Founder of aiHQ · Towards Data Engineering

Batch and Streaming: Two Sides of the Data Architecture Coin

Every system that processes data eventually confronts one fundamental question: When is the data complete enough to be moved to the compute…

Jul 17, 2026 · 5 min read · 28 claps #data-engineering #stream-processing #data-architecture

TCH ECO SPT

KI Kiran

Batch is Dead: Building a Real-Time Ad Click Engine in a Weekend

If you are still running batch jobs overnight to calculate ad click metrics and billing data, you are lighting money on fire. Advertisers…

Jul 15, 2026 · 5 min read · 0 claps #stream-processing #apache-flink #apache-kafka

TCH

PA Patryk Ziobroń

Stateful Order Fill Matching — Part 5: Apache Flink versus Kafka Streams

A practical comparison of Kafka Streams and Apache Flink on the same stateful order-lifecycle workload.

Jul 07, 2026 · 20 min read · 48 claps #apache-flink #kafka-streams #stream-processing

TCH ECO

YA Yash Jain · AlgoMart

Stream Processing in System Design: Processing Data Continuously for Real-Time Decision Making

Modern systems do not wait politely for a nightly batch job anymore. Users click, payments move, sensors emit signals, logs keep flowing…

Jul 06, 2026 · 6 min read · 44 claps paywalled #stream-processing #system-design-interview #software-development

MDA

SA Sahin Yelkenci

Kafka Stream Eğitim Serisi — Bölüm 33 — Threading Model Derinlemesine.

Modül 10 — Üretim ve Operasyon

Jul 01, 2026 · 6 min read · 0 claps #kafka-streams #threading-model #stream-processing

SA Sahin Yelkenci

Kafka Stream Eğitim Serisi — Bölüm 29 — Spring Kafka Streams.

Modül 8 — Spring Boot Entegrasyonu

Jun 30, 2026 · 7 min read · 0 claps #kafka-streams #spring-boot #spring-kafka

TCH MDA AI SCI

ME ömer çakan

Python ve Rust’ın Gücüyle Gerçek Zamanlı Veri İşleme: Bytewax Nedir?

Veri mühendisliği tarafında gerçek zamanlı veri (streaming) işleme süreçleriyle uğraşırken genellikle Apache Spark Structured Streaming…

Jun 30, 2026 · 3 min read · 0 claps #data-engineering #python #data-science

ECO MDA

NI Niteesh K R

Process Functions and UDFs in PyFlink — Unlocking Flink’s Real Power

A practical guide to user-defined functions and process functions that help you build real production-grade streaming logic.

Jun 29, 2026 · 4 min read · 1 claps #stream-processing #apache-flink #flink-streaming

HUM

SA Sahin Yelkenci

Kafka Stream Eğitim Serisi — Bölüm 26 — Exactly-Once vs At-Least-Once. EOS v2, idempotency.

Modül 7 — Doğruluk ve Dayanıklılık

Jun 28, 2026 · 7 min read · 0 claps #kafka-streams #exactly-once-semantics #at-least-once

SA Sahin Yelkenci

Kafka Stream Eğitim Serisi — Bölüm 24 — State’i Dışarı Açmak.

Modül 6 — Interactive Queries

Jun 27, 2026 · 7 min read · 0 claps #kafka-streams #interactive-queries #state-store

SA Sahin Yelkenci

Kafka Stream Eğitim Serisi — Bölüm 18 — GlobalKTable ve KTable–KTable Join.

Modül 4 — Join’ler

Jun 24, 2026 · 7 min read · 0 claps #kafka-streams #globalktable #ktable-join

TCH MDA

MO Morad Abbou Azaz

Streaming is Not Fast Batch. Here are Five Principles That Prove it

Some basic principles according to Kleppmann’s “Designing Data-Intensive Applications” book

Jun 23, 2026 · 6 min read · 0 claps #stream-processing #kafka #distributed-systems

SA Sahin Yelkenci

Kafka Stream Eğitim Serisi — Bölüm 12 — Pencere Türleri. Tumbling, hopping, sliding, session.

Modül 3 — Zaman ve Pencereleme

Jun 21, 2026 · 8 min read · 0 claps #kafka-streams #windowing #stream-processing

SA Sahin Yelkenci

Kafka Stream Eğitim Serisi — Bölüm 11 — Zaman Semantiği.

Modül 3 — Zaman ve Pencereleme

Jun 20, 2026 · 8 min read · 0 claps #kafka-streams #event-time #stream-processing

SCI

PA Patryk Ziobroń

Stateful Order Fill Matching with Kafka Streams - Part 4: Horizontal Scaling, Partitions, and …

An experimental look at partition ownership, workload distribution, and horizontal scaling in Kafka Streams.

Jun 11, 2026 · 7 min read · 2 claps #kafka-streams #apache-kafka #java

TCH HUM ECO AI

MU Muhammad Naufal Dzakki Rauf

The Hardest Part of Stream Scoring Isn’t the Model

How rules, XGBoost, and anomaly detection combine into an auditable fraud decision system, and why the decision logic matters more than…

Jun 11, 2026 · 5 min read · 4 claps #data-engineering #fraud-detection #machine-learning

VE Vedant Thakkar

How Kafka Transactions Actually Work: A Walk Through the Two-Phase Commit

From a lost producer ack to atomic writes across two topics — what really happens under the hood.

Jun 09, 2026 · 7 min read · 0 claps #kafka #distributed-systems #software-engineering

TCH ECO

MU Muhammad Naufal Dzakki Rauf

Building a Fraud Pipeline: Why Ingestion Is Harder Than the Model

Three producers, one Kafka topic, a dead-letter queue, and why idempotent upserts quietly keep everything alive

Jun 08, 2026 · 5 min read · 0 claps #data-engineering #apache-kafka #stream-processing

GEN MKT

RE Renu Tewari

Musings on Data: Part 2

Renu Tewari

Jun 01, 2026 · 12 min read · 0 claps #stream-processing #data-lakehouse #logs

ART TCH DSN MDA

ST Stéphane Derosiaux · Conduktor

Kafka Partitions are the wrong ordering abstraction. Keys are.

Bye “Parallel Consumer” and Why the real Kafka unit of work was never the partition.

Jun 01, 2026 · 7 min read · 14 claps #apache-kafka #data-streaming #data-engineering

GEN

YA Yashash Lokesh · numaproj

Numaflow 1.8: Ordered Processing, Tracing, and a Smarter MonoVertex

Three things kept coming up in Numaflow issues and Slack threads this past cycle: “how do I keep my messages in order?”, “why is this one…

May 29, 2026 · 5 min read · 152 claps #numaflow #stream-processing

ART TCH MDA

JO Joe Thiongo

Understanding Apache Flink Architecture: How Distributed Stream Processing Works

Apache Flink is an open-source distributed stream processing framework engineered for stateful computation over both unbounded (streaming)…

May 24, 2026 · 4 min read · 3 claps #apache-flink-cluster #data-engineering #distributed-systems

ART TCH

SA Sayantan Kundu

A Practical Tour of Apache Flink: Architecture, Slots, and How It Stays Consistent

The previous post walked through the late-event problem: the bug class that quietly breaks naive sliding-window aggregations on Redis, and…

May 17, 2026 · 13 min read · 18 claps #apache-kafka #apache-flink #stream-processing

SA Sayantan Kundu

The Late-Event Problem: How Sliding Windows on Redis Break — and Why Stream Processors Like Flink…

It’s 2 AM. PagerDuty fires. A user moved ₹5 lakh through your platform in the last five minutes and your fraud alert never triggered.

May 17, 2026 · 9 min read · 33 claps #apache-flink #distributed-systems #stream-processing

MDA TCH

JA Jatin Gupta

Building pipeline-runner-streams: A Small Library for Serious Node.js Stream Pipelines

If you process files or streams in Node.js — NDJSON, logs, ETL jobs — you have probably used stream/promises.pipeline. It is solid. It is…

May 12, 2026 · 4 min read · 4 claps #software-engineering #nodejs #backend-development

TCH MDA

MA Manoj Kumar

️Batch vs Streaming: When to Use What

Not every system needs real-time. And not every delay is a problem.

May 03, 2026 · 4 min read · 1 claps #data-engineering #data-processing #batch-processing

TCH ECO ART SCI

RI RisingWave Labs · Towards Dev

Building a Polymarket-Style Prediction Engine with RisingWave

Polymarket and similar platforms have proved something simple and powerful: price behaves like probability. Traders want to bet on…

Apr 26, 2026 · 6 min read · 0 claps #stream-processing #data-engineering #sql

TCH MDA MKT

PI Pinjari Akbar

Real-Time Data Streaming with Azure Databricks: Mastering PySpark Structured Streaming and Delta…

Introduction: Why This Matters in Modern Data Engineering

Apr 26, 2026 · 4 min read · 0 claps #azure-databricks #pyspark #stream-processing

MDA

KA Katya Gorshkova

Hands-On with Flink — Part 5: Managing State

Previous parts:

Apr 23, 2026 · 7 min read · 8 claps #apache-flink #apache-kafka #stream-processing

ART TCH AI

CO Community contribution · Real-Time Data Evolution

Apache Flink Alternatives in 2026: When SQL Is Enough

Flink is powerful but demands Java expertise, operational overhead, and a separate serving database. These alternatives handle the common…

Apr 23, 2026 · 12 min read · 5 claps #data-engineering #system-architecture #apache-flink

ART MDA TCH

CO Community contribution · Real-Time Data Evolution

Kafka Streams vs RisingWave in 2026: When You Need a Database

Kafka Streams is a Java library for stateful stream processing. RisingWave is a streaming database with a SQL interface. They overlap in…

Apr 22, 2026 · 9 min read · 1 claps #data-engineering #apache-kafka #stream-processing

ART TCH MDA

CO Community contribution · Real-Time Data Evolution

Materialize Alternatives in 2026: Open Source Options for Streaming SQL

Materialize pioneered streaming SQL materialized views. In 2026, open source alternatives have caught up. In several dimensions, they have…

Apr 21, 2026 · 11 min read · 0 claps #data-engineering #stream-processing #data-architecture

TCH

GA Gavin J.

Unlocking Stream Processing: Message Brokers & Data in Motion | DDIA Chapter 11, Part 1

Welcome back to our journey through Martin Kleppmann’s masterpiece, Designing Data-Intensive Applications (DDIA). In Chapter 10, we…

Mar 18, 2026 · 2 min read · 0 claps #data-engineering #stream-processing #kafka

AI ART LIF

JO John Maguire

Your “Second Brain” Is Just a Filing Cabinet With a Search Bar

Why the hottest concept in AI productivity is already obsolete — and what comes next

Mar 13, 2026 · 12 min read · 0 claps #cognitive-architecture #multi-agent-systems #knowledge-graph-embedding

ART TCH MDA

AB Abhishek Bharti · Flipkart Tech Blog

High-Risk, High-Scale: Guaranteeing Ad Budget Precision at 1 Million Events/Second

Flipkart serves thousands of sponsored ads across various pages and with search queries. Advertisers are charged based on the impression…

Jan 22, 2026 · 6 min read · 11 claps #adtech #stream-processing #apache-flink

SPR MDA MKT

RA Rajat Amate

Event Stream Windows: Tumbling, Sliding, Session — A Deep Dive

I’ve spent the past few months working with streaming data at scale, and one thing became crystal clear: understanding windowing is the…

Jan 02, 2026 · 8 min read · 0 claps #stream-processing #windowing #large-scale-system

MO Modexa

Event-Driven Python: Exactly-Once with Faust + Kafka

A practical playbook for “effectively once” processing in Python — using Kafka transactions, idempotent keys, and Faust Tables without…

Nov 26, 2025 · 5 min read · 12 claps #kafka #python #faust

EN Enrico Piovesan · WebAssembly — WASM Radar

🛰️ Weekly WASM Watch: The Top 5 You Shouldn’t Miss This Week

Fresh projects, releases, and signals from the WASM world — WASM Radar, October 30, 2025

Oct 31, 2025 · 10 min read · 0 claps #webnn #wasix #component-model

TCH

MA Mayuresh K · Python in Plain English

Apache Flink for Python Developers: A Practical Introduction to Stream Processing

An introduction to Apache Flink’s stream processing capabilities using Python, covering core concepts, practical examples, and how it…

Oct 24, 2025 · 13 min read · 16 claps paywalled #python-programming #kafka-streams #stream-processing

AI MDA MKT

ED Edward Wright · VorTECHsa

AIS ingestion at Vortexa with Redpanda Connect

This blog talks about streaming data ingestion in a data analytics business, how standard tools can replace bespoke software and how the…

Oct 10, 2025 · 11 min read · 28 claps #stream-processing #ai #big-data

TCH MKT

AM Amila De Silva

Tuning an Azure Stream Analytics Pipeline: A Step-by-Step Guide

Step-by-step guidance to identify bottlenecks, tune Event Hubs and Stream Analytics Jobs, and scale reliably to high event rates.

Jul 17, 2025 · 11 min read · 20 claps #stream-processing #azure-event-hub #azure-stream-analytics

TCH MDA MKT

AM Amila De Silva

Scaling Stream Analytics Jobs: Identifying and Fixing Bottlenecks

Tips for scaling Streaming Analytics Jobs for Higher Throughput

May 11, 2025 · 11 min read · 10 claps #azure-stream-analytics #stream-processing #azure

TCH

TD TDengine

Stream Processing in Time-Series Databases | TDengine

Stream processing is a natural extension of the capabilities of a time-series database and offers new insights from real-time data…

Feb 25, 2025 · 5 min read · 0 claps #tdengine #stream-processing #timeseries

TCH MDA

PU Puripandasc Ai

Exploring ksqlDB: An In-Depth Guide to Stream Processing

Introduction

Feb 05, 2025 · 3 min read · 0 claps #streaming #ksql #kafka