#data-streaming

Hands-on with Flink — Part 8: Flink SQL in Application Mode

Previous parts:

I Built a Stream Processor and Accidentally Became a VP

I learned that clarity and accountability are what drove my success

Reaching Kafka Across VPCs is an Addressing Problem

The structural reason VPC peering collapses with Kafka and the solution that fixes it.

Your Kafka Infrastructure Just Became Optional

Five hops between an event and a queryable row. Snowflake says you only need one.

How to Rule Apache Kafka

Kafka is great at moving data. Terrible at knowing what should happen to it.

Flink restart strategies: Stop letting your Job loop forever

The default retry config will loop your job into oblivion. Here’s what to set instead.

How Modern Map Routing Engines Calculate Journeys in Milliseconds

Beyond Dijkstra: How OSRM and Valhalla Decouple Static Topology from Real-Time Traffic to Solve Global Pathfinding at Scale

PostgreSQL 怎麼邊查邊吐資料？用 Server-Side Cursor + FETCH 實作 data Streaming

什麼是 Streaming？

From Workflow to Streaming Pipeline: Modernizing a Legacy Query Engine in Kotlin

A few days ago I started a side project called Helianthus.

Your Kafka is encrypted but a stolen password still reads everything.

If one of your Kafka credential leaked today (a single producer or consumer password) what would the person holding it actually be able to…

Hands-On with Flink — Part 6: Calling LLMs from Flink

Previous parts:

The Real-Time Revolution: Why Blockchain Needs Data Stream Processing

Blockchain is an extremely data-driven technology because the primary function of a blockchain is to store, check, and coordinate…

Whats new in Apache Kafka 4.3

Hello Pals,

One Topic, Many Event Types: A Kafka Multiplexing Pattern with Confluent Flink

Introduction

The Kafka Industry Has an Ownership Problem.

A streaming platform nobody can govern will not become a streaming platform agents can operate on.

Kafka Partitions are the wrong ordering abstraction. Keys are.

Bye “Parallel Consumer” and Why the real Kafka unit of work was never the partition.

AI needs a Kafka Operations Platform. (2/2)

AI for cost optimization, security auditing, migration planning, and operational reasoning across Kafka environments.

The Architecture of Speed: Why Sportsbook Data Pipelines Dictate Profitability in 2026

The competitive landscape of 2026 has transformed B2B data services from a mere utility into a primary strategic asset for sportsbooks. Not…

Real-Time Streaming from Amazon Kinesis to Snowflake with Openflow

Introduction

The Data That Powers the World Is in Motion with Confluent

Confluent Platform for IBM Z & LinuxONE — and why it changes everything for the enterprise

The Real-Time Data Stack in 2026: Architecture, Tools, and Trade-offs

The modern data stack assumed batch. The real-time data stack assumes streams. Here is how the components fit together in 2026, which tools…

ClickHouse vs RisingWave in 2026: OLAP vs Streaming SQL

ClickHouse is the fastest OLAP database for historical analytics. RisingWave is the leading streaming SQL database for real-time data. They…

Best CDC Tools for Real-Time Analytics in 2026

Change data capture has split into two categories: pure CDC connectors that ship events to Kafka, and integrated CDC databases that…

Architecting Trust: The Philosophy and Patterns of Modern Data Platforms

Building a data platform is rarely just a technical challenge; it is a business imperative. In enterprise-scale environments, the gap…

Real-Time Feature Store in 2026: Beyond Batch ML Pipelines

Traditional feature stores (Feast, Tecton) were designed for batch ML training. AI agents in 2026 need features fresh to the second…

Streaming Log Data Across Networks: Building a Real-Time Log Pipeline in .NET and Blazor

A deep dive into how we built a chunked NDJSON streaming pipeline that moves API logs from a remote client application, through a relay…

When AI Meets Real-Time: Building Intelligent Data Streaming Pipelines with Apache Flink

The Problem With Batch Intelligence

Streaming Schema Changes: Why You Must Sync Schemas Before You Stream Data (and How to Do It…

In modern data architectures, streaming pipelines (CDC with Debezium, Kafka Connect, Flink, or custom Python replicators) are the backbone…

Real-Time Data Streaming for Live Video: A Decision-Maker’s Guide to Interactive Streaming…

The pervasive infusion of interactivity into live video streaming across every corner of cyberspace has opened a new era in video commerce…

Hi everyone,

We’re excited to share something we’ve been working on for a while — a simple, lightweight Kafka UI and observability tool that makes it…

System Design Learning Journey: Kafka Integration in GO

The concepts of System Design become much clearer when you see them in action

From Kafka Events to Real-Time Alerts: A Hands-On Guide with ksqlDB

Learn how to combine Kafka topics, AVRO schemas, and ksqlDB queries to detect suspicious patterns in real time for event monitoring.

The Great Confluence of Streaming

Governance, not infrastructure, is the bottleneck for AI and innovation. By enabling governed self-service and AI-ready data, proxies are…

Is Sparkplug B Essential for IIoT Data Streaming?

Let me start with the short answer: No, Sparkplug B is not a “must” in every IIoT data streaming project. But it’s also not a technology…

Hiç Apache Pulsar Denedin mi?

Apache Pulsar Nedir?

7 Dynamic Table Plays for Streaming AI Features

Turn Snowflake into a low-lag feature factory — without rewriting pipelines.

Real-Time Data Streaming at Scale: A Practical Guide to AWS Kinesis Streams

Enterprise applications have shifted from batch processing to real-time demands — fraud detection in milliseconds, instant recommendations…

Kafka® Use Case: Streamlining Real-Time Stock Data Sharing

Real-Time Data Streaming with Azure Stream Analytics:

Turn Your Batch ETL Pipeline into Real-Time Analytics with Azure Stream Analytics

Data Engineer on Azure — DP-203 [Part 12] — Data Streaming

This time we are going to look into data streaming, some techniques on how to handle it.

Data Engineer on Azure — DP-203 [Part 11] — Data Streaming

This time we are going to looking into data streaming, some technic on how to handle it.

Dissecting Data Streams: My Journey Using Python to Monitor and Analyze IoT Network Traffic

From passive packet sniffing to anomaly detection pipelines — a deep dive into building a full-fledged Python-based network traffic…

Notes from PyGrunn 2025: The View from the Stage (and the Audience)

Reflections on speaking at PyGrunn 2025: streaming beyond Java heavyweights, FastStream demos, and inspiration from fellow speakers

Exploring Pattern Recognition using MATCH_RECOGNIZE

by Charles Tan

How we migrated the current batch-processing log pipeline to a near-real-time streaming system

I’m Manh, Data Engineer from the Data Platform Team at Classi corp - an education service for teachers and students in Japan.

What is Kafka Streams?

Kafka Streams is a client library for building applications and microservices, where the input and output data are stored in an Apache…

Kafka Summit London 2024 Review: The main topics

Our review of Kafka Summit London 2024: Analytics, Self-Serve, SQL

How to stream data output from Large Language Models (LLMs): A Comprehensive Guide

The whirlwind of AI advancement surrounds us, presenting a rapid transformation. In a mere span of four months, our concerns shifted from a…

CoreDump: SQL — the love and hate story.

Monitor Tabby: Welcome to CoreDump, the one stop for techies. In today's episode Stewie, talks about his love and hate relationship with…

Lenses SQL Kafka stream processors scale at Kubernetes

What’s great about Lenses SQL engine for Apache Kafka is that not only is supported by an intuitive user interface, but also how it scales…