Video: Supercharge Your FlinkSQL Job: Use a Hybrid API Approach to Achieve Perfection (Hebrew)

Apache Flink’s SQL interface almost sounds like a dream: write standard SQL — get a full-blown streaming app. However, dreams and reality rarely align, and FlinkSQL achieved a questionable level of production readiness. In this session, we’re exploring the known upsides and downsides of FlinkSQL. From the enormous velocity and traction FlinkSQL can create in your organization, to sleepless nights of manual state interventions and unexpected data type inference. The good news is, there are ways to bypass obstacles and tap the most out of FlinkSQL’s benefits by using a Hybrid Flink API approach. We examine the architecture of a fault-tolerant FlinkSQL job (using DLQ), bypassing inherent FlinkSQL’s data-type limitations, and finally, showing how we can set up our job to handle a massive load. Finally, we’ll also share step-by-step code examples of how to achieve it.

About the speaker: Gal Krispel is a Data Platform Engineer at Riskified, specializing in streaming technologies such as Apache Kafka and Apache Flink. He focuses on building scalable, real-time data pipelines that power Riskified’s core products. Gal is particularly interested in making complex data architectures accessible and efficient across the organization. His work spans real-time analytics, event-driven design, and the seamless integration of stream processing into large-scale production systems.

[embed]