#etl

TCH AI

CD CData Software · CData Software

Connect S3, ADLS & GCS to ETL Tools: 6 Proven Ways

Most data engineering teams don’t struggle to store data in Amazon S3, Azure Data Lake Storage (ADLS), or Google Cloud Storage (GCS). These…

Jul 16, 2026 · 6 min read · 5 claps #etl #cloud-storage #data-engineering

TCH

HJ hjtc365

Learn Apache Airflow by Building a Real Data Pipeline: The Complete 14-Part Series

Stop watching tutorials that only teach isolated concepts.

Jul 11, 2026 · 3 min read · 0 claps #apache-airflow #data-engineering #etl

TCH

RI Ritza Editor · Tech Stackups

Getting started with Bruin

Most data teams end up with a stack that looks like this:

Jul 10, 2026 · 10 min read · 0 claps #data-engineering #etl #sql

TCH

VI vishnu.m

AWS for Data Engineering, Explained Like You’re Not an Engineer (Yet)

Why did every company suddenly move to the cloud, and what does AWS actually do for a data engineer? Let’s break it down with real-world…

Jul 09, 2026 · 8 min read · 0 claps #aws #etl #aws-ec2

GEN TCH MDA

ME Median Talenta Raya

Proses ETL (Extract, Transform, Load) dalam Pembangunan Data Warehouse

Dilansir dari laman sis.binus.ac.id, Data Warehouse merupakan suatu sistem yang digunakan untuk melakukan penyimpanan, pengelolaan, dan…

Jul 05, 2026 · 5 min read · 0 claps #etl #talend-open-studio #data

TCH AI

TE TechLTE World

Why Does LTE Need the Random Access Procedure?

Every time your smartphone powers on, moves into a new cell, or reconnects after losing coverage, it must first establish communication…

Jun 29, 2026 · 3 min read · 0 claps #etl #5g #telecommunication

TCH MKT

QA Qaiser Anoosh Mughal

Building a Cloud-Ready ETL Pipeline with Python: From Raw Data to Analytics-Ready Insights

In the modern data-driven landscape, organizations generate vast amounts of information from disparate sources. Before this data can be…

Jun 26, 2026 · 9 min read · 0 claps #data-engineering #etl #python

TCH

BH Bhavana Siddappa Hemanth · Deloitte UK Engineering Blog

Data Transformation: A Deep Dive into DBT

In today’s data-driven landscape, it is important to turn raw data into reliable, useful information. As Deloitte practitioners, we are…

Jun 23, 2026 · 13 min read · 7 claps #dbt #etl #dbt-core

TCH MDA

NO Norma Mg

Ejemplo de Optimización de consultas SQL transformando un modelo Relacional a Dimensional

Introducción

Jun 23, 2026 · 4 min read · 0 claps #olap #etl #kpi-dashboard

TCH

CD CData Software · CData Software

Reduce Veeva Vault to Amazon S3 Sync Latency

If you have ever run data pipelines between Veeva Vault and Amazon S3, data latency is a common challenge. Vault defaults to a single daily…

Jun 22, 2026 · 6 min read · 0 claps #veeva-vault #etl #aws

TCH AI SCI

SA Saddam Hussain · Towards Data Engineering

From Transactions to Insights: Understanding OLTP, OLAP, and the Data Pipelines Behind Them

In my previous articles, I talked about data, where it comes from, why organizations collect it, and where it gets stored. I also discussed…

Jun 21, 2026 · 8 min read · 0 claps #data-engineering #data-science #database

TCH

HA Hafiq Iqmal · Towards Data Engineering

The 100 Million Row Question: Designing a ~50 GB ETL Pipeline Into PostgreSQL

A practitioner walkthrough of a Google interview prompt, treated as the extract, transform and load problem it really is, with runnable…

Jun 20, 2026 · 20 min read · 0 claps paywalled #postgresql #etl #big-data

AI TCH HUM DSN

VA Valentin Podkamennyi · Data Science Collective

Modern AI Implementation Requires Standard ETL Engineering

Why treating embedding pipelines as standard data infrastructure is essential for moving AI prototypes into reliable production environments

Jun 19, 2026 · 5 min read · 151 claps paywalled #ai #data-engineering #etl

AI TCH

ZA Zahir Mohideen

Strange issue with Informatica Secure Agent Installation

Most of the time , copy and paste is a life saver. Sometimes , this could get tricky.

Jun 19, 2026 · 2 min read · 0 claps #iic #informatica #agent-installation

AI ART TCH

CD CData Software · CData Software

Batch Integration vs. Real-Time Integration: Pros, Cons, and AI Impacts

Every enterprise data pipeline reflects a choice, whether deliberate or inherited: how quickly does data need to move from where it lives…

Jun 19, 2026 · 8 min read · 0 claps #data-integration #real-time-data #artificial-intelligence

TCH SPT

AG AGirlCalledChi

The evolution of data engineering and the role of ELT tools

Evolution is peculiar to nature, and as such, the field of data engineering has also continued to evolve over the years; we first started…

Jun 18, 2026 · 5 min read · 2 claps #etl #data-engineering #airbyte

TCH

AN Ancybjohn

What I Learned About Azure Data Factory (ADF): A Beginner’s Technical Guide

A breakdown of Microsoft’s cloud data integration service, explained simply, without the jargon overload.

Jun 18, 2026 · 6 min read · 0 claps #aure #azure-data-factory #data-engineering

TCH MKT

BL BladePipe

Before You Pick a Data Pipeline Tool in 2026, Read This

I work on BladePipe now, but before that I spent years building and maintaining data infrastructure across analytics, change data capture…

Jun 18, 2026 · 10 min read · 3 claps #data-pipeline #data-engineering #etl

AI TCH DSN

TI Titus Yory

Building a Real-Time ETL Monitoring Prototype with SSIS, Python, and Gemini AI

Introduction

Jun 18, 2026 · 8 min read · 2 claps #python #ai #sql-server

AI ECO TCH LIF MKT

DA Data Trends by Kleene.ai

Do you spend all day babysitting AI? 🍼

The productivity numbers look great on paper. The reality is that most of those saved hours are going straight back into feeding, checking…

Jun 17, 2026 · 4 min read · 10 claps #etl #data-management #ai

TCH

AP Apeksha Telecom

Private 5G Network Training 2026: Complete Hands-On Course for Telecom Engineers — Your Career…

Introduction Private 5G Network Training 2026

Jun 17, 2026 · 16 min read · 0 claps #5g-technology #5g #5g-network

TCH MKT

MY MyData Insights

CRM Integration: SAP, Dynamics 365 & the Operations Gap

Sales books the order in CRM. Operations finds it three days later in SAP after a manual re-key. By then the customer has changed delivery…

Jun 17, 2026 · 8 min read · 0 claps #sales-integration #crm-integration #etl-pipeline

TCH

VE Vembry Fiska

Extracting NetSuite Data via REST API: Why Your ETL Strategy Matters More Than Your Code

A practical guide to date range strategies, chunk sizing, and load patterns — with real Python examples built on SuiteQL and OAuth 1.0a

Jun 16, 2026 · 16 min read · 0 claps #netsuite #python #data-engineering

TCH

CD CData Software · CData Software

10 Steps to Build a Reliable Odoo-to-Snowflake ETL Pipeline

Customer data continues to move fast, and the business decisions backed by that data shouldn’t fall behind. Organizations commonly rely on…

Jun 15, 2026 · 6 min read · 0 claps #odoo #snowflake #etl

TCH

KR Krushikreddynallamilli

Snowflake’s Most Underrated Operator: Using IS NOT DISTINCT FROM for Reliable Insurance Data…

When people talk about Snowflake, the conversation usually revolves around Dynamic Tables, Snowpark, Streams, Tasks, or query performance…

Jun 15, 2026 · 4 min read · 0 claps #snowflake #sql #data-engineering

TCH LIF

MA Mark Anderson

Why StarRocks Is Better Than DuckDB for ETL and Data Transformation Pipelines

DuckDB has quietly become one of the most loved tools in the modern data stack, and nowhere more so than in transformation. The dbt-duckdb…

Jun 14, 2026 · 9 min read · 0 claps #starrocks #duckdb #etl

TCH

AN Andreretorta

Construindo um Pipeline de Dados ETL com Pentaho PDI e Modelagem Dimensional

Da extração de arquivos CSV/XLSX e de um sistema OLTP em PostgreSQL à carga de um Data Mart de Vendas em Star Schema — um projeto completo…

Jun 14, 2026 · 37 min read · 0 claps #pentaho #etl #data-warehouse

TCH

ZA Zahir Mohideen

Enhancement in Direct Path Insert — Oracle 23 ai Feature

This is one another nice feature introduced in 23ai.

Jun 14, 2026 · 1 min read · 0 claps #oracle-database #etl #sql

TCH

DW Dwicky Feri

Meet Quilt: Local Visual ETL That Compiles to SQL

Background

Jun 14, 2026 · 4 min read · 0 claps #etl #duckdb #workflow

TCH MDA

AV Avinash Thakur

NDJSON Explained: The JSON Format Built for Big Data and Streaming

If you’ve worked with APIs, log processing, Apache Spark, Kafka, Elasticsearch, or large-scale ETL pipelines, you’ve probably encountered…

Jun 14, 2026 · 4 min read · 1 claps #data-engineering #big-data #json

TCH HUM

TA tanmay bhurkhunde

Building My First ETL Pipeline Changed How I Think About Data Engineering

When I first started learning Data Engineering, ETL pipelines seemed fairly straightforward.

Jun 13, 2026 · 4 min read · 0 claps #data-engineering #google-cloud-platform #etl

ART TCH

AK Akshay

Building Production-Grade Data Engineering Systems: A Reference Architecture Series

Part 1 of 5 | Series: Reference Architecture for Production Data Engineering

Jun 11, 2026 · 4 min read · 4 claps #data-engineering #spark #apache-flink

TCH HUM AI SCI

FA Fahimkiyalah

Why data cleaning matters more than most beginners think

When I started learning Data Science and Data Engineering, I often found myself asking the same question:

Jun 11, 2026 · 2 min read · 3 claps #data-science #data-engineering #etl

TCH

RA Rahul Kumar

Mastering Modern MuleSoft Batch Processing (Part 2): Under the Hood & Engine Configuration

In Part 1, we established why standard synchronous loops like For Each crumble under the weight of massive datasets and why the Batch Job…

Jun 10, 2026 · 4 min read · 0 claps #mulesoft #batch-processing #etl

AI MKT TCH

DA Data Trends by Kleene.ai

Morgan Stanley bets big on AI agents 🤖

Banks are discovering that their AI ambitions are only as good as the data underneath them. This edition we look at why 80% of banks are…

Jun 10, 2026 · 4 min read · 4 claps #ai #agentic-ai #predictive-analytics

ART TCH SPT

CD CData Software · CData Software

The Complete Guide to Configuring SQL Server Replication

Replication is a mechanism that automatically propagates changes from a source database to a destination database, ensuring every system…

Jun 10, 2026 · 7 min read · 0 claps #sql-server #data-replication #data-engineering

ART TCH

MI Mihitha Bandara

Designing an ETL Application: Why I Started with a Modular Monolith Before Microservices

What I learned while building the first version of an ETL application, planning the project structure, and thinking about future…

Jun 10, 2026 · 16 min read · 3 claps #etl #data-engineering #software-architecture

TCH MKT AI SCI

KA kalkidan Gebrehiwot

Data Engineering Fundamentals: ETL vs. ELT

Understanding how data moves through an organization is a core competency for any analyst or engineer. At the center of this, there are two…

Jun 10, 2026 · 2 min read · 0 claps #etl #data-science #analytics

TCH HUM

CH Chanithi Rasnakawewa

Understanding a Real-World Batch Data Pipeline

When I first started learning Data Engineering concepts, I thought a data pipeline was something simply moving data from one place to…

Jun 09, 2026 · 4 min read · 2 claps #data-pipeline #batch-processing #airflow

TCH MKT SPT

VE VeloDB, Powered by Apache Doris

Apache Doris 4.1 Spill to Disk: Running Memory-Intensive Queries Without OOM

Picture an analytics team running its end-of-quarter close. The ETL job joins three years of transaction history against a customer…

Jun 09, 2026 · 12 min read · 0 claps #apachedoris #database #olap

TCH ECO

SH Shivam Porwal

Lakeflow Connect: Managed Ingestion Without the Pipeline Tax

The pipeline tax most teams pay quietly

Jun 09, 2026 · 9 min read · 5 claps #databricks #data-engineering #data-pipeline

TCH AI

LA Lakmal Deshapriya

How do I design a secure Data/File transfer platform as a SaaS application?

After about three and a half years of effort, I have just submitted my PhD thesis titled “Context-aware Classification of Static Analysis…

Jun 09, 2026 · 7 min read · 4 claps #system-design-interview #system-design-concepts #etl

TCH

LE Leo Leon

You’re Moving Data You Never Had to Move.

Five Snowflake patterns that quietly kill your external ETL — and the gotchas nobody warns you about.

Jun 08, 2026 · 4 min read · 0 claps paywalled #snowflake #data-engineering #etl

TCH

CH Chanithi Rasnakawewa

Dockerizing a Batch Data Pipeline: What I Learned by Building One

When I first started working with Docker, it felt like one of the more challenging tools to understand. While I could execute Docker…

Jun 08, 2026 · 3 min read · 1 claps #docker #containers #containerization

TCH

CH Chanithi Rasnakawewa

Understanding Apache Airflow Through a Real Batch Data Pipeline

When I first started my internship, Apache Airflow was one of the tools that seemed complicated from the outside. I understood Python…

Jun 08, 2026 · 4 min read · 0 claps #apache-airflow #orchestration #data-pipeline

TCH

DH Dhaarini

DBT+duckDB

# Getting Started with dbt + DuckDB: A Practical Guide for Modern Data Analysts

Jun 08, 2026 · 5 min read · 0 claps #dbt #duckdb #sql

TCH

MA Maaz Ghazanfar

How to Build Clean ETL Pipelines in Node.js with etlkit

Stop writing messy data scripts. Here’s a better way.

Jun 07, 2026 · 3 min read · 4 claps #nodejs #etl #data-engineering

SCI TCH LIF

IG Ignacio Gaete

The biggest telescope on the planet: A tour of the ELT and Paranal Observatory

As a resident of Antofagasta, Chile, I have a very close relationship with the Atacama Desert. It is the driest non-polar desert in the…

Jun 04, 2026 · 2 min read · 0 claps #etl #telescope #alma-telescope

TCH

TE TechLTE World

RACH Failure Causes in LTE and 5G: Understanding the Root Causes of Random Access Failures

Introduction

Jun 03, 2026 · 2 min read · 0 claps #rach #etl #5g

ART TCH

PO Poorva Patil · helpshift-engineering

Migrating from a Monolithic Orchestrator to Apache Airflow

Before we knew better

Jun 03, 2026 · 6 min read · 9 claps #apache-airflow #data-engineering #software-architecture