How I Integrated AI Into My Data Journey — And Stopped Worrying It Would Replace Me

The Tool Has Changed. The Motivation Hasn’t. Curiosity is still the only thing that matters

Not a Medium member, no worry, you still can learn from here.

When I saw Claude Code could self-debug accurately, I was surprised but worried. Working in a field that changes this fast, I sometimes ask myself: Will I be replaced someday?

Instead of worrying about it, I decided to see if I can integrate with AI to make myself more competative.

I started exploring how to collaborate with AI -using it as a learning partner rather than a shortcut. What I found is that it genuinely accelerated my growth as a data engineer, in ways I hadn’t expected, and I want to share these 5 strategies here.

If you’re a data analyst, analytical engineer, or data scientist, the same approaches apply.

1. AI can help me define learning priority

When I first started learning data engineering, I watched 30–50 YouTube videos and read many articles to understand what I needed to know to become a data engineer. The knowledge list I should pick up is pretty long. I need to know data modelling, ETL/ELT patterns, orchestration, cloud warehouses, batch processing, streaming, Spark, dbt, Airflow, Redshift, BigQuery, Databricks.

I felt overwhelmed — and honestly, I questioned whether I was even capable of making this transition.

AI won’t replace us

Thus, I turned to AI and saw if it could help me prioritize.

I gave AI a direct prompt: “I’m transitioning from data analysis into data engineering. I have three years of SQL experience and know Python. What should I realistically learn first, and what can I wait on? Be honest about the timeline.”

The answer was clear: start with data modelling concepts and ETL/ELT patterns. These are the foundational skills that almost every data engineering role actually needs.

Streaming and distributed computing like Spark?

I’d need them eventually — but many companies don’t require junior engineers to know them on day one.

That was practical guidance YouTube never gave me, because YouTube rewards the flashiest topics, not the most foundational ones.

I also validated this when I started integrating data engineering work into my day-to-day. Companies still need structured data for reporting. They still need people who understand how to normalize and denormalize it. That’s where a data engineer actually earns their place.

2. Use analogies to break open abstract concepts

I found that the terminology in data engineering is sometimes abstract.

For example, when I first encountered ELT, I couldn’t picture what was actually happening. So I asked AI: “Could you explain ELT with a daily-life analogy that aligns with how it actually works?”

Photo by Bernd 📷 Dittrich on Unsplash

AI described it like a busy shipyard. Hundreds of ships arrive daily — there’s no time to inspect and sort every container before unloading. So you get everything off the ship first, into the warehouse, and sort it later. That’s ELT: load everything into cheap, large storage first, then transform it once it’s safely there.

The analogy isn’t a perfect one-to-one match with the technical definition — but it is like an ice-breaker, where I could go deeper.

3. Map new tools against what I already know

When I started exploring cloud warehouses, I kept hearing about AWS Redshift and Google BigQuery. I’d been using DuckDB for local work, and I knew all three were data warehouses — but I didn’t know the difference between them. So I asked AI:

Compare DuckDB, BigQuery, and AWS Redshift — 
the syntax differences for basic CRUD operations, 
the different use cases, 
and a real-world reason you'd choose each one 
based on team size, data volume, and client needs.

That strategy helps me map the new tool against what I already know. It helps me absorb the differences quickly rather than reading three separate documentation pages in isolation. I may need to spend 2 or 3 days to understand the cloud data warehouse without the help of AI, but now it cuts down to a few hours when I link similar concepts to the tool I am familiar with.

from confused to clear

Another example is about Terraform. There are three common files in the Terraform setup: main.tf, outputs.tf, and variables.tf. I felt scared because the files looked complicated.

What I did was to ask AI what the equivalent files are in Python.

AI told me variables.tf is like config.py and main.tf is the main.py to execute our scripts, and outputs.py is like return or print.

Suddenly, I didn’t feel Terraform was difficult and gained more confidence in learning it.

Photo by Austin Distel on Unsplash

Understanding what the industry actually looks like

Portfolio projects teach you the technical skills. What they don’t teach you is how companies actually make decisions.

One habit I developed was asking AI how different types of organisations approach the same problem. When I built a pipeline, I’d ask:

How does the data modelling approach differ between a large tech company, a government agency, and a small startup? What constraints drive those differences?

In a real data team of 10–20 people, how would the work I’m doing get divided? Where would my pipeline fit?

These questions changed how I think about tool choices. In a real team, picking a tool isn’t purely a technical decision — it’s about team capacity, compliance requirements, budget, and what the business actually needs now versus in two years. Portfolio projects alone don’t teach me to think that way.

5. Building projects without letting AI do the thinking

After building up foundational knowledge, the most valuable step is building something. I mention this last deliberately: without some vocabulary, we won’t be able to ask AI the right questions — and it is difficult for us to evaluate whether its answers are actually correct.

Before writing any code, I usually have a long planning conversation with AI.

For example, when I built a DBT project, I asked:

• What do I need to know before I start?
• What loading and transformation strategies should I consider?
• How should I structure the project — staging, marts, and so on?
• How do I apply DBT tests to make the project more professional?

That conversation gave me a starting point and the vocabulary to make better decisions as I built, and it also provided me a great opportunity for architectural design.

Photo by Amsterdam City Archives on Unsplash

But building a project won’t be as smooth as I thought; there is always something broken, and I viewed it as my real learning time.

At one point, I was struggling with DBT’s naming — I didn’t understand the difference between sources.yml, profiles.yml, and dbt_project.yml. I just copied and pasted what AI returned to me into my project, but when I tried to update mydbt naming into retail_dbt, everything crashed.

I even accidentally overwrote a config that referenced the old name in multiple places.

I spent 20 hours figuring out what I did wrong, but that crash forced me to actually understand what each file does and how those names actually mean in the data warehouse.

I went back to AI — not to ask it to fix the error, but to ask it to explain what each file controls and why changing one name broke another.

That conversation taught me more than the three hours of tutorials that preceded it.

When errors come up, I don’t ask AI to fix them. I ask it to explain where the change is and why.

Copying a working fix without understanding it doesn’t improve your skills. Understanding why something failed is where the real knowledge gets built.

Why I’m not worried about replacement

After all of this, I’ve come to a clear conclusion: AI won’t take our jobs — and it’s worth continuing to invest in levelling up.

What AI can’t do is understand the specific constraints of your organisation, or ask the right questions when a stakeholder can’t fully articulate what the business actually needs.

Translate a vague requirement into a concrete data solution requires understanding the business context, the team’s capabilities, the existing infrastructure, and the tradeoffs involved.

You can’t get that from a chatbot. It comes from real experience and real failure.

The gap in experience between a senior data engineer and someone earlier in their journey won’t disappear suddenly with AI.

AI won’t replace you

A key mindset for your data career — and beyond

Before AI, this industry was already changing constantly. People were talking about Hadoop; now the conversation is Databricks.

The uncertainty we feel today isn’t new — it’s a permanent feature of working in technology. The tools will change again. What you’ve invested in understanding won’t disappear. It transfers.

The engineers who get replaced won’t be the ones who used AI. They’ll be the ones who stopped being curious about what they didn’t yet understand.

Thanks for reading until here.

If you like this article and want to show some love:

• Clap 50 times — each one helps more than you think! 👏
• **Follow me**, so you won’t miss it when a new article is published
• You can buy m**e a Coffee** to support me further.
• Let’s connect with me at **LinkedIn or lhungen@gmail.com to chat more about data!**