I Built an AI-Powered ETL Pipeline That Tells Ethiopian Farmers Exactly When to Harvest

How I combined real weather data, crop-specific scoring logic, and Groq AI to solve a problem that costs smallholder farmers their income every single season.

The Problem Nobody Is Talking About

Across Ethiopia and East Africa, millions of smallholder farmers face the same silent crisis every harvest season.

They have no data-driven tool to tell them when exactly to harvest their specific crop.

So they guess. They check weather apps built for city commuters. They follow generic advice that was never designed for maize, or wheat, or Ethiopian coffee. And the consequences are brutal:

• Harvest too early — your crop weighs less, quality drops, income shrinks
• Harvest too late — unexpected rain or humidity destroys weeks of hard work
• Use generic advice — wrong for your specific crop entirely
• Keep no records — same costly mistakes repeated every season with no way to improve

This is not a technology problem. It is an information problem. And information problems are exactly what engineers are supposed to solve.

So I built a solution.

What I Built

A fully automated, crop-aware ETL pipeline that runs every Monday at 06:00 without any human input. It fetches real weather data, scores harvest readiness from 0 to 100 using rules specific to each crop, gets AI-generated farming advice tailored to that exact crop and region, and saves everything to structured files for trend tracking over time.

Five crops supported: Maize, Wheat, Rice, Coffee, Sorghum.

One line to switch between them.

Zero cost to run.

The Architecture — How It Actually Works

The pipeline follows a classic ETL pattern — Extract, Transform, Load — with a scheduler sitting on top. Here is what each stage does and why it matters.

Extract — Fetch Real Weather Data

Every run starts by pulling a live 7-day weather forecast from the Open-Meteo API for any Ethiopian farming location. The coordinates are stored in a .env file — change the latitude and longitude once and the pipeline works anywhere in the region instantly.

What comes back from the API:

• Maximum daily temperature for 7 days
• Total daily rainfall for 7 days
• Maximum daily humidity for 7 days

No API key required. No signup. Completely free. This was a deliberate choice — the farmers this tool serves should never be blocked by a paywall.

Transform Part 1 — Crop-Specific Scoring

This is the core intelligence of the system, and it is what makes this project genuinely different from a weather app.

Every crop has its own ideal harvest conditions stored in a file called crop_config.py. Each crop entry contains a temperature range, a maximum rainfall threshold, and a humidity window. The scoring works like this:

• Temperature in ideal range → +40 points
• Rainfall below threshold → +30 points
• Humidity in ideal range → +30 points
• Maximum possible score → 100 points

Here is why this matters. Take the same weather conditions — 28°C, 0.5mm rain, 75% humidity — and run it through two different crops:

Maize rules: 28°C is within range ✅ +40, rain is fine ✅ +30, humidity is borderline ⚠️ = Score: 85/100 — harvest now

Wheat rules: 28°C is too hot ❌ +0, rain is fine ✅ +30, humidity is too high ❌ +0 = Score: 30/100 — do not harvest this week

Same weather. Completely different harvest readiness. This is why generic advice fails farmers.

Switching crops requires changing exactly one line in the .env file:

CROP=maize ← change this to wheat, rice, coffee, or sorghum

Every other file in the pipeline stays untouched.

Transform Part 2 — Groq AI Farming Advice

Once the score is calculated, the pipeline sends the crop name, score, weather averages, and crop-specific ideal ranges to Groq AI running LLaMA3–70B. The model returns 3 to 4 sentences of plain-language advice tailored to that exact crop and location.

Not generic advice. Not boilerplate. Advice that references the actual conditions, the actual score, and the actual crop — written in language a farmer can act on immediately.

Example output for maize at 85/100:

“Conditions are excellent for maize harvest this week in Addis Ababa. Temperatures are well within the ideal range and rainfall is minimal — your maize should be dry and ready to pick. Aim to harvest Tuesday through Thursday before humidity rises slightly toward the weekend. Store in a dry ventilated location away from direct ground contact.”

Load — Save Everything to Disk

Every pipeline run saves results in two formats:

weekly_scores.csv — one new row per run containing the date, city, crop, score, weather averages, and AI advice. This file grows week by week and becomes a historical record farmers and advisors can use to spot seasonal patterns.

2026-W15_maize.json — a full detailed report for that specific week and crop, containing the score, interpretation label, all weather data, and the complete AI advice paragraph. Named by week number and crop so every run produces its own traceable file.

Scheduler — Runs Itself Every Week

One command starts the scheduler. It fires the full pipeline immediately on startup for testing, then waits and fires again automatically every Monday at 06:00. No cloud infrastructure. No cron job configuration. Leave the terminal open and walk away.

The Design Decision I Am Most Proud Of

Separating configuration from logic.

crop_config.py holds all the crop-specific rules. transform.py reads from it. pipeline.py orchestrates everything. None of these files know about each other's internals.

This means:

• Adding a sixth crop takes 10 lines in one file
• Changing a scoring rule touches zero pipeline logic
• Switching regions requires editing three values in .env
• Every stage can be tested independently

This is not just clean code. It is the difference between a script and a system.

What This Project Taught Me

How to design a pipeline where each stage has exactly one responsibility and hands clean structured data to the next stage. How to make a system fail loudly and clearly when something breaks — because silent failures in automated systems are the most dangerous kind. How to think about configuration as a first-class concern, not an afterthought.

And most importantly — how to build something that solves a real problem for people who genuinely need it.

What Is Next

• A simple web dashboard so farmers can see their score without reading JSON files
• SMS delivery of the weekly advice using Africa’s Talking API
• Expanding to more East African crops — teff, cassava, sweet potato
• Multi-location support so advisors can monitor several farms at once

Final Thought

The best engineering projects are not the most technically complex ones. They are the ones where the gap between the problem and the solution is obvious once you see it — and you wonder why nobody built it sooner.

This is one of those projects.

If you are working on AgriTech, data tools for emerging markets, or AI applications with real-world impact — I would love to connect.

Built with Python, Open-Meteo, and Groq AI. Designed for Ethiopian smallholder farmers.

Tags: #Python #ETL #DataEngineering #AgriTech #AI #Africa #GroqAI #LLaMA3 #OpenSource