Meet DAGenie: Purpose-Built for DAGs. Engineered for Speed.

In the world of data-driven applications, DAGs (Directed Acyclic Graphs) are everywhere — task execution engines, workflow orchestrators, ETL pipelines, CI/CD systems — you name it.

Yet, when it comes to efficiently storing, querying, and managing DAG data, most systems rely on general-purpose databases not tailored for the unique characteristics of DAGs.

That’s why I built DAGenie.

What is DAGenie?

DAGenie is a lightweight, disk-persistent database designed from the ground up to work with DAG data. It is:

• Blazing-fast.
• Equipped with a SQL-like query language for DAG-centric queries.
• Operated through a sleek command-line interface (CLI) for both server and client.
• Extensible for real-time DAG analytics, scheduling, and more.

DQL: DAG Query Language

DAGenie comes with its own intuitive DQL syntax — a familiar, SQL-like language — but with optimizations tailored for DAGs.

Examples of DQL Queries:

🗂️ Basic SELECT:

SELECT * FROM dag WHERE dagid='dag1';
SELECT id, name, status FROM dag WHERE status='pending' ORDER BY status ASC LIMIT 5;

🧮 Aggregates:

SELECT COUNT(*) FROM dag WHERE status='completed';
SELECT AVG(duration), MAX(retries) FROM dag WHERE dagid='dag42';

📊 Group By:

SELECT status, COUNT(*) FROM dag GROUP BY status;

✍️ Mutations:

INSERT INTO dag (dagid, id, name, payload, status)
VALUES ('dag1', '1', 'FetchData', '{"url":"api"}', 'pending');

UPDATE dag SET status='completed' WHERE id='1' AND dagid='dag1';

DELETE FROM dag WHERE status='failed';

DAGenie supports ORDER BY, GROUP BY, LIMIT, and even COUNT, SUM, AVG, MIN, MAX for numeric fields like duration, retries, etc.

CLI Magic: Server + Client

DAGenie offers both server and client CLIs for seamless interaction.

Start the TCP Server

dagenie serve --port 9090

DAGenie TCP server running at :9090

Connect with the TCP Client

dagenie connect --host localhost --port 9090

Connected to localhost:9090
DQL> SELECT * FROM dag WHERE dagid='dag1';

Features:

• 🟩 Auto-completion for SQL commands
• 🟩 Multi-line query support ending with ;
• 🟩 Command history via arrow keys
• 🟩 Results displayed in colorful tables (white borders, green text)
• 🟩 Handles missing responses gracefully

Under the Hood: Technical Highlights

• Language: Go (Golang)
• Storage Engine: Disk-based, low-latency key-value store
• Architecture: Modular Parser → AST → Executor
• Performance: Optimized for fast DAG scans, conditionals, and aggregations
• CLI: Built with cobra, readline, and tablewriter for a rich UX

Target Use Cases

• Workflow orchestration systems
• DAG-based analytics pipelines
• Task scheduling engines
• Custom DAG modeling tools

What’s Next?

• REST + gRPC APIs
• DAG visualizations (Web UI)
• Integration with Prefect, Airflow, etc.
• Query planner optimizations for large DAGs
• SDKs in Go, Java, Python, C++, .NET, Rust etc.

Open Source Coming Soon

I built DAGenie to empower developers and data engineers who need speed and precision when working with DAGs. The project will soon be available on GitHub, and I’m looking for collaborators, feedback, and contributors!

Join the DAG revolution.

DAGenie — Purpose-Built for DAGs. Engineered for Speed.

Stay tuned for the public release, and feel free to connect if you’d like early access or want to collaborate!