The Question “What Is ADM?” Changed My Path

A simple TOGAF question led me to develop an identity layer instead of a retrieval layer in my AI system.

[If you cannot access the full article, please click here]

Image generated using AI (ChatGPT) by the author.

Hello For the past few months, I’ve been working on my own local AI system and sharing my notes and progress on it.

My initial goals were:

To add my EPUB books to the system To index my Medium posts To upload my PDF documents To ask questions and get answers

Over time, the system has grown with the notes I’ve added, and I’m still adding more.

Naturally, the first thing that came to my mind was:

I need to improve my retrieval.

Like in many RAG projects, I was spending my time on the following topics:

Embedding models Chunk structures Reranking mechanisms Vector database optimizations Search quality

Whenever the system gave a poor response, my reflex was always the same:

“I need to improve the retrieval side.”

For a while, it actually worked.

But then a different problem arose.

It All Started with a TOGAF Search

One evening, I did a TOGAF search on my Telegram app.

/ara togaf # Search

The system returned 11 different PDFs.

1. An Introduction to the TOGAF Standard 2. Guide to Enterprise IT Architecture 3. TOGAF Pocket Guide …

I selected the first document.

/detay 1 #detail

The system correctly displayed the document ID.

Then I generated a summary.

/ozet #summary

Everything looked normal.

Then I asked a very simple question.

/soru ADM nedir #question What is ADM?

That’s when I realized what was missing.

The Problem Isn’t Retrieval — It’s Identity

For a human, this question is crystal clear:

What is ADM?

Because the human knows this question pertains to the TOGAF document they just selected.

However, many traditional RAG systems don’t know this.

They resubmit the question to the entire information universe.

They run a new retrieval.

It performs a new search.

Technically, this seems correct.

From the user’s perspective, however, the context is lost.

This is where the concept of the “Identity Layer” emerged.

Identity Layer

The system now needed to remember the following:

Which PDF was selected?
Which book is being reviewed?
Which Medium article is being read?
Which information asset is the user currently working on?

That’s why I changed the architecture.

Old structure:

Question
↓
Retrieval
↓
Answer

New structure:

Search
↓
Select
↓
Remember Identity
↓
Generate Summary
↓
Ask a Question

Here's how the workflow works now:

/search togaf
↓
/detail 1
↓
/summary
↓
/ask What is ADM?

This small change completely transformed the user experience.

Because the system no longer forgets what I’m working on.

Another Unexpected Problem Arose

As the number of commands increased, another problem emerged.

At first, there was only:

/ara #search

Later, the following were added:

/detay, /ozet , /sor, /pdfara,  /pdfdetay,  /pdfozet,  /mediumara,  /mediumozet
#/details
#/summary
#/ask
#/pdfsearch
#/pdfdetails
#/pdf-summary
#/mediumsearch
#/medium-summary

At a certain point, even I started getting the commands mixed up.

For this reason, I improved the system’s initial usability layer.

I added the

/yardim #help

command.

Users can now view all commands and sample usage flows whenever they want.

For example:
/ara  togaf #search togaf
/detay 1 #detail
/ozet   #summary
/sor #ask What is ADM?

Typical scenarios like these are available directly on the help screen.

This feature, which may seem minor, has become quite important as the system has grown.

Viewing the Evolution of the Architecture in the Commit History

I tracked all the changes I made during this process in Git.

For example:

042d10a feat: add universal summary engine v1 skeleton 
ca0b141 refactor: route selected pdf summary through universal summary engine v1 
dcaef9e feat: add selected pdf qa v1 
818072d feat: add sor command for selected entity qa 
f5907bc feat: add yardim command reference

These commits actually represent a much larger transformation.

The system is no longer just a retrieval engine.

It is gradually evolving into a Knowledge Operating System.

What’s Next?

My next goals:

Universal Summary Engine
↓
EPUB integration
↓
Medium integration
↓
Unified Question Answering
I've also added a new sprint to my roadmap:
Repository Governance V1
- GitHub repository structure
- README
- CHANGELOG
- ROADMAP
- Architectural documentation

Because as the system grows, governance becomes just as important as the code.

Conclusion

For a long time, I thought that better retrieval would solve every problem.

Today, I think differently :) based on my own experiments.

Retrieval finds the information but cannot preserve the context.

True value often emerges from preserving the context.

Retrieval finds the information. The identity layer preserves the context. And more often than not, what makes an answer valuable is the context rather than the information itself.

One recent lesson surprised me. For months, I thought I was building a better information access system. Looking back, that wasn’t the case. I was slowly laying the groundwork for an Information Operating System. The next steps are already clear:

• Universal Summary Engine for EPUB
• Universal Summary Engine for Medium
• Unified Q&A
• Repository Management V1
• GitHub-based project management The system is still evolving. But the direction shifted on the day a simple question forced me to think about identity…
• **You can read my other articles on Medium.**

I hope you enjoy reading this, and I’d really appreciate it if you could share any relevant experiences you have in the comments.