AI Problems From the Last 20 Years That Became Irrelevant — And Today’s AI Problems That May Eventually Become Irrelevant Too

Organized by ChatGPT

AI Problems From the Last 20 Years That Suddenly Became “Who Cares?”

1. Handcrafted Feature Engineering

This used to be the central problem.

For images:

• SIFT
• SURF
• HOG

For audio:

• MFCC

For NLP:

• n-grams
• POS-engineered features
• rule-based linguistic features

Back then, the core of AI research was:

“What features should humans design?”

Then deep learning arrived, and suddenly:

“Why not let the network learn the features itself?”

This was one of the biggest “problem evaporations” in AI history.

2. “How Should We Add Attention?”

Around 2016–2019, this was huge.

People explored:

• soft attention
• hard attention
• hierarchical attention
• visual attention

…countless variants.

Then Transformers arrived, and the field basically said:

“What if everything is attention?”

And after scaling laws kicked in:

“Model scale matters more than attention tricks.”

A huge amount of research into complicated attention mechanisms effectively evaporated.

3. The Long-Term Dependency Problem in RNNs/LSTMs

This used to dominate sequence learning research.

Topics included:

• vanishing gradients
• memory cells
• forget gates
• GRUs

For over a decade, this was a major frontier.

Then Transformers appeared, and suddenly:

“If it can’t parallelize efficiently, it’s painful.”

The center of gravity shifted rapidly.

Today, dedicated LSTM specialists are relatively rare.

4. “Winning Image Classification Benchmarks by 0.3%”

Late-stage ImageNet culture.

Around 2014–2020:

• ResNet
• DenseNet
• EfficientNet

…competed intensely.

But today, the focus shifted toward:

• LLMs
• multimodal systems
• agents

Pure image classification lost relative importance.

Turns out:

“Detecting cats vs dogs with ultra-high accuracy”

did not fundamentally change the world.

5. The GAN vs VAE Wars

Around 2016–2021.

At the time:

• “GANs are the future”
• “VAEs are blurry”
• discriminator design wars

…were enormous topics.

Then diffusion models arrived and rapidly took over.

A huge amount of GAN-centric research suddenly lost momentum.

6. “Reinforcement Learning Will Solve Everything”

The post-AlphaGo hype era.

Around 2016–2019, there was a real feeling that:

“RL can become the foundation of general intelligence.”

But in practice:

• sample inefficiency
• real-world environments
• reward design
• training cost

…became major obstacles.

Meanwhile, the LLM paradigm exploded.

RL survived — especially through RLHF — but the idea that RL alone would dominate AGI lost momentum.

Current AI Problems That Might Become Irrelevant in the Future

Now this gets even more interesting.

1. Prompt Engineering

This one has a dangerous smell.

Right now we have:

• prompt tricks
• role prompting
• chain-of-thought prompting
• mega-prompts

But as models improve, we may move toward:

“You can just talk naturally and it still works.”

Meaning:

“Prompt wizardry”

may lose much of its long-term value.

Kind of like how hand-written SQL expertise became less central over time.

2. Tiny Optimizations in RAG Systems

Currently, people obsess over:

• chunk size
• rerankers
• retrieval fusion
• hybrid search

But if we eventually get:

• massive context windows
• better memory mechanisms
• end-to-end memory integration

then:

“External retrieval engineering”

may shrink dramatically in importance.

3. Overcomplicated Agent Architectures

Right now we see endless work on:

• planners
• tool routers
• reflection loops
• debate systems

But historically, AI repeatedly shows that:

handcrafted architectures often get crushed by scaling.

Meaning:

many complex agent frameworks may eventually become historical curiosities.

4. Benchmark Optimization

This happens over and over again.

Examples:

• GLUE
• SuperGLUE
• MMLU
• GSM8K

At first, each benchmark feels like:

“This is humanity’s final exam.”

Then a few years later:

“Wait… is this just benchmark memorization?”

AI history repeats this cycle constantly.

5. Explicit “Human-Like Personality Modeling”

For decades, researchers tried:

• emotion architectures
• symbolic personalities
• dialogue state management

But LLMs unexpectedly produced highly convincing personality effects from relatively simple next-token prediction.

Which raises the question:

“Do we actually need carefully engineered human simulation systems?”

The answer may increasingly become “not really.”

The Pattern That Repeats Throughout AI History

This is the really important part.

In AI, what often disappears is not merely:

“the solution to the problem”

but:

“the problem itself.”

In other words:

• the field doesn’t just improve the method,
• the entire framing becomes obsolete.