AI 1.06 — Full Throttle and Everyone’s Suddenly Interested:
Impact on AI by Researchers and Communities
AI 1.06 — Full Throttle and Everyone’s Suddenly Interested:
Impact on AI by Researchers and Communities
A Brief History of ML Jargon that Newbies run into, and other big words:
The second wave of AI ended on note of overlap with the dotcom boom and the birth of the modern search engines. The promise was the combined the power of the internet and the cloud bringing researchers and interested parties both together.
And bringing them together it did, and perhaps more beautifully chaotic than anyone could have anticipated. All of the populace who had access to internet came together as one, in almost everything us human beings were interested in. Early interests in AI was no exception.
For decades, Artificial Intelligence had been dominated by a symbolic approach. It had more to do with rules than feeding on data, but this shifted. It meant moving more towards data than it relied on rules.
Vladimir Vapnik and Alexei Chervonenkis were Soviet-Russian pioneers in the field of computer science and engineer. As early as the 1960s, they had laid the theoretical groundwork for their work on statistical learning theory. But it was Vapnik’s development of the Support Vector Machine (SVMs!) in the 1990s at Bell Labs that gave the field a practical, powerful tool.
https://www.geeksforgeeks.org/machine-learning/support-vector-machine-algorithm/
SVMs were able to take data points, find the optimal boundary between categories and classify new data with remarkable accuracy. They excelled at text classification, image recognition and bioinformatics. A very popular part of the “Classification” approaches learners usually run into today.
Around the same time, Judea Pearl was formalising Bayesian networks, graphical models that could represent probabilistic relationships between variables. His 1988 book, Probabilistic Reasoning in Intelligent Systems, became a foundational text. Bayesian networks allowed machines to handle uncertainty natively, to not just say “this is a cat” but to say “there is an 87% chance this is a cat given what I can see.” Pearl would later win the Turing Award in 2011 for this work.
Then came the ensemble methods. Leo Breiman introduced Random Forests in 2001, a technique that built hundreds of decision trees on random subsets of data and let them vote on the answer. It was robust, resistant to overfitting and quite effective. Random Forests, along with gradient boosting methods that followed, became the go-to approach for structured data problems in industry and in the Kaggle competitions that would spring up a decade later. I include myself in this long list of ML learners who did this and called themselves ML Scientists. Little did we not know this was just tip of the iceberg.
What all of these methods had in common was a philosophical departure from the AI of McCarthy and Minsky. They did not try to encode intelligence. They let it emerge from data.
This was the bridge. Without SVMs and Bayesian networks and Random Forests, without proving that statistical, data-driven approaches could outperform hand-crafted rules, the core deep learning revolution of the 2010s would have had no foundation to stand on, and no audience willing to believe in it.
The dot-com boom of the late ’90s had put everything on the internet. After the bubble burst and the dust settled, what remained was infrastructure, bandwidth and an ever-growing population of curious minds with access to everything.
The content and whereabouts of AI research, previously tucked away in university libraries and expensive journal subscriptions, were suddenly within reach of anyone with a browser, a stable internet connection and a very strong willingness to learn.
This accessibility would go on to change the trajectory of AI research in ways that no single laboratory or institution ever could.
Professors First, Legends Always:
The year was 2003. A professor at Stanford University had started teaching his CS229 Machine Learning course. He was embraced with massive crowds and high interest from his pupils. His ML course was rigorous, grounded in linear algebra and probability, and it covered everything Machine Learning.
https://www.coursera.org/specializations/machine-learning-introduction
This was the beginning of Andrew Ng’s journey to what he has become today.
He was very popular from the get-go, but it was when the lectures were recorded and uploaded online that things changed dramatically. Everyone who claims to have a grip on Machine Learning has 100% sat through the meticulous math that Ng has explained in his videos, and to this day, his Coursera course remains the single most watched and highest rated course on the internet. Any learner who has dabbled in AI would have definitely installed Octave on their systems to tune into Prof. Ng’s ML tutorials.
I count myself in the list of learners who have scribbled along to his gradient descent to recommendation system explanations.
His research output was no exception. In 2011, alongside Jeff Dean and others, Ng published on large-scale unsupervised learning using deep neural networks, famously training a network across 16,000 CPU cores on unlabelled YouTube thumbnails. The network learned to recognize cats, entirely on its own, without ever being told what a cat was.
The paper, Building High-level Features Using Large Scale Unsupervised Learning (2012), became one of the most cited works of the decade. It demonstrated that scale and data could unlock capabilities that hand-crafted features never could.
For a generation of self-taught engineers and researchers around the world, Andrew Ng was the gateway into Artificial Intelligence.
Alongside Coursera, Udacity’s cofounder Sebastian Thrun was also a research legend on his own right. He taught robotics at Stanford. In 2005, he and his team built Stanley, the self-driving car, and won the 2005 DARPA Grand Challenge. This would also help him co-found Udacity in 2012.
https://www.nbcnews.com/id/wbna9621761
His own landmark paper, Stanley: The Robot That Won the DARPA Grand Challenge (2006), detailed the sensor fusion and machine learning pipeline that allowed an autonomous vehicle to navigate 132 miles of unrehearsed desert terrain.
ImageNet and the Quiet Breakthrough:
While the Massive Open Online Courses (MOOCs!) were busy building an army of new learners, something extraordinary was being assembled in an academic lab.
Fei-Fei Li, a computer science professor at Stanford, had embarked on an ambitious and seemingly thankless project since 2006: building ImageNet, a database of over fourteen million hand-labelled images. It spanned more than twenty thousand categories, but the idea it was built on was deceptively simple.
If machines needed to learn to see, they needed something to look at. Ie, Data.
ImageNet became the benchmark. Every year from 2010, the ImageNet Large Scale Visual Recognition Challenge (ILSVRC) invited researchers from around the world to pit their algorithms against each other in image classification. Error rates came down steadily, but nothing prepared the community for 2012.
A University of Toronto team led by Geoffrey Hinton and his students Alex Krizhevsky and Ilya Sutskever submitted AlexNet, a deep convolutional neural network that crushed the competition, reducing the error rate from 26% to 16% in a single leap. It was a moment that reignited interest in neural networks — the very same neural networks that Minsky had cast doubt upon decades ago.
The importance of this cannot be overstated. Fei-Fei Li’s ImageNet was an academic labour of love. Hinton had been championing neural networks for thirty years when most of the field had looked the other way.
The research renaissance was not driven by corporate labs with billion-dollar budgets. It was driven by professors, graduate students and a shared belief that the problems were finally solvable.
The 2012 breakthrough did not come from a tech giant’s research division. It came from a university and more importantly, a strong audacity to learn. The takeaway was clear: deep learning worked, and the data to prove it was painstakingly put together by researchers who then became legends.
Kaggle, Hackathons, and people who just showed up to these things:
The research was surging, but so was something which research thrived on. Ie, Communities.
In 2010, an Australian named Anthony Goldbloom founded Kaggle, a platform that hosted machine learning competitions. Companies and researchers would post datasets and problems, and teams from around the world would compete to build the best predictive models. Prize pools ranged from bragging rights to serious money, but the real currency was learning.
Kaggle became a proving ground. University students in Bangalore could go head-to-head with PhD researchers in Berlin. Self-taught programmers in Lagos were outperforming seasoned data scientists in London. The playing field, for the first time, was truly level.
What mattered was not your institution or your pedigree, but your ideas, your code and your tenacity. What were you worth was a question that could easily be answered with a computer and a stable internet connection.
At its peak before 2020, Kaggle had amassed over five million registered users — a staggering number for what was essentially an academic competition platform.
Hackathons were sprouting up everywhere. From weekend-long AI challenges at university campuses, to global events sponsored by research councils, the culture of building and competing had taken root.
Communities like fast.ai, founded by Jeremy Howard and Rachel Thomas in 2016, went even further. Their philosophy was quite radical, which was to make deep learning accessible to people with only one year of coding experience. Their free online courses, built atop a practical, code-first teaching style was able to produce thousands of capable practitioners who would have otherwise not even considered the field.
The Explosion of Research:
With more people entering the field came more research. In just six years, the number of AI-related publications on arXiv grew more than sixfold — from 5,478 in 2015 to 34,736 in 2020. AI publications represented 3.8% of all peer-reviewed scientific publications worldwide in 2019, up from 1.3% in 2011.
The growth was not just in volume, it was pace at which it was done. New techniques, new architectures, new applications were being published at a speed that made it impossible for any single researcher to keep up.
And it was coming in from everywhere. From every major country and region, peer-reviewed AI papers came from academic institutions mostly but also from corporations and individual contributors. China surpassed the United States in the share of AI journal citations in 2020 for the second time, briefly overtaken the US in the overall number as early as 2004.
The field was no longer an American or European affair. It was a global effort, fuelled by the very accessibility and community that the MOOCs and competitions had created.
AI conferences became events unto themselves. NeurIPS, ICML, CVPR and AAAI saw attendance figures that would have been unimaginable a decade prior. COVID forced most of these conferences to go virtual in 2020. The number of attendees across nine major conferences nearly doubled not despite COVID, but because of it.
The barrier of plane tickets and visas had been the last wall standing, and the virtual world we built for ourselves knocked it down.
The Tapering and the Tease:
There was no stopping this storm now. The Gartner cycle was slowly climbing to reach its peak, there was no sign of tapering of the exponentially increasing curve.
And taper it did not. By the late 2010s, deep learning had proven itself. The question was no longer whether it worked, but how far it could go.
Recurrent neural networks gave way to attention mechanisms. Sequence-to-sequence models showed promise. And then, in 2017, a research paper titled Attention Is All You Need, introduced the Transformer architecture and everything changed.
https://arxiv.org/abs/1706.03762
Then began the story of how machines finally started learning our languages, of conversational AI and a revolution that began with the letters G, P and T.
References and Further Reading:
메타데이터
- post_id
- 323dd2df8ac3
- slug
- ai-1-06-full-throttle-and-everyones-suddenly-interested-323dd2df8ac3
- url
- https://medium.com/the-ai-society/ai-1-06-full-throttle-and-everyones-suddenly-interested-323dd2df8ac3
- canonical_url
- https://medium.com/the-ai-society/ai-1-06-full-throttle-and-everyones-suddenly-interested-323dd2df8ac3
- author_url
- https://medium.com/@vishnukumarvh10
- status
- ok
- fetched_at
- 2026-06-12 18:14:10
