Summary of the most interesting World Economic Forum discussion panel on AI

Great insight from top professionals.

Photo by Evangeline Shaw on Unsplash

Last week’s World Economic Forum (WEF) was significant for its focus on AI. Numerous intriguing discussions and interviews covered various aspects of AI, including sustainability, safety, and the future of jobs. While there were many noteworthy panels, one, in particular, stood out to me due to its in-depth exploration of the evolving landscape of generative models and AI as a whole. Titled ‘The Expanding Universe of Generative Models’ this discussion featured a lineup of top AI experts, each bringing a unique perspective to the table. For a comprehensive summary of the AI topics discussed, you can visit the dedicated WEF subpage.

TL;DR i.e. my main 5 takeaways from the panel

• We are running out of data that is of sufficient quality to advance current LLMs.
• So far, LLMs are learning from text. We’re still unable to make use of video data for training, while LLMs are trained on far less data than what a child processes through their optical nerve in the first four years of life.
• Advancement will come from future agents interacting with the physical world through our wearables, sensors, augmented reality, self-driving cars, etc. Such agents will be able to learn about the cause-effect rules of the world.
• Benchmarking AI against human intellect is an extremely challenging and probably unfair task, especially given the completely different ways of processing information. This should be taken into account when finally agreeing on the definition of AGI.
• Lobbyists are working hard to impose heavy regulations on open-source models.

Introduction to panelists:

• Yann LeCun: A pioneering figure in AI, particularly known for his work in deep learning and neural networks. Currently Chief AI Scientist at Meta.
• Andrew Ng: Renowned for his influential role in the rise of online education in AI (co-founder of Coursera) and founding of the Google Brain Project. Founder of deeplearning.ai
• Daphne Koller: A leading figure in the application of AI in biomedicine and the co-founder of Coursera. Founder and Chief Executive Officer at Insitro Inc
• Kai-Fu Lee: A notable AI expert with a focus on machine learning and its implications in the global technology landscape. CEO at 01.AI
• Aidan Gomez: Known for his significant contributions to the development of advanced AI models, co-author of the groundbreaking paper on the transformer architecture ‘Attention is all you need”. CEO at Cohere.
• The panel was moderated by Nicholas Thompson, CEO of The Atlantic, known for his expertise in technology and its intersection with society.

Below are the main points that I took note of, each sections being a separate question from the moderator.

1. Future directions and current limitations of AI

• Andrew Ng: Highlighted the text revolution in 2023 and predicted 2024 to be focused on video. He emphasized the evolution towards autonomous agents capable of solving complex problems over time, using various web resources and plugins.
• Daphne Koller: Discussed how we are just beginning to utilize data. Having almost used all of the available web-scale data, she envisaged a future where AI agents, enhanced by augmented reality, self-driving vehicles, and medical data, become more capable by interacting with the world and learning the rules and phenomena governing it.
• Yann LeCun: Addressed the data limitation, noting that current large language models (LLMs) are trained on far less data than what a child processes through their optical nerve in the first four years of life. He agreed with Daphne and argued for the need for breakthroughs in learning from sensory data and new architectures.
• Aidan Gomez: Pointed out the growing challenge in improving models, where expert knowledge is increasingly required to gather high quality data, contrasting with a time when almost anyone could contribute to model enhancements and publically available datasets were enough.

2. Should we be building machines more intelligent than humans?

• Yann LeCun: Compared AI research to the early goals of aviation (“the goal of building airplanes was not to be faster than birds, but to learn the principles of aviation”), emphasizing understanding the principles of learning and intelligence rather than surpassing human capabilities per say.
• Daphne Koller & Kai-Fu Lee: Offered a different perspective, focusing on using AI to solve complex problems that may be beyond human capability, not necessarily to understand learning and intelligence.
• Aidan Gomez: Expressed a desire to use AI to enhance human life, regardless of whether it leads to achieving artificial general intelligence (AGI).
• Andrew Ng: Discussed the unfairness of comparing digital intelligence to human intelligence by means of current benchmarks, suggesting that this should be considered in defining AGI.

3. Open Sourcing Models

• Yann LeCun: Advocated for open sourcing foundational models, citing its role in accelerating AI progress and ensuring safety and diversity. He pointed out that while OpenAI uses open-source tools (PyTorch, transformer architecture), they maintain proprietary elements to drive the field forward.
• Aidan Gomez: Proposed a hybrid approach to open sourcing, balancing the need for innovation and business interests.
• Andrew Ng: Emphasized the general benefit of increased intelligence (both human and artificial) in society. He also raised concerns about access to infrastructure and the influence of powerful lobbyists in Washington D.C. on open-source policies.
• Daphne Koller: Argued for the necessity of open-source models for foundational advancements, while acknowledging the role of closed-source models in specific applications.

I highly recommend anyone interested in the subject to spend the 45 minutes listening to the discussion. It offers much more insight, examples and analogies.