02Learning

What is Learning?

Learning can be understood as the ability to incorporate new information and adapt to the environment. Whether it is a biological organism or an artificial system, learning allows us to adjust behaviors and decisions based on new stimuli or experiences.

Both humans and AI aim to achieve several key cognitive capabilities through learning: Classification, Prediction, Generalization, Problem-Solving, and Adaptation.

How Do Humans Learn?

Humans possess a unique ability to learn from their surroundings, from others, and from their own actions:

• Innate Abilities: Skills encoded in our genetics, present from birth.
• Imitation Learning: Acquiring skills by observing and imitating others.
• Trial and Error: Experimenting with different actions until we reach a goal.
• Formal Learning: Structured learning through explicit rules and systems.
• Abductive Learning: Learning by insight or deduction, making novel connections.

In each of these modes, humans rely on System 2 thinking when learning something new, whereas once mastered, skills shift to the more automatic System 1.

How Machines Learn

• Supervised Learning: The machine is given labeled data and learns by comparing predictions to actual outcomes.
• Unsupervised Learning: The machine identifies patterns and structures within unlabeled data.
• Self-Supervised Learning: The machine generates its own labels from input data, finding patterns and relationships to create tasks that guide learning.
• Imitation Learning: Learning by observing and mimicking expert behavior, especially effective in sparse reward settings.
• Reinforcement Learning: Learning through interaction with an environment, using feedback in the form of rewards and penalties.

Reinforcement Learning with Human Feedback (RLHF)

After a model has trained using a self-supervised learning approach, its behaviours are ranked by humans. This process is surprisingly similar to how formal education is applied in human learning—formal education in humans serves to refine previously learned behaviors, aligning them with societal norms, moral values, and cultural expectations.

What are we learning from the engineering of powerful AIs?

Avoid Overfitting: Just as diverse training data prevents overfitting in AI, humans benefit from diverse experiences and exposure to multiple perspectives.

Model Distillation: This mirrors how cultural and generational knowledge is passed down through teaching, storytelling, or written records.

Imitation vs. Rule-Based Learning: AI systems learn more effectively through imitation learning than through rigid, rule-based systems, highlighting the importance of good role models.

Reinforcement Learning: Humans also learn through reinforcement, and this can be enhanced by providing more structured feedback and creating environments where failure is seen as a learning opportunity.