RL 1: Biological Foundations and Law of Effect (1898-1949)
Tracing the history of AI back to hungry cats and Pavlovian dogs, exploring the origins of the law of effect and modern reinforcement learning.
Stock photo for illustration only, not from the actual event
- Reinforcement learning traces back to 1898 psychologists without writing code.
- Edward Thorndike discovered the law of effect through wooden crate cat experiments.
- Ivan Pavlov popularized predictive signaling responses before receiving rewards.
- Six core concepts were discovered by observing animal behaviors for decades.
Before GPUs, neural networks, or even transistors were a rumor, there was a hungry cat inside a wooden crate trying to reach a piece of fish. The main character of this era is not an engineer, but a loose group of psychologists poking at animals to answer why living things act the way they do.
They never wrote a line of code or tried to build AI. Yet, between 1898 and 1949, they reverse-engineered the core loop that every modern reinforcement learning system runs on. This post tells that story with no math yet, focusing purely on the loop and the four people who discovered its pieces.
Reinforcement learning describes any system that learns by doing and seeing what happens, rather than being shown correct answers through trial, failure, and adjustment. The learner is called the agent, while everything outside it is the environment, operating through an interconnected loop.
Stock photo for illustration only, not from the actual event
Understanding the historical foundations of reinforcement learning reminds us that today's complex AI algorithms did not emerge from abstract mathematics alone. They stem from decades of systematic animal behavior observation, reflecting fundamental adaptation principles that still drive modern artificial intelligence.
The agent needs a policy, which is a strategy for choosing actions rather than a fixed script, serving as a rule of thumb for specific situations. It also requires a sense of value to determine how good a state is based on what it leads to rather than immediate rewards.
In 1898, a graduate student named Edward Thorndike was annoyed by animal literature full of charming anecdotes about clever pets. Thorndike suspected people were flattering their pets and that animals were doing something else entirely that nobody had measured.
The experimental setup was simple: a hungry cat in a small wooden crate with a door that opened only when pulling a string, pressing a lever, or stepping on a pedal, with fish sitting outside in plain view. Trial one was pure chaos as the cat clawed, shoved paws, yowled, and thrashed at loose items until it accidentally hit the lever.
"If an action is followed by a satisfying state of affairs, the bond between that situation and that action is strengthened. If followed by discomfort, it is weakened."
Edward Thorndike
By trial twenty, the cat walked in, hit the lever, and strolled out. Thorndike noticed there was never a sudden lightbulb moment of reasoning. Instead of escape times dropping off a cliff instantly, plotting escape time against trial number yielded a slow, gradual curve where useless behaviors quietly faded.
Source: Dev.to
Found something wrong in this article? Report an issue with this article
Comments
Leave a Comment