In this video, color cards (red-blue-yellow) are the stimulus that represents stimulations from the environment.
The green card stands for reward. If the card is shown on the right, it means a reward is given to the robot.
Otherwise it means, there is nothing. There are also, three actions (movements) that are associated with related
stimulus. In this experiment, what the actions are not important, the important thing is they are different.
In this video, the robot is shown a color card. It reacts to this card with a movement. If the movement is the desired
one (which will be associated to the color card) then the reward is given. In this situation, this generates an
expectation error since robot doesn't expect something as a result of its action. If the selected movement is not the
desired one, the robot doesn't get a reward then it continues to select movements randomly.
After selecting the right action for a given color card, the reward is given and reinforcement is begun. Through the reinforcement,
robot learns to associate the color card with a desired movement. After learning is accomplished, if the color card is desired to be associated
to another movement, then the robot chooses the previously associated movement to the given color and
doesn't get a reward. This also generates an expectation error, since the robot previously associated the stimulus
with an action, and expects for a reward. This situation reinforces the robot to select another movement for the given color card.
Through this process, a stimulus is associated with a movement, and the association is rearrange to associate the stimulus
with another movement.
For long version of video