Realization of Visual Representation Task on Humanoid Robot, Darwin-OP



In this project, the representation of an object is generated by spiking neural networks (SNN) in a self-organizing way. In addition, the generated representations are used to actuate the humanoid robot, Darwin-OP, to accomplish a task. So, the perception of the robot on an object is built in a biologically-plausible way with a computational model.

Part of this project is prepared as a final project of Robot Intelligence Course (BLG 622E) at Istanbul Technical University. The project report can be found here. Computational model part of this study is presented in National Neuroscience Congress 2016. The poster can be found here.


Video of the Cortex SNN Experiment



The task in this video is similar to the one recorded in the previous video. The model is Basal ganglia-Thalamus-Cortex model, but differently the decision part, Cortex, is modeled with spiking neural network model with using Izhikevich spiking neurons.
Before the video, robot is first trained to associate the blue colour with the head movement. After this, in the video robot is urged to rearrange the association and trained to associate the blue colour with a new action, arm movement. Robot rearranges the association due to not getting reward to the head movement.
One of the important difference of model. is the experiment time. Real time factor of the experiment on 1.6 GHz Intel Atom Processor is 0.01. After the colour is presented, robot waits for the action decision from the model to move its actuators. Therefore, the computation power of the robot need to be increased to use more complex models of Basal ganglia-Thalamus-Cortex loop in future experiments.

For video


Implementation of Matching Stimulus-Movement Experiment on a Humanoid Robot



In this study, we implemented a working memory task on a humanoid robot, Darwin-Op, using Basal ganglia-Thalamus-Cortex action selection model and temporal difference learning rules. The task is structured upon matching the stimulus with a predefined action. When a colour card (R-Y-B) is shown to robot, it is urged to select an action. If the right action is selected, a reward is given to reinforce the robot‘s right action. This reward generates an expectation error and this error updates the parameters of action selection model. After matching a stimulus and action, in order to force the robot to adapt to a new action, nothing is given to right action. This urges the robot to rearrange the connection between the stimulus and the action.

Video of the Experiment



In this video, color cards (red-blue-yellow) are the stimulus that represents stimulations from the environment. The green card stands for reward. If the card is shown on the right, it means a reward is given to the robot. Otherwise it means, there is nothing. There are also, three actions (movements) that are associated with related stimulus. In this experiment, what the actions are not important, the important thing is they are different.
In this video, the robot is shown a color card. It reacts to this card with a movement. If the movement is the desired one (which will be associated to the color card) then the reward is given. In this situation, this generates an expectation error since robot doesn't expect something as a result of its action. If the selected movement is not the desired one, the robot doesn't get a reward then it continues to select movements randomly.
After selecting the right action for a given color card, the reward is given and reinforcement is begun. Through the reinforcement, robot learns to associate the color card with a desired movement. After learning is accomplished, if the color card is desired to be associated to another movement, then the robot chooses the previously associated movement to the given color and doesn't get a reward. This also generates an expectation error, since the robot previously associated the stimulus with an action, and expects for a reward. This situation reinforces the robot to select another movement for the given color card. Through this process, a stimulus is associated with a movement, and the association is rearrange to associate the stimulus with another movement.

For long version of video