|
Research interests
My research interests include reinforcement learning, action selection and navigation based on reward in rats and autonomous robots. I recently got my PhD (sept. 2007) where I studied, under the supervision of Dr. Sidney Wiener and Dr. Agnès Guillot, the respective roles of the rat medial prefrontal cortex and ventral striatum in reward-based learning and shifting navigation strategies.
My research investigations include scientific techniques such as computational modelling, electrophysiology in freely moving rats and autonomous robotics. My work is particularly focused on Actor-Critic models and their possible extension to combine goal-directed behaviors and habit learning within cortico-basal networks. My work tries to relate this functional distinction to anatomical segregations of motor, associative and limbic subparts of the basal ganglia, and to the distinction in bioinspired robotics of navigation strategies (planning, locale, taxon) and their associated learning mechanisms.
|
Skills
-
Modeling (Neural Networks, Reinforcement Learning, Planning, Bayesian Modeling, Genetic Algorithms, EM algorithm, Self-Organizing Maps, Mixtures of Experts)
-
Programming (C/C++, Java, Basic, Php, Assemblor, Fortan, Caml, Prolog, Python)
-
Data analysis (Matlab, R, Spike2, Klustakwik, Klusters, SPAD)
-
Neurophysiology (Extracellular multiunitary recordings in behaving animals)
-
Animal Behavior (Conditioning and spatial navigation)
-
Autonomous robotics (E-puck, Lynxmotion, iCub humanoid robot, Psikharpax rat-like robot), middleware (Webots, Arduino, ROS, YARP, CSLU Toolkit, SpikeNet)
Publications
International journals with scientific committee
-
Benchenane, K., Peyrache, A., Khamassi, M., Wiener, S.I. and Battaglia, F.P. (2010). Coherent theta oscillations and reorganization of spike timing in the hippocampal-prefrontal network upon learning. Neuron, 66(6):921-36.
-
Peyrache, A., Benchenane, K., Khamassi, M., Wiener, S.I. and Battaglia, F.P. (2010). Sequential reinstatement of neocortical activity during slow oscillations depends on cells' intrinsic excitability. Frontiers in Systems Neuroscience, 3:18.
-
Peyrache, A., Benchenane, K., Khamassi, M., Wiener, S.I. and Battaglia, F.P. (2010). Principal component analysis of ensemble recordings reveals cell assemblies at high temporal resolution. Journal of Computational Neuroscience, 29(1-2):309-25.
-
Peyrache A., Khamassi M., Benchenane K., Wiener SI. & Battaglia FP. (2009) Replay of rule-learning related neural patterns in the prefrontal cortex during sleep. Nat Neurosci. Jul;12(7):919-26.
-
Khamassi M., Mulder AB., Tabuchi E., Douchamps V., Wiener SI. (2008) Anticipatory reward signals in ventral striatal neurons of behaving rats Eur J Neurosci 1849-66 .
-
Khamassi, M., Lachèze, L., Girard, B., Berthoz, A., and Guillot, A. (2005). Actor-critic models of reinforcement learning in the basal ganglia: From natural to artificial rats. Adaptive Behavior, Special Issue Towards Artificial Rodents, 13(2):131-148
-
Meyer, J.-A., Guillot, A., Girard, B., Khamassi, M., Pirim, P., and Berthoz, A. (2005). The Psikharpax project: Towards building an artificial rat. Robotics and Autonomous Systems, 50(4):211-223 .
-
Zugaro, M. B., Arleo, A., Déjean, C., Burguière, E., Khamassi, M. , and Wiener, S. I. (2004). Rat anterodorsal thalamic head direction neurons depend upon dynamic visual signals to select anchoring landmark cues. European Journal of Neuroscience, 20(2):530-536 .
International conferences with scientific committee
-
Khamassi, M., Quilodran, R., Enel, P., Procyk, E. and Dominey P.F. (2010). A computational model of integration between reinforcement learning and task monitoring in the prefrontal cortex. Doncieux, S. et al. (eds) Proceedings of the 11th International Conference on Simulation of Adaptive Behavior (SAB2010), pp. 424-34, Springer LNAI 6226.
-
Benchenane, K., Peyrache, A., Khamassi, M., Wiener, S.I. and Battaglia, F.P. (2009). Coherent oscillations and learning-related reorganization of spike timing. 4th International Conference on Cognitive Systems, CogSys10. January 27 & 28, 2010, ETH Zurich, Switzerland.
-
Dolle L, Khamassi M, Girard B, Guillot A, Chavarriaga R (2008) Analyzing interactions between navigation strategies using a computational model of action selection. Lecture Notes in Computer Science 5248 Springer, publisher. p 71-86
-
Battaglia, F.P., Benchenane, K., Khamassi, M., Peyrache, A., and Wiener, S.I. (2007). Neural Ensembles and Local Field Potentials in the Hippocampal-prefrontal Cortex System During Spatial Learning and Strategy Shifts in Rats . In Proceedings of the first International Conference on Cognitive Neurodynamics (ICCN), Shanghai , China .
-
Khamassi, M. , Martinet, L.-E. and Guillot, A. (2006). Combining self-organizing maps with mixture of experts: Application to an actor-critic model of reiforcement learning in the basal ganglia. To appear in From Animals to Animats 9: Proceedings of the Nineth International Conference on Simulation of Adaptive Behavior, Rome , Italy , 25-29 sept. 2006 .
-
Filliat, D., Girard, B., Guillot, A., Khamassi, M. , Lachèze, L., and Meyer, J.-A. (2004). State of the artificial rat Psikharpax. In Schaal, S., Ijspeert, A., Billard, A., Vijayakumar, S., Hallam, J., and Meyer, J.-A., editors, From Animals to Animats 8: Proceedings of the Eighth International Conference on Simulation of Adaptive Behavior, pages 3-12, Cambridge, MA. MIT Press .
-
Khamassi, M. , Girard, B., Berthoz, A., and Guillot, A. (2004). Comparing three critic models of reinforcement learning in the basal ganglia connected to a detailed actor in a S-R task. In Groen, F., Amato, N., Bonarini, A., Yoshida, E., and Kröse, B., editors, Proceedings of the Eighth International Conference on Intelligent Autonomous Systems, pages 430-437, Amsterdam, The Netherlands. IOS Press .
-
Dollé, L., Khamassi, M. , Girard, B., Guillot, A. and Chavarriaga, R. (submitted). Analyzing interactions between cue-based and map-based learning in a Morris water maze using a computational model of selection between navigation strategies.
Book chapters
-
Khamassi M, Wilson C, Rothé R, Quilodran R, Dominey PF, Procyk E (2011). Meta-learning, cognitive control, and physiological interactions between medial and lateral prefrontal cortex. In Neural Bases of Motivational and Cognitive Control, edited by R. Mars, J., Sallet, M. Rushworth, and N. Yeung, Cambridge, MA: MIT Press, in press.
-
Battaglia FP, Peyrache A, Khamassi M, Wiener SI (2008). Spatial decisions and neuronal activity in hippocampal projection zones in prefrontal cortex and striatum. In Mizumori S. (Ed) Hippocampal place fileds: Relevance to learning and memory. Oxford University Press, publisher.
Peer-reviewed National Conferences
- Khamassi, M., Quilodran, R., Enel, P., Dominey P.F. and Procyk, E. (2010). Role of the frontal cortex in solving the exploration-exploitation trade-off. In Proceedings of the 5th French Neurocomp Conference.
Conferences and Workshops with minimal peer-reviewing
-
Caluwaerts, K. and Grand, C. and N'Guyen, S. and Dollé, L. and Guillot, A. and Khamassi, M. (2011). Design of a biologically inspired navigation system for the Psikharpax rodent robot. International workshop on bio-inspired robots, Nantes, France.
-
Khamassi, M. and Lallée, S. and Enel, P. and Procyk, E. and Dominey P.F. (2011). Human- Robot Interaction with the iCub Humanoid Robot using a Neuro-Inspired Model of Reinforcement Learning. International workshop on bio-inspired robots, Nantes, France.
-
Khamassi, M. and Lallée, S. and Enel, P. and Procyk, E. and Dominey P.F. (2011). Gestion de l'incertitude dans le monde réel avec un modèle d'apprentissage par renforcement neuro-inspiré. Journées Nationales de la Recherche en Robotique Humanoïde, Toulouse, France.
Scientific vulgarization article
Abstracts and posters
-
Khamassi, M. , Mulder, A.B., Tabuchi, E., Douchamps, V. and Wiener S.I. (2007). Actor-Critic models of reward prediction signals in the rat ventral striatum require multiple input modules . Society for Neuroscience Abstracts, San Diego , USA .
-
Peyrache, A., Benchenane, K., Khamassi, M. , Douchamps, V., Tierney, P.L., Battaglia, F.P. and Wiener, S.I. (2007). Rat medial prefrontal cortex neurons are modulated by both hippocampal theta rhythm and sharp wave-ripple events. Society for Neuroscience Abstracts, San Diego , USA .
-
Benchenane, K., Peyrache, A., Khamassi, M. , Tierney, P.L., Douchamps, V., Battaglia, F.P. and Wiener, S.I. (2007). Increased firing rate and theta modulation in medial prefrontal neurons during episodes of high coherence in the theta band of hippocampal/prefrontal local field potentials (LFP) in behaving rats. Society for Neuroscience Abstracts, San Diego , USA .
-
Battaglia, F.P., Peyrache, A., Benchenane, K., Khamassi, M. , Douchamps, V., Tierney, P.L. and Wiener, S.I. (2007). Rat medial prefrontal cortex neurons are modulated by both hippocampal theta rhythm and sharp wave-ripple events. Society for Neuroscience Abstracts, San Diego , USA .
-
Battaglia, F., Khamassi, M. , Peyrache, A., Douchamps, V., Tierney, P., and Wiener, S. (2006). Spatial and reward correlates in medial prefrontal neurons of rats acquiring and switching strategies in a y-maze. In Society for Neuroscience Abstracts. Atlanta , USA .
-
Wiener, S., Khamassi, M. , Peyrache, A., Douchamps, V., Tierney, P., and Battaglia, F. (2006). Transitions in behaviorally correlated activity in medial prefrontal neurons of rats acquiring and switching strategies in a y-maze. In Society for Neuroscience Abstracts. Atlanta , USA .
-
Battaglia, F.P., Khamassi, M. , Douchamps, Tierney, P.L., Wiener, S.I. (2005). E EG correlations between hippocampus and prefrontal portex in rats performing a decision-making spatial task. Society for Neuroscience Abstracts, Washington DC , USA .
-
Mulder, A.B., Tabuchi, E., Khamassi, M. and Wiener S.I. (2005). Reward site associated activity in the ventral striatum of behaving rats. Society for Neuroscience Abstracts, Washington DC , USA .
-
Wiener, S.I., Arleo, A., Déjean, C., Boucheny, C., Khamassi, M. and Zugaro, M.B. (2004). Optic field flow signals update the activity of head direction cells in the rat anterodorsal thalamus. Society for Neuroscience Abstracts, San Diego , USA .
-
Khamassi, M. , Girard, B., Guillot, A. and Berthoz, A. (2003). Mécanismes neuromimétiques d'apprentissage par renforcement dans l'architecture de contrôle du rat artificiel Psikharpax. Poster presented at the french conference on artificial learning (CAp) within the frame of the AFIA platform, Laval , France , 1-4 July 2003 .
PhD and Master thesis
- Khamassi, M. (2007). Complementary roles of the rat prefrontal cortex and striatum in reward-based learning and shifting navigation strategies. PhD thesis in cognitive science, Université Pierre et Marie Curie, Paris 6, France .
- Khamassi, M. (2003). Un modèle d'apprentissage par renforcement dans une architecture de contrôle de la sélection de l'action chez le rat artificiel Psikharpax. Master thesis in cognitive science, Université Pierre et Marie Curie, Paris 6, France .
Teaching
2008-2011 Neuro-inspired reinforcement learning, Master of Artificial Intelligence, at Univ. Paris 6 (2h)
2010- Learning & decision-making: elements of modelling, Master of Neuroscience, at Univ. Lyon 1 (1h30)
2010- Bio-inspired action selection & reinforcement learning, Master of Mechatronics, at Univ. Paris 6 (1h)
2006 Artificial and cognitive life at ESIEA, Laval (18h).
2006 Computer Science at La Sorbonne, Paris (18h).
2006 Computed Science at Université Paris 6 (38h).
2006 Artificial and cognitive life at ESIEA, Ivry (12h).
2005 Computer Science at La Sorbonne, Paris (36h).
2004 Computer Science at Conservatoire National des Arts et Metiers, Paris (20h).
2004 Computer Science at La Sorbonne, Paris (36h).
2003 BioInformatics at Université d'Evry, France (12h).
Invited talks
2011 : Invited speaker at International Conference on Decision Making, Allahabad, India.
2010 : Role of the frontal cortex and striatum in reward-based learning: model-based analysis of extracellular recordings in behaving animals, INCM, Marseille, France.
2010 : Bio-inspired reinforcement learning: from neurobiological data to robotic applications, LRI, Orsay, France.
2010 : Integration of reinforcement learning and task monitoring in the prefrontal cortex, Computational Neuroscience Day, ENS Ulm, Paris, France.
2010 : Role of the medial prefrontal cortex-hippocampus system in learning rule-shifting behaviour, with Karim Benchenane, INSERM CRNL, Lyon, France.
2009 : Integration of reinforcement learning and task monitoring in the Anterior Cingulate Cortex, Lab. Cognitive Neuroscience, ENS Ulm, Paris, France.
2009 : Applications robotiques de modèles neuro-inspirés d'apprentissage par renforcement, Lab. Robotique GREYC, Université de Caen, France.
2008 : Actor-Critic models: from ventral striatal reward-related activity to robotics simulations of rat behavior, GDR Neurosciences de la Mémoire, Aussois, France.
2008 : Actor-Critic models: from ventral striatal reward-related activity to robotics simulations of rat behavior, INSERM U846, Bron, France.
2008 : Actor-Critic models: from ventral striatal reward-related activity to robotics simulations of rat behavior, Okinawa Institute of Science and Technology, Japan.
2008 : Modeling the substrate of competing spatial orientation strategies, ICEA project evaluation meeting, Budapest, Hungary.
2007 : Actor-Critic models: from ventral striatal reward-related activity to robotics simulations, Neural Coding Workshop, Third day in Computational Neuroscience, Paris , France .
2006 : Concurrent and parallel neuromimetic learning systems for navigation, Derby , UK.
2004 : Actor-critic models of reinforcement learning in the basal ganglia: From natural to artificial rats, University of Toyama , Japan .
2004 : Modélisation des mécanismes d'apprentissage par renforcement dans les ganglions de la base du rat artificiel Psikharpax, ESPCI, Paris, France.
Other Activities related to Research and Universities
• Review Editor for the international journal Frontiers in Neurorobotics.
• Reviewer for international journals: Neural Computation.
• Reviewer for international conferences: SAB, SSSC.
• Scientific communication and interviews for Radio France Culture (« Science et Conscience » program), TV France 5 (« Le magazine de la santé » program), TV Arte (« Global Mag »), TV France 3 Rhônes-Alpes, AFP, Reuters, Le Point, « Des Robots et Des Hommes » Day at Cité des Sciences. • Board Member (2002-2005) and President of Cognivence (2003-2004), association of students and young researchers in cognitive science in Paris ' area.
• Board Member of FRESCO (2003-2005), national federation of students and young researchers in cognitive science.
• Co-organizer with Vincent Jacob (principal organizer) of the Forum des Laboratoires en Sciences Cognitives d'Ile-de-France (April 2, 2004).
• Principal organizer of the Forum des Sciences Cognitives ( April 15, 2005 ) gathering together laboratories, companies and students, concerned with cognitive science.
• Member of the regional committee in Paris (CloEG) for the organization of the french « Etats Généraux de la Recherche 2004 ».
• Co-organizer of the regional workshop in Paris for the french « Etats Généraux de la Recherche » (June 21, 27 and 28, 2004).
• Member of the national committee (CIP-CloEG) for the synthesis of texts elaborated regionally during the french « Etats Généraux de la Recherche 2004 ».
|
: mehdi.khamassi AT
isir.upmc.fr