I argue for the importance of developing a statistical neuroscience in the spirit of Ludwig Boltzman's statistical mechanics. Neuronal phenomena occur and are observed at different scales, spatial and temporal. Large populations of neurons behave in ways that cannot be accounted for in a straightforward fashion by the activities of their constituent neurons. In my presentation I explain how such a theory could be developed thanks to recent developments in mathematics, in particular the theory of Large Deviations. I then show on a simple example of a population of firing rate neurons how this program can be achieved and what kind of results it may produce.