A contradiction exists within the neurosciences on the issue of sex. On the one hand, it is increasingly clear that there are important differences in brains as a function of sex, and to ignore these differences is a disservice to both men and women. On the other hand, research that draws on findings in neuroscience to explain sex differences in behaviour has been called out as ‘neurosexist’. In this article I’ll make the case for keeping sex in the neurosciences, outline some of the ways research can be misinterpreted (by both scientists and the public), and clarify what neuroscience can, and can’t, tell us about men and women.