Even since Galileo, philosophers have tried to explain why science has been so successful. All of the great philosophers in history have taken an interest in epistemology – the philosophy of knowledge – and the most important philosophers of recent centuries have grappled with the problem of scientific knowledge. At the same time, probability theorists and mathematicians have also grappled with the problem of scientific experimentation, giving rise to modern statistical methods of experimentation and analysis. Is there then a ‘right’ way to do science, or is science a matter of ‘anything goes’ as Paul Feyerabend has argued (Feyerabend 2010). Feyerabend was writing in reaction to the rigidity of the methodology of Imre Lakatos, who in turn was reacting to the writings of his teacher Karl Popper. But this context has been lost in some recent popular histories of science, which have presented science as a wholly anarchic enterprise (Brooks 2012). It is my contention that there is not a single right way of doing science, but nor does ‘anything go’. Rather, a plurality of approaches to science is possible.