The relationship between a highly developed scientific theory and the phenomena it describes, predicts or explains is extraordinarily complex and multifaceted. It is typical for philosophical accounts of theories to focus on only one facet or another. In particular, contemporary work on the theory-world relation tends to focus on abstract formal frameworks (for theory reconstruction or presentation) and formal or informal accounts of the role of scientific models. Missing from these perspectives are accounts of the methods of theory development and their relationship to experience and experiment (there are exceptions, an important example being the work of Oliver Darrigol, e.g., Darrigol (2008)).
A famous example of the problem with building a philosophical perspective on the theory-world relation without accounting for the empirical origins and constraints on scientific theories is the applicability problem that aries from Eugene Wigner's famous The Unreasonable Effectiveness of Mathematics in the Natural Sciences. Wigner claims that "the miracle of the appropriateness of the language of mathematics for the formulation of the laws of physics is a wonderful gift which we neither understand nor deserve." (Wigner, 1960) The applicability problem, then, is to explain this miraculous appropriateness. Mark Steiner (1998) takes up this question and also argues for the miraculousness of the applicability of mathematics given the nature of the very successful methods used to develop quantum theories.
The views that Wigner and Steiner support, however, abstract theory development from its historical context, focusing only on the limited time frame within which a particular theory is successfully formulated. I argue that a proper account of theory development must understand the development of scientific theories as part of a complex and continuous historical process, with deeper and broader theories developing stagewise on the support of earlier empirically successful theories as well as new experimental data and results.
A consequence of this view is that a full understanding of theory development will require patching an account together from a mosaic of episodes of theory-making from the history of science. This is obviously not work that one person can do alone. Thus, my focus is on attempting to understand the reasoning processes used in particular cases of theory development, and attempting to explain 1) how the reasoning involves prior theory and experience/experiment and 2) why it is rational.
Formally Aided Phenomenology
The one historical episode that I have studied in some level of detail is the development of theories in optics, electricity and magnetism in the 19th century. In studying this episode I was able to identify a reasoning pattern where equations, methods or results known to be valid in one context are adapted by analogy into another context. One aspect of the pattern is that empirically validated equations or relations together with a speculative hypothesis are used as constraints to identify a characteristic set of equations that is consistent with them. In mathematics terms this is a solution to an inverse problem. In philosophical terms, it is very close in form to a transcendental argument. In this way, I argue, the physicists in the 19th century were engaged in a kind of formally aided phenomenology, where phenomenology is understood in the sense of Kant (elucidating the conditions necessary for the possibility of experience).
This is just one historical episode, among many, in one area of physics, among many, but it serves to illustrate that at least some of the reasoning methods in theory discovery are quite subtle and have a distinct rationale to them. They are certainly considerably more sophisticated than methods of enumerative induction.
Viewing the development of scientific theories as the result of a complex interplay of prior theories or models, empirical equations or relations, and new data, provides one way to demystify the "magic" of theory discovery. Another is to understand scientific theories not as revealing the ontology or architecture of the world but rather as characterizing the behaviour of phenomena under a certain range of conditions. By viewing theories in this way, we resist the temptation to reify the elements or objects of a theory, and resist any good reason to suppose that a new theory will all of a sudden get the ontology or architecture exactly right.
On this view, then, quantum theories, are regarded as characterizations of the behaviour of matter-energy stuff under certain ranges of circumstances, not as revealing the "fundamental" structure and ontology of the world. Such a view is, of course, in line with the view that all quantum theories are effective field theories, where this term is understood to extend beyond its technical meaning in quantum field theory. I argue that on such a view, it becomes quite natural to understand the success of the analogical methods used to develop quantum theories as methods of extending known theory to capture the behaviour of physical phenomena in a broader domain.
I do not take the perspectives presented here to be at all conclusive concerning the nature of physical theory and its relation to the world. I merely mean to show that by understanding the nature of our knowledge differently, we can understand how the methods we use to extend our knowledge can be rational and powerful, even extraordinary and surprisingly fruitful, without being either inexplicable or magical.