Perhaps I have been neglecting Duhem in my recent selections. The difficulty for me is that I have only three books by him, a biography by Martin, and the works by Jaki... and I do not know French... and as enticing as it is, I have no time to learn French at present.
So for now, I will try to proceed with what I do have.
Here, then, is a brief view of Duhem's thought...
...Pierre Duhem held firmly to separating physics from metaphysics: he saw in the history of physical theories, whether they were based on continuous or discontinuous images, or whether they were of the field or atomic type of physics, a proof of our radical inability to reach the depths of reality. It was not that Pierre Duhem, a convinced Catholic, rejected the value of metaphysics; he wished to separate it completely from physics and to give it a very different basis, the religious basis of revelation. This preoccupation with a complete separation of physics from metaphysics led him, as a logical but curious consequence, to be ranked, at least with respect to the interpretation of physical theories, among positivists with an energetistic or phenomenological tendency. In fact, he summarized his opinion concerning physical theories in the following conclusion: "A physical theory is not an explanation; it is a system of mathematical propositions whose aim is to represent as simply, as completely, and as exactly as possible a whole group of experimental laws."
Physical theory would then be merely a method of classification of physical phenomena which keeps us from drowning in the extreme complexity of these phenomena. And Duhem, arrived at this positivist and pragmatist conception of nature bordering closely on the conventionalism (commodisme) of Henri Poincaré, was in complete agreement with the positivist Mach in proclaiming that physical theory is above all an "economy of thought." For him all hypotheses based on images are transitory and infirm; only relations of an algebraic nature which sound theories have established among phenomena can stand imperturbably. Such, in the main, is the essential idea which Duhem produced about physical theory. It certainly pleased the physicists of the school of energetics, his contemporaries; it certainly is also favored by a great number of quantist physicists of the present day. Others were already finding it or will still find it a little narrow, and will reproach it for diminishing too much the knowledge of the depth of reality which the progress of physics can procure for us.
We must be fair and emphasize the fact that Duhem did not fall into the extremes to which his views might perhaps have led him. He believed instinctively, as all physicists do, in the existence of a reality external to man, and did not wish to allow himself to be dragged into the difficulties raised by a thoroughgoing "idealism." Hence, taking a position which is a very personal one at that, and separating himself on this point from pure phenomenalism, he declared that the mathematical laws of theoretical physics, without informing us what the deep reality of things is, reveal to us nonetheless certain appearances of a harmony which can only be of an ontological order. In perfecting itself physical theory progressively takes on the character of a "natural classification" of phenomena, and he made precise the meaning of the adjective "natural" by saying: "The more theory is perfected, the more we apprehend that the logical order in which it arranges experimental laws is the reflection of an ontological order." In this manner, it seems, he had been led to mitigate the rigor of his scientific positivism because he felt, and we think justifiably so, the force of the following objection: "If physical theories are only a convenient and logical classification of observable phenomena, how does it come about that they can anticipate experiment and foresee the existence of phenomena as yet unknown?" In order to answer this objection he really felt that we must attribute to physical theories a deeper bearing than that of a mere methodical classification of facts already known. In particular, he was clearly aware, and some passages of his book show this to be so, that the analogy of the formulas employed by physical theories bearing on different phenomena most often do not reduce to a mere formal analogy but may correspond to deep connections among diverse appearances of reality.
Such in the main is the conception which Duhem propounded concerning the scope of physical theories - an idea more subtly nuanced in the end than one might first believe. ...
[Prince Louis de Broglie, 1953, from the Foreword to Duhem's The Aim and Structure of Physical Theory, ix-x. Translated by Philip P. Weiner from the second edition, 1914]