I'd like to add my opinion in the form of the following two quotes from Feynman (whom Schwinger remembered in an epitaph as "the outstanding intuitionist of our age"):
"We know a lot more than we can prove."
"Mathematics is to physics as masturbation is to sex."
I really only agree with the first quote, which, if you think about it, says something quite deep.
For our proof-obsessed brethren, I recommend the book Proofs and Refutations by Lakatos. How is it, exactly, that we "know" things? Can we ever be entirely sure that a given proof is error free? If not, how is it different from having only a probabilistic (e.g., confidence level) sense of truth?
It is their attempts to deal with unwanted and unexpected refutations - to preserve a valuable theory in the face of imperfect axioms and proof methods - that teach mathematicians the true depths of their conceptions and to point the way to new and deeper ones.
Lakatos shows this by an account of the historical development of the concept of proof in mathematics and by showing in historical detail how certain valuable 'proofs' were preserved in the face of refutation. To this point Lakatos shows that the 'proofs' of the truth of Euler's number are no proofs at all. The great mathematician Euler noticed that for any regular polyhedron the formula V-E+F=2 holds where V is the number of vertices, E is the number of edges and F is the number of faces. Euler's and his successors' proofs fall before any number of counterexamples. Does this prove that the theorem is 'incorrect?' Or does it mean... that their concept of what constituted a regular polyhedron was deficient. Lakatos shows how these conceptions were modified over a couple of hundred years as counterexample after counterexample were faced.