The atheist or agnostic claims that one ought not to believe in God if there is no scientific way to verify His existence. If we were to set this out in a simple syllogism, it would say:
We ought not to affirm the existence of anything for which there is no physical evidence.What's wrong with this argument? Well, an argument can be faulty either in its structure (form) or its content (matter). The form of the argument is sound: the premises lead to the conclusion, provided the premises are true. But are the premises true? Nope.
God is a thing for which there is no physical evidence.
Therefore, we ought not to posit the existence of God.
The biggest problem is with the premise "We ought not to posit the existence of anything for which there is no physical evidence." This premise assumes that only physical things, things able to be detected by observation and verified by the scientific method, exist. It claims that our only sure basis of knowledge is empirical science, that we cannot say that we know anything beyond what observation tells us. But this is not true. There are all kinds of things we know to be true that cannot be established by the scientific method.
For one, there are truths of our own interior experience. It is true that right now, I feel fine. It is true that I love my fiancee. It is true that you feel hungry. It is true that you hate the Lakers. All of these things are true, but there is no scientific experiment one can run to verify the truth of these things. They are not subject to empirical observation.
For another, there are moral truths. It is wrong to injure innocent parties. It is wrong to steal. We know these to be true, but we don't know that by observing human behavior and drawing the conclusion that these things are wrong. We don't derive our morals from behavior; we apply our morals to behavior. We don't determine their truth with test tubes and telescopes.
Nor are the very truths used by science to do its work. Science draws conclusions based on observation; but the rules of reason that science uses to draw those conclusions are not themselves based on observation. The Law of Identity (A equals A, A does not equal B) or the Law of Non-Contradiction (a thing cannot both be and not be at the same time and in the same respect) are two obvious, intuitive truths that structure our thinking and that we use to examine and evaluate our observations. Mathematical truths are not demonstrated by science, either. What experiment do you run to prove that two plus two equals four? It is pre-observational truth, what we call a priori. Truths based on empirical observation, like scientific laws, are called a posteriori. To use the argument above, you must deny all a priori truth; but if you try to do that, you cut your own legs out from under you. Certain a priori truths provide the condition for the possibility of science. The existence of these truths alone prove that not everything that is is demonstrable by science.
Indeed, the claim "It is true that only that which can be discovered by empirical observation (a posteriori) is true or real" is itself not an a posteriori claim, but rather an a priori one. The claim refutes itself!
This is all to say that the this materialist empiricist atheist must concede the fact that there are truths beyond those with which science deals. With that, the atheist must admit the possibility of things existing outside of the sensor range of empirical science. Then maybe, just maybe, there's a God after all.