Re mathematical truths.
We can know whether mathematical statements are true or false because they are made within systems which we have defined rigorously, with known axioms. eg for Jeremy's example within the system of number theory.
Similarly with science, we build models trying to best represent the world that we experience, and can get predictive results using those models. Those results are correct or not (or sometimes undecidable) within those well defined models, but whether they are absolutely true or false, in reality, we can never know. In practice we develop a model and stick with it if it proves useful, until we find a more useful one.
Other philosophies are available.