Scientific theories are not based on direct evidence at all, in most cases. You know that and hope that no other person will question you on the matter.
I suspect the problem here is that you are not aware of what a scientist means by the word "theory".
In everyday usage "a theory" is an - as yet - untested idea. Too much Sherlock Holmes! The word a scientist would use is probably
hypothesis, but a hypothesis is an idea, a concept,
which is potentially testable. A hypothesis test will either allow the hypothesis to be confirmed or rejected.
If a hypothesis has been tested then it gives rise to further tests. All of these tests give results. After a sufficient number of successfully tested hypotheses have been tested then a pattern or framework which explains the results may develop. It is this explanatory framework which is the theory.
When it has been developed sufficiently, the theory will enable predictions to be made. If the predictions are supported empirically, then the theory is developed.
"Evolution" is a theory, which means it is not just a speculative guess. It is the consequence of thousands of studies, experiments, observations. These thousands of studies, in general, support each other. The theory of evolution gives just about as complete and accurate an understanding of the development of homo sapiens as it is possible to get.
The limiting factor when studying evolution in homo sapiens is the length of time taken by successive generations to reproduce. There are other species which have a short reproduction cycle. Successive generations can be studied in a fairly short time and evolution can be seen in action.
As Torridon has stated, the ability to digest milk, the possession of a white skin and blue eyes are all evidence of mutations in the human genome and hence evidence that evolution is real.
