What would stop them - the cost of living.
If you aren't working then you can't claim benefits so you'd have to have enough independent wealth to be able to afford to live in the UK without working. And given that the cost of living is low in Poland, but high in the UK there won't be many who would be able to afford to do that - so economics prevails. I'm not sure of the figures but I doubt there are many retired and/or deliberately non working Poles in the UK.
But actually if they have enough independent wealth to be able to live in the UK, without working and without relying on benefits, why wouldn't we want them here. They would be contributing to the UK economy by spending their independent wealth here. We don't seem to stop independently wealthy Americans, or Russians or Chinese coming and living here and spending their fortunes here and none of those are in the EU - why would it be different for EU nationals.
You are being reasonable, and I am being completely unreasonable (as I am with my systems in destruction tests).
Again I am testing with silly numbers.
So 20 million people arrive, they have no money , no means of working (and for this test they do not care)
Do we let them starve?
Do we round them up and ship them off?
If not, then we have to house and feed them. The system only stops people coming when the country is on its knees.
This seems like a bad system, and one to which the exit people can play to. The immigration non system is floored.
I could pose the same question about the USA. What if everyone wanted to live in Florida from the US, and I guess there is nothing to stop that happening other than system failure.
It's the system overload or failure that I usually try to prevent, but it seems that in this and the US case, there is no protection, and things just get very bad to the point where people leave because things get bad.