In any case, the movement of labour is determined by the economy. In times of boom, shortages of labour are created, which suck in labour from abroad. Traditionally, in the UK this was from Ireland, which supplied unskilled and semi-skilled labour, which dug roads, and so on.
Hence, the idea of controlling immigration is quite weird, since you can't control boom and bust in the economy, and hence the flow of labour which expands and contracts accordingly.
So if you put a freeze on labour movements, you run the risk of freezing the economy.