I think that if you have convinced yourself that sex is only permissible within marriage, and you haven't actually got married yourself, then believing that people who are having sex outside of marriage are really having a miserable time must be a way of seeking consolation for what you are missing out on.