Aristotle has a cosmological argument for why there is a universe.
And like every Cosmological argument, it's special pleading - the universe had to come from something, therefore I will posit something that didn't have to come from nothing, but that's not allowed to be the universe because reasons.
The New Atheists don't so ultimately their position is to say it just is or to just ignore the problem.
The New Atheists say 'we don't know', in the main. Some of them are a little more hopeful and say 'we don't know yet'. 'Science' currently has one or two promising hypotheses that work mathematically, but more evidence is needed.
Krauss tries to make science into a cosmological argument but can't. But even that is slightly better than Hawking who although not strictly a new atheist, the bloke who argued the death of philosophy and it's replacement with science. I believe Feser attacked Krauss for claiming that science is atheistic and since science describes the truth atheism is somehow the truth.
That argument sort of falls down given that it's not what Krauss says, it's just a pompous straw man.
Krauss says he thinks it's easier to be an atheist as a scientist, and a scientist as an atheist, but the doesn't say either is necessary.
That sort of argument is the sort I have countered by substituting the phrase science with Brobat toilet cleaner. Feser makes a similar argument......You see great minds think alike.
And, like you, his best arguments are against the things that no-one has said.
O.