That's right. The idea of atheism comes after the idea of theism so if the baby has no concept of Gods then it can't choose to be an atheist, or entertain anything along those lines.
The idea of atheism, yes, but the atheism comes before the idea of it.
Before the invention of the car no-one had any cars - the fact we didn't have a word or an idea for them doesn't change the fact they didn't have one.
Before we have a word for the idea of god we can't believe in that idea of god - the absence of belief is atheism, it doesn't matter why you have that lack of belief.
It might be worth, in some circumstances, clarifying that sort of inherent, ignorant atheism, I suppose, but it doesn't change the classification, just refines it.
O.