Here's a data point, I am a white male and I have no fear of AI at all.
I do have irritation that so many articles of this nature come down to "white men bad, everybody else good". It's not as if all white men or even most white men are at the top of the intelligence tree.