Don't go to unsafe neighborhoods, that are problematic for everybody. But, I don't think you are going to be racially vilified if you are a visitor. I lived there for 7 years and never was anybody explicitly racist to me. There may be undercurrents of racism, but because it is socially unacceptable to be racist, most people wont show it even if they have those feelings.
So, I don't think you need to worry about racism. Just take precautions like not going to unsafe places, the same way you would do in China.
Moreover, there are so many Asians everywhere in US (maybe not in villages, but almost every city has its share of Asians.), you don't need to worry about being singled out.