Steve:- imho a Jap calling you "Korean" is a pretty major insult.
(Japs' culture as a whole hates Koreans, not just a recent thing.)
I guess it boils down to the context of then current conversation.

But don't go around telling any Japs they look like Koreans.

Especially the ones with tattoos...