It seems like humans like to categorize ourselves, make prejudgements, and have large prejudices just by nature. Unfortunately this results in many people labeling themselves, others, and just generally creating toxic and grouped/boxed environments. Personally, I think we should all see ourselves as human as our main characteristic and quit boxing ourselves with labels.
It doesn't really bother me most of the time, you've got to remember how much worse people have it to you and judge if what's happening to you is really as bad as you think.