Simply because our brains are like that, whether we like it or not. It's not that people who judge fromappearance are dumb, it's just that it's the first thing we learn when we see a new person. Judging off appearance first isn't that wrong, it only becomes wrong if you don't change your opinion on the person after you actually get to know what they're like. Read about theHalo effect, it seems like it would interest you.