sexism, racism, and other forms of oppression are not based on being “unkind” to certain kinds of people. That is a teeny, tiny, minuscule effect of oppression.
Oppression is the systematic, institutionalized, and ubiquitous devaluing and disenfranchising of particular groups. It’s not just some guy telling some girl that she’s stupid. Its the entire culture telling all girls that they are are shallow, brainless objects who are only valuable in the desires of a man. Racism isn’t some white person telling some black person they’re worthless. It’s the entire culture basing their concept of beauty on whiteness and talking about “the black community” as synonymous with crime and poverty.
Saying we shouldn’t worry about racism and sexism is basically saying “I want to ignore the fact that some people are systematically disenfranchised because it makes me uncomfortable.” But guess what? Those -isms are still there. It may sound nice to say “but we’re all people!” and yeah, we are. But not all of us get treated like people.
Your words do not exist in a vacuum. They do not exist without context. Oppression is not a matter of kindness or unkindness. Its about power structures constantly shitting on certain groups while giving others immense privileges. And that’s not going to go away if we all just hold hands and sing kumbaya together.