I think this is a completely logical reasoning. I'd wager that if you had led with this explanation instead of jumping into calling people racist you would have made a better impact.

Also, this reasoning is more along the lines of "may add to racist caricatures" rather than "this is a racist caricature". In that case it makes sense to "avoid just in case" rather than a directly racist act in itself.

If your goal is to educate people to make an actual impact, then avoiding jumping into hyperbole and attacking people directly should be your goal.

Remember to attack the argument or action, rather than the individuals.

Additionally, when someone jumps right into hyperbole and aggressive language towards people, it becomes more of a show as if the goal to be seen as righteous rather than being focused on education and making a real impact.