It's white man's burden BS slipping back in, where people (like Robert E. Lee) would say "We don't want to enslave these people, it's just our duty to civilize them."

I assume it's some kind of mechanism for maintaining their cognitive dissonance.