Content moderation on the internet is really tricky. It’s an absolutely brutal job when humans have to do it and the scale of the work is almost impossible to comprehend (though AI can help them quite a bit now). It’s also tricky to understand the entire ecosystem, leading to questions like “If they can shut down x, why can’t they just get rid of child porn?“.
I recently read Adam Aleksic’s book “Algospeak“, and he presented another angle that I hadn’t really considered; if you block some speech to try to help protect certain groups, you may mistakenly hurt their ability to discuss those same issues. From the book:
In an unfortunate catch-22, this problem happens because minority groups’ identities are considered “politically sensitive issues” by the algorithm. The keywords used by Black creators may overlap with the keywords used to talk about racism, because racism is frequently directed against the Black creators. But since discussion of racism is suppressed in the algorithm, many Black creators are inadvertently silenced. The same problem happens with homophobia and the queer community, or ableism and the disabled community. Now these groups can’t even talk about the very problems that content moderation is trying to protect them from.
Moderation on the internet is a such a complex issue, and things like this make it even moreso. Aleksic’s book unpacks issues like this and opened my eyes to a lot of things, and I encourage you to check it out.


