The Dangers of Context Collapse
Context collapse is a favorite tool of Internet trolls and other disruptive influences, but it has so many other implications. It’s important to avoid it, and to recognize where context collapse is leading to incorrect beliefs or actions.
Context collapse is a favorite tool of Internet trolls and other disruptive influences1, but it has so many other implications. It’s important to avoid it, and to recognize where context collapse is leading to incorrect beliefs or actions.
Context collapse itself is the phenomenon of highly-contextual information being used, purposefully or otherwise, in an ambiguous manner which leads to confusion. A pretty common example of this is the word “theory;” in colloquial speech it means the same thing as “hypothesis” in scientific speech. This leads to a very common chain of reasoning: the theory of evolution (in the scientific sense) becomes “just a theory” (in the colloquial sense), which then is purposefully used to sow seeds of doubt in people using the phrase colloquially, when scientifically-speaking it’s established fact.
This ends up being a huge issue in a lot of unexpected places, which can often result in extremely unfortunate results which can negatively impact peoples' livelihoods, even with the best of intentions.
As a rhetorical device
The phrase “African-American” gained a lot of traction in the 90s as a politically-correct alternative to “Black” (which was, in turn, seen as more progressive than “colored”). The intention was to refer to Americans who are of an African ethnic background, due to the very specific historical and systemic issues which affect Black Americans who came to this country by means of slavery.
However, it didn’t take long for this phrase to be context-collapsed by people who use it to mean “an American who came from Africa,” including and especially white immigrants who came from South Africa — often the ones who were willing participants in or benefactors of Apartheid.
There’s a current trend on Twitter for people praising Elon Musk for being a “successful African-American,” often calling people who are opposed to him “racists” because of their opposition to him. It’s a clear and flagrant context collapse being used to attempt to silence people on a completely ridiculous basis, which some people start to actually believe as being factual.
It muddies the waters, and it causes endless confusion, with some people actually starting to believe that Elon Musk literally has a Black ethnic identity because he’s “African-American.”
Discourse and trolling
There are many social avenues on the Internet where context collapse, willful or otherwise, is used to attack people based on beliefs that are unrelated, or possibly in direct opposition, to positions they actually hold.
I can think of two silly examples in my own experience, and I have seen this happen with much more severe implications to others.
For example, when the J. J. Abrams Star Trek reboot released, I wrote a review critical of it. One of my points of contention was the use of time travel as a reboot device, in a way that destructively attempted to erase the original timeline, where it was specifically trying to undo the amazing continuity that existed before. This led a few people to attack me in other threads as not being a true Star Trek fan, because in their mind I’d said that time travel doesn’t belong in Star Trek, and had therefore been unaware about the many time travel plots in Star Trek.
Another example, in a thread about mixed drinks I talked about how my favorite way of making a margarita was keeping it simple, with tequila, agave nectar, and lime juice, the reason being that tequila is fermented from agave nectar in the first place and it pairs very well. This somehow became “fluffy only drinks hard liquor mixed with sweetener” and was used to attack my character in… bizarre ways.
Other examples that I see all the time are widespread discourse and callout posts on Twitter and Mastodon. For example, someone will post a thing about the difficulty in classifying and abolishing child pornography, which someone then interprets as being against the abolishment of child pornography, which then ratchets to “this person supports child pornography and pedophilia,” which then leads to everyone interpreting the original thread under that lens and then the person is forever branded a pedophile.
In one particularly ironic example of that last one, the person was trying to raise awareness of context collapse as an issue, and the end result was them becoming context-collapsed!
And, unfortunately, it becomes difficult to defend the people in this situation because any defense of them is seen as “defending pedophilia,” continuing the chain of context collapse.
In “The Horrifying Panopticon of West Elm Caleb” (which is well worth a watch), Sarah Z uses a phrase which really stuck with me: “The abuser adopts the language of the abused.”
This comes into mind whenever I see the term DARVO being used to attack people who are simply trying to defend themselves. DARVO is short for “Deny, Attack, Reverse Victim and Offender.” The idea behind it is that when someone is being called out for being an abuser, the tendency is to attack the accuser and call themselves the true victim of this attack.
It’s a good phenomenon to be aware of, but the context collapse of the term “DARVO” then leads to its weaponization against people even further.
(To be clear, I am not saying that DARVO isn’t a thing! It absolutely is! But it isn’t so cut-and-dried and accusations of it are often themselves a form of abuse, or are used to enable it.)
For example, I have seen many situations where someone will be accused of being a sexual predator, and when they deny the allegations, that leads people to shout “DARVO!” because they’re saying that any of the behaviors listed is a problem, not that it’s all of the behaviors together.
In some particular cases of this, there will be circumstantial evidence, or maybe even some actual wrongdoing on the part of the accused (but not to the extent that they’re being accused of), and any attempts at defending themselves or attempting to clarify it is then labeled as DARVO and is therefore more proof of their own guilt.
I’ve also seen situations where the actual issue between the two parties is resolved, but then someone else decides to carry a torch on the accuser’s behalf, and then the accused questions this third-party participant’s motives in doing so, which then triggers the “RVO” part (even though “victim” and “offender” come from a different context!) and therefore becomes another case of “DARVO.”
Algorithmic context collapse
Another troublesome situation occurs when well-meaning humans write algorithms which end up leading to a chain of events which severely and negatively impact creators and their livelihood.
One particularly common case of this: LGBTQIA+ and its related terms are often classified as being about “sexuality” (when there’s a broad spectrum of topics that it’s about, not just sex and sexuality). There’s a broad swath of systemic and social implications to members of the LGBT community, which affects people in daily life, not just in terms of sexual attraction. For example, asexuality is about a lack of sexual attraction which in turn leads to many forms of discrimination, and gender identity-related topics get lumped in with sexuality when the larger implications about it are purely societal, not sexual.
However, LGBT-related terms get thrown into the “sex and sexuality” bucket, and “sex and sexuality” often gets thrown into the “pornography” bucket.
In the mid-2000s, Amazon delisted all LGBT-related books from search results, including books intended for children, because of this. This was only discovered because a children’s author noticed their books weren’t appearing in any search results, and they had to go on a large, public crusade to get the issue fixed.
There are ongoing issues with YouTube demonetizing, delisting, and sometimes even deleting LGBT-related content because a word used gets interpreted as pornographic or sexual (or it’s maliciously flagged as such by bad actors who want to game the system).
This also leads to a vicious cycle where positive LGBT-related content is suppressed or marked as pornographic, which makes it harder to find and also sets up discourse about how LGBT identities are inherently pornographic as well. This then ratchets into ridiculous beliefs like “gay teachers are grooming their students” which then leads to the horror show of the current push towards “Don’t Say Gay” bills.
I am a huge proponent of Bandcamp. They are by far the best place for musicians to sell their music and for fans to buy it. They have the best metadata, the most generous revenue split (especially on Bandcamp Friday), the most amazing post-sale support and library access, and so on. I have been on their platform since nearly the beginning, and have also done my part to convince others to move over to them. Even with the recent and poentially-troublesome acquisition by Epic Games I still think it’s the best thing around. (Although I do hedge my bets, of course.)
Last night I discovered that none of my music was showing up on the Bandcamp search, however, nor was it appearing in the discovery feed. Which meant that a huge source of potential revenue and visibility was lost to me, and it made no sense what was going on.
So, let’s talk a bit about how Bandcamp artists are set up.
The artist has top-level tags; these are intended to just list what specific genres they work in. This is a problem for my music because you are limited to five genre tags and one broad category, and I make music in… way more than five genres.
Fortunately, you can also apply tags at the album and song level. These tags are meant to be both content-specific genres and also topic tags for what the song is about. This is very helpful for music discovery, because it lets people search on their specific topics of interest, such as furry or lo-fi or coffee or the like.
Anyway, since tags can be genres, it makes sense for song tags to be applied to album tags, and for album tags to be applied to artists, right?
Okay, so what happens when you write a couple of songs that are political and about the alt-right, and tag them “alt-right?”
Well, those songs are about the alt-right, but they aren’t in an “alt-right” genre. But the system doesn’t know that they’re topic tags and not genre tags, so they’d might as well propagate to the albums, right?
And the system sees that this album is tagged “alt-right,” so obviously whatever “alt-right” means, this artist does something about it, right? So let’s propagate the tag to the artist…
…the end resulting being that my entire artist account was tagged as “alt-right,” which meant that the moderation algorithm saw that, believed me to be alt-right, and delisted all of my music.
It’s kind of hilarious, except that this means that my last several albums never got any discovery at all, and who knows how many sales and potential new fans I lost out on as a result!
Basically, this incredibly well-meaning system ended up flagging me as being alt-right, because I wrote two songs critial of the alt-right, and this greatly impacted my potential livelihood. And I had no idea any of this was happening — for at least two years I wasn’t showing up in search results or the discovery queue, because I’d been ironically labeled something that I was protesting.
Fortunately, I have a friend who works for Bandcamp who was able to escalate this issue, and I’ve been re-listed, and they’re re-evaluating this system and the processes involved.
Unfortunately, I can’t help but imagine how many other artists have been affected by this without knowing!
And I doubt that any alt-right-positive bands would actually be tagging themselves as “alt-right,” as that’s not a label that they use for themselves.
So this was a well-meaning moderation decision put in place by a human but never properly evaluated, with far-reaching implications that have the opposite effect of the intent.
This is a perfect example of the old saying, “The road to hell is paved with good intentions.”
I still love Bandcamp and I’m glad that I was able to get this situation resolved, for me. But resolving it just for me doesn’t fix the problem.
Should Bandcamp list their tags which lead to moderation and delisting? Probably not; that would be too easy to game by the very people who they’re intending to delist. But the system definitely needs to work differently than it does. A band singing a song about chocolate doesn’t mean the band is made of chocolate.
At the very least they need to separate genre tags from topic tags to at least provide better context, not propagate topic tags upwards, and not categorically delist an entire account because of a self-provided tag without context.
I’ve heard from my friend that they’re working towards a suitable form of compensation for this issue. My hope is that they’ll write a blog post about this issue with a link to my Bandcamp page and the songs which triggered the bad moderation, as well as finding and sharing some other artists who were impacted, and ideally putting all of the impacted content back into discovery for a little while to make up for it.
With all that said, I am incredibly grateful to my friend for taking it upon herself to fix this situation so quickly after it was discovered, and for Bandcamp for correcting the issue. I am disappointed that the situation happened in the first place, but I am confident that Bandcamp will do what it takes to make it right.
Whenever someone is making accusations towards the beliefs or character of someone else, it’s worth looking into the source to make sure that there isn’t context which mitigates this. No matter how many people you see favoring a particular take on things, it’s possible that there’s a major, unfortunate case of context collapse taking place.
Content moderation should be done by humans, not by algorithms. Algorithmic moderation is helpful at scale, but all final decisions must be reviewed by someone who has the context, and if a moderation is potentially impacting the livelihood of the people being moderated, they must be aware of this and have a path forward to get it reevaluated, again, by a human who is aware of context.
Context collapse is easy to have happen from the most innocent or well-meaning/well-intentioned of sources, and the implications can be horrifying to those who don’t deserve it.
This article previously mentioned “the alt-right” here, and this had the unfortunate effect of immediately (and ironically) muddying the waters for the ongoing conversation that resulted. This is the only place where I have removed the use of that phrase, and I hope that it makes it more clear that this article was never intended as a discussion of the alt right to begin with. ↩
Before commenting, please read the comment policy.
Avatars provided via Libravatar