Caution about increasing toxicity

De-escalate tense threads and reduce toxicity

Our confidence rating

Tentative

Share This Intervention

Last Modified

August 5, 2024 8:06 AM

What It Is

Using a classifier trained on a forum's conversations, alert users when a thread they are about to reply to is likely to become tense or toxic and give them real feedback on whether their reply will reduce or add to the tension.

Civic Signal Being Amplified

Welcome
:
Ensure user safety

When To Use It

Interactive

What Is Its Intended Impact

By making well-intentioned users aware of rising tensions in a thread and giving them feedback on the likely impact of their reply, users can tailor and adjust their comments to de-escalate tensions.

Evidence That It Works

Evidence That It Works

Chang et al. (2022) developed and tested a browser extension tool, "ConvoWizard", that alerts Reddit users if a thread they are about to reply to is "getting tense" (or not) and then gives those users real-time feedback on whether their reply will likely increase or decrease tension. The tool's predictive model was trained specifically on threads in a given subReddit using moderator labels. The researchers tested the tool with 14 volunteer participants, using a within-subjects design that randomly assigned when ConvoWizard would "pop-up." Matching participants' replies on similar threads, researchers observed that participants took more time to craft their reply on tense threads when ConvoWizard popped up as opposed to when it didn't. Participants also tended to revise their replies to be more likely to de-escalate conversations. (Note: all effects we include are statistically significant, unless otherwise stated.)

While these initial results are promising, given the small sample size and within-subject design further testing is needed to confirm that ConvoWizard is broadly effective.

Why It Matters

Given the nature of online communication, conversations can frequently escalate into toxic exchanges, even among well-meaning users. A tool that makes those users aware of the risk of escalating toxicity can help well-intentioned users not be inadvertent contributors to toxic exchanges.

Special Considerations

There are two considerations and a risk for tools that use ConvoWizard's approach. First, ConvoWizard's effectiveness may be due to its predictive model which uses discussions in a specific subReddit, so it may be less effective in mass-forums (Twitter, Facebook) where models could be less precise. Secondly, the existing tool and tests assume that a forum's participants voluntarily use the tool; there is no evidence that a ConvoWizard tool would be effective for users who did not choose to use it. Moreover, as the authors note, there is a risk that ill-intentioned users could misuse the tool to escalate tensions and toxicity.

Examples

This intervention entry currently lacks photographic evidence (screencaps, &c.)

Citations

Thread with Caution: Proactively Helping Users Assess and Deescalate Tension in Their Online Discussions.

Chang, J. P., Schluger, C., & Danescu-Niculescu-Mizil, C.
Proceedings of the ACM on Human-Computer Interaction

Is this intervention missing something?

You can help us! Click here to contribute a citation, example, or correction.

Further reading

Back to Prosocial Design Library