Twitch has released a tool called Suspicious User Detection to protect streamers and their chat from abuse.
The Suspicious User Detection tool is said to be developed in response to hate raids and harassment on the platform, particularly among streamers from marginalized groups.
The machine learning is being used to catch users who are avoiding channel-level bans by highlighting potentially suspicious activity in chat.
When a user bans someone from their channel, they should be permanently barred from the channel’s community. Unfortunately, bad actors frequently create new accounts, return to Chat, and continue to harass others. The said tool is said to help users identify users based on a variety of account signals.
Suspicious activity can be flagged at two levels. Any message in chat sent by ban evaders will be visible only to streamers and their moderators, while messages sent by potential ban evaders will be visible but flagged to streamers and moderators. Messages can then be deleted, and users can be banned.
Across all streams, the tool will be turned on by default. Streamers, on the other hand, will be able to turn it off or change the settings to be more cautious, such as preventing any and all messages flagged by the tool.
The one thing users should be aware of, especially around launch, is that no machine learning system will ever be 100% accurate, which is why suspicious activity bans aren’t automatic.