YouTube’s algorithm routinely suggests videos that violate its own guidelines

The non-profit Mozilla Foundation has released new data suggesting that YouTube continues to recommend videos to users that violate its own content policies. The Next Web reported on the findings.

The organization came to its findings using crowdsourced data from RegretsReporter, a browser extension that users can install and use to report information on harmful videos and what led them there. More than 30,000 YouTube users have used the extension to report harmful content.

According to Mozilla’s latest research, 3,362 videos were flagged as “regrettable” between July 2020 and May 2021. Of that total, 71 percent of the videos that users deemed regrettable were surfaced by YouTube’s recommendation algorithm. A video can be deemed regrettable for various reasons, such as if it constitutes spam or hate speech.

Unusual transparency — The findings are useful because YouTube’s algorithm is essentially a black box. Every user potentially sees something different in their feeds, so it’s nearly impossible for outsiders to quantify how well YouTube is performing its stated mission of ridding itself of harmful content. One person might be led down a rabbit hole of extremist content, while another never sees content of that nature at all.

To that end, Mozilla says that YouTube users in non-English countries were 60 percent more likely to report finding videos they considered disturbing. This may indicate YouTube expends more energy and resources policing English-language content while letting its algorithms have freer rein where other languages are concerned.

User attention — Like any other major platform, YouTube relies on its algorithm to show users content they might like and keep them on the service as long as possible. You start watching one clip from your favorite show, and the recommended section accompanying the video shows you more related clips to keep you hooked. In the past, YouTube has said its algorithm drives about 70 percent of daily watch time.

One adverse consequence of algorithmic recommendations is that they can be based on crude signals that encourage bad behavior. Videos that keep people watching and engaged have historically received a boost from the algorithm, but these types of signals could potentially benefit sensationalist videos designed to outrage people and in turn, generate ad revenue.

YouTube has responded over the years with tweaks that are supposed to help, like suppressing “borderline” videos that toe the line between what’s acceptable and what violates its terms of service. The company only ever provides vague data to quantify its success, however. Mozilla hopes its efforts will push lawmakers to seek greater transparency into how YouTube decides to program its algorithm, and what incentives that creates.

Overall, it’s very much a game of cat-and-mouse to keep YouTube safe when billions of hours of video are uploaded each day. That’s really the problem with the platform model — allowing anyone to upload almost anything has made YouTube into a giant, and moderating it closely is at odds with the company’s own goals. So it will always try to use algorithms, especially considering how much success it’s had with them when it comes to identifying copyrighted content, for instance.

But computers still aren’t great at understanding context or nuance, and that’s why human moderation remains essential, otherwise, companies can easily end up violating their own policies in the incessant pursuit of attention.