A confidential internal review reportedly found that one teenager was placed in a control group where protections against repetitive harmful content were intentionally inactive.
TikTok withheld an algorithm safety update from millions of people in the United States while it studied how the change would affect user engagement, according to a confidential company document reviewed by Bloomberg.
The experiment reportedly left about 10% of TikTok’s US users—roughly 15 million people at the time—on an older version of the recommendation system. The remaining 90% received an updated algorithm designed to prevent users from becoming trapped in repetitive streams of potentially harmful content.
TikTok had publicly announced in December 2021 that it was developing ways to interrupt clusters of videos involving subjects such as sadness, loneliness, extreme dieting and breakups. The company said that individual videos might not violate its rules but could create a negative experience when repeatedly recommended together.
However, the newly reported document indicates that the safeguard was not immediately activated for everyone.
Safety feature became part of an engagement experiment
TikTok reportedly used an A/B test, sometimes described internally as a backtest, to compare users receiving the new safety system with those remaining on the previous algorithm.
Such experiments are commonly used by technology companies to measure how product changes affect user behaviour. In this case, however, the feature being tested was intended to reduce repetitive exposure to sensitive and potentially dangerous material.
According to the document, TikTok wanted to determine how the update affected daily active users and other engagement measurements. The test reportedly continued for between six months and a year, significantly longer than the short experiments commonly associated with minor product changes.
Among the users assigned to the control group was Chase Nasca, a 16-year-old from New York who died by suicide in February 2022.
A 14-page internal review prepared by TikTok’s safety and algorithm teams in March 2023 found that Nasca’s account had entered a recommendation loop dominated by videos involving depression, hopelessness, loneliness, self-harm and suicide.
The review stated that TikTok’s strategies for preventing filter bubbles did not apply to his account “by design” because he had been included in the control group.
Thousands of videos examined
TikTok’s internal teams reportedly examined 7,563 videos delivered to Nasca during the final two weeks of his life.
Approximately 73% contained recurring themes involving emotional distress, relationship problems, mental-health struggles or suicide awareness but did not violate TikTok’s rules. Nearly 10% were found to breach company policies prohibiting content that promotes or normalises suicide and self-harm.
The findings do not, on their own, establish that TikTok or its recommendation system directly caused Nasca’s death. They do, however, raise serious questions about the risks of excluding users—particularly minors—from safety updates for the purpose of measuring engagement.
Nasca’s parents later brought a wrongful-death case against TikTok. The lawsuit was dismissed after the company argued that Section 230 of the US Communications Decency Act protected it from liability for third-party content recommended through its platform. An appeal remains pending, according to the reporting.
Internal review recommended major changes
The confidential review reportedly recommended reducing future control groups from 10% of users to less than 1%. It also proposed limiting such experiments to seven days rather than allowing the same users to remain without an update for months.
Even a 1% control group would leave at least 1.8 million US users without a particular safety feature during testing, the document reportedly acknowledged.
TikTok described the 2022 experiment as a routine test intended to determine whether the safety changes were functioning as expected. The company also said it has since refined the way it conducts similar tests, although it did not publicly provide detailed information about those changes.
Questions over TikTok’s statements to Congress
The timing of the internal review has also renewed questions about what TikTok disclosed to US lawmakers.
TikTok CEO Shou Zi Chew appeared before the House Energy and Commerce Committee on March 23, 2023, approximately two weeks after the company examined Nasca’s account. During the hearing, Chew described the safety and well-being of teenagers as a core priority for the platform.
In written answers submitted to Congress on May 4, TikTok said it did not believe use of the platform had caused users to die by suicide. When lawmakers asked the company to identify cases in which algorithmic A/B testing had harmed users, TikTok responded that its community standards applied to all content shown during such tests.
The answers did not mention the control-group experiment or the findings concerning Nasca’s account, according to Bloomberg’s reporting.
In a statement addressing the confidential document, TikTok said it remained deeply committed to protecting users, particularly teenagers. The company said it continues to invest in content-detection technology, enforcement teams and other trust-and-safety operations.
The disclosure adds to wider scrutiny of how social-media platforms test changes to recommendation systems and whether users are adequately informed when safety measures are being evaluated against growth and engagement targets.
It also presents technology companies with a difficult question: whether safety systems should ever be treated like ordinary product features when withholding them could expose unsuspecting users to foreseeable risks.
