The trust & safety paradox: Why spending more on safety hasn't made social media safer

Every major social media company now runs an entire industry's worth of infrastructure dedicated to keeping users safe. Trust and Safety (T&S) teams, content moderation vendors, age-verification partners, crisis-response hotlines baked into apps, machine-learning classifiers trained to catch grooming and self-harm content: the spending is real, and it isn't small. And yet the headlines keep coming, with lawsuits, regulatory settlements, congressional hearings, and a steady drumbeat of reporting on the harm these platforms cause, especially to kids.

This is the paradox at the center of the industry. Safety investment and safety failure are rising together, not trading off against each other. Understanding why requires looking past the dollar figures and into how these companies are actually built.

The latest, biggest example: Meta's $17 billion settlement

In late August 2026, Meta agreed to one of the largest settlements in the history of Big Tech litigation. A bipartisan coalition of 51 attorneys general announced a proposed settlement resolving a lawsuit alleging that Meta designed and deployed harmful features on Instagram and Facebook that drive compulsive use by children and teens, all while misleading users, their families, and the public about the existence and severity of those risks.

The numbers are staggering on their own. Meta said the settlement includes a payment of approximately $18 billion, distributed in annual installments over a 10-year period, money earmarked to fund state youth online safety initiatives. The case had been consolidated from a 2023 lawsuit in which California, Colorado, Kentucky, and New Jersey represented a larger group of states alleging Meta designed its apps to be addictive to kids and knew the risks but hid that information from the public. The states also alleged Meta violated federal children's privacy law by collecting data on children under 13.

What makes this case such a clean illustration of the paradox is what Meta agreed to change going forward, because those changes describe, in granular detail, safety work the company apparently hadn't done at scale before now. Under the deal, Meta committed to a default two-hour daily time limit for teens that can only be turned off with a parent's permission, cumulative across Facebook and Instagram, along with a default nighttime usage block and, more broadly, enhanced age-assurance measures to prevent children from accessing the platform or age-restricted content, plus new tools to help parents protect their kids online. Meta will also bring on an independent auditor with expansive access to information and resources and the right to raise concerns directly with the attorneys general.

In other words, the settlement itself functions as an admission that meaningful guardrails (defaults, not opt-ins) were missing from products used by tens of millions of teenagers, at a company that has spent years telling the public and regulators it took youth safety seriously.

Zoom out, and the pattern predates this case by years. Meta's own regulatory filings show a long trail of enforcement actions: a $725 million settlement in 2022 tied to prior data-practices litigation, a $5 billion FTC penalty and consent order in 2020 requiring significantly enhanced privacy compliance processes, and a separate California settlement in December 2025. And the current wave isn't over. A New Mexico attorney general's case, expanded to include content-moderation claims, was scheduled for trial in September 2026, and a separate New Mexico public-nuisance case had already produced a $567 million abatement-fund order in August 2026.

Each of these came after, not before, Meta had built out substantial safety infrastructure. That's the uncomfortable part.

The money being spent on safety is not small

That last point deserves numbers, because the scale of industry safety spending is genuinely large, even before this settlement's $17 to $18 billion is added on top.

Meta has said that since the 2016 U.S. election it has spent more than $13 billion on safety and security efforts, and that it employs over 40,000 people, including contractors, working on those issues. That figure was disclosed in 2021, in a blog post published specifically to push back on Wall Street Journal reporting, based on leaked internal documents, that showed the company's own researchers flagging serious problems with harmful content that went unfixed despite the investment.

TikTok has made similar commitments. The company told the U.S. Senate Judiciary Committee in 2024 that it expected to invest more than $2 billion in trust and safety efforts that year, with a significant share going toward its U.S. operations, and that it employed more than 40,000 trust and safety professionals worldwide. At the same time, TikTok has been shifting much of that effort toward automation. By 2025, the company said roughly 80 percent of the content it removes for violating its policies was caught by automated systems rather than human moderators, and it laid off hundreds of human moderation staff as part of that transition, even as it faced a wave of state lawsuits over the platform's effects on young users.

Google, Snap, and other major platforms don't break out comparable trust and safety figures as cleanly in public disclosures, but industry-wide, the pattern is consistent: tens of billions of dollars and tens of thousands of people, concentrated at exactly the companies now facing the largest lawsuits and settlements over the harms those investments were supposed to prevent.

Why investment and harm can rise together

A few structural reasons explain why more safety spending hasn't translated into fewer scandals:

The business model and the safety model pull in opposite directions. Engagement (time spent, notifications opened, sessions per day) is the metric that drives ad revenue. Safety features that reduce engagement, like time limits, friction before sharing, or weaker algorithmic recommendations, work against the thing the company is otherwise optimized to maximize. T&S teams can build excellent tools, and can spend billions of dollars doing it, but they rarely get to set the defaults that matter most, because defaults are a product decision, and product decisions answer to growth targets first.

Scale defeats even well-resourced systems. Billions of pieces of content are uploaded every day, across dozens of languages and cultural contexts, reviewed by a mix of automated classifiers and human moderators who are themselves stretched thin. No amount of budget makes this a fully solved problem. A safety team can catch the overwhelming majority of violations and still let through the harm that ends up in a lawsuit, because "overwhelming majority" at that scale still means enormous absolute numbers of misses.

Safety spending and legal liability serve different masters. Money spent on classifiers, moderators, and industry partnerships with groups like the National Center for Missing & Exploited Children or the Tech Coalition genuinely reduces certain harms. But it doesn't retroactively fix product decisions made years earlier, like addictive notification design, infinite scroll, or algorithmic amplification of engaging-but-harmful content, that are now the subject of litigation. Companies can be simultaneously investing heavily in future safety and being sued for past design choices. Both things are true at once.

Disclosure gaps compound the problem. Several of the recent cases hinge less on whether harm occurred and more on whether the company told the truth about what it knew. That's a reputational and legal risk that safety spending alone can't buy down; it requires actually changing internal incentives around what gets escalated, studied, and disclosed to the public and regulators.

Why companies don't just do the bare minimum

Given how much money gets spent on safety, it's fair to ask why companies haven't already done the comparatively cheap things the lawsuits describe, like flipping a default time-limit toggle. A few reasons that gap persists:

1. Most safety spending funds a different problem than the one lawsuits are about. The bulk of Trust & Safety budgets goes toward content moderation: catching CSAM, terrorism content, scams, and spam. That work is genuinely hard and expensive, and it absorbs most of the budget. But recent lawsuits aren't primarily about moderation failures, they're about product design: infinite scroll, notification design, algorithmic engagement optimization, and settings that default to on rather than off for minors. Flipping a default is technically cheap, but strategically expensive, because it costs engagement. That makes it a product and growth decision rather than a line item T&S teams typically control.

2. Defaults are decided by product and growth teams, not safety teams. A safety team can recommend turning a feature off by default for minors, but that recommendation competes against a growth team whose goals are tied to engagement metrics and who usually has more organizational leverage. Spending on safety buys the ability to catch what's already illegal or clearly against policy. It doesn't buy safety teams a veto over the product defaults that drive engagement.

3. Litigation risk was, for years, priced as a routine cost of doing business rather than a near-certain outcome. Thirteen billion dollars in safety spending sounds enormous until it's set against annual ad revenue in the hundreds of billions, or even against a single settlement of $17 to $18 billion, which is roughly comparable to everything Meta says it has spent on safety since 2016. If the internal calculation is a possible future settlement, discounted and spread over a decade, against giving up billions in engagement revenue today, the settlement can look like the cheaper option.

4. Voluntary safety changes and court-ordered ones are different categories internally. A company can build genuinely good moderation tools and still resist committing to specific defaults, because a voluntary feature can be quietly rolled back or tested away if it hurts engagement, while a legally mandated default cannot. That distinction is likely why the Meta settlement includes an independent auditor with reporting rights directly to the attorneys general: the plaintiffs are betting that voluntary commitments don't reliably survive contact with a quarterly earnings call, so this one isn't voluntary.

What the settlement actually signals

The Meta settlement is being described by the plaintiffs as a turning point: changes that will reduce the risk of harm from Meta's platforms, delivered within months rather than years. If the mandated defaults hold up (genuine time limits, genuine age verification, genuine auditor access), it will be a meaningfully different product than the one that prompted the lawsuit.

But it's worth sitting with the paradox rather than resolving it too quickly. The lesson isn't that Meta's safety teams have failed, or that Trust & Safety spending is theater. It's that safety work bolted onto a growth-optimized product will keep losing to the product's core incentives until safety defaults are legally or structurally mandated rather than voluntarily offered. That's arguably the real function of settlements like this one. They don't just punish past behavior; they force safety decisions out of the "nice to have" column and into the terms of a binding consent judgment, which is often the only place, in this industry, where safety reliably wins.