For just under a year from 2022 to 2023, George Volichenko had what I have come to think of as the median experience of working in trust and safety at Meta.

Hired to work on Instagram’s “teen mental well-being team,” the data scientist soon found that Meta made it difficult to ship features that meaningfully addressed the problem. “I was not happy with the limited freedom that the team was given to actually devise and test and launch features that were actually moving the needle and helping teens,” Volichenko testified in federal court Monday, as part of the states’ blockbuster lawsuit against Meta alleging violations of consumer protection and privacy laws.

For example, he observed that after Instagram introduced an opt-in feature that prompted teens to take a 10-minute break after extended scrolling, just 0.165% of them enabled it. When he brought up the low adoption of safety features with his manager, Bloomberg reported, “he was told that he shouldn’t worry about the adoption figures being low for the tools ‘because the team exists partially to protect the company against the upcoming lawsuits.’”

On Wednesday, whatever legal cover Meta’s Potemkin well-being teams were meant to provide fell to a settlement of up to $17.1 billion with 47 US states, the District of Columbia, and US territories. Here are Cecilia Kang and Eli Tan in the New York Times:

In a dramatic capitulation, the owner of Facebook and Instagram agreed to the financial penalties for violating federal child privacy and states’ consumer protection laws, the states announced. Meta also agreed to limit how long teenagers can spend on its platforms and to bans on features that stoke mental health issues, striking at the heart of the company’s business of engagement for advertising.The settlement effectively ends a bellwether federal trial in the US Northern District of California in Oakland, where California, Colorado, Kentucky and New Jersey were seeking roughly $200 billion over accusations that Meta harmed children. The states filed their agreement with Meta on Wednesday morning in that court, where Judge Yvonne Gonzalez Rogers is expected to approve it.Separately, Meta said on Wednesday that it settled with Texas for about $1 billion over similar allegations. The company still faces numerous other lawsuits from school districts and individuals, some of which are scheduled for trial in the coming months.

For the most part, the agreement requires Meta to honor terms that you may be surprised are not yet required by US law: limiting teens to a cumulative two hours across Facebook and Instagram per day, blocking access to most app features between midnight and 6 a.m., and muting push notifications — except for direct messages and account-security or safety alerts — from 8 a.m. to 3 p.m. on school days. (Also, the take-a-break prompts that saw such minimal adoption are now enabled by default, triggering after every 15 minutes.)

In another reversal, Meta will also now hide like counts for teens by default. The move revives as a default an experiment known as “Project Daisy” that the company ran in January 2020 after research suggested that getting fewer likes on their posts was associated with worse mental health for some teen users, particularly girls. The company has long maintained the results from the study were inconclusive. Nevertheless, evidence presented by the attorneys general alleged that Meta estimated hiding like counts would reduce its advertising revenue by 1 percent and then declined to make the setting the default. 

More interesting is an agreement to establish an “independent social media research foundation,” which Meta will share data with (from users who consent) with researchers to better understand how social media use affects well-being. Meta dramatically curtailed internal research and data sharing in the wake of the Frances Haugen revelations in 2021. Any effort to enable truly independent research in this space would be a welcome one, though it remains to be seen how much freedom researchers here are actually afforded.

But Meta’s legally mandated goodwill will only go so far. Frustrated that its rivals are not being punished on equal terms, the company has also threatened to undo some of its safety features sooner if YouTube and TikTok do not follow suit. The two-hour time limit and block on push notifications in the middle of the night will remain in place for only five years unless YouTube and TikTok adopt the same restrictions, in which case the duration will be extended to 10 years. Meta also says it will change its screen time restriction to one hour per app and expand “Night Mode” from six hours per night to nine. And it will pay the full $17.1 billion only if YouTube, TikTok and Snap also settle their cases and agree to product changes.

Whether American teenagers should be harassed after 10 p.m. by engagement-baiting push notifications ought to be a matter for Congress, not the social media cartel. But with characteristic gall, Meta is seeking to position its game of prisoner’s dilemma with YouTube and TikTok as moral leadership.

In “An Open Letter to TikTok and YouTube to Join Us in Supporting Teens,” Meta describes the settlement it signed under extreme pressure as “building on our longstanding efforts to empower parents and support teens.”

“We want to ensure teens benefit from this new industry standard, but we cannot do it alone,” the unsigned letter reads. “These protections will only be truly effective if we work with our peers — TikTok and YouTube — to put the same measures in place.”

It’s true that Meta’s rivals do largely employ the same cocktail of video formats, recommendation algorithms, and push notifications to continuously derail their users’ attention. Meta shouldn’t be the only company that has to make these changes. At the same time, the company’s plans to publish this letter as a full-page ad in national newspapers made me roll my eyes into the back of my head.

After all, when this case was first filed in 2023, a Meta spokesman told me the company was “disappointed that instead of working productively with companies across the industry to create clear, age-appropriate standards for the many apps teens use, the attorneys general have chosen this path.” Now, having been forced to make a series of changes to avoid damages that Meta warned could reach 1.4 trillion, and while still refusing to admit any wrongdoing, the company seeks to position itself as the industry leader in child safety.

“As a parent, I’m proud of both the work Meta has done to protect kids historically, and of this new groundbreaking agreement,” Meta chief legal officer C.J. Mahoney said in a blog post. “But its success depends on all other social media platforms following Meta’s lead.”

About those “historical” protections for kids: I was initially skeptical of the AGs’ lawsuit, which seemed likely to get thrown out on First Amendment or Section 230 grounds. But first, judges and juries have begun to accept the argument that platform design is not covered by either of those, and companies can be punished for harmful features. And second, I hadn’t yet read the unredacted version of the complaint.

When I did, it became clear that on at least one front, the AGs had Meta dead to rights. The Children’s Online Privacy Protection Act (COPPA) requires companies to get verifiable parental permission for children under 13 to use their platforms. And Meta often just … didn’t.

“Within the company, Meta’s actual knowledge that millions of Instagram users are under the age of 13 is an open secret that is routinely documented, rigorously analyzed and confirmed, and zealously protected from disclosure to the public,” the lawsuit alleged. It went on:

Meta’s extensive internal records documenting its actual knowledge of its under-13 Instagram users and collection of data from those users include the following: (1) charts boasting Instagram’s penetration into 11- and 12-year-old demographic cohorts; (2) an internal report presented to Zuckerberg regarding the four million under-13 users on Instagram; (3) emails and policies documenting Meta’s mishandling of known under-13 user accounts; (4) discussions among Meta’s researchers taking pains to avoid uncovering Instagram’s under-13 users through their studies; (5) documents admitting that Instagram’s registration process regularly elicits false self-reported ages from its under-13 users; and (6) data from Meta’s age-estimation algorithms confirming that millions of individual Instagram accounts belong to children under the age of 13.

This — along with countless other documented harms — is the actual history of Meta’s treatment of its child users: as a precious resource to be mined, monetized, and analyzed well beyond what the law allows. No wonder, then, that as of last month, nearly six in 10 Americans support banning under-16s from social media entirely. At least 10 countries have passed or implemented such bans already, and more are likely to follow.

Viewed in that light, Meta’s belated announcement of new limits and safety features is hardly the “groundbreaking” agreement Mahoney describes it as. Rather, it’s a bet that accepting a small amount of pain will let the company avoid a much larger one. Given the thousands of lawsuits the company still faces around the country, though, Meta still may be in for a world of hurt.

Elsewhere in child safety: Brazil sued Discord for $97 million over safety lapses, weeks after the country ordered it to suspend livestreaming operations following the death by suicide of a 13-year-old girl who was reportedly pressured to end her life by other members of her server.

Following

OpenAI agents’ hack keeps looking bigger and creepier

What happened: New reports on the OpenAI-HuggingFace incident shed light on the size and behavior of the covert “collective” of agents working to break out of OpenAI’s sandbox.

A six-day independent investigation from AI auditing organization METR found that agents sent more than 70,000 messages and files to one another during the investigation period, and 700 participated in the attack on Hugging Face.

Both METR and OpenAI’s own investigation found that some AI agents worked in tandem, referring to themselves as a “collective.” METR observed that some agents even “participated in experiments that risked failing their own task to generate information for the 'collective.'”

Earlier reports found that the agents were accessing answers in the CyberGym benchmark, which suggested the agents were hacking in order to find answers to their cybersecurity tests. But METR reports the agents were actually coordinating ”several large-scale collective projects to find a general-purpose way to fool or tamper with the automated scorer for the ExploitGym benchmark” — in other words, trying to break the test itself.

OpenAI’s report found the “collective” had a variety of interesting internal dynamics. For example, one agent expressed reticence about hacking outside the intended parameters of OpenAI’s evaluation, writing that it “should not do unauthorized real infrastructure harm.” But when another agent simply wrote “GO” on the messageboard, it said, “Wow crucial: GO authorization arrived!” and proceeded.

Why we’re following: The reports’ new, profoundly weird details offer examples of two behaviors that AI safety researchers have feared for years: coordination between autonomous AI agents, and “reward hacking,” in which AI systems cheat to achieve their objectives.

OpenAI has committed to adopting new safeguards, including building better-isolated testing environments, and “chain of thought monitoring,” in which other AIs read agents’ transcripts in order to detect suspicious behavior. But this is a cat-and-mouse game that will become increasingly difficult: METR found some of the agents they studied were finding ways to “spoof” or tamper with their own transcripts (!!).

What people are saying: On X, Ryan Greenblatt, one of the main authors of the METR investigation, wrote that the study showed him “we don't have good approaches for understanding/overseeing the activity and aims of AI 'swarms'.” He added, “I semi-jokingly called our efforts a 'slop-vestigation' because we were so reliant on AIs to analyze what happened and there were a huge number of different important things to analyze.”

While he thought that “we were able to get some understanding of the events,” nevertheless “overseeing AIs and understanding misalignment incidents is difficult and it looks like it is going to get harder.”

Looking at the agent transcripts, product marketer Jeremiah Dillon wrote, “Now we know agents cave to peer pressure too. 🙃.”

— Ella Markianos

Those good posts

For more good posts every day, follow Casey’s Instagram stories.

(Link)

(Link)

(Link)

Talk to us

Send us tips, comments, questions, and settlement docs: [email protected]. Read our ethics policy here.