Meta can use web scrapers to crawl and collect content from websites “associated with” business accounts on its platforms, according to its new commercial terms of use.
The update, set to go into effect on October 30, will include news publishers with business accounts. These accounts allow companies or public figures to manage Facebook pages, Messenger accounts, Instagram profiles, or WhatsApp channels from one place. While journalism isn’t the only industry that will be affected by this update, virtually every news organization has at least one Meta business account, including Nieman Lab.
The new terms broadly cover business accounts that manage pages and groups, run advertisements, push marketing messages, and use the platform’s business account dashboard. As written, the new terms could be used to argue that Meta can scrape content from news sites and use that content to train the company’s AI products, according to three legal experts who reviewed the terms at Nieman Lab’s request.
In early October, Meta alerted users to the update in a blog post and on the Meta Business Suite. The updated terms explicitly state that Meta can use the content it collects from associated websites to “train” Meta products. Elsewhere in its terms of service, the company defines “AI at Meta” as one of those products.
Here’s the new language about scraping in full:
You authorize us (or any entity acting on our behalf) to use automated software (such as web scrapers, bots, robots, spiders, crawlers, and user agents) and create test credentials to access, index, cache, analyze, collect, retrieve, or crawl the domains, web pages, or URLs (and any content available through them) that are associated with your products or services, or that are made available through your or users’ use of Meta Company Products and services, and to periodically pull updates from such domains, web pages, or URLs. Meta may use any content from such domains, web pages, or URLs to provide, develop, improve, research, train, and maintain the integrity of, the Meta Company Products, services, and features provided under these Commercial Terms and any other applicable supplemental terms.
“For news publishers, this clearly represents permission to Meta to train AI on any content from their sites and possibly to deploy some of that content, even in competition with the publishers themselves,” said James Grimmelmann, a professor of digital and information law at Cornell Law School. “On the broadest possible interpretation, this would be used to develop a comprehensive archive of all content that Meta would then serve as it wanted to directly or through AI.”
Currently, there is no option for business accounts to opt out of these new terms ahead of October 30. The update reads, “If you do not agree to the updated terms, please stop all access or use of our Products for business or commercial purposes.”
In short, “publishers either have to trust that Meta doesn’t mean the full extent of what these terms say, or they have to stop [using business accounts],” said Grimmelmann.
Meta declined to answer a list of questions about these updated terms. A public affairs spokesperson at the company said in an email that language around crawling was previously scattered across different terms of use, including terms about using generative AI in ads, and this update consolidates them in one place. He also said that the new terms will not change how Meta crawls publishers in the future, and that “content owners can still manage how their content is accessed by using robots.txt.”
On its developer page, Meta currently has instructions for how to block its crawlers using robots.txt, a file that tells bots which parts of a website they can and cannot access.
However, legal experts said the new language creates an opening for Meta to argue its commercial terms supersede other blocking instructions. The update also states that Meta can create “test credentials” in order to scrape websites associated with business accounts. Grimmelmann said this would technically allow Meta to “sign up as a fake user” to bypass registration walls on a publisher’s site to access content. Meta declined to answer Nieman Lab’s questions about whether the “test credentials” would bypass paywalls or other paid content.
Meta’s decision to tether access to its business products to broad scraping permissions echoes a concept in antitrust law called “tying,” according to Benjamin Sobel, a law professor at the University of Wisconsin. “If you want to use the Meta products that require acceptance of the commercial terms, you have to also acquiesce to the scraping and crawling permissions exacted in those terms.”
Sobel compared the move to Google’s recent policies around its AI search products. In order to appear in Google’s traditional search results, publishers now have to consent to their content also being used in AI Overviews and AI Mode.
Meta’s new commercial terms come on the heels of the company’s rollout of its paid subscription service, Meta One. The cash-for-links program restricts the number of external links that some accounts on Facebook and Instagram can post without subscribing. Put another way, Meta has begun charging users to post links to the same content that it will be permitted to crawl for free. Earlier this month, Press Gazette reported that some news publishers are currently exempt from the link limit. Meta declined to answer Nieman Lab’s question about the service.
Nieman Lab first learned about Meta’s new commercial terms through a reader tip. Stephen Beale, the founder and editor of The Steampunk Explorer, got in touch when he saw a notice of the upcoming change in his Meta Business Suite dashboard.“Traditionally, you have this idea that Meta might be using content [posted] on Facebook or Instagram for training,” Beale told Nieman Lab. “But this was different because they’re saying that [since you are] a business user of Facebook, they have the right to go out and grab information from basically wherever you have a presence.”
Beale has long depended on Facebook as a way to connect with his audience and refer traffic to his site. Fans of steampunk — a retrofuturistic subgenre of science fiction — skew older, he said, and the community is “totally entrenched on Facebook.” For many, their only interactions with the community happen on Facebook or at in-person events that they learned about on Facebook.
Beale said he’s seriously considering shutting down his accounts before October 30.
“I know my site has been scraped to death,” he said. “But it just seems like such an arrogant, in-your-face move on [Meta’s] part… It just makes me angry and I don’t want them to get away with it.”
Are you considering changing how you use Meta-owned platforms? Let us know at [email protected].