Signal Policy & RegulationAccess ControlsPlatform & EcosystemLicensing & Monetization X — Discovery ·

Meta Strips Contested AI Training Language from Commercial Terms Following Publisher Backlash

Open source ↗Open source ↗ Archived copy ↗ Saved copy · private ↗

Summary

Meta updated its commercial terms to remove language that permitted the company to use content from external domains and URLs to train and develop Meta Company Products. The rollback follows criticism from legal experts and media observers who argued the clause attempted to bypass publisher robots.txt restrictions to scrape content for AI training. Meta stated that the consolidated terms applied solely to its ads and business products and reaffirmed that web crawlers continue to respect robots.txt directives.

Key facts

  • Meta removed a commercial terms clause stating that ‘Meta may use any content from such domains, web pages, or URLs to provide, develop, improve, research, train, and maintain the integrity of, the Meta Company Products’.
  • Legal scholars, including Cornell law professor James Grimmelmann, and Nieman Lab interpreted the language as granting Meta rights to use scraped publisher content for AI training and supersede robots.txt blocking instructions.
  • Meta spokesperson Daniel Roberts stated that ‘the crawling terms we consolidated only applied to our ads and business products, and we’ve clarified that in the terms themselves.’
  • Roberts affirmed: ‘As always, content owners can use robots.txt to signal their preferences. This is not about scraping news sites, and these updates don’t give us new rights that supersede those preferences.’
  • Andy Stone, Meta’s vice president of communications, emphasized that ‘There is no change to how Meta uses web crawlers; any suggestion to the contrary is false.‘

Insight

Platform attempts to broaden commercial data-use clauses often collide with publisher sensitivities around AI training rights, forcing companies to explicitly decouple standard ad-tech crawling language from AI development permissions and re-pledge allegiance to robots.txt.

Implication

Publishers and site operators maintain robots.txt as an acknowledged mechanism for signaling crawl preferences to Meta without risk of commercial platform terms superseding local crawler directives.

Why it matters

When platforms consolidate commercial terms, ambiguous definitions of AI training rights can quietly erode standard opt-out conventions, making explicit language rollbacks critical for preserving existing crawl control frameworks.

Evidence