Meta
stable for 49 days · 0 material events tracked · 1 snapshot in history
Documented user-agents (5)
Each distinct UA this vendor publishes on its docs page, extracted by Haiku from the latest snapshot. New UAs appearing or scope changes here are the high-signal events to watch.
| User-agent | Purpose | Scope / when it fires | Opt-out |
|---|---|---|---|
FacebookExternalHit | Crawl shared content on Facebook, Instagram, and Messenger to gather title, description, and thumbnail | Content shared on Meta family apps, gathers metadata for display in social feeds | User-agent: facebookexternalhit / Disallow: / in robots.txt (may bypass for security checks) |
Meta-WebIndexer | Improve Meta AI search result quality and content relevance for AI responses | General web crawl to index content for Meta AI search capabilities | User-agent: meta-webindexer / Disallow: / in robots.txt |
Meta-ExternalAds | Crawl web for improving advertising and business-related products and services | Web indexing for advertising and business intelligence purposes | User-agent: meta-externalads / Disallow: / in robots.txt |
Meta-ExternalAgent | Train foundation AI models and improve products by indexing content directly | General web crawl for AI model training and product development | User-agent: meta-externalagent / Disallow: / in robots.txt |
Meta-ExternalFetcher | Fetch individual links at user request to support agentic AI capabilities and task completion | User-triggered fetch for helping AI navigate websites and complete tasks | robots.txt rules may be bypassed as fetches are user-requested |
Events
No material changes detected yet.