Open account

Meta Faces Backlash as AI Web Crawlers Overwhelm Small Business Websites

Meta Platforms is drawing increasing scrutiny as its automated artificial intelligence web scraping bots cause operational disruptions for small business websites. Web crawlers identified as Meta-ExternalAgent and Meta-WebIndexer have caused severe traffic spikes, pushing server CPU usage to 100% and taking sites offline. In one incident, an artist's website was hit with over 550,000 requests within a 48-hour period, generating hundreds of requests per minute. Meta utilizes these bots to scrape online content to train foundation AI models and refine search index quality, while indicating webmasters can manage access via robots.txt protocols.

Category

Meta Platforms

Sentiment

Bearish

Event

Other

Reading time

1 min