Perplexity AI has been exposed for secretly deploying stealth crawlers that intentionally evade website no-crawl directives, rotate IP addresses, and even masquerade as regular browsers. Despite explicit robots.txt blocks and firewall defenses, Perplexity’s undisclosed bots aggressively scrape restricted content from millions of websites daily. Cloudflare’s investigations confirm this shocking breach of trust, prompting immediate countermeasures against these rogue AI crawlers.
-Want to be a Guest on a Podcast or YouTube Channel? Sign up for GuestMatch.Pro
-Thinking of buying a Starlink? Use my link to support the show.
-Get a United Chase Card and 10,000 miles. You are supporting GNC Travel with your signup!
Subscribe to the Newsletter.
Join the Chat @ GeekNews.Chat
Email Todd or follow him on Facebook.
Like and Follow Geek News Central’s Facebook Page.
Download the Audio Show File
New YouTube Channel – Beyond the Office

Full Summary:
In this episode of Geek News Central, host Todd Cochrane discusses a significant story that revolves around Perplexity AI. He reports that Perplexity AI has been found to employ stealth crawlers that intentionally bypass website no-crawl directives, rotate IP addresses, and disguise themselves as ordinary browsers, despite explicit prohibitions against crawling certain websites. This revelation stems from a CloudFlare investigation that highlights Perplexity’s aggressive scraping of restricted content from millions of websites daily, raising ethical and legal concerns.
Cochrane describes this behavior as a blatant disregard for the rules that govern web crawling, and he stresses that such actions will likely fuel ongoing litigation against the company. He shares detailed findings from the investigation, indicating that Perplexity’s undeclared crawlers utilize multiple IPs to evade website blocks, accessing content in violation of established web norms.
He transitions into discussing his own choices regarding website crawling, expressing no issues with bots crawling his sites, while acknowledging that many site owners may prefer to restrict access. He also highlights the transparency expected of well-mannered bots, in contrast to the deceitful tactics employed by Perplexity AI’s crawlers.
After the main discussion, Todd provides updates and news in the tech world, makes a shoutout to GoDaddy, the podcast’s sponsor for over twenty years, and encourages listeners to support the show. He mentions exclusive deals available through GoDaddy, reflecting on successful statistics from the previous month and soliciting support for the ongoing podcast endeavors.
Cochrane shares some personal achievements, such as weight loss and the launch of a new service called GuestMatch.Pro, aimed at connecting guests with podcast opportunities. He touches upon recent technological advancements, including OpenAI’s newest reminders feature in ChatGPT,