Perplexity-User: Perplexity’s user-triggered fetcher
Perplexity-User visits a page when a Perplexity user’s question needs it, and generally ignores robots.txt. How to identify and control it.
Updated
Perplexity-User is the agent Perplexity uses when a user’s question leads it to visit a page, so it can give an accurate answer and link the page in its response. Perplexity says that since a user requested the fetch, it generally ignores robots.txt rules. To restrict it, you need a firewall or CDN rule, not robots.txt.
| Owner | Perplexity |
|---|---|
| Purpose | User-triggered |
| robots.txt token | Perplexity-User |
| Follows robots.txt | Generally ignores it (user-initiated) |
| Published IP ranges | www.perplexity.com/perplexity-user.json |
| If you block it | A robots.txt block generally has no effect; use a firewall rule. |
Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko; compatible; Perplexity-User/1.0; +https://perplexity.ai/perplexity-user)What does Perplexity-User do?
Perplexity describes it as supporting user actions within Perplexity: when users ask a question, it might visit a web page to help provide an accurate answer and include a link to the page in its response. It is not used for web crawling or for training data. The crawler that builds Perplexity’s index is PerplexityBot.
Can I block it?
Not reliably with robots.txt. A group like the one below documents your wishes, but by Perplexity’s own statement it is generally not followed:
User-agent: Perplexity-User
Disallow: /To actually stop it, block requests that carry the Perplexity-User user agent and come from the IP addresses Perplexity publishes for it, at your web application firewall or CDN.
What happens if I block it?
Perplexity can’t open your page while answering a user, so it can’t quote or link it for that answer. As with any user-triggered fetcher, the people affected are actively asking about your topic, often about you.
How do I verify it?
Compare the IP with perplexity.com/perplexity-user.json. Perplexity recommends combining user-agent and IP checks in WAF rules.
Should you block Perplexity-User?
For public pages, there is little reason to. If you have content you don’t want any AI agent to read, put it behind a login. Review all your AI bot settings with the AI crawler checker.
Sources
Common questions
Why does Perplexity-User ignore robots.txt?
Perplexity's position is that a user requested the fetch, much like a person opening the page in a browser, so the fetcher generally ignores robots.txt rules. Robots.txt was designed for automatic crawlers.
Is Perplexity-User used for training?
Perplexity says it is not used for web crawling or to collect content for training AI foundation models.