striker/pitstopavto-su-v2
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

chore(robots): add ChatGPT-User + OAI-SearchBot + Applebot-Extended
Some checks failed
deploy / deploy (push) Has been cancelled
Details

Browse Source
This commit is contained in:
Dmitry Gusev
2026-05-23 23:47:42 +03:00
parent 602ecd871d
commit 381a9a90a4
1 changed files with 6 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files

6
public/robots.txt
View File

@@ -4,8 +4,14 @@ Allow: /
# AI / LLM crawlers — явный allow для прозрачности # AI / LLM crawlers — явный allow для прозрачности
User-agent: GPTBot User-agent: GPTBot
Allow: / Allow: /
User-agent: ChatGPT-User
Allow: /
User-agent: OAI-SearchBot
Allow: /
User-agent: ClaudeBot User-agent: ClaudeBot
Allow: / Allow: /
User-agent: Applebot-Extended
Allow: /
User-agent: anthropic-ai User-agent: anthropic-ai
Allow: / Allow: /
User-agent: PerplexityBot User-agent: PerplexityBot