striker/pushkinohistory-ru-v2
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

chore(robots): add ChatGPT-User + OAI-SearchBot + Applebot-Extended
Some checks failed
deploy / deploy (push) Has been cancelled
Details

Browse Source
This commit is contained in:
striker
2026-05-23 23:46:06 +03:00
parent d2bd3647d4
commit 3104eaf1ce
1 changed files with 6 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files

6
public/robots.txt
View File

@@ -4,8 +4,14 @@ Allow: /
# AI-crawlers
User-agent: GPTBot
Allow: /
User-agent: ChatGPT-User
Allow: /
User-agent: OAI-SearchBot
Allow: /
User-agent: ClaudeBot
Allow: /
User-agent: Applebot-Extended
Allow: /
User-agent: anthropic-ai
Allow: /
User-agent: Google-Extended