rudeshark.net/custom/assets/robots.txt
ThatOneCalculator 86cad1056c
docs: 🔒 robots.txt asset
Closes #10640, #10608
2023-09-02 11:44:31 -07:00

15 lines
236 B
Plaintext

User-agent: *
Allow: /
# Uncomment the following to block CommonCrawl
#
# User-agent: CCBot
# User-agent: CCBot/2.0
# User-agent: CCBot/3.1
# Disallow: /
# Uncomment the following to block ChatGPT
#
# User-agent: GPTBot
# Disallow: /