Robots.txt can be used to exclude AI bots like ClaudeBot, GPTbot, and PerplexityBot. A number of news and publication websites have blocked AI bots, and GPTbot is the most blocked bot from Tom Capper’s research. Whether blocking AI bots is the right move for you and whether this will be honored by all AI bots is still under review and up for discussion.
Sa Su
Thanks for your visit our website.