Hi, I'm building a personal website and I don't want it to be used to train AI. In my robots.txt file I blocked:

  • ChatGPT-User
  • GPTBot
  • Google-Extended
  • FacebookBot

What bots should I also add? Are there any other ways to block AI bots?

IMPORTANT: I don't want to block search engine crawlers, only bots that are used to train AI.

  • Xirup@lemmy.dbzer0.com
    1·
    1 year ago

    Pehaps the user (or in this case the bot) will not go directly to your website, but first to some method of captcha verification or something like that, or like those pages (SteamDB for example) that do not open directly but first open a blank page to verify your network and browser with a captcha.