 
      
    | You are here | coryd.dev | ||
| | | | | sixcolors.com | |
| | | | | Six Colors by Jason Snell, Dan Moren and friends | |
| | | | | www.jeremiak.com | |
| | | | | How I sniffed the user agent in an edge function to prevent some AI crawlers from accessing my site. | |
| | | | | tsak.dev | |
| | | | | With the recent news of OpenAI's web crawler respecting robots.txt and the ensuing scramble by seemingly everybody ensuring their robots.txt is blocking GPTBot, I was thinking if there wasn't a better solution to help our future AI overlords make sense of the world. As I am hosting all my sites on a tiny NUC using nginx and having previously played with its return directive I decided to reuse the same trick for visits of GPTBot. | |
| | | | | www.toptal.com | |
| | | Get a fast, lightweight website with a static site generator. Find out how these systems compile templates into static HTML pages. | ||