Outer Web | Explore

/explore

Click through on any links that interest you or select the planets on the right to continue exploring the Outer Web.

You are here		tsak.dev Feeding GPTBot
\|	\|	sixcolors.com Excluding your website from Apple's AI crawler - Six Colors	3.3 parsecs away Travel
\|	\|	Six Colors by Jason Snell, Dan Moren and friends	3.3 parsecs away Travel
\|	\|	coryd.dev Go ahead and block AI web crawlers	3.2 parsecs away Travel
\|	\|	AI companies are crawling the open web to, ostensibly, improve the quality of their models and products. This process is extractive and accrues the benefit to said companies, not the owners of sites both small and large.	3.2 parsecs away Travel
\|	\|	www.andrlik.org TIL - Block AI bots with a CloudFront function · Ministry of Intrigue	2.9 parsecs away Travel
\|	\|	It is now clear that at least some AI companies are ignoring robots.txt that forbid them from scraping a site. Robb Knight wrote up a great guide for explicitly blocking those scraping bots via your Nginx config. However, this site is currently served by AWS CloudFront, which means that the content gets served without the request touching the source server. I was sure there had to be a way to do something similar with a CloudFront function, so I set out to try.	2.9 parsecs away Travel
\|	\|	www.matsimitsu.com Publish articles from iA Writer to your static site - Blog - Matsimitsu.com	12.4 parsecs away Travel
\|		My hope with this project is that with less friction to write and publish a note/article, I'll do it more.	12.4 parsecs away Travel