/explore

Click through on any links that interest you or select the planets on the right to continue exploring the Outer Web.
You are here

 www.dannyguo.com
How to Prevent a Website Page From Showing Up in Search Results
| | mmazzarolo.com
Generating a robots.txt in Next.js | Matteo Mazzarolo
3.7 parsecs away

Travel
| | A robots.txt file tells search engine crawlers which pages or files the crawler can or can't request from your site. Next.js does not generate a robots.txt out-of-the-box, so here are a couple of options on how to create it.
| | www.marginalia.nu
For Webmasters @ marginalia.nu
2.4 parsecs away

Travel
| | [AI summary] A small non-profit search engine operator addresses crawler fingerprinting, robots.txt compliance, indexing delays for specific TLDs and hosts, and requests community links.
| | seirdy.one
Opting out of LLM indexing - Seirdy
1.9 parsecs away

Travel
| | I added an entry to my robots.txt to block ChatGPT's crawler, but blocking crawling isn't the same as blocking indexing; it looks like Google chose to use the
| | jabberwocking.com
The New York Times has sued OpenAI for copyright infringement. They should lose. Kevin Drum
11.7 parsecs away

Travel
| The New York Times is suing Microsoft and OpenAI for training their AI software on Times stories: In a complaint filed Wednesday, the Times said the technology companies exploited its content without permission to create their AI products, including OpenAIs humanlike chatbot ChatGPT and Microsofts Copilot. The tools were trained on millions of pieces of Times content, the suit said, and