AI usage

The goal of this project is to improve the connectivity and discoverability of real, human authors of web content. So it’s probably important to talk about AI and its use on the Outer Web.

AI summaries

The most obvious use of AI on this project got its own post. When we come across a webpage that has no description, we add it to a queue for a locally-run LLM to generate one. Having dogfooded our site we know firsthand how important it is to know what a post is about before clicking through.

To the extent that a given post is more than a single sentence, AI summaries are really more of an exercise in classification - the generative part is just because classification output isn’t easy to read. Since we algorithmically exclude posts that are not contentful, there’s no danger of the AI summary stealing a click from the destination post.

AI tagging

There are some standardized ways to tag webpages but, like with descriptions, adoption is far from universal. Our LLM summary pass also suggests tags that are used in the similarity computation that defines the structure of the Outer Web. This is notionally equivalent to a traditional classifier. Our similarity heuristic could survive without tags, but its much better with them.

Embeddings

In addition to tags and raw text content, page similarity is determined by cosine similarity between text embeddings (whose values are a product of machine learning).

Pixel art

The retro pixel art used throughout this site is a combination of gen-ai and hand editing. We’d rather write code than create a dozen gas giants pixel by pixel. We could probably create a neat algorithm to do this but this was the path of least resistance. Not that hand editing pixel art is a cakewalk.

Probes

We’re continuing to develop our set of indieweb discovery features. Our first approach, Navicomputer, asks the user to paste the text of a webpage and let us retrieve similar content. This was developed with the objective of enabling web authors to easily find and link each other, but it’s not great for casual browsing. For casual page discovery we followed up with Exploration Mode that lets the user choose a category and explore from there.

We recently added a keyword/tag search feature called Probes that’s less demanding than full-post search and more granular than Exploration Mode. Probes is largely identical to tag search UX found elsewhere on the web with two exceptions:

1. The results aren't exhaustive - not every post with a given keyword shows up in that keyword's results.
2. Probes doesn't pull from a curated list of tags but rather snippets of the pages in the Outer Web

Give it a try!

AI descriptions

Anyone who has been rickrolled or jumped through hyperspace to a planet that has been destroyed by a giant laser will tell you this: it’s nice to look before you leap. Since the Outer Web is basically a large network of hypertext leaps, we try to give people a good idea of whats on the other end of their click. So both Navicomputer and Explore show the linked page’s title and description like this:

Arrakis IRL
A gallery of my August trip to the Pismo Dunes.

Neither the internet nor the Outer Web requires either of these fields, so for pages without an RSS or Open Graph description, we’ve started adding LLM-generated descriptions:

Exonematology
[AI summary] This post speculates on the viability of populating Tatooine from the Star Wars series with sandworms from the Dune series. It discusses the impact on civic centers, moisture farming, and intergalactic trade.

2030

Dogfooding and discovery

I was verifying the Outer Web link for Web 2.0 Sucks! and stumbled upon its neighbor Manifesto by V.H. Belvadi. This, in turn, led me to Belvadi’s lastest post about hosting this month’s IndieWeb Carnival.

What is IndieWeb and what is its carnival, you might ask.

• IndieWeb is an advocacy movement for the small web, it provides helpful guides and technologies for self-publishing on the web
• The IndieWeb Carnival is their monthly blog prompt with a rotating host

This month’s question: Where do you see the IndieWeb in 2030?

The web in 2030

Gut check: my 2025 personal site looks about the same as it did in 2015 and 2005. The content has evolved, the tooling has evolved, but for the reader it’s structurally the same. Having reviewed many, many sites during the development of this project, I can say my own experience isn’t an isolated one.

Writing: The backbone of the indieweb consists of personal/semi-professional websites crafted by cool people who have hobbies, jobs, and responsibilites. I expect that the next five years will see the blogosphere gain and lose some voices but they’ll largely fit the same mold. Even if enshittification causes people to flee web2 platforms in droves, the fundamentals of blog writing will largely remain the same.

Medium: I also don’t expect the core web experience to change: html, scripts, images, videos. Incumbent content delivery methods have momentum and, well, there’s nothing wrong with traditional web pages. Microblogging, bring back webrings, nownownow, backlinks, gemini, atproto… it’s tough to gain traction in the small web’s fragmented ecosystem. And many of these approaches require critical mass to be of value to their user base.

And so while I don’t expect the indieweb to experience a major structural change in the next five years, I do expect it to experience a subtantial topographical change. In five years I hope to see an indieweb that is denser and more discoverable.

• Denser: there will be more links (and more linking). As writers embrace links pages, webrings, and a general sense of community, we’ll have an easier time traversing the blogosphere using the trusty old href.
• More discoverable: finding content on an arbitrary subject won’t be such a challenge. Densification combined with benign external services such as this site, Marginalia, Scour.ing, and Indieseek will make it easier for readers to enter the indieweb and find what they are looking for. Then maybe hang around and read some other cool stuff.

The indieweb will still be alien to people who cannot be severed from infinite-scroll platforms and video shorts. It will still be an enemy of the web built around profiteering and endless javascript. But as the core web offers less and less authenticity, people will seek noncommercial, human-created content.

Subprompts

Belvadi provided a handful of subprompts in case the big question was too open-ended for carnival participants. They’re great questions in their own right and I found them to be an excellent way to revisit my answer to the main prompt.

Are you building something for the IndieWeb? This would be a great time to showcase it, whether alpha, beta or production-ready.

As it happens, yes! This site. It’s production-ready for its current feature set and it’s growing by the day. In the context of the 2030 question, plenty of others are attacking the same connectivity and discoverability problems in different ways.

What radically new idea do you have for the IndieWeb that you haven’t been able to communicate to someone with the skills to build it?

Let’s all add a footer to every blogpost that links to three other webpages with similar content. Let’s link to people we don’t know (or don’t necessarily know).

A simple example, here are The Outer Web peers of Belvadi’s manifesto post:

• IndieWeb principles / developer-principles
• The IndieWeb is for Developers
• WordPress and IndieWeb
• Why have a personal website?

How better can the IndieWeb stand up to the CorpoWeb in the coming years?

I mentioned above that people will seek content that isn’t AI-generated, SEOed, and delivered via algorithm. Not everyone will, but as the corpoweb grows more and more plastic, users will seek flesh and blood. The indieweb simply needs to remain independent and stick to its values.

Web Discovery

Entrypoints to the authentic web

If you’re here, we hope you’ve checked out Explore or its more orderly cousin Explore Sorted. There are a few other great indieweb discovery projects, here are some of our favorites:

Posts about web entrypoints

• A look at search engines with their own indexes has a list small web search projects with helpful commentary 🪐
• Notes on the “indieweb” #2: Where do I find things to read? discusses RSS, newsletters, and microblogs 🪐
• How I Discover New Blogs lists a variety of indieweb discovery sites 🪐
• /g/’s based sites contains a categorized list of alternative search engines and other points of interest
• Surf the Web lists webrings, link directories, blogstumblers, and alternative search engines 🪐
• Entrypoints to the Indieweb discusses indieweb discoverability challenges and lists a few stumbler and search sites 🪐

A few blog posts about the web

• My IndieWeb Journey: Building, Sharing, and Owning Your Online Presence. (2025) 🪐
• The IndieWeb’s next stage? (2024) 🪐
• We Need to Bring Back Webrings (2023) 🪐
• What happened to blogging for the hell of it? (2023) 🪐
• A blog post is a very long and complex search query to find fascinating people and make them route interesting stuff to your inbox (2022) 🪐
• Some thoughts on walled gardens (2022) 🪐
• The IndieWeb for Everyone (2022) 🪐
• The IndieWeb Mixed Bag (2021) 🪐
• The small web is beautiful (2021) 🪐
• The Small Website Discoverability Crisis (2021) 🪐
• Web 2.0 SUCKS! - My manifesto and feelings of Web 2.0 (2021) 🪐
• Ringy Dingy (2020) 🪐
• The Dark Forest Theory of the Internet (2019) 🪐
• An Introduction to the IndieWeb (2017) 🪐
• How the Blog Broke the Web (2017) 🪐
• Indie Web (2015) 🪐
• Why the Indie Web movement is so important (2014) 🪐
• The Slow Web (2012) 🪐

On the replacement of independent websites by social media:

Those who would give up essential liberty to decide whom to follow, for getting a little temporary convenience in exchange, deserve neither to choose what to read nor convenience - adiian