busybee General Articles

Some extremely-imprecise history

In the beginning, there was the Internet. The Internet itself was a collection of a bunch of purpose-built pieces of software for folks to communiate and collaborate with one another; you’d share files using FTP, you’d email people with, well, Email, large-scale discussions would take place on Usenet, realtime chat happened via talk or IRC, and people would share their status updates via a .plan file which would be viewable via the finger command. It was all very academic, very scattered, and very difficult to actually participate in.

Then the World Wide Web happened, and suddenly this became a very easy way for people to communicate. Their company or university would provide people the ability to publish “pages,” in the form of HTML documents with various formatting features (usually written in Notepad and published somewhere like http://example.com/~bob/my_cool_homepage.htm), and eventually this grew the ability to add links to other resources such as images and navigation metadata.

One of these metadata formats that eventually emerged was RDF, and then there was much arguing over its specifics, which led to various flavors of RSS and Atom, and there were several pieces of web-logging software such as Movable Type and Wordpress, and this practice would eventually come to be known as “blogging” for silly reasons, and the proliferation of syndication feeds (primarily RSS and Atom, which people also called “RSS” for similarly silly reasons) meant that there were also a number of so-called “feed readers” that people used to keep up with other peoples' blogs.

One of the most popular readers, Google Reader, was loved by everyone, so of course Google killed it off, because they wanted to completely control the social media landscape with Google+, which almost nobody loved, so they killed that off too. And when Reader died, the tech press collectively declared RSS to be dead and everyone moved to Facebook and Twitter which, as it turns out, was not a great long-term strategy.

Fortunately, RSS never really died. It continued to be used to syndicate and subscribe to content by many people without even knowing it; many social platform such as Wordpress and Tumblr continued to support RSS out of the box (for publishing, anyway), podcasts never stopped using RSS as their primary mechanism, and some folks even continued to use RSS as their primary means of keeping up-to-date with their favorite websites.

Anyway, fast forward a few years to when a bunch of nerds decided that blogging is Good, Actually, and could also be better than the corporate social networks we’ve all been stuck with for a while.

Blogging, evolved

So, IndieWeb basically is a continuation of the original idea behind blogging: you have your own website under your own control where you post your own content, and then provide mechanisms for other folks to subscribe to that content and also post their own responses.

It is, by its very nature, inclusive, and from a technical standpoint, simple, but not trivial.

The most basic thing you need in order to publish is a space in which you can post HTML that is under your own control. Ideally this means owning your own domain (which can be cheap or free) and having your own webhosting (which is also pretty cheap). For example, you can get cheap shared hosting with Dreamhost or Nearly Free Speech, or you can set up a website with tilde.club or Neocities and have something that’s pretty darn close (although it really is better to have your own domain name if possible; fortunately, domains are as little as $free if you’re willing to accept some limitations, and a typical domain will only cost $5-10 per year these days).

Then, you need a reader of some sort to follow other people. There’s many, many choices out there and it all comes down to preference. I personally use Feed on Feeds, but there are plenty of others out there with different designs and various pluses and minuses. There’s even mechanisms for converting feeds into an email inbox, and also many email clients have feed readers built in.

But just having a site isn’t quite enough to really participate in the IndieWeb, so there’s a few other things you need.

Feeds

Your site should provide some form of subscription feed. For the best compatibility, this should probably be RSS or Atom. However, there’s also a big push in IndieWeb to support what’s called h-feed… but more on that later.

Actually, more on that now!

Metadata

I’m sorry, but this is going to be extremely geeky and technical, out of necessity. Fortunately it isn’t vital that you understand this directly, but I tend to overdo it on technical details so bear with me. If fiddly technical details make your eyes glaze over, though, feel free to just skip ahead.

HTML isn’t just a layout language, it’s also a semantic language. That is, it doesn’t just provide formatted text, it also provides information about what the text means, from a structural perspective. The way that things link to each other, what every bit of text represents, and so on.

There are many standards for actually formatting the metadata within a page, but the one that IndieWeb most strongly encourages is called microformats2, usually shortened to “mf2.”

The idea here is that your page markup provides hints to other things as to what everything means, using different non-visual CSS classes. For example, if your main page looks something like:

<html><body class="h-feed">

<article class="h-entry">
    <header>
    <h1 class="p-title"><a href="article-19257.html" class="u-url">Here is an article</a></h1>
    <p>Posted by <data class="p-author">fluffy</data> at <time class="dt-published" datetime="2022-02-02 02:20">February 2, 2022</time></p>
    </header>

    <div class="e-content">
    <p>Hello!</p>
    </div>

</article>

</body></html>

then tools which consume mf2 can see this page and tell that you have a feed (h-feed) with a single article (h-entry), and knows exactly what the canonical location is (u-url), who wrote it (p-author), when it was published (dt-published), and the HTML content of the entry (e-content).

The advantage of this is that there’s no need for a separate feed (or separate metadata at all, really) in its own syntax; all of the data that goes into a feed is just there on the HTML page that already exists (contained in the top-level h-feed element, in the above example). It also means that programs can automatically get information about any mf2-supporting page without needing to special-case everything or make use of a complicated API. In effect, the HTML is the API.

What’s more, there’s plenty of libraries out there for automatically parsing mf2 out of HTML; for example, I use mf2py in my Python code. There’s also some online tools that you can use with mf2 such as granary and pin13.

I don’t want to go into too much detail about mf2 itself here, though, because mf2 can get incredibly complicated and there’s so much more to talk about! For more information, please see the microformats wiki.

Notifications

So, one of the best things that traditional social media provides are mechanisms for receiving notifications; roughly-speaking, there are two major kinds of notifications, namely subscription updates (i.e. “This person you’re following posted a new article”) and responses (i.e. “Your friend posted a reply to your article”).

There are two major notification mechanisms in use in the IndieWeb: WebSub (for subscriptions) and Webmention (for responses).

WebSub, being for generic notifications of subscription updates, doesn’t require any specific relationship between the subscriber and the publisher. It’s also largely optional; most feed readers will, by default, check a page for updates on a regular basis. The purpose to WebSub is more to tell a feed reader that there’s new content available right away, so that it can get the update immediately instead of waiting for an hour (or more) for the next update to happen.

Webmention, on the other hand, is a mechanism that relies on a relationship between two pages to work; first, a page is created or updated with a link that declares a relationship. For example, this page has a link on it that looks like:

<a class="u-syndication" href="https://news.indieweb.org/en">

which means that I want this page to be reposted (or at least linked to) on the public IndieWeb English news feed.

After this page goes up, I then tell the IndieWeb News webmention endpoint that this page exists and links to it with a syndication-type URL. The webmention endpoint then retrieves this page, verifies that the link exists, parses the mf2 on the page to get things like the title and author, and then adds the link to the news index.

But this is just one of many things that webmentions can do! For example, the main use of webmentions is so that anyone can put a link to this page with a microformat like u-in-reply-to or u-like-of, and then after they tell my webmention endpoint about it, that response will appear on my webpage through various bits of plumbing that aren’t worth going into.

Webmention itself is also very flexible. There are tools such as brid.gy which monitor major social networks that don’t directly support Webmention, and then converts various reactions on there into webmentions for me to see! For example, if someone posts a comment about this on Reddit, or a reply on Mastodon, or any number of other things, and brid.gy sees it, it’ll publish a webmention on that site’s behalf.

You can also use Webmention replies to start a conversation with someone. Every URL can theoretically receive a mention (it doesn’t even have to be an HTML page!), and by convention, people can tag others into a conversation by posting an unlabeled link (which is sometimes called mention-of in some software stacks) to their website.

This mechanism can also be used to send someone something approximating a “direct message” on the other social platforms, by sending it from a page which isn’t visible to the wider Internet; however, either the mention page will have to be at an unguessable URL (which isn’t completely secure) or there needs to be some other mechanism for the endpoint to be able to see the protected page, which doesn’t currently exist (although there’s a few ideas that are being worked on). Generally-speaking, the best way to send a DM is via an out-of-band communication method like email or your favorite chat system.

Anyway! The particular Webmention endpoint I (and a lot of IndieWeb folks) use is called webmention.io and is run by Aaron Parecki as a community tool. There are a number of tools which interface with it to provide further functionality; for example, many people use webmention.js, which I wrote specifically for this purpose, and makes it easy to embed webmention.io’s mentions into any webpage, even on sites which are statically-generated or hand-written!

I also make use of webmention.io’s notifications feed to see replies in my feed reader.

But there’s no requirement to use webmention.io. There are several other endpoints available, including ones which are built in to specific blogging software… but more on that later!

Social interaction

Earlier I mentioned using a feed reader to subscribe to other peoples' posts.

However, IndieWeb also has a concept of a more integrated reader experience (called a “social reader”) with a pair of protocols, Micropub and Microsub. I don’t personally use these protocols (yet!) but they’re still worth talking about briefly.

Microsub provides a mechanism for a user to manage the feeds they’re subscribed to, and to retrieve the current items of their subscriptions and view them in a reader.

Micropub provides a mechanism for a user to publish things directly to their website from a third-party client, which could very well also be a Microsub reader. So while you’re using a Microsub reader, you might want to post a reply to a blog post, or publish a “like” response, or syndicate it to another news site, or so on; the Microsub reader can use Micropub to in order to do this, if your site supports Micropub of course.

Also Micropub isn’t just for use in feed readers — it can also be used as a first-line posting client for publishing new pages to your site from anywhere! For example, you can have a piece of software that monitors your public photo albums on Flickr and then uses Micropub to repost them to your site, and there are also a number of mobile Micropub clients that enable IndieWeb posts from your cellphone, for example.

Anyway, this pair of protocols makes it a lot easier for any random IndieWeb site to be a more direct participant in the IndieWeb. I personally see them as “optional, but recommended.”

Authentication, identity, and privacy

There’s also another really important aspect to talk about: how you get identified to other sites.

The core protocol for this is called IndieAuth, which is an extension to OAuth, which is in turn the authentication method used by a lot of things right now (for example, most social platforms use OAuth as the mechanism for letting a client or app access your account).

The idea behind IndieAuth is that you use your own website as the “key” to log in to another website. So for example, if I want to sign in to an IndieAuth-enabled website, I sign in as https://beesbuzz.biz/, and then there’s a multi-step handshake between that website and my own to verify that it’s really me at the keyboard. Once this happens, the remote site sees that I “am” beesbuzz.biz and grants me permission to do anything that would be allowed by that.

A pretty common example of things that IndieAuth allows is users to edit the IndieWeb wiki or sign in to IndieWeb chat. Some folks (such as myself!) also use IndieAuth as an access method for accessing audience-restricted posts.

But audience-restricted posts have one pretty major problem: feed readers can’t get to them, because they’re just running in the background and can’t ask the user to sign in. What do we do about that?

Well, there’s another protocol, IndieAuth Ticket Auth (sometimes just “Ticket Auth”/“TicketAuth” for short), which helps to solve that problem as well! With TicketAuth, if someone wants to grant you permission to something private, they can send a ticket to you, and then your ticket endpoint redeems the ticket for an authentication token which it can then use for authenticated access to your site on your behalf.

For example, if Alice sends a ticket to Bob, then Bob’s feed reader can use that ticket to retrieve a version of Alice’s blog feed that’s just for them, and includes all of the private posts Alice wants them to see but doesn’t want to make public.

This can also be entirely automatic; for example, whenever someone uses IndieAuth to log in to this site, this site will see if they support TicketAuth and, if so, send them a ticket outright. That way, if I later decide to add them to one of my protected-access groups, their ticket-supporting feed reader will automatically start to receive the protected items they have access to.

Unfortunately there aren’t a lot of things that currently support TicketAuth, but hopefully that will change over time, as it’s (IMO) a great model for providing access to private content without requiring reinventing the world or needing everyone to participate on the same closed social network.

Comparison with ActivityPub

So, this is a bit of a tangent, but there’s been a lot said about ActivityPub for the future of social networking.

There is nothing about IndieWeb that makes it contradict ActivityPub; in fact a lot of IndieWeb sites also support ActivityPub in addition to the IndieWeb protocols! This site has minimal ActivityPub support (I use Bridgy Fed, which uses Webmention to drive a basic ActivityPub adapter, which you can subscribe to on Mastodon at @beesbuzz.biz@beesbuzz.biz) and I plan on adding more comprehensive support, eventually, to give it an even better experience.

ActivityPub is nice in that it’s gaining a lot of adoption, but there’s a lot that I personally prefer about IndieWeb. For example, the fact that IndieWeb protocols are all separate and composable — they’re like Lego bricks that you can use together, rather than needing to be all-or-nothing up-front — makes it a lot easier to ease your way into supporting IndieWeb. You also only have to support the parts that make sense to you; there’s no requirement to support push notifications, for example, and since the core is just HTML with microformats, there’s no need for complicated APIs for anything to actually read the data. People don’t need “an IndieWeb client” to view this website, or “an IndieWeb server” to interact with it.

The IndieWeb community also provides a lot of tooling so that you don’t even need to host it all yourself. As mentioned, there’s webmention.io for supporting Webmention, you can use IndieLogin to get (very basic) IndieAuth support, and things like brid.gy, webmention.app, and Comment Parade exist to make it much easier to send responses to other people.

More choices are better, of course, and some aspects of ActivityPub work really well for what it was intended to do! But there’s no reason to be all-or-nothing about these things, and I personally feel that IndieWeb covers way more use cases in a way more humane way.

The fact that you can fully support IndieWeb from a statically-hosted website that’s running on a server you don’t have administrative control over? That makes a huge difference, to me, anyway.

What about community?

A thing that commonly comes up is the question of community, both in terms of the upside (discovery) and the downside (moderation and toxicity). It’s worth noting a few things along these regards.

For starters, many of the social networks out there are built based on the assumptions that all social networks should be based on how a handful of social networks have worked. For example, Twitter used to have a public global timeline, and later moved to algorithmic-based discovery; Mastodon emulates this via having both global and instance-based public timelines, and Tumblr provides algorithmic discovery.

But these elements ignore that what’s made for the most successful interactions in these spaces is not the platform telling you what to look at, but other users doing so. How many people do you follow on Twitter or Mastodon based on random happenstance, and how many do you follow because someone else you follow boosted something and you saw that they were someone worth following? Or because someone posted a nice or thoughtful response?

The same thing happens in the IndieWeb. People boost other peoples' posts, via bookmark/like links or public responses to them, and there’s various syndication feeds you can follow, such as IndieWeb News and Indieweb.xyz. And of course people can link to IndieWeb sites from any social network as well; I get a lot of incoming traffic from Mastodon, Reddit, and Hacker News, because something I wrote resonated with someone who decided to post there. (And those are just the ones I know about thanks to brid.gy!)

Also, because IndieWeb sites are hosted by individuals, they can control exactly what content shows up and how. I moderate all of my comments and will remove spammy Webmentions (which are actually quite rare, at least for now), and because of this there’s less of an incentive for people to troll since it’s unlikely that their trolling will appear in public anyway. Sure, I occasionally get the rare toxic response or attempt at spamming, but I can control when and how they appear on my site, whereas on other social platforms, your only choices are to individually block the other person and report them (and hope that your report gets seen by the people who can make decisions, and that they decide to actually help out).

I also feel like the lack of discovery is also linked to a lack of toxicity; there’s less of a drive-by “reply guy” effect, and things also tend to only spread within the audience for which it was intended in the first place. Even on the posts of mine which have gone widespread, the amount of toxicity that make it back to me is pretty limited (even including the webmentions I get from Reddit and Hacker News).

Of course, as IndieWeb grows, this will probably change, and there will need to be better tools for managing community and toxicity — but it’ll always be under the control of the people publishing to the web, not gatekept by a third-party moderator who might not have your own comfort or safety at heart.

Granted, there are also some very large holes left in this mode of thinking. Doxing and threats have to be handled externally, via contacting other peoples' hosting providers or getting the law involved, for example, and the public-by-default nature of feeds means that restricting an audience is (currently) difficult (although, again, I’m optimistic about Ticket Auth helping to solve this problem).

But as they say, an ounce of prevention is better than a ton of cure, and as the community grows, the tools will also evolve. And remember that growth of community also means growth of a mutual support network! We’re all in this together, and that’s worth a lot.

The initial setup can also be quite challenging, but I hope that over time this, too, becomes easier. And on that note…

How to get started

Okay, so that’s a lot of talking about what IndieWeb is at a very weird mix of extremely abstract and detailed, but nothing about how to actually get started with IndieWeb! And I know that’s confusing and weird, and you might just want to dive in to joining IndieWeb.

So, there’s a few different ways you can get started with IndieWeb.

The easiest way is to sign up for micro.blog, which is an inexpensive ($5/month) service that gives you a fully-fledged IndieWeb site, with support for IndieAuth, Webmention, and Micropub, right out of the box. It also integrates directly with Mastodon, and provides some form of subscription mechanism (although I’m not clear on whether they use Microsub or something else).

If you have a bit more technical savvy, a good place to start is by getting shared hosting somewhere such as Nearly Free Speech and installing some blogging software that supports IndieWeb. For example, there is a Wordpress plugin that gives you IndieAuth, Webmention, and Micropub out of the box, and there’s a few other IndieWeb-focused blogging platforms such as Known.

If you’re willing to get even more technical, there’s a lot of resources out there for adding IndieWeb functionality to static site generators such as Pelican, Jekyll (which is also usable on GitHub Pages), or Hugo.

You might also consider running a site with Publ, the publishing system that runs this website. At present, IndieWeb setup with Publ is still a bit involved (as it doesn’t yet have built-in support for Webmention or Micropub, and only supports IndieAuth as a consumer rather than a provider), but my hope is that, with more users of it, it becomes worthwhile to develop it further to make it easier to set up and use.

(I also swear that someday I’ll finally start on Subl, the subscription/reader dashboard!)

Finally, it would also be amazing if more of the smaller/upstart social networks would join in. There is nothing that prevents Mastodon or Cohost from being an IndieWeb provider of some sort. Both of them are already well on the way there!

I realize that this all seems very daunting; people have a hard enough time choosing a Mastodon instance, and that’s nothing compared to having to choose things like a hosting provider and a software stack and so on. But look at it this way: when you get a cellphone, you have to choose which model of phone and which carrier to go with, right? And if you’re putting on an outfit you need to decide what style to wear, and which top and/or bottom best suit that and fit your body and form of expression. But none of these decisions are permanent, either, nor should they be.

Just as there’s entire communities of folks with opinions on cars and phones and fashion, there’s also an entire community of folks with opinions on software and hosting solutions. You don’t have to go it alone. (And just like with phones and clothing, it isn’t strictly required that you make a choice or even use anything, but you’re probably better off if you do!)

A guide to social networks looking to implement IndieWeb protocols

Here are the core components and protocols you absolutely should support (IMO):

• IndieAuth (as a provider, so that you can provide an external client API and also so your users can sign in to other IndieAuth-supporting sites)
• Feeds (in both directions, ideally supporting atom+h-feed as a publisher and as many formats as possible as a subscriber)
• TicketAuth (in both directions), because privacy is important
• Webmention (in both directions)

And you probably should also support these:

• Microsub (allow users a choice in “dashboard”/reading interface)
• Micropub (allow users a choice in posting interface)
• WebSub (in both directions)

Things that are nice to consider, but not so useful in the big picture:

• IndieAuth (as a consumer, probably just to support private access, or maybe to support third-party login to your site)
• ActivityPub + Webfinger (to interoperate with Mastodon in its current state — ideally Mastodon would support IndieWeb directly!)

Also, the IndieWeb wiki has a page on what it means to be IndieWeb-friendly (and why it’s a good thing).

tl;dr

The IndieWeb is not a platform, it is an ethos. Build your own website the way you want it to be built, and use open protocols to interoperate with other websites. Own your own experience, and make it what you want it to be, not what’s given to you.