RSS LJ

September 7, 2014

Let's build a social network! (, , )

by fluffy at 4:43 PM

Lately I've been on a huge RSS-evangelism kick. I don't want to see it die out. It's an open standard that was built to enable the free sharing of ideas on the web in a transparent, simple, free manner, without any advertisers dictating what you could or should see.

Unfortunately, advertisers are what fund the systems that make it easy to get and share online, and as a result, the various getting-online services have a vested interest in keeping people captive within those networks; the last thing they want is to make it easy for people to import or export feeds that can be used on other systems.

The thing is, there is a totally-open distributed social network out there already, hiding in the aether; you just need to find it. Here's my guide to connecting to it without too much work. I'll try to keep things simple, but feel free to make suggestions for making it even simpler!

Technical Note

Many times in this article I refer to RSS, because that's a term that more people are familiar with, but what I really mean by that is "RSS or Atom." Atom is another social stream format that is similar to RSS but also a lot more expressive (and has a few major advantages for this). In general, the specific choice of format doesn't make much difference for what I'm talking about, but I just thought I'd get that out of the way.

What you'll need

You will need some sort of web hosting setup that supports PHP, which is most of them these days. For these purposes, something like Dreamhost is pretty good. I've had my ups and downs with them, but there's a few reasons why I still recommend them for personal site hosting:

  • They have a really easy-to-use panel, including managed one-click installs of a lot of useful software
  • They provide PHP and MySQL hosting, which are the easiest way to get going with a lot of stuff
  • They also give you a shell account, which is well worth learning how to use!
  • They let you set up "sub-accounts" so that you can share your hosting with your friends
  • They give you one free domain registration if you purchase an annual plan (and you can set up unlimited subdomains, again great for sharing with friends)

First steps: set up your hosting

After you get your hosting (which could be via Dreamhost, or via a generous friend, or your university, or whatever!) you'll have a place to put your things. This will have some sort of address for it, be it a domain of your own (such as example.com) or a hosted directory on a shared server (such as example.com/~yournamehere or yourname.example.com) or whatever. This will be the entry point into all things you.

There will be at least two things that you need to set up: a feed reader (equivalent to your "inbox" or "timeline" or "feed" or "dashboard") and a blog (equivalent to your "wall" or "posts" or whatever). You'll probably want to plan out how you have your things laid out a little bit; for example, my social setup is as follows:

But this isn't how it has to be! You could easily set things up to be, for example, if you just have a ~username directory on someone's domain, you might go for:

  • Top site: http://example.com/~fred/
  • Blog: http://example.com/~fred/blog/
  • Dashboard: http://example.com/~fred/feeds/

Your timeline/feed

The first thing you'll want to set up is a way of subscribing to other peoples' content. For this, I prefer FeedOnFeeds, an open-source RSS reader. It's not the prettiest thing around but it's incredibly lightweight, easy to set up, and is being somewhat actively developed in an open-source manner. (There is an official site for it but it's pretty outdated; the original author stopped working on it and now it's being developed by an unassociated group of volunteers, myself included.) Here is what it looks like:

FeedOnFeeds' user interface

To set it up, first you need to download the files. You can just download a snapshot of the files and then upload them to where you want your dashboard. (More advanced users would want to use git to download it, because that makes upgrading way easier, but that's well beyond the scope of this article.)

After you've uploaded the site files, you'll also want to copy the fof-config.php.dist file to fof-config.php and edit it to set things up for your situation. You'll probably need to make the following changes:

  • Remove the // at the beginning of the line that says: // define('USE_SQLITE', 1);
  • Change the line that says define('FOF_DATA_PATH', '/var/fof-data/');. This just tells FeedOnFeeds where to keep its database. Where this should go depends on your hosting set up; on most hosting accounts you can just create a directory in your account called, say, fof-data, and then point that to /home/username/fof-data. For example, if your account username is "dinorawr" then that line would read:
    define('FOF_DATA_PATH', '/home/dinorawr/fof-data/');
    although the specifics will vary based on your webhost's setup.
  • Point your web browser to the uploaded install.php and follow the rest of the instructions. For example, if you uploaded FeedOnFeeds to be at http://example.com/~bob/feeds/ then you'd point your browser to http://example.com/~bob/feeds/install.php

And now you have a feed reader and can subscribe to practically anything. To follow people, you just have to click on the "Add Feeds" link and copy their website address (for example, http://beesbuzz.biz/blog/) into the feed form. FeedOnFeeds also provides a couple of ways to make subscribing even easier; just read the instructions at the "Add Feeds" link.

(You'll also probably want to set up background updates. To do this you just set up a scheduled task that loads the update URL which is e.g. http://example.com/~bob/feeds/update-quiet.php; generally you'll want to set your webhost to do this. Most hosting providers provide a service for this, usually called "cron" but sometimes called something else. The FeedOnFeeds documentation has a bit more information on this.)

Sometimes a website will have an RSS feed but FeedOnFeeds can't find it automatically (because of the thing I talked about with advertisers and lock-in, and also because a lot of sites just aren't set up with feed autodiscovery). If you run into this, sometimes there will be a little RSS icon somewhere on the page (it usually looks like this: RSS), and you can use that link to subscribe, or sometimes it's hidden even more severely. There are various articles out there on how to find other services' RSS feeds, though.

Basic reblogging/sharing

FeedOnFeeds provides a sharing feed, so that you can share items from your inbox to other people. But this has to be turned on! Just go to the "prefs" section of FeedOnFeeds, and then under "Sharing," set it to "share tagged as 'shared' items." It will look something like this:

FeedOnFeeds sharing configuration

After you save your preferences, the link will be active. (My sharing page is over yonder.)

Full blogging

Now, this is where things can get a lot more fun, but also a lot more complicated. There are a lot of blogging systems out there that you can use, each with pluses and minuses. Here's a quick rundown of them:

  • WordPress: This is very popular, and Dreamhost makes it incredibly easy to set up (via a 1-click installer). There's a lot of templates available for it as well. However, historically it's had a lot of security issues, and it also tends to need a lot of server resources. Customizing the layout is also pretty tricky.
  • Pelican is lightweight and simple and works well out-of-the-box and takes practically no server resources to run, and customizing it is easy if you know basic HTML and CSS. However, it also requires quite a bit of technical skill to actually publish with, since you have to upload your entries as files to certain parts of your site and then run a publish command.
  • Octopress is similar to Pelican, only it's even more complicated to set up and work with. I wouldn't recommend it. Also, it needs a newer version of Ruby than what Dreamhost currently supports.
  • Movable Type: This is what I use. It's stable, robust, fast, and very powerful, but also requires a lot of technical know-how to set up well, current versions cost money (but 5.x is still free), and it needs Perl and a real database to work. I would definitely not recommend it for beginners.

These are just ones I've used. Dreamhost has one-click installers for a bunch of blogging systems, though, and you might find one of them amenable to your needs! (For example, Textpattern looks pretty promising.)

Anyway, once you have a blog, you just publish stuff and then people can use the RSS feed to subscribe.

You can also set up comment engines, although I wouldn't recommend it, and some blog engines support "trackbacks" (a means of getting notified when someone responds to one of your entries), although that tends to get abused by spammers so I wouldn't recommend it either. One way to keep track of conversations is to just look at your site referrers, which most webhosts will provide in your "site statistics" or the like.

Pulling it all together

Some folks like to keep their own articles separate from their shared links. This is fine! People can just subscribe to your blog feed and to your sharing feed separately. But you might want to make a page that gathers everything together (including public posts on other sites and so on), or at least put the various feeds on a sidebar or whatever.

You might also want to provide a feed that contains all your other feeds, just so that people can subscribe to one thing and get everything (especially if you add more stuff). You can use a simple feed glommer for this purpose, or you can use something fancier that does things a bit more elegantly.

My dream for the future

I want this entire guide to become unnecessary. I want to see people work on a blogging-and-syndication-and-communication platform that ties all the above together. Here is my wishlist for such a thing:

  • Easy deployment; the fewer steps and less customization necessary, the better
  • Easy publishing and content management; a rich-text editor, easy template customization, categories optionally with their own templates and URL schemata, etc.
  • A simple stream-of-content inbox (like FeedOnFeeds) that also provides the ability to prioritize items, possibly by a personalized, private machine-learning algorithm
  • Supports RSS and/or Atom feeds for everything
  • Able to produce the following sorts of content:
    • New, original content (articles, photos, media, etc.)
    • Bulk republications of subscriptions (primarily for republishing your social output elsewhere)
    • Republication of existing items (both from subscribed feeds and from ad-hoc shared URLs)
    • Scheduled posts (both time-based and queue-based)
    • Posts that expire (or at least disappear) automatically (for e.g. sharing videos or links that are in the moment)
  • OpenID support for authentication (federation of external servers as well as providing its own simple server)
  • Friends-only/private-publishing functionality (probably with a combination of authentication and per-friend private feeds, and if we don't mind extending RSS/Atom, adding private-key crypto for the summary; I have some ideas about how that could work but that's a subject for an article on its own)
  • Down the road, PubSubHubBub (because pull kind of sucks)
  • A trackback-type notification mechanism that generates inbox items (ideally with a spam filter and maybe some form of auth)
  • And most importantly, only use open protocols for the above so that people can join in however they want

Also, there isn't really any reason why the subscription and sharing/blogging portions need to be in a single piece of software; any blogging platform that supports simple link-sharing-type publishing could be just fine, and it's pretty easy to write a "share to whatever" plugin for FeedOnFeeds anyway. There's also no reason that the publishing platform can't just provide an "inbox" feed (with trackbacks/linkbacks, auth requests, etc.) for the owner's feed reader!

Essentially, all of the protocols for the above already exist. There's no reason everything has to be centralized and tightly-controlled; there is absolutely nothing new about any of the technology I've written about here. We just need it to be easy for people to do this stuff.

Let's take back the Internet!

Comments

#16829 fluffy 09/07/2014 10:12 pm
    For what it's worth, I am well aware of the Diaspora project, but I don't think that their model really fits what I'm going for here. It's a very interesting approach to making what is basically a distributed Facebook-type social network, but it still exists as its own little walled garden where, as far as I can tell, it just happens to replicate that content across every instance of that garden.

    What I'm looking for here is something that sits on top of the web as it already is, and keeps things open and simple and doesn't force anyone into any particular implementation of things. Items don't have to live on in perpetuity, they don't have to be pushed and replicated everywhere, and they certainly don't need any specific software or connection point or API or whatever to get data; anything that speaks RSS or Atom is applicable, and so you can connect to this network with whatever feed readers you want - and there's a hell of a lot of them out there.

    (Private items will always be more difficult; I see requiring some sort of OpenID provider on the reader's behalf and a bit of wrangling to make it work well while still protecting user privacy. But OpenID is also very simple and open and has a billion easy-to-deploy implementations, and hell, you probably have at least one OpenID provider without even knowing it!)

    Basically, this is about having everything in one place, integrated together in a neat little package of simple standards that everyone already supports, rather than having yet another standard to choose from. The sites you already love don't need to do anything different to work in this setup. And people who already have RSS readers won't have to do anything different either. The only people who should be worried are the ones who shouldn't have your data in the first place.
    #16830 fluffy 09/08/2014 06:26 am
      Also I apologize for continuing to edit this article well after-the-fact but more ideas keep on popping into my head.
      #16831 fluffy 09/08/2014 07:07 am
        So, for an example of how authorized feeds might work:

        1. Alice visits Bob's blog, and sees a handy "be my friend" link. This link takes Alice to an OpenID login page.
        2. Alice completes the OpenID login with her identity URL (example.com/alice); Bob's blog now sees Alice as being logged in, and all links to the RSS feed are now made specific for Alice's identity (e.g. with an embedded auth token in the URL). These feeds would have something like "This feed is for your use only; please do not share!" in the description.
        3. Bob gets an inbox notification saying "Hey, a new user has logged in with the URL example.com/alice." In addition to providing a link to that URL (so Bob can see who it is), Bob can now place Alice into whatever permission groups he wants.
        4. Alice ends up subscribing to Bob's blog using the custom URL; now the permission groups that Bob gave Alice dictates what shows up in Alice's customized feed.
        5. Bob's blog system now knows who's subscribed, and he can see and do a bunch of stuff based on that (Cathy's subscription lapsed, Doug should get into the permission group for Eric's surprise birthday party, Francis has been a jerk lately and probably should be taken off of the Deep Thoughts list, etc.)
        6. Another nifty thing is that if someone logs in via OpenID a few times but their personal feed hasn't been used in a while, the blog can say "Hey, did you know you can subscribe to this blog at http://example.com/bob/feed/01234-56789-ABABDCCDF00F?" That way, they know they can subscribe, and it gently points people towards subscribing with the right feed (in case they previously subscribed anonymously).

        Bob can also be privacy-conscious and only allow a brief excerpt (or even just "personal entry") into the friends-only feeds, to make it more difficult for people to accidentally (or "accidentally") re-share non-public content.

        This could all be done pretty easily with an easy-to-embed PHP library, which would then in turn work really well with any static blog generator out there. Or, in other words, I just might work on such a prototype to replace the kludgy friends-only stuff on this very blog. Smile
        #16836 pat 09/15/2014 08:10 am
          RSS feeds are awesome! I think they are powerful and easy to work with but way too obscure. Without any kind of coding they helped me get all kind of advantage for using internet tools to run a business. A lot of beginners don't seem to know what they are, big loss... too bad igoogle was pulled, but now I like using netvibes for that. It sucks if they are purposely underpromoted for commercial reasons.
          #16838 fluffy 09/24/2014 06:04 pm
            Oh, another kind of obvious thing to add: API access, so that it's easy to build mobile clients/views/etc. for reading and blogging. Although that's just an icing thing, and it'd be better to just have a good responsive mobile UI in the first place.
            #16839 fluffy 09/29/2014 07:52 pm
              Ah, nice, there's an improved protocol for Trackback/Pingback called webmention (found out about via l.m.orchard)
              #16876 fluffy 12/13/2014 10:12 pm
                So I finally got enough gumption to start up an assembly project for this. Because I don't have the kind of time/energy to work on code myself but I still want it to happen and I want to do what I can to make it happen.
                #17019 fluffy 09/10/2015 11:28 pm
                  So I let this stuff lay fallow for, er, a year... but now I'm starting up the publication portion in earnest. For now I'm working on the data model stuff because I want that to be Right, and then I'll work on simple commandline-based publication stuff and a basic template for things. The admin interface is going to be a HUGE pile of work, so I'm holding off on that for now...