So hey, I thought it was weird that nobody had been posting comments on my blog in a while. Turns out comments were just, like, broken, and nobody told me, for some reason.
The problem turned out to be that Isso currently doesn’t work on Python 3.8 (or at least, the current released version, which is ridiculously outdated, doesn’t), and it was easy to roll it back to Python 3.7, thanks to poetry’s pyenv integration. So, score another one for poetry.
But why don’t people actually tell me when they’re having problems with my site? Do people just assume that if something’s broken it’s broken on purpose? Because I mean… no?
Anyway, comments are fixed now.
So, one of the things with the Isso migration is that I finally came up with a better way of handling thread IDs to keep them actually-private. And part of that is the mechanism to rehash them.
Which is good, because I keep on accidentally leaking the dang secret sauce. The first time was when I updated my sample templates with the comment hash generation (and I accidentally left the HMAC key intact), and the second time was when I started building a new Publ-based website and decided to start with my actual
app.py as the basis, HMAC key and all, never mind that I later ended up removing about 90% of the beesbuzz.biz custom routes and the Authl config since they’re not actually needed for this site. Yeesh.
Anyway, whatever. Someday I’ll learn my lesson (and maybe I’ll even go so far as to make the HMAC key not even be checked into code!), but today is not that day.
As far as I know, all of the comments have been restored and mechanically updated to work correctly. It’s pretty neat that I actually have comments dating back to 2003, that have survived four separate comment systems! (Movable Type, phpBB, Disqus, and now Isso.) And some of the oldest ones hadn’t been visible for years, since I never got around to migrating them over to my comics section before.
I also now have a script to automatically rehash the thread IDs in case the HMAC key leaks, as it did yesterday when I accidentally forgot to redact it from the sample templates repository, oops. I doubt anyone saw that but now it doesn’t matter if they did.
I do want to make a final migration script to try adding thread nesting to comments which quote other comments. I have a good idea of how to do it but it’s gonna be tricky and since Isso apparently uses oldest-to-newest sort on comments I don’t know how useful it’ll be, anyway. But I like doing that sort of thing.
I also have automated backups of my comment database, as well as having it checked into a git repository so I can do simple checkpointing whenever I do something funky with a migration (and it means I can also run the migration on my local machine instead of having to worry about hecking something up in production). And of course since Isso runs as its own systemd unit I can easily take it down while I’m doing a thing. (If you ever notice my comments completely vanishing for a while, that’s probably what happened. Unfortunately there isn’t any easy way to show a reasonable message when that’s what’s going on.)
So, now I feel a lot more confident in the privacy and longevity of my comments. Which is good because I have a lot more private stuff to talk about. 😛
Because my original import from phpBB to Disqus got botched, and the Disqus to Isso import lost a bunch of useful information, I ended up just going back to my old phpBB database and reimporting it directly into Isso. It mostly went well but there’s a few things that I need to go back and fix. This is my TODO list:
<a href>stuff that got converted to
Defunge the weirder bits of BBCode where e.g.DONE
[quote:abcde]so it didn’t get converted to HTML (example)
Clean up some older comments where I was a lot more accepting of Problematic Things (not gonna link to any but yeah they’re there)done, I think
- If possible, reparent comments based on
[quote]s (way easier said than done, I’ll probably have to do that manually)
Update: generate a new comment secret key and fix the thread IDs, because I made an oopsDONE Looks like when I did the reimport of phpBB stuff I accidentally removed some of the earliest Disqus-based comments (example, also) so I’ll have to do a bunch of reconciliation for that, fun fun…DONE Also some of my earliest journal comics had comments posted via Movable Type’s comment system rather than phpBB, so I’ll want to also migrate those over (which I never got around to doing back when I was still using Movable Type to run my website); back then I just had “native” MT comments rendered in the MT template, which was Good Enough and I figured I’d get around to fixing it later. Well, it’s later. And that’s done. Even though I’m up way later than I meant to be. Oops.
Oh, and since I set up monsterid for the default avatars I feel like I should try to track down the email addresses of the folks who were posting to Disqus and fill that stuff in wherever possible.
I promise at some point I’ll get back to blogging about stuff other than the website itself.
Okay, instead of trying to modify Isso to support thread IDs that are separate from page URIs, I ended up leveraging the way that Publ request routing works and just made all thread IDs consist of a
/<signature>/<entry_id> path, where
<signature> is computed from an HMAC signature on the entry ID and a secret key. So, now the thread ID is only visible to people who have access to the entry in the first place (as long as my signing key never leaks), and the fact that Isso only uses the thread ID when generating a reply email link isn’t a problem.
So, for example, this entry has an entry ID of 4678, and the generated thread ID is (for example)
/890824f4d450d4ac/4678, so when someone gets a reply notification the email will say something like:
such-and-such <email@example.com> wrote:
Link to comment: http://beesbuzz.biz/890824f4d450d4ac/4678
which will then redirect back here.
It’s not ideal, of course, but it works well enough.
Of course, to do this I had to migrate all of my thread IDs again, but hopefully this is the last time I’ll have to do that, and it also takes care of all my legacy Movable Type-era thread IDs. It does set a bad precedent that I’ll have to migrate thread IDs more in the future if I ever change my publishing system but the fact I was able to get away with not doing that for so long is a pretty good testament to my laziness, which I ended up having to pay interest on in the future anyway. So, lesson learned.
Also, this approach is even better privacy than what I was hoping to get out of the Disqus method; as it stood before, someone on my friends list (or who saw an
Auth: * entry) could have theoretically figured out the way I was determining private thread IDs and used that to explore comments on entries they don’t have access to, and also there was an issue that if I ever took a public entry private, its thread ID would remain the same as when it was public. But this way, it’s unguessable as long as my HMAC key never leaks, and if my HMAC key does leak I can just reset it and regenerate the thread IDs. (Edit from the future: Ha. Haha. Ha hahaha ha haha. Ha.)
This approach is also useful for things other than Publ; my advice to anyone who’s using Isso for comments is that instead of using the actual entry URI as the thread ID, they should have some sort of stable mechanism for forwarding an opaque thread ID to the actual entry, and use that. This just happened to be really easy to implement for Publ since Publ already supports opaque ID chasing.
So, Disqus has served me pretty well for quickly embedding comments into my website, but there are a few pretty big downsides to it:
- No support for private/hidden threads
- No way to disable random discovery of hidden threads, by design
- They’re trying to make the whole Internet into their own forum rather than providing “just” a comment system (not that anyone even uses it the way they intend)
- Their UX keeps getting more and more cumbersome and annoying
I’m going to look into alternative comment systems, ideally ones I can self-host. Isso looks promising, if a bit sparse. So does Schnack. (I’m going to try Isso first because its setup/requirements are far less onerous.)
Anyway, thanks Passerine for bringing the privacy leak issue to my attention. I figured there was probably something like that lurking in the shadows, but I didn’t think it was quite so close to the surface…