Publ’s insurmountable technical debt

Back when Publ was brand-new I was using a fairly common ORM for managing my database. It was okay but I had some issues with how the project was being run, particularly in how the core maintainer treated bug reporters who were reporting things that he didn’t personally agree with.

So I switched to Pony ORM, which has a much nicer, more-Pythonic API, and whose maintainers are friendly and helpful. It’s missing a few important functions such as migrations but for Publ it doesn’t matter so much and they were always claiming that migrations were coming “real soon now” for a while.

Unfortunately, some changes in Python 3.11 broke Pony completely, and progress on getting it working again has been, well, slow.

Back when I switched to Pony I mused about just dropping SQL entirely and moving to a more primitive indexed data store, such as lmdb. But I really do not have it in me to completely redo all that stuff in Publ again, especially with all of the weird regressions that always occur due to subtle differences in locking behavior and so on.

Sometimes I think how now that I actually know how Publ should work I should rewrite it with better testability and a more actively-developed Markdown processor (because misaka has also been abandoned and is in permafreeze status), and if I’m going to do that, maybe I should use it as an excuse to finally learn Rust because honestly the Python ecosystem is kind of a mess right now.

I’ve put so much work into Publ but keeping the project going just feels so… difficult.

Or I mean I could switch to any number of existing site generators out there but then I’d be giving up all of the things that makes Publ unique (friends-only/access-controlled entries, variadic templates, top-notch template-driven image renditions, etc.).

This is one of those things where the decision would be a lot easier to make if people were actually helping to fund Publ development or if it had any actual, y'know, users.

Things that have gotten tricky with Publ:

  • Unmaintained/abandoned Markdown engine, with a big chunk of Publ’s own code written specifically to that engine’s quirks and features
  • Poor automated test coverage and an architecture which doesn’t lend itself to improving it
  • Way too much dependence on Flask, which also limits adding features I’ve been wanting to add
  • Awful wish-it-were-async image rendering pipeline (fundamentally limited by how Python’s WSGI interface works)
  • Lots of sharp edges with actual deployment, especially regarding Python versions combined with Python’s cavalcade of incompatible dependency/deployment management systems

Things that I’d also like to be able to support, which is made difficult by the current architecture:

  • Object-storage backend (S3/B2/et al)
  • More “cloudy” deployment, with proper support for load balancing/multi-region availability/fleet deployment/etc.
  • A proper content management API (beyond “put shit on the filesystem and hope watchdog doesn’t fuck up”)

but then thinking about that also makes me like, grumpy, because that’s supporting use cases other than why I designed Publ.

Part of me just wants to like. Build something completely different, like an IndieWeb version of Cohost or something. I mean that was the original design of Publ to begin with: it was meant to be basically self-hosted Tumblr with better access control. And when I first started working on it, I hadn’t yet learned about the Indieweb or any of its protocols, and I think if I’d set out to build an Indieweb-first publishing system I’d have made some very different decisions along the way.

Gah, I dunno. Things are working “fine” for now. Maybe I’ll just kick the can down the road until the current issues become untenable. Maybe Pony will fix Python 3.11 compatibility, and maybe it’ll turn out that migrating the Markdown processing to mistletoe or Mistune or whatever won’t be that bad.