Preventing bot scraping on Publ and Flask Code

This morning I was once again thinking about how to put some proper antibot behavior onto my websites, without relying on Cloudflare. There are plenty of fronting proxies like Anubis and Go Away which put a simple proof-of-work task in front of a website. This is pretty effective, but it adds more of an admin tax (and is often quite difficult to configure for servers that host multiple websites, such as mine), and sometimes the false positive rates can have some other bad effects, such as disallowing feed readers and the like.

I started going down the path of how to integrate antibot stuff directly into Flask, using an @app.before_request rule that would do much of the same work as Anubis et al, but really the various bots are very stupid and the reason the challenge even works is because they aren’t running any JavaScript at all. This made me think that a better approach would be to have it just look for a simple signed cookie, and if that cookie isn’t there, insert an interstitial page that sets it via form POST (with a Javascript wrapper to automatically submit the form).

But then I realized, Publ already provides this sort of humanity test: the login page!

UPDATE: This approach is a bit obsolete; here is a better approach that uses HTTP 429 responses (which also serve the purpose of signalling to crawlers that they are unwelcome). I also no longer recommend the g.is_bot approach to removing page elements, as Publ now has user.is_bot as a built-in function that works better with caching.

Read more…

I FOUND IT fluffy rambles

I finally found the song I was looking for: it was “Lolita” by Moneyshot off the album Bliss.

Which I CANNOT FIND ANYWHERE ONLINE. Spotify doesn’t have it, iTunes Music doesn’t have it, Musicbrainz doesn’t have it, even Discogs doesn’t have it. CDDB probably has it (given that I’m sure someone submitted it when they ripped the CD back in the day) but they don’t make it easy to search. And FreeDB doesn’t.

But anyway, mystery solved.

Read more…

Building a lyric search engine fluffy rambles

Y'all probably know that my views on AI are somewhat nuanced. I’m not 100% “AI BAD!!!” but I’m also hesitant to rely on AI for a lot of things, and generally do not care for generative AI or any situation where you need AI to “reason” on things.

But, recently I’ve wanted to remember the name of a song that I listened to a lot, and where the lyrics I can remember don’t come up in any of the major lyrics databases. I listen to a lot of obscure indie music that tends to get lost by the major platforms, and I’ve been packratting music for decades now.

Further, it’s only fairly recently that music started to get lyrics embedded into the id3 tags (thanks to bandcamp really pushing for that) and even the streaming platforms have taken forever to pick it up. So a lot of the music I listen to has never had its lyrics entered in any sort of machine-searchable way.

But, hey, there are plenty of AI models for vocal extraction and text transcription… so why not actually use them?

Read more…

My Zelda tier list fluffy rambles

I (not so) recently1 finished Echoes of Wisdom and after a ramble about the Zelda games I’ve played and enjoyed, Star implied that I should do a tier list.

So, here we go, way too much effort put into telling the Internet my opinions on things: how I feel about every main series Zelda game so far2.

Read more…

I persevere fluffy rambles

I’m 47 now.

Or rather, I’ve been 47 for the past two days. I didn’t really want to post about it on my actual birthday, partially because I was super busy that day with two VRChat gigs (following immediately after the previous day’s choir show), but partially because I spent the first half of the day super depressed, particularly over the state of the world, but also over a bunch of interpersonal conflicts I’ve been having which seem to rhyme with one another, and also learning that a good friend of mine is in a really bad situation right now.

And how so many people I feel close to are hurting.

And how of course I share a birthday with a fucking tyrant who was using his birthday as an excuse for a jingoistic military parade, and how completely reasonable it would be that most of my friends would be out protesting that, rather than coming to see me perform silly songs about mental health in VRChat.

Read more…

May is/was ME/CFS awareness month fluffy rambles

I only just learned from eladnarra’s blog that May is ME/CFS awareness month, which I guess is fitting given that throughout all of May I’ve been getting painfully aware that I may have been developing it on top of everything else.

Basically, ever since I got COVID last June I’ve been especially prone to fatigue, vertigo, brain fog, shortness of breath, a persistent cough, and a whole bunch of other long COVID signs. The worst of it comes and goes, but this month has been particularly bad, especially on the fatigue front. And ME/CFS is a very common part of long COVID.

Read more…