RSS LJ

September 7, 2014

More thoughts on Bagel Voice (, )

by fluffy at 1:22 AM

I actually might not give up on Bagel Voice so soon after all. My original intention for it was to offer an XMPP console (to get push notifications for "free"), and it really wouldn't be that much work to port it to Twisted, which has XMPP support. (It does make deployment a bit more involved, but things like wsgi and Passenger ease the pain too.) At one point I thought I might port it to node.js but you know what, the more I use Python, the more I like Python.

Incoming call integration is still problematic due to a couple of aspects of iOS's security design (this is one of the few places where Android is better), but iOS 8 interactive notifications fix (or at least mitigate) the major issue, and I can always just keep having the SIP proxy dial my real cellphone number instead.

So, I'm still thinking about continuing to develop it.

September 4, 2014

Reminder Me (, )

by fluffy at 2:57 AM

For a long time I've been needing a service that gently reminds me to do errands around the house in a way where I can just get around to doing it, but keep getting reminded if I forget (things like cleaning the cat box, watering the plants, etc.). I was using an iCal calendar for this but that didn't quite work out for me.

But, hey, I still use an RSS reader, so I made a simple RSS reminder-generating service. And you can fork and install it yourself, too.

August 17, 2014

Bagel Voice (, , )

by fluffy at 8:32 AM

The thing that put me down the Bagel Voice rabbit hole in the first place was being unable to get cellphone reception at my desk at work or at home and using Google Voice + VoIP for my incoming calls, and then realizing that I was paying a lot of money for voice and data service that I hardly ever used, and then I started thinking about how I could get voice calls and text messages during the 90% of the time that I have WiFi coverage without spending more than necessary for the mobile part, all the while still having something like Google Voice but without the Google. I figured that a piecemeal data-coverage-based web app would be sufficient for my needs, and that I could fill in the gaps with a mobile access point.

Turns out, affordable mobile access points have crappy coverage too, and dealing with a web app for texting is really obnoxious. I could make it less obnoxious, but the work necessary to get a better level of integration is just not worth it.

Anyway, iOS 8 finally supports WiFi calling (and presumably texting), and T-Mobile has a $30/month plan that's amenable to my needs and not outrageously expensive (and provides unlimited minutes when doing WiFi calling, negating the other reason I went down the VoIP service rabbit hole it still charges for airtime, unfortunately, but I still have my VoIP service for outgoing calls anyway), so I'll probably just switch back to a regular iPhone on T-Mobile when iOS 8 comes out. (Or maybe sooner, if the iPhone 6 comes out first — I have no desire for an iPhone 6 due to it being allegedly just as big as the Galaxy Nexus, but that will drive the price of used iPhone 5s down substantially.)

Twilio is pretty fun to play with, though, and I have some other app ideas that make sense for it. And of course anyone else can feel free to fork Bagel Voice if they want to pick up where I left off.

July 31, 2014

Embedding binary resources with CMake and C++11 ()

by fluffy at 1:25 PM

The problem

Let's say you want to make a single-binary application that has embedded resources (images, GLSL shaders, etc.). Let's say you want to automatically wrap your resources in a storage container to make it easier to deal with stuff. Let's also say that you might even be using CMake as your build system.

CMake doesn't provide a way of making a custom build rule, and using extern data is a little unwieldy. So here's an easy-ish way to do both parts, making use of C++11 language features (and a scary preprocessor hack).

The C++11 bit is also useful on its own, even if you aren't using CMake, although things will have to be adapted to your build system of choice.

(Note that this only applies to UNIXy toolchains, such as what you'd find on Linux/Android, OSX/iOS, mingw, Cygwin, and so on. Visual Studio users are on their own.)

July 25, 2014

A simple SIP multiple-ring proxy (, )

by fluffy at 11:55 PM

I've been using Twilio as my dial-in device (DID) for my VoIP service, but it has a pretty major downside in that it only supports either a single SIP endpoint or multiple-ring of multiple trunked phones. A single SIP endpoint is much nicer because it reduces the latency somewhat, and also reduces costs considerably.

So, in order to support multiple-ring functionality on Twilio using SIP, you need to set up a very simple SIP proxy. Here is the basic configuration I'm using for that.

February 17, 2014

My call switchboard app progress ()

by fluffy at 11:34 PM

I've been calling the app "Voicebox" so far but it turns out that this completely uncreative name is also taken... by another Twilio app. So I'll have to come up with another name to avoid confusion.

Also, a bit of a setback in my overall plan: it turns out that Twilio can only forward to a single SIP endpoint at once, plus it turns out that CallWithUs won't easily do SIP call termination from arbitrary peers (and I can't figure out how to get it to accept Twilio's peer). Since I need at least two SIP devices active at any given time (iPod Touch and Obi) that means I still need something that will proxy a single incoming call to multiple SIP endpoints — and CallCentric still fits the bill. (Using the $1.50/month DID.)

But all in all things are going well.

October 10, 2013

The problem with select() vs. poll() ()

by fluffy at 8:43 AM

The UNIX select() API should have been deprecated years ago. While unsafe operations like sscanf(), sprintf(), gets(), and so forth all provide compile-time deprecation warnings, select() is also incredibly dangerous and has a more modern, safer replacement (poll()), but yet people continue to use it because they misunderstand the problem with it.

The problem is that it doesn't scale. In this case, "not scaling" doesn't mean it's bad for performance, "not scaling" means it will destroy your call stack, crash your process, and leave it in a state that is incredibly difficult to debug.

May 18, 2013

FeedOnFeeds updates ()

by fluffy at 3:03 PM

Plenty of updates to FeedOnFeeds since the last time I posted about it. You can of course just read the changelog there. The main changes to care about:

  • Item purging should now behave correctly on feeds with missing or broken timestamps
  • It now works with PHP 5.4
  • The dynamic update estimation algorithm is a bit better, and also has metrics logging to make it easier to experiment with changing the algorithm

As always, the best way to install or update is to just clone or pull from the git repo.

At some point I should put together an actual site for this stuff, and maybe provide actual versioned snapshots too.

March 23, 2013

InsufficientOxygenException (, , , )

by fluffy at 5:29 AM

I had a dream in which I was sitting on a train and talking to an ostensible coworker (nobody I know in real life) about programming languages and some of the interesting stuff going on in performance evaluation, when suddenly she asked me, "Dude, are you okay?"

"Yeah, why?"

"Then why are you turning bat-cave black?"

I suddenly realized that I was turning blue with hypoxia, and promptly woke up to find myself not breathing.

After a few seconds of hyperventilation, my first thought was: "Uncaught exception propagated up the stack."

February 19, 2013

Reviving FeedOnFeeds ()

by fluffy at 12:42 AM

My favorite web-based feed aggregator of all time, FeedOnFeeds, had a few flaws which made me give up on it and switch to using a different feed reader. However, that other feed reader was getting increasingly complex, and the person maintaining it made it very clear that he saw his users as a liability rather than an asset. So I tried a few more feed readers, but kept on going back to remembering how good FeedOnFeeds was, aside from a few flaws.

Unfortunately, FoF has succumbed to a lot of bit rot, namely it makes use of a lot of functions which have been (finally!) deprecated in PHP 5.3, and it comes bundled with a very old version of SimplePie. There were also a few things which it never handled at all well to begin with, such as entries with malformed HTML (which wreaked a lot of havoc). Meanwhile, there was a fork called FoF-Redux which turned into a complete rewrite, and they never got it to a particularly working state (and what they did have working showed that they completely missed what made FoF so great to begin with).

So I did what any good OSS zealot would do, and forked FoF. If you want to play with my modernized fork, just do:

git clone http://git.beesbuzz.biz/FeedOnFeeds.git/

I also have a Github instance if that's more your cup of tea.

So far the changes I've made have been:

  • Updated to the latest SimplePie
  • Removed a bunch of deprecated cruft (there's probably still a bunch still in there)
  • Added a plugin for HTML Purifier (as well as the associated library)
  • Added a plugin for sharing items on Twitter (because, you know, del.icio.us is a bit... outdated)

Let's make the best damn feed reader ever work even better than ever!

February 7, 2013

In which compilers are smarter than me (, )

by fluffy at 2:54 PM

So I spent all day doing some fiddly optimizations on my network datagram protocol to try to optimize the alignment-matching characteristics of data transfers (similar to the ones I did in Sprung). At best I managed to beat out default mismatched alignment for the fairly old version of glibc that we're still running in production. However, my team lead suggested I build it on Ubuntu with a modern glibc and see just how pointless that work was. I did. Now I'm sad.

This is what happens when all my optimization tricks are based on late-90s x86 and early-00s ARM, because that's the world in which I've spent most of my time-critical optimization work. So, not only are CPUs so fast that you "don't have to" think about this stuff anymore, but the underlying system libraries and processor μops have already fixed these problems anyway, especially for streaming reads.

(To be fair, these tricks are still very much valid in the mobile/embedded space. But that's not a space I'm working in anymore.)

December 4, 2012

Shuffling lists in MPI ()

by fluffy at 12:00 AM

Let's say you want to shuffle a list in MPI (one of the scripting languages used on MUCKs).

The MPI documentation provides this code snippet for shuffling a list:

{lsort:{&list},v1,v2,{gt:{dice:100},50}}

This method, while easy to implement, is WRONG, WRONG, WRONG. It does a terrible job of actually shuffling a list's elements. It will tend to bias certain elements of the list to be in certain positions, and so it won't actually be all that random.

A much better approach to shuffling a list's elements is like so:

{parse:pfx,{lsort:{parse:col,{&list},{dice:900,1,99}{&col}}},{midstr:{&pfx},4,-1}}

The way this works is that for every entry in the list, it prefixes it with a random number from 100 to 999 ({dice:...} generates 1-based numbers, like dice rather than any other language's RNG), then it shuffles the list based on that number, and then strips that number off. It's a little silly, but it will at least generate pretty good shufflings, as long as your list has no more than 30 elements. (Once you exceed 30 elements, it will start to bias the alphabetically-lower entries to higher up in the list for reasons not worth getting into.)

Of course, if you want to improve the entropy, you can increase the number of digits in the prefix, for example:

{parse:pfx,{lsort:{parse:col,{&list},{dice:90000,1,9999}{&col}}},{midstr:{&pfx},6,-1}}


which will generate pretty good uniform selections for lists of up to around 300 elements. (Essentially, the square root of the range parameter is how many numbers you can generate without having to worry about collisions, so adding two digits to the prefix will add one digit's worth of uniform entropy.) Of course if you're handling lists of 300 or more elements chances are the MPI interpreter is going to barf anyway.

There's other ways you can generate the prefix which will improve the entropy a little bit (such as {left:{dice:10000},5} but they won't help that much, and also will make it more likely to exceed the MPI instruction count limit (which I have actually run into a few times).

Of course, if you were using a real language, you could use an iterative in-place shuffling algorithm but that would just make too much sense.

October 24, 2012

Sprung memories (, , , )

by fluffy at 9:37 PM

The Sprung soundtrack just came up on iTunes shuffle for me, and it's been about 8 years and for some reason I feel like talking about some of the fun technical stuff. It wasn't a great game and it wasn't an amazing use of the DS technology, but there was some pretty cool internal stuff about it, even if it wasn't really visible in the end product.

I figure that since it was released over 8 years ago now and apparently Ubisoft doesn't even have the source anymore there's no harm in talking about things. (No, Nintendo, I'm not sharing any secrets specifically about the DS, and I never did to begin with, despite what your clueless lawyers told Ubisoft's clueless lawyers.)

I feel like I've written about this before but if so I feel like writing about it again, so bear with me.

June 11, 2012

Password security update (, , )

by fluffy at 10:17 AM

Thanks to the recent spate of websites' password hash lists getting leaked and a lot of my internal shame over the fact that phpBB2 still uses unsalted md5 for its auth, I finally got around to fixing phpBB2 to salt its damn passwords. It's shameful how many PHP apps don't use the built-in password_hash() function and just use md5() [WARNING: many of the comments on that page show a DANGEROUS lack of understanding of how this works!] instead - because people just don't understand security (even though this is stuff that's been known since the 1940s or something). It's stupid.

Anyway, I debated either replacing it with password_hash() and requiring everyone to change their password, or just wrapping the existing md5 values up in a crypt and making all future passwords a password_hash(md5($password)) instead, and the latter won out.

If (like me) you're running an old phpBB2 instance and don't want to upgrade to phpBB3, I've provided a handy guide for doing this yourself.

March 2, 2015 I updated this to use the newer (available as of PHP 5.5) password_hash/password_verify API that is far superior to crypt.

May 4, 2012

Some Mandelbrot rambles (, )

by fluffy at 9:04 PM

A lot of folks out there know about the Mandelbrot set. But that knowledge is often based on pretty abstract stuff, and is limited to the fractal as a two-dimensional thing that doesn't really make much sense. The other day I decided to screw around with some basic GPU programs, and I figured I'd finally write some stuff about this.

This is of course very basic to anyone who has studied fractals before, but the description of what's going on is largely secondary in this case.

(Warning: There are a lot of very large images in this post.)

May 3, 2011

Vala (, )

by fluffy at 8:24 PM

A while ago I learned about an interesting language called Vala, which is basically C++ if it were redesigned from the ground up with only the good parts of C++, plus useful parts of other more-modern languages like C# and JavaScript. Every now and then I look at the tutorial again and it's just getting better and better. And it is designed to maintain ABI compatibility with C, and actually is just a front-end that generates and compiles C code, meaning it's also perfectly-suited for embedded programming, especially on platforms where C++ has been stupidly hobbled (such as Android NDK).

I especially like the way it handles generics, the way all Vala references are equivalent to boost::shared_ptr (with a keyword to make a reference weak - without any need to explicitly lock it), the way that you can still do explicit memory management if you need to (only with a very nicely-implemented equivalent of std::auto_ptr to make even that stuff easier), it supports contract-based programming features (including requiring explicit nullability for object parameters), and that it supports true closures (unlike Java's quasi-closures) with a syntax that isn't ridiculously horrible (unlike in Javascript, C99, or C++0xB2011). It also has a very nice syntax for function pointers delegates, and also has some syntactic features that are there specifically to support asynchronous message handling.

It's a shame that people only seem to think that it's suitable for Gnome, probably because it grew out of GObject and is part of the Gnome project. It's not! It's a general-purpose language, that just happens to also provide GTK and Gnome bindings as part of its standard library. But it's also compiled, C-compatible (and therefore C++-compatible), and doesn't require a ridiculously large and fragile runtime library.

Maybe someday I'll have an opportunity to actually do a project with it. For now I just feel like I keep on staring through the window at the shiny toy that I wish I could use.

February 16, 2011

Any mod_rewrite gurus out there? (, , )

by fluffy at 10:50 PM

As an attempt at getting rid of some stupid spammers who are also trying to exploit nonexistent scripts on my site, I'm trying to redirect all requests that contain /../ in the original request URI (which are basically guaranteed to be an exploit attempt) elsewhere. However, it looks like mod_rewrite is only performing the substitution rule on the resolved path (i.e. it's treating /foo/../bar/ as /bar/) despite the fact that the REQUEST_URI environment variable still has the original /../-containing expression in it. I know my regexp is written correctly because the rule is working on other things (such as QUERY_STRING and HTTP_REFERER).

So, does anyone know if there's any configurations to mod_rewrite which might fix this behavior?

It's not a big deal but I've noticed a high correlation between the spammers who get through and the folks who are trying to find stupid CGI exploits and I figure anything that trips them up can only help.

February 14, 2011

C++ wishlist: Abstract-on-subclass (, )

by fluffy at 6:50 PM

In C++ you can mark a method as being abstract pretty trivially, e.g.:

class A {
  virtual foo() = 0;
};

which makes it so that you can't instantiate anything that derives from A until the method foo has an implementation. This is a very useful pattern.

What it's missing, however, is a way of insisting that every instantiable instance of A must provide its own implementation of foo; for example:

class A {
  virtual foo() = 0;
};

class B: public A {
  virtual foo() {}
};

class C: public B {
};

It would be nice if in this case, there were a way to make it so you can't instance C until it gets its own implementation of foo.

Of course, the situation I've run into where this would be exceptionally useful (having a complex DOM which needs per-class script bindings) is kind of nichey, and I can't really see it adding much value to the language as a whole. Still, I can dream...

March 19, 2010

Using inner classes to simplify complex cleanup in C++ ()

by fluffy at 5:55 PM
A lot of my work involves writing C++ code which uses fairly low-level C APIs which require stateful initialization and cleanup at various points (for example, binding a resource, performing an operation, and then unbinding the resource when it's done). The usual approach to this sort of thing is like this:
bool MyClass::useResource() {
    if (bindResource(mResource) != RESULT_OK) {
        //! Binding failed; return failure
        return false;
    }

    if (doSomethingComplex() != RESULT_OK) {
        //! Our complex operation failed; return failure (but first unbind the resource)
        unbindResource(mResource);
        return false;
    }

    if (doSomethingElse() != RESULT_OK) {
        //! Another complex operation failed; return failure (but first unbind the resource)
        unbindResource(mResource);
        return false;
    }

    // We were successful; unbind the resource and return success
    unbindResource(mResource);
    return true;
}
This is a bit annoying to deal with, especially if the unbind procedure is fairly complex. What's even worse is if you have multiple resources which get bound and need to be unbound, especially if there are multiple steps between those resource bindings. So here is a way to simplify this sort of thing.

October 10, 2009

Geometry Images ()

by fluffy at 7:52 PM
Back when I was in grad school I was doing a lot of research on graphics, and had come up with an obvious-in-retrospect geometry representation format, which I called an "NMMesh," for an NxM Mesh of points; at the same time, X. Gu at MIT had been working on a similar concept, which he called a "Geometry Image." Of course, I never got around to writing any papers, whereas Gu had, and so his (much better) name caught on. For a few years they were a darling concept in the graphics research community.

Recently I'd been messing around with my research 3D engine for various reasons (mostly of nostalgia), and I came to realize that even now there are several things I had been doing with geometry images that still don't appear to be well-known for them. So, I'd might as well write some of it up.

Older »