busybee fluffy rambles

Sliding Index controllers (Knuckles) down my wrists

This is what I tried at first, since other musicians I know swear by it and it works well for them.

The first problem with this approach is that the controller on my right hand still occasionally hits the strings, which can be really annoying while playing.

The second, much bigger, problem, for me, is that having the constant pressure on the tendons of my left hand causes a massive pain flareup including my fingers freezing up for a bit, usually right in the middle of a song. Not ideal!

Bifrost Pulse

I backed these haptic gloves on IndieGogo. Unfortunately I couldn’t quite get them to work, and at the time the driver wasn’t actually usable. More recently I’ve pulled them out of the box I had them sitting in and the batteries are trash and one of them had its tracker connecting bolt fall inside the case, so it needed a bunch of repairs, which wasn’t easy because they are not built to be serviced. I still don’t know if the driver’s actually usable (so far it just crashes a lot when I try to do anything with it), and Bifrost has put the project on hold for a while to focus on his positional tracker project.

Unfortunately, all of these things are basically just expensive hobby projects and there’s no guarantee they’ll ever actually become usable.

I also have my doubts that the Pulse gloves will actually sit reliably on my hands while doing the fast motions needed for guitar playing, especially since I’ll have to disengage the fingertips in order to actually play.

Folks who have been successfully using their Pulses have complained that even on a good day the battery only lasts around two hours (due to the unoptimized firmware and particularly the current constant stepper motor chatter) which would be a problem for most of my concert activities.

With the batteries in their current state I can only get them to even stay powered on for about 5 minutes at a time, but I’ve ordered new batteries which will hopefully hold out a bit longer. Still, I’m not optimistic.

Mounting Index controllers to the backs of my wrists

I’ve tried a few different approaches for this.

My original attempt was to get some Velcro straps that I attached to a flexible wrist compression sleeve, which worked okay at first but the controllers kept on working their way out of the straps and would just start flopping around and hitting things, and I tried various high-friction foam tapes but those didn’t help and just made a gunky mess.

My second attempt was to make some 3D-printed brackets that I clamped the controllers to. One of the clamps was slightly too tight and ended up damaging the finger tracking (and possibly positional tracking) on one of my controllers, which I’ve had to swap out (at great expense) for another controller. The clamp itself also holds the controller far enough away from my wrists that it adds a significant moment of inertia to them, making actually playing the guitar quite difficult.

Both approaches also suffered from making it quite awkward to work the sticks and press buttons. Getting on and off the stage was doable, but doing things like modifying sound settings or entering my desktop overlay (necessary for managing the recording and stream) became pretty much impossible in the moment.

Shiftall FlipVR

This is the most recent thing I’ve tried, out of desperation. I couldn’t find anyone who had actually reviewed these damn things, which are expensive (currently over $600/pair due to tariffs, although I managed to get a used pair for “only” $420 after shipping), and I figured I’d take a chance on them.

As SteamVR controllers go they are pretty godawful:

• I could not adjust the thumb clusters to fit my hands, and they kept on sliding around anyway, making it very difficult to actually control my avatar
• The way they mount to my hands puts a lot of pressure on the tendons, causing the same problems as trivially sliding Index controllers down
• Flipping them away was a bit unreliable and the thumb clusters would happily drift back to where my thumbs are, getting in the way of playing the guitar
• The hilt of the right controller is still long enough that it hits the strings unless I’m very careful about how I strum/pick (severely limiting my play style)
• Even at best, the thumb clusters still added significant moment of inertia, also making the actual playing more difficult

Basically it was the worst of all worlds for the above situations. After I do a video review demonstrating the problems I’ll be selling this on eBay and hoping it becomes someone else’s problem (and that I won’t have lost too much money on this).

Next up: Ultraleap Leap Motion

This is the next thing I’m trying. The Leap Motion 1.0 is easy to find pretty cheaply (I found one for around $50 in supposedly working condition), although its range and field of view are pretty limited. I wanted to get a 2.0 but those are currently running for well over $600 on average.

The reason this one’s interesting is because there’s an opensource driver (leapify) that allows for hybrid operation, where if you’re holding your SteamVR controllers, it lets the controllers win, but if you put them down, the Leap Motion takes over for the hand tracking. Plus, I’ve seen videos of people specifically playing guitar with a Leap Motion and having their hands be tracked properly.

I have some open questions about how well Leapify will work, though. It’s intended to be mounted to the HMD (fortunately there’s a 3D-printable mounting system for the Bigscreen Beyond, and it shouldn’t be too hard to design a Leap Motion 1.0 mount for it) but it isn’t clear if my hands will generally be visible to the Leap Motion if I do it that way.

There’s also some hints that Leapify has a mode where I can set the Leap Motion on a table and have it track my hands from a distance, but that will require calibrating it to the SteamVR coordinate space (which it might be able to do automatically, but the docs are extremely limited) and also I’m not sure if there will be enough range to make it comfortable.

Another annoying thing about this is that the Bigscreen Beyond only has a single USB-C accessory port (which is normally taken up by the headphones, because for some damn reason they decided not to put a built-in DAC and ordinary headphone jack), but I found a minimal USB-C hub that might work, and also a less-minimal USB-C hub that might work less-annoyingly. So I’ve ordered a bunch more stuff and hopefully I can get something to work.

But I’m skeptical that the Leap Motion will be able to see my hands reliably enough for this to make a difference.

Other things to consider

ContactGlove2 appears to be a workable solution, being haptic gloves with actual driver support, and which keep the fingertips open (unlike Bifrost Pulse) and have lightweight physical thumb clusters which can be properly moved out of the way (unlike FlipVR). However, they’re fucking expensive, at $600 before tariffs and shipping (which means probably closer to $1000 all-in right now).

EOZ has the Immersive VR Gloves which are much more reasonably-priced, although they’re currently very backordered and in “incubator” mode (which is to say, early alpha, same as Bifrost Pulse), and it seems like the gloves themselves might interfere with guitar playing. It also requires a separate button module for the thumb clusters and there’s no information about where to get those or how much they cost. Fortunately the EOZ Discord is a font of information about this stuff and I’m sure someone there could answer my questions.

There’s also the possibility of downgrading my own experience and using the Quest 2 and its optical hand tracking when I perform. This would actually be a slightly better headset fit for me (rather than my custom facial interface on the BSB, which is fairly uncomfortable to actually use), but it’s a more complicated setup, especially where audio is concerned (although TBH that would probably be better than my current audio setup for a bunch of reasons), and the optical hand tracking isn’t super great anyway.

I could also see if I can get something to work with my old Kinect and Amethyst, but that’s built around fullbody and doesn’t seem to have any sort of hand overrides.

The thing you’re probably screaming at your monitor for me to try

Okay so you’re probably wondering why I don’t just attach some Vive/Tundra Trackers to my hands and use it as added fullbody tracking points.

I actually did try that at one point. Unfortunately, VRChat’s FBT doesn’t work that way; it will only track hand locations from actual SteamVR controllers (tracking points don’t count), and what’s even more fun is if there’s no controllers detected, it just disables all tracking for the arms entirely.

There’s an open feature request but it hasn’t garnered any attention from the VRChat team.

Really, this would be the ideal solution for a whole bunch of reasons. It’s affordable and simple, and would also theoretically work with any FBT tracking implementation (such as SlimeVR, HaritoraX, april tags, mocopi, etc.). But it requires specific support from VRChat at present.

Theoretically it should be possible to trick VRChat into thinking an FBT tracker is a fully-fledged SteamVR controller and then use some alternate input method to actually control the player, but that feels super sketchy and would almost certainly cause problems with normal controllers, and I haven’t found anyone actually manage to get that approach working, anyway.

The pragmatic solution: just don’t care

People who are watching me in-world never notice my hands not moving, and it’s not like they actually properly line up on the guitar anyway.

I want my recordings to look nicer, but I’m not sure how much nicer they can be anyway.

Conclusion

Gosh this hobby is expensive.