Some more things about setting up VRChat avatars.

## Matrix Third Room

Back when the VRChat debacle took place, I’d rambled quite a lot about the idea of a standards-based truly-distributed VR system. I probably mentioned how back in the day I was designing something like that, and was intending to use XMPP as the actual transport mechanism; in my more recent reevaluation of the idea I was thinking that using WebRTC to stream the actual realtime data (voice, character animation, etc.) would be the way forward.

Well, the Matrix folks just announced Third Room, which is the same basic concept, running on Matrix and WebRTC! They’ve also made a bunch of other great technology choices along the way.

This is pretty exciting and I’m looking forward to seeing where it goes. At least from a 20,000-foot view it hits all of the right notes for me. For example, using glTF for all object interchange, focusing on in-world editing, and allowing for (apparently optional) world persistence.

Hopefully this can disrupt the stranglehold VRChat has on social VR and will also be a fun, compelling experience in its own right.

## Plaid!

I reached another big milestone on my avatar!

Then I used it in an area with a low-resolution mirror and noticed some really bad texture seams around the edges of the pigmentation map. And I’m wondering if the pigmentation map approach is really all that useful for the avatar in realtime.

## Surface texturing idea

Right now I’ve been trying to make some more of the pigmentation maps for the critter avatar, and in particular I want to make one for the splotchy coloration (which would also be possibly useful for calico and blue giraffe). But hand-drawing this on a disjoint surface is pretty obnoxious.

## VRChat avatar, referred colormaps edition

Today I put my literal shower thought into action, and redid the pigment mapping stuff to use a color lookup table instead. And hoo boy, were there a lot of pitfalls; namely:

• Unity defaults to loading color textures as sRGB, and does so by linearizing the color space
• The default wrapped texture lookup causes all sorts of fun boundary conditions to happen
• Default texture compression completely hecks up texture values which rely on precise lookups
• Oh right, interpolation can’t solve precision issues

I don’t actually need the pigment map to look good in untrusted mode: I can have the material itself provide a fallback texture! There’s nothing stopping me from putting a _MainTex slot on the shader which just does nothing on the shader, but which would be used by the Standard fallback; I can use other texture slots for the pigment map and palette. So obvious.

Also, when laying out the palette, it would be helpful to group related things next to each other, which makes mipmapping play more nicely (and also makes authoring eaiser, although still not as easy as just fiddling with sliders in the Unity editor, unfortunately).

## Come to think of it…

It would probably make a lot more sense to just do all of the color schemes as indirect lookups on a “swatch” texture. The math is a lot simpler, it would allow other things like baking gradients in easily, and would also give a lot more flexibility when it comes to various interactions; for example, I’d be able to encode arbitrarily many skin colors into things, which would allow for argyle and such as well. Plus it would give me even more color slots to work with, so I could also have separate colors for the lips, tongue, nose, and inside of the mouth (which are currently stuck using the same color).

Also, doing that would only require two of the channels in the pigment map, which would free up the other two channels for the occlusion and subsurface maps, which would make everything way more efficient overall.

Yeah, I like this idea.

So, today I had a great idea for making a custom shader for my VRChat avatar which would make it much easier for me to generate color schemes for my avatar. I called the technique a “Pigment Map,” for lack of a better term; basically I used the four channels of a texture as fuzzy “bits” in a palette lookup, and set things up cleverly so that I could modulate between a bunch of different colors based on surface pigmentation and indirect lookup thereof. The idea is that I wouldn’t need to bake out a bunch of textures for, say, red-and-black plaid, purple plaid, green plaid, etc., and could just have mappings for high-level colorations like plaid, stripes, splotches, and so on. Y'know, as one does.

## Some VRChat avatar creaton tips, so far

I’ve been working on a critter avatar for VRChat, as mentioned previously. It’s been an interesting process, with a bunch of elation mixed with a whole lot of frustration.

Someday I’ll write up a collected thing with a bunch of what I’ve learned, but here’s some salient things before I forget.

## Critter avatar progress

I hadn’t made much progress on my avatar, as mentioned previously, because I wasn’t feeling all that up on building stuff for VRChat for a number of reasons. But I’ve finally gotten the urge to start working on it again, and I’ve made quite a bit of progress.

The main thing is over the past day or so I’ve set up all of the visemes and some useful configuration shapekeys1 (namely being able to adjust the torso configuration), and have also looked at some plugins that will supposedly make my life easier. In particular, my friend Lagos pointed me to Cat’s Blender Plugin which seems to provide some very nice improvements to the VRChat avatar creation workflow.

I’m at a point where I think the next steps are rigging and UVing, and those require destructively applying my mesh modifiers, which is a point of no return. I think the Cat’s Plugin workflow is set up such that instead of doing that to my base model, I’m supposed to export it as a .fbx and then reimport it into a new project (or as a new object in the same project, I suppose) and then do the next steps from there. Which makes some amount of sense.

## Lack of VRChat build progress

Recently a couple of folks have asked me if I’ve made any progress on my VRChat avatar, and no, I haven’t. There’s a bunch of reasons for that, but a big one is that I just don’t really feel like the amount of time/energy it’d take is worth it in the end.

In particular, VRChat’s avatar system requires a lot of fiddly work to go into it, and a lot of very hyper-domain-specific knowledge, because a lot of things in VRC are very shoddy hacks on top of the “expression” system which was clearly built for entirely different purposes than what it’s being used for.

The more I work on modifying the Winterpaw mouse for my needs, the more I learn about just how… weird VRC is, and given that the future of the platform itself is easy to doubt, I’d rather focus on building stuff that’s a bit more transferable between systems. So for now I’m just modifying the mouse.

## VRChat continued

It seems like I chose a very… interesting time to get into VRChat, and also I’m taking a break from it for reasons not related to the current debacle.

I also have a lot of thoughts about VRChat as a platform, and where we can go from here.

## VRChat

For the last few days I’ve been spending quite a lot of time in VRChat. I’ve also been making stuff: