Some more things about setting up VRChat avatars.
Back when the VRChat debacle took place, I’d rambled quite a lot about the idea of a standards-based truly-distributed VR system. I probably mentioned how back in the day I was designing something like that, and was intending to use XMPP as the actual transport mechanism; in my more recent reevaluation of the idea I was thinking that using WebRTC to stream the actual realtime data (voice, character animation, etc.) would be the way forward.
This is pretty exciting and I’m looking forward to seeing where it goes. At least from a 20,000-foot view it hits all of the right notes for me. For example, using glTF for all object interchange, focusing on in-world editing, and allowing for (apparently optional) world persistence.
Hopefully this can disrupt the stranglehold VRChat has on social VR and will also be a fun, compelling experience in its own right.
I reached another big milestone on my avatar!
Then I used it in an area with a low-resolution mirror and noticed some really bad texture seams around the edges of the pigmentation map. And I’m wondering if the pigmentation map approach is really all that useful for the avatar in realtime.
Right now I’ve been trying to make some more of the pigmentation maps for the critter avatar, and in particular I want to make one for the splotchy coloration (which would also be possibly useful for calico and blue giraffe). But hand-drawing this on a disjoint surface is pretty obnoxious.
Today I put my literal shower thought into action, and redid the pigment mapping stuff to use a color lookup table instead. And hoo boy, were there a lot of pitfalls; namely:
- Unity defaults to loading color textures as sRGB, and does so by linearizing the color space
- The default wrapped texture lookup causes all sorts of fun boundary conditions to happen
- Default texture compression completely hecks up texture values which rely on precise lookups
- Oh right, interpolation can’t solve precision issues
I don’t actually need the pigment map to look good in untrusted mode: I can have the material itself provide a fallback texture! There’s nothing stopping me from putting a
_MainTex slot on the shader which just does nothing on the shader, but which would be used by the Standard fallback; I can use other texture slots for the pigment map and palette. So obvious.
Also, when laying out the palette, it would be helpful to group related things next to each other, which makes mipmapping play more nicely (and also makes authoring eaiser, although still not as easy as just fiddling with sliders in the Unity editor, unfortunately).
It would probably make a lot more sense to just do all of the color schemes as indirect lookups on a “swatch” texture. The math is a lot simpler, it would allow other things like baking gradients in easily, and would also give a lot more flexibility when it comes to various interactions; for example, I’d be able to encode arbitrarily many skin colors into things, which would allow for argyle and such as well. Plus it would give me even more color slots to work with, so I could also have separate colors for the lips, tongue, nose, and inside of the mouth (which are currently stuck using the same color).
Also, doing that would only require two of the channels in the pigment map, which would free up the other two channels for the occlusion and subsurface maps, which would make everything way more efficient overall.
Yeah, I like this idea.
So, today I had a great idea for making a custom shader for my VRChat avatar which would make it much easier for me to generate color schemes for my avatar. I called the technique a “Pigment Map,” for lack of a better term; basically I used the four channels of a texture as fuzzy “bits” in a palette lookup, and set things up cleverly so that I could modulate between a bunch of different colors based on surface pigmentation and indirect lookup thereof. The idea is that I wouldn’t need to bake out a bunch of textures for, say, red-and-black plaid, purple plaid, green plaid, etc., and could just have mappings for high-level colorations like plaid, stripes, splotches, and so on. Y'know, as one does.
Someday I’ll write up a collected thing with a bunch of what I’ve learned, but here’s some salient things before I forget.
I hadn’t made much progress on my avatar, as mentioned previously, because I wasn’t feeling all that up on building stuff for VRChat for a number of reasons. But I’ve finally gotten the urge to start working on it again, and I’ve made quite a bit of progress.
The main thing is over the past day or so I’ve set up all of the visemes and some useful configuration shapekeys1 (namely being able to adjust the torso configuration), and have also looked at some plugins that will supposedly make my life easier. In particular, my friend Lagos pointed me to Cat’s Blender Plugin which seems to provide some very nice improvements to the VRChat avatar creation workflow.
I’m at a point where I think the next steps are rigging and UVing, and those require destructively applying my mesh modifiers, which is a point of no return. I think the Cat’s Plugin workflow is set up such that instead of doing that to my base model, I’m supposed to export it as a .fbx and then reimport it into a new project (or as a new object in the same project, I suppose) and then do the next steps from there. Which makes some amount of sense.