[WIP] NPR Shading in Houdini using Gaussian Splatting

Long-time back I wrote an article about how stylized rendering techniques were using the famous Spider-man into the spiderverse movie:

[embed]Spider-man Into the Spider-verse — a balance between art and technology You have watched the new Spider-man animated movie, you are still at awe at the beautiful creation. You want to try to…medium.com

This was back in 2023, a lot of things have changed since then —

1. I have gotten a lot more used to a software named Houdini
2. Gaussian Splatting has come across as a new way to render scenes in real-time
3. My general knowledge about graphics and shaders have come a long way
4. New features like COPs(Copernicus) in Houdini enables a lot more cool looks and shaders

Today I want to see if I can create a stylized version of a traditional 3D scene in Houdini using some of the above techniques

Step 0: Input Scene Pixar’s Kitchen Set

[embed]Downloads and Videos - Universal Scene Description 26.05 documentation Getting the Expression for a Relationship-Mode Collectionopenusd.org

ref: https://kaolin.readthedocs.io/en/v0.1/notes/usd_tutorial.html

Step 1: Convert Scene into Gaussian

This is the cool part, with the introduction of gaussians splatting. It is a really cool way to represent and render a scene. It is more of a particle field way of representation rather than triangles, so traditional techniques like rasterization or ray tracing don’t really translate directly to them.

There is a well know paper from 1996: Painterly Rendering for Animation

[embed]Walt Disney Animation Studios - Painterly Rendering For Animation The official site of Walt Disney Animation Studios.disneyanimation.com

The idea is simple, you sample a bunch of points across the scene and then draw strokes which combine color and contour of the shape to represent a brush stroke. We use the same to represent each point has a gaussian and the contour and color represent the shape and size of the gaussian. Which already gives a very artistic look to the scene.

Renderer natively using Houdini VK viewport

All of this possible natively in Houdini, thanks to a Beta feature “Bake GSplat” SOP node, that converts our gaussian representations into Houdini’s GSplat Attribute that is now natively rendered in Houdini VK viewport render

[embed]Bake GSplat Houdini 21.0 Nodes Geometry nodes Converts to Houdini's GSplat Attributes. This feature is still under development. The…www.sidefx.com

Step 3: Adding Stylized Rendering using COPs

We are trying to achieve multiple effects:

1. Outline
2. Half-tonning
3. Line Hatching
4. Mis-registration

To get that distinctive spider-man look!

3.1 Outline:

Analytical Scene Outline vs Gaussian Scene Outline

Inverted Outline to get black outlines | COPs Network

Final Results with Outline on Gaussians:

3.2 The rest of the effects to be continued…

TBD

References and Tools Used

• • hou — Houdini’s Python scene API; I scripted the stage/splat/comp builds and read parm values back from the GUI’s own callbacks, which is how I caught the empty-AOV bug (render vars authored with N instead of ray:hitN).
• hython — Houdini’s headless Python interpreter; I ran every build script through it and fired safe in-memory verify renders (drop res, render, never save the hip).
• husk — the standalone out-of-process USD renderer; it ran my background finals and taught me the hard boundary that husk can’t see a live COP, plus exposed the slap comp polarity flip (black ink in viewport, white under husk).
• hoiiotool — the bundled OpenImageIO CLI; I pulled per-channel min/max/mean stats and image diffs to prove what renders actually contained, the “trust numbers not thumbnails” tool.
• APEX — the general graph compiler behind blocktogeo/husk; it compiles your COP block into an executable program for husk, and it’s not just character rigging despite the docs living there.