Mixing Traditional Geometry with Gaussian Splatting For Creating Photorealistic Environments

What do you do when making something perfectly real requires you to make it imperfect? Don’t worry, we’re not talking about the grand scheme of things — only about creating environments in 3d, as for digital artists, that challenge has long defined how virtual worlds are built.

Until recently, there were two approaches, each with its own troubles: manually building worlds from scratch or scanning reality through photogrammetry.

Manual 3d modeling offers total creative control, but it struggles to capture the natural randomness of the real world. Environments often look too clean and organized, and missing even minor imperfections — like warped floorboards or uneven wear — can give a scene away as an artificial space. Achieving convincing realism this way demands enormous time, labor, and specialized teams, making production slow and expensive.

Fractals work great for creating natural environments, but things get much harder when you try to build cities with detailed buildings.

Photogrammetry, on the other hand, may capture real-world detail effortlessly but also brings serious technical problems. For starters, it’s a major logistical operation when it comes to scanning large-scale environments.

Scanned assets are also often very dense, with millions of polygons that must be painstakingly simplified through retopology before they can run in real-time engines.

Lighting is another major issue: shadows and highlights from the original photos are baked directly into textures, breaking immersion when assets are reused in different lighting conditions. On top of that, photogrammetry struggles with reflective or transparent materials like glass, mirrors, and polished metal, which frequently produce distorted or unusable geometry.

In short, manual modeling fights realism with human precision, while photogrammetry captures realism but delivers overwhelming, inflexible data — leaving artists stuck between artistic control and technical feasibility.

3D Gaussian Splatting (3DGS) has introduced a third path — a volumetric, point-based approach that we talked about in length in another article, so today, we’ll focus on the new hybrid workflow made possible by this innovative technique.

Namely, you can now mix traditional, high-precision geometry with 3D Gaussian Splatting to get the best of both worlds: surgical architectural precision nestled within a photorealistic environment. That’s something we never really had before, so let’s see a bit about how it came to be and what it promises.

3D Gaussian Splats can be integrated into DCC tools and game engines to create realistic, interactive environments. Image and GIF by © 2025, Amazon Web Services.

The Question of Light: Why Is Mixing Traditional Geometry and 3DGS Challenging?

For starters, the traditional geometry relies on Bidirectional Reflectance Distribution Functions (BRDF) and shaders to simulate how light hits a hard surface. It calculates the relationship between three key variables: the incoming light direction, the viewer’s camera angle, and the “normal”, which is the direction the surface is facing.

By processing these, the BRDF determines exactly how much light reaches your eye, defining whether a pixel looks like dull plastic, shiny chrome, or rough stone. In traditional 3d modeling, we use Physically Based Rendering (PBR) to tweak these BRDF parameters (like metalness and roughness) in real-time. This allows the computer to dynamically recalculate how a 3d model looks if you add a light bulb or move the camera.

Now, the new approach using Gaussian Splats represents scenes with a dense cloud of volumetric elements rather than rigid geometry. Each Gaussian encodes where it exists in space, how large it is, what color it contributes, and how it blends with its surrounding elements to produce continuous surfaces and fine detail.

To create such a scene, multiple photographs of an object or environment are captured from different viewpoints. An optimization process then adjusts millions of Gaussians so that, when projected back into each camera view through differentiable rendering, they reproduce the original images as closely as possible.

But the same problem as with photogrammetry remains. Gaussian Splatting “bakes” the reflections into the data rather than calculating them on the go, making the fusion of traditional geometry and 3DGS the current challenge of photorealistic rendering.

Illumination is baked into the Gaussian splats as per-splat radiance captured during reconstruction, with no support for dynamic lighting or relighting. GIF by © 2026 Alpha Pixel

On the bright side, we are already seeing promising research toward Relightable Gaussian Splatting. Emerging efforts are focused on “unbaking” this data by decomposing the captured radiance field into its fundamental components — albedo (base color), normals (surface direction), and material properties.

The working theory is that by estimating these intrinsic properties, artists will be able to strip away the original “baked” sunlight and replace it with dynamic lights within Unreal or Unity. This transition from “static capture” to “dynamic asset” will be the game-changer that will make 3DGS indistinguishable from traditional, high-end geometry in any lighting scenario.

Integration Mechanics: How Geometry and Splats Interact

The primary benefit of the hybrid approach is efficiency. In traditional pipelines, you’d buy a low-poly city kit or spend weeks modeling surrounding blocks. The result? It often looked like a “video game” version of reality — flat, repetitive, and lifeless.

With the “Splat” way, however, you fly a drone around the site for ten minutes. You feed that footage into a 3DGS engine. Within an hour, you have a 3d environment that includes the exact lighting of that day, the grime on the pavement, and the specific way the sun hits the glass of the neighboring building.

Furthermore, when establishing a hybrid pipeline, you don’t need to replace anything — the power of this workflow comes from the combination. You still get to use your high-precision 3ds Max or Revit models, only this time you drop them into the captured environment.

1. Capture & Process: You can use tools like Luma AI, Polycam, or Postshot to create your 3DGS asset (usually a .ply or .splat file representing a full real-world environment) from photos or video.
2. The Bridge Into Real-Time: Bring that Splat into a real-time engine like Unreal Engine 5 or Unity. With plugins such as Luma’s UE5 plugin or Splats v2, these point clouds render efficiently and interactively.
3. The Alignment: Because Splats are generated from real-world photos, they have inherent scale. You can drop your FBX building into the scene and align it to the “Splat” ground plane.
4. Shadow Catching: The biggest challenge is making the building feel “grounded.” You do this by placing “invisible” geometry (Shadow Catchers) that matches the Splat’s floor, allowing your building to cast realistic shadows onto the photorealistic ground.

And there you have it: a building that belongs in a scene that is very photorealistic and low effort. However, certain limitations still remain. When using Gaussian splats, you may lose some of the fine details, not to mention you’ll have much less artistic control compared to working with traditional geometry.

Also, you’ll need to invest in some storage as these 3DGS files are quite big, and rendering large scenes will require significant memory resources.

This render shows a 3D building model placed within a real location represented as a Gaussian Splat. Image by © 2026 Chaos Software

Managing All That

As we mentioned, Gaussian Splatting is powerful, but it’s a data nightmare. These files are notoriously heavy, unstructured, and difficult to preview. Unlike a standard FBX, you can’t just “look” at a .ply file in a standard Windows folder.

This is where the need for a Digital Asset Management solution like Connecter Suite becomes quite apparent. Since you’re now mixing traditional geometry with massive Splat files, your library can quickly become heavy and messy, with numerous scattered asset versions and duplicate files.

• Visual Organization: Connecter allows you to organize your high-poly FBX models alongside your 3DGS captures. You can create and assign Custom Previews for your Splats so you don’t have to load a 2GB file just to see which city block it is.
• Version Control: Splatting is iterative. Connecter helps you manage the different versions of your site captures, ensuring the whole team is using the “cleaned” version of the scan.
• Metadata & Tagging: You can tag assets by location, weather conditions, or scan quality, and make them quick and easy to locate. If you need a “Rainy London Street” Splat to go with your “Modern Office” FBX, you can find each of those files in seconds.

[embed]

With Connecter, we stay at the forefront of innovation — like our support for USD files and the Node Graph Editor for MaterialX, letting you create or edit your .mtlx files directly in Connecter, among many other features.

Exciting new functionalities for Gaussian Splats are coming soon! Subscribe now to be the first to know when we release developments that are truly worth your attention.

If you’re looking for a way to take control of your digital assets, our DAM might be a good fit for your needs. Contact our support team to find out more about its organizational and collaboration features.

Disclaimer: Some of the images in the article are generated by Midjourney.