On the 19th of December, we were joined by Nathan Saucier from the Leiden Learning & Innovation Centre to dive into Gaussian Splatting – a new technology which has significantly lowered the bar when it comes to creating realistic 3D environments able to be physically navigated by visitors. In his highly detailed and insightful presentation, Nathan clarified the ins and outs of Gaussian Splatting, how it works, and its practical applications within a research setting.
What is Gaussian Splatting?
Previously, if one wanted to allow people to visit a location without entering the physical space, a common solution was 360° video. A major drawback of this method is, however, that the viewer remains rooted in one spot – precisely that of the camera while shooting the video. While this issue could be said to be somewhat solved by existing VR applications, the use of VR introduces a new limitation, namely a lack of realism.
Gaussian Splatting addresses both these issues, creating detailed 3D environments modelled after real locations in which visitors can move around. In addition, collecting the required footage is not very time-consuming. Unfortunately, this footage takes considerably longer to render than with that captured by photogrammetry, yet Nathan believes this to be far from a stumbling block on the whole.
Providing a more technical explanation, Nathan explains that Gaussian Splatting is a radiance field rendering technique. The creation of Radiance fields is, in essence, a method of reconstructing a 3D space using 2D images through deep learning. By providing a specialised program (Nathan uses Postshot) with a large number of images from the location you to be 3D modelled (taken at different heights and at short intervals while walking through the space), the program will be able to determine where in the environment each image belongs and how the images relate to one another. In the deep learning stage, a point cloud with textures (Gaussians) mapped over it will be created. This forms the 3D environment.
Gaussian Splatting for research
Together with Jelle Brands (a researcher and lecturer at the Institute of Criminal Law and Criminology at Leiden University) and the municipality of Leiden, Nathan has been working on a research project focussing upon people’s perceptions of safety in different environments. For this, he used Gaussian Splatting to create the 3D environment of a tunnel for cars and bikes (as seen in the video above) and an alley, both of which are located in Leiden.
The aim is to expose people to these environments, testing how their feelings of safety change under various simulated conditions, including low light and the presence or absence of graffiti and trash. Some obstacles remain, however, as using models created through Gaussian Splatting in VR is not yet possible with all desired possibilities. While we can explore the space, we cannot yet interact with it, or have the model behave like a real-life space.
What’s next?
While Gaussian Splatting has already vastly improved the quality of 3D environments and made the process less labour-intensive, room for improvement remains. Aside from the expansion of its capabilities in VR, Nathan notes that there seems to be a limit to how good the splats can get. After a certain point, providing the program with more images or time does not significantly improve the outcome.
Nevertheless, Gaussian Splatting is a relatively new tool and, taking the large increase in research papers on radiance fields as an indicator, we’re likely to see improvements to the technology soon.
Eager to get started with Gaussian Splatting yourself? Have a look at the YouTuber Olli Huttunen and radiancefields.com, as recommended by Nathan!
