100.

FIG. 6 presents a process 600 for generating a splat representation at a lower LoD in accordance with some embodiments presented herein. Process 600 is implemented by splat generation system 100.

Splat generation system 100 includes one or more devices or machines with processor, memory, storage, network, and/or other hardware resources that are configured to generate different LoDs encodings of a splat representation in less time and/or using fewer resources than traditional iterative splat training techniques. Splat generation system 100 may be part of a 3D graphics system or 3D rendering engine. For instance, splat generation system 100 may be integrated in a 3D gaming device, a 3D streaming platform, and/or other provider of spatial computing experiences.

Process 600 includes receiving (at 602) a splat representation of a 3D asset or 3D content. In some embodiments, the 3D asset or 3D content may be originally defined with primitives of another 3D format. For instance, the original 3D asset or 3D content may be originally encoded with meshes for a mesh encoding, points for a point cloud encoding, or other primitives of a 3D format other than a 3D Gaussian splat format. In some such embodiments, the splat representation is generated from converting the primitives of the 3D asset or 3D content to the splats of the splat representation. In some other embodiments, the 3D asset or 3D content is a physical object or scene that is imaged or captured from different viewpoints, angles, or perspectives. The captured images of the 3D asset or 3D content may be used to generate the splat representation. For instance, the received (at 602) splat representation may be generated based on a traditional iterative splat training technique that receives the captured images as inputs and that performs different splat generation iterations until the generated set of splats reconstruct the structure and visual detail in the images with a threshold degree of accuracy. In any case, the splat representation is defined by a distributed set of semi-transparent ellipsoids or splats that are distributed across a 3D space and layered over one another to reconstruct the structure and detail of the 3D asset or 3D content. The splats may correspond to Gaussians that are defined with a position or coordinate in the 3D space, a covariance matrix that defines the shape and orientation of the splat, spherical harmonics that define colors and/or other visual characteristics of the splat, and an opacity or alpha value for the splat transparency. The splats may be defined with other parameters or attributes.

Process 600 includes partitioning (at 604) the 3D space of the splat representation into volumetric subregions or voxels. The partitioning (at 604) may include dividing the 3D space using one or more volumetric grid structures. In some embodiments, the partitioning (at 604) may include subdividing the voxels produced from a first partitioning into smaller and smaller sets of voxels for subsequent partitionings. In some such embodiments, each voxel may be subdivided into a set of two or more smaller sized voxels and the subdividing may be repeated on the two or more smaller sized voxels until the voxels reach a minimum size or contain a threshold number of splats. The different sized voxels associated with a different partitioning (at 604) may be used to generate the splat representation at different LoDs.

Process 600 includes defining (at 606) a tree-based representation of the splat representation based on the partitioning (at 604) of the 3D space. The nodes at each level of the tree-based representation may correspond to the voxels of a particular size generated from a particular partitioning (at 604) of the 3D space. More specifically, the nodes at each lower level of the tree-based representation may correspond to smaller and smaller sized voxels that are generated from a further subdividing of the voxels represented at the next higher level of the tree-based representation.

Process 600 includes determining (at 608) the centroid or central moment for the splats in each voxel. The centroid or central moment for a given voxel may be determined (at 608) from analyzing the positions of the splats in that voxel. In some embodiments, determining (at 608) the centroid or central moment may include defining a central moment matrix based on the splat positions and/or weighting assigned to the splats in each voxel. The weights may be assigned based on values of the non-positional splat parameters (e.g., opacity, splat area, etc.) and/or determined based on the visual contribution of each splat to the voxel.

Process 600 includes performing (at 610) an eigen-decomposition of each voxel's central moment. The eigen-decomposition solves the central moment matrix and/or converts the raw distribution of splats in a voxel to a geometric summary that includes eigenvectors defining the orientation of splats in the voxel and eigenvalues defining the spread of the splats in the voxel.

Process 600 includes defining (at 612) a substitute splat for each particular voxel based on the eigenvectors and eigenvalues derived from the eigen-decomposition of the particular voxel's central moment matrix. The substitute splat defined (at 612) for a particular voxel represents and replaces the subset of splats from the splat representation that are positioned in the particular voxel. Defining (at 612) the substitute splat includes defining the center position of the substitute splat based on the computed centroid for the splats in the voxel, defining the orientation of the substitute splat to match the direction or orientation of the splats in the voxel based on the eigenvectors, and defining the shape and scale of the substitute splat to match the overall shape formed by the splats of the voxel based on the eigenvalues. In this manner, the substitute splat represents the structure formed by the splats in the voxel accurately and with less loss than defining the position, orientation, and shape of the substitute splat by merging or averaging neighboring splats in order to generate the substitute splat.

Process 600 includes defining (at 614) the visual characteristics for each substitute splat based on the visual characteristics of the splats that are replaced by that substitute splat in a given voxel. In some embodiments, splat generation system 100 defines (at 614) the visual characteristics for a substitute splat of a particular voxel by averaging or a weighted averaging of the visual characteristics of the splats in that particular voxel. For instance, splat generation system 100 may define (at 614) the color values or spherical harmonics for the substitute splat by averaging or calculating a weighted average of the color values or spherical harmonics of the splats in the particular voxel. The weighted average may include assigning a higher weight to or calculating a larger color contribution from splats that occupy a greater area or volume within the particular voxel and/or splats that have greater opacity and less transparency. Splat generation system 100 may define (at 614) the opacity of the substitute splat by determining the cumulative opacity of the splats in the particular voxel. For instance, splat generation system 100 calculates the cumulative average opacity of different splats that are layered over another at different parts of the particular voxel and defines (at 614) the opacity of the substitute splat based on the cumulative average opacity. Other techniques may be used to define (at 614) the substitute splat visual characteristics. For instance, the substitute splat opacity may be defined based on the mean or weighted mean of the replaced splat opacities. Moreover, splat generation system