Caching Multiple Objects As One Mesh

Introduction

One of the strongest features of the XMesh geometry caching system is its ability to save arbitrary numbers of animated objects as a single mesh.

Most 3D DCC applications like Autodesk Maya and 3ds Max process individual high-resolution meshes faster than large numbers of low-resolution objects. This is due to both the internal overhead of processing the hierarchy of components (transforms, shapes etc.) before the final mesh can be displayed or rendered, and because graphics cards and renderers are better optimized to handle a single heavy mesh.

Originally, XMesh was developed primarity to help in situations where tens or hundreds of thousands of animated objects with changing topology have to be displayed and rendered. Both the viewport playback and the render performance were important to consider, and XMesh was designed to be positioned as the link between the group of Modeling/Animation/Simulation/Effects departments and the Lighting & Rendering department. In other words, what was cached to XMesh was usually meant to be the input of the renderer with only lighting, shading, timing and transformation changes applied to the XMesh Loader.

This concept is different from the design of alternative solutions like Alembic where the whole hierarchy is cached and restored. In other words, if you save 1000 animated objects to an Alembic cache, you still need 1000 Alembic loaders to restore the scene. This has significant implications on the performance characteristics of these solutions.

In the following example, we will see how XMesh can help with consolidating large numbers of animated objects into a single external cache, and how it compares in terms of playback and file storage to Maya itself and to Maya’s Alembic cache.

The Basic Scene

For this example, we will use a simple MEL script to create an array of animated pCube primitives. A pCube has by default only 6 polygons / 12 triangles and 8 vertices. We will place the pCubes side by side and animate their height randomly over the course of 100 frames to produce unique motion in each object.

seed 123;
string $cubes[]={};
currentTime 1 ;
for ($z = 0; $z < 40; $z++)
{
   for ($x = 0; $x < 25; $x++)
   {
        string $obj[] = `polyCube -ch on -o on -w 2 -h 5 -d 2 -cuv 4` ;
        move -r ($x*2) 0 ($z*2);
        setKeyframe { ($obj[1]+".h") };
        setAttr ($obj[1]+".height") (`rand 1 10`);
        autoKeyframe;
        $cubes[`size $cubes`] = $obj[1];
   }
}
currentTime 100 ;
for ($aCube in $cubes)
{
    setKeyframe { ($aCube+".h") };
    setAttr ($aCube+".height") (`rand 1 10`);
    autoKeyframe;
}

Executing this script will produce 1,000 animated pCubes. Changing the $x or $z loop limits would let us produce more or less objects to test the scalability of the caching solutions.

Caching 1000 Objects

In our first test, lets work with only 1,000 pCubes. This amounts to 8,000 vertices and 12,000 tris which is really a very low-resolution scene.

Native Playback Performance

• Run the script above in an empty Maya scene.
• Set the Playback End time to 100 to be able to play back from 1 to 100 and see the whole animation.
• Change the viewport shading to Smooth Shade All.
• Zoom out to see all objects in the viewport.
• Save the scene as an MB Maya Binary file for future reference.
• Use a stop watch and play back the 100 frames - on the i7 machine this tutorial was developed with, the playback time was around 11 seconds.

Caching To XMesh

• Stop the playback.
• Press CTRL+SHIFT+A to select all objects.
• Click the XMesh Saver icon on the XMesh shelf.
• Make sure the “green” mode “Save ALL Objects As ONE MESH in WORLD SPACE Coordinates” is selected.
• Enter “Boxes1K” for the name of the sequence.
• Enter “v001” for the Revision.
• Press the Playback Range button to set the saving interval to the scene range.
• Expand the Channels To Save panel and uncheck “Velocity” (we don’t need to cache that channel because the topology is not changing and the Velocity could be extrapolated from neighboring frames at loading time if needed).
• Prepare a stop watch and click the SAVE XMESH SEQUENCE TO DISK button while starting the stop watch.
• On the same i7 system, the saving to XMesh took around 26 seconds.
• Looking at the Boxes1kv001folder creared in the Base Path, the total size of all files in the folder is 2,794,557 bytes.

Caching To Alembic

For comparison, let’s save the same objects to an Alembic cache file.

• Go to the Pipeline Cache > Alembic Cache > Export All To Alembic…
• Set the Cache time range option to Time Slider
• Enter a file name like “Boxes_1k_v001”
• Prepare the stop watch and click the Export All button while starting the stop watch.
• The saving time on the same i7 machine was 98 seconds.
• The resulting ABC file should be around 73,059,059 bytes.

Playing Back With An XMesh Loader

Let’s use the same scene to see how our caches are comparing to the playback time of the native Maya primitives.

• Press CTRL+SHIFT+A to select all objects
• Press the DEL key to delete them all.
• Click on the XMesh Loader icon on the XMesh shelf.
• Navigate to the location of the XMesh cache and pick the .MEL file generated by the saver - this is a MEL Script that will create the XMesh Loader. Alternatively, you can click any of the .XMESH files and the result will be the same.
• Play back the resulting scene and stop the time.

On the same system, the playback time for the 100 frames was 8.8 seconds. This is slightly faster than the native scene playback of 11 seconds.

Playing Back From Alembic Cache

• Select the XMesh Loader and delete it.
• Go to Pipeline Cache > Alembic Cache > Import Alembic… and select the ABC file saved previously.
• Once the 1,000 objects are recreated, play back and stop the time.

On the same system, the playback time for the 100 frames was 46 seconds which is over 4 times slower than the native Maya playback and over 5 times slower than XMesh.

Caching More Objects To Test Scalability

• Using the same steps as above but with different $x and $z variables, we can benchmark Maya, XMesh and Alembic using 2,000, 3,000 and more objects.
  • The expectated outcome is that doubling the number of objects will also double most of the results from our previous test.
  • The reality might be slightly more… strange.

Saving Times in Seconds

Count	XMesh	XMesh Factor	ABC	ABC Factor	ABC/XMesh
1,000	25.89	1.0	97.9	1.0	3.781x
2,000	46.31	1.788x	247	2.522x	5.333x
3,000	67.00	2.588x	426	4.351x	6.358x
10,000	204.00	7.879x	4080	41.675x	20.000x

As you can see, saving twice as many objects in XMesh takes less than 2x more time, and the same is true for three times more objects - instead of a factor of 3 when compared to 1000 objects, we get only 2.588x. In the case of Alembic though, saving two times more objects takes about 2.5x more time, and saving three times more objects takes over 4 times longer.

As result, the more objects you save, the faster XMesh becomes in comparison. With 1000 objects it is already nearly 4 times faster than Alembic, with 2,000 objects it is over 5 times faster, with 3,000 objects it is already over 6 times faster, and this continues on as we increase the number of objects to be saved.

File Sizes in Bytes

Count	Maya	Maya Factor	XMesh	XMesh Factor	ABC	ABC Scaling	ABC/XMesh
1,000	1,722,276	1.0	2,794,557	1.0	73,059,059	1.0	26.143x
2,000	3,442,536	1.998x	5,420,366	1.939x	145,853,985	1.996x	26.908x
3,000	5,130,536	2.978x	8,070,697	2.888x	218,577,317	2.991x	27.082x
10,000	16,619,272	9.649x	26,372,624	9.437x	721,588,041	9.876x	27.361x

Here we have a comparison of the Maya .MB file size, the XMesh folder size and the Alembic ABC file size.

Note that the Maya MB file stores only the procedural definition of the geometry and animation (pCube primitives and Height parameter animation keyframes), while both XMesh and Alembic save explicit samples of the whole mesh on 100 frames.

As you can see, the XMesh cache is less than two times the size of the Maya procedural description of the animated objects! When compared to Alembic, XMesh is consistently about 26 to 27 times smaller. This is partly due to the lack of binary compression in Alembic, and partly due to the storage mechanism used.

An interesting experiment is the compressing of the caches using ZIP. Compressing the 1,000 objects Alembic file using WinRAR to .ZIP archive produces a file with size of 8,757,828 bytes. This is still larger than the XMesh cache of 3,000 objects (8,070,697 bytes).

Playback Times in Seconds

Count	Maya	Maya Factor	XMesh	XMesh Factor	ABC	ABC Scaling	ABC/XMesh	Maya/XMesh
1,000	10.83	1.0	8.8	1.0	45.9	1.0	5.209x	1.229x
2,000	20.77	1.917x	10.03	1.138x	91.8	2.001x	9.156x	2.070x
3,000	30.46	2.812x	11.19	1.270x	146.0	3.180x	13.04x	2.722x
10,000	106.00	9.787x	19.00	2.156x	711.00	15.49x	37.421x	5.578x

When playing back the animation in the viewports, Maya scales more or less linearly - if 1,000 objects play back in about 10 seconds, then 2,000 and 3,000 objects play back in 20 and 30 seconds respectively. This is very similar in the case of Alembic, except that it is around 4.5 times slower than the native Maya scene.

On the other hand, XMesh produces almost consistent results with very little overhead related only to the number of polygons to be redrawn. After all, in all cases it is a single XMesh Loader being played back, so there is no linear dependency on the number of scene objects. As you can see, each 1000 objects seem to add only about 1.2 seconds to the previous playback time!

In general, XMesh is already an order of magnitude faster than Alembic with 2,000 objects, and the difference becomes even larger as the object number increases. As we reach 10,000 objects, XMesh is already over 37 times faster than Alembic, running only about two times slower than with 1,000 objects. The speedup over the native Maya scene is also over 5.5 times, which makes it a great option for improving playback of scenes with large number of moving objects that don’t need to be editable individually. Given that Alembic with 10,000 objects is nearly 7 times slower than the original Maya scene that was cached, it is clearly the wrong choice for handling large numbers of individual objects through a single cache.

The following video demonstrates the viewport playback speed of XMesh, Alembic and the Maya source scene with 3,000 animated pCubes. It uses a slightly modified version of the script that produces some wave-like height distribution.

Accelerating The XMesh Viewport Display

The XMesh Loader provides several additional and much faster display modes. Other than the built-in display options of Maya, these XMesh Loader modes do not simply reduce the display after all data has been loaded, but can load only those channels of the cached data that are actually needed. This can result in significant speed up of viewport updates at cost of reduced information.

Faces Display Mode

• The first mode to try is the Faces option.
• This mode loads only two mesh data channels - the Vertex List and the Face List.
• All other channels including Smoothing information, Velocity, Material assignments and Mapping coordinates will not be loaded, resulting in a mesh that is fast, but cannot display most of the shading info.
• Using the 3,000 pCubes example scene, displaying the whole mesh in Faces mode increases the frame rate from 9 FPS to 20 FPS.
• The Faces mode also provides a Percentage value which can be set to anything between 0.0 and 100.0.
  • Setting the Percentage to 50.0 hardly affects the display quality, but increases the frame rate to 25 FPS.
  • Going down with the Percentage can produce up to 30 FPS at cost of visual quality.

Vertices Display Mode

• This mode is even faster since it requires only one channel to be loaded and displayed - the Vertex List.
• The display in the viewport is just a point cloud, but when loading very heavy geometry with millions of vertices, it can be enough to get a good idea what the whole geometry looks like without showing any polygons.
• The frame rate when drawing the 3,000 pCubes cache with 100% Vertices is around and over 135 FPS.
• Reducing the Percentage can be useful in the cases of high-resolution meshes, but doesn’t really help much with our test scene which is actually rather low-res anyway.

Bounding Box Display Mode

• The fastest available mode in the XMesh Loader is the Bounding Box display.
• Other than the regular Bounding Box display of Maya which uses the incoming Mesh to determine the size of the box to be drawn, the XMesh loader does not load ANY binary data from disk!
• The actual Bounding Box values are stored as plain text in the .XMesh header file which is in XML format.
• As result, the frame rate in this mode with ANY XMesh cache is over 400 FPS, regardless of the actual content of the cache.
• Obviously, the amount of information is also reduced severely, but it is often enough to determine the location and overal size of the mesh to be rendered.

Note that these modes apply to the viewports only and do not affect the rendering in any way. In other words, you can display Vertices at 1% or Bounding Box, and still render the complete mesh in the final rendered image.

The following video shows the XMesh Loader playback in the three display modes with a different wave animation of 3000 pCubes for visual reference: