XMesh File Sequences

Overview

  • An XMesh file sequence uses multiple files per frame, resulting in a large number of files per sequence.
  • This has several positive implications:
    • Unchaging channel data can be stored just once for the whole sequence, reducing disk usage;
    • Multiple machines can save to the same sequence in parallel for faster turnaround;
    • A sequence can be saved in multiple runs, adding more frames to existing ones if needed;
    • Sub-ranges of an existing sequence can be resaved / updated without the need to resave the whole sequence;
    • Corrupted or missing data can be fixed easily by resaving an individual frame without the need to resave the whole sequence;
    • Multiple CPUs can save and compress individual data channels of the same frame in parallel;
    • Multiple CPUs can decompress and load individual data channels of the same frame in parallel

and so on.

The XMesh File Sequence Structure

The XMesh File Sequence consists of two types of files: XMesh Header files, and XMDat binary channel data files.

The .XMesh Header Files

  • The .XMesh file is a human-readable XML file used as the “header” of the sample.
  • One .XMesh header file is written for every sample in the sequence.
  • Every .XMesh file is usually only a couple Kilobytes in size and contains the following information:
    • XMesh Version (currently 1)
    • Vertex Count
    • Face Count
    • Vertex List data file
    • Face List data file
    • Bounding Box min. and max. values
    • Zero or more Vertex Channels like mapping coordinates, normals. velocities etc.
    • Zero or more Face Channels like Smoothing info, Material assignments etc.
    • Source Scene Frame Rate
    • Source Scene Units

.xmdat Binary Channel Data Files

  • For each data channel requested to be cached, a separate binary data file will be stored for each sample, unless the data is identical to a previous sample.
  • In that case, the XMesh header file will reference the existing data file from the earlier sample, and no data will be saved for the current sample to avoid data duplication.
  • In the following example, the header file of frame 0042 contains a reference to the Boxes3K__verts0042.xmdat file describing the animated Vertex List, but all other channels are still referencing frame 0001 because there were no topology changes and thus the Face List, Smoothing, MaterialIDs and Textute Mapping Coordinates are also unchanged during the whole sequence:
../../_images/xmeshmy-fileformat-xml.png
  • Each binary data channel file is compressed using ZLib to further reduce disk space usage.
  • As result, the data footprint of an XMesh file sequence storing all geometry data is often smaller than even the Autodesk PointCache format which stores merely the vertex list in an uncompressed binary file without inter-frame optimization.
  • In most cases, XMesh file sequence is several times to an order of magnitude smaller than an equivalent Alembic .ABC file.

Sampling Step

  • The XMesh file format allows for sub-frame sampling.
  • In other words, multiple XMesh files can be saved between full frames to reproduce the exact motion and rotation of objects or allow for higher-quality retiming.
  • Currently, the number of sub-sample steps is limited to specific values to ensure easier data exchange with 3ds Max which uses Integer Ticks as sub-frame samples and has certain limitations to the actual steps that can be taken.
  • The frame number for full frame samples is padded to 4 digits, for example XMeshTest_0001.xmesh.
  • The frame numbering format for sub-sampling uses a comma character to denote the decimal point, e.g. XMeshTest_0001,5.xmesh represent a half-frame sample between frames XMeshTest_0001.xmesh and XMeshTest_0002.xmesh