Particle Saving Workflows

Overview

• The saving of particle data to file sequences was originally necessary because the early stand-alone version of Krakatoa did not have any other connection to 3ds Max.
• When Krakatoa was implemented as an integrated 3ds Max Renderer plugin, the ability to save particles to disk was left as a special render mode to ensure compatibility with the stand-alone version.
• In early beta versions prior to Krakatoa for 3ds Max 1.0.0, Krakatoa could load only a single particle sequence from disk.
• When the PRT Loader object was introduced to the early beta cycle, it proved to be an invaluable tool for reusing, retiming, combining, reducing, deforming, transforming and culling particle assets saved to disk.
• Also, it provided an accelerated workflow when dealing with typically history-dependent data from systems like Particle Flow where getting the particles from frame 100 would require pre-calculating 99 more frames.
• This turned Krakatoa from a simple point-based volumetric renderer into a more complex system with the ability to cache, manage and manipulate particle data.
• Thus, the ability of Krakatoa to save particles from any source to disk, load them again, reprocess and render them or resave them to disk opens a whole new world of possibilities to reuse and modify particle data.

Differences Between Saving and Rendering Particles

• The Particle Saving process is slightly different from the Particle Rendering process:
  • Other than in Rendering mode, particles don’t have to be loaded into memory (there is an exception to this rule - see further below) but are written to the particle file directly and discarded from memory.
    • Since Krakatoa v1.6.0 this happens in groups of 50,000 particles to allow multiple threads to perform material/map evaluation, deformations and Krakatoa Channels Modifiers processing on all CPUs/cores available.
    • In previous versions, Krakatoa would process, save and discard one particle at a time which resulted in slower performance with materials and KCMs.
  • Shading controls that do not modify the content of the Particle Data Channels but operate at render time including the Phase Functions (Shaders), Lighting and Final Pass Density values, Motion Blur, Depth Of Field, Environment Reflections etc. will not affect the content of the particle file.
  • As mentioned, there is an exception to the rule - when the option to “Compute Lighting And Copy Lighting Channel Into Emission Channel When Saving” found in the Save Particles rollout is checked, the saving process will behave like the Lighting process in that it will load all particles into memory, sort them relatively to each light and will use the Lighting Pass Density (or Final Pass Density, if used for both passes) to determine the attenuation.

Saving 3ds Max Particles

The following is an overview of the particle saving process when using 3ds Max particle systems as the sources.

Switching To Save Mode

• The saving of particles is currently implemented as a special render mode called “Save Particles To File Sequence”.
• The mode is selected from a drop-down list found to the left of the large RENDER button in the Main Controls rollout of the Krakatoa User Interface.
• When Krakatoa is switched to this mode, it stops producing image output and switches its output to particle files instead.
• The RENDER button will change its label to SAVE PARTICLES and the group of controls around it will be put into a red border to denote the fact that Krakatoa will be saving particle files instead of generating images.

Specifying Particle Sources

• Just like with rendering particles, the Save Particles To File Sequence mode of Krakatoa respects the Particle Source options found to the left of the SAVE PARTICLES button in the Main Controls tollout.
• These options can be used to filter the objects by general classes - see the Particle Sources topic for details.
• Hidden particle source objects will be ignored and not saved, so you can hide individual objects or layers of objects to prevent them from saving.
• Non-renderable objects will also not save (since the saving is considered a form of rendering)
• Particle sources that are disabled at render time like Particle Flows, PRT Loaders etc. will also not generate particles for saving.
• You can use the Presets arrow to the right of the Source options to select pre-defined sets of particle source types, add the current selection as a preset, remove the current selection from the presets list if it is already there and also make the current selection the default for the Save Particles mode.

Switching Source Defaults

• If the “Switch to User Defaults on ‘Particle Render Mode’ change” option is checked, switching between “Render Scene Particles” and “Save Particles To File Sequence” modes will also automatically switch the source particle types.
• Here is how to set up different default sources for Rendering and Saving.
• Let’s say that you save mostly Particle Flow particles but typically render PRT Loaders.
• Switch Krakatoa to “Render Scene Particles” mode.
• Uncheck all options and check the >PRT Loaders option.
• Click the [>>] Presets button and select Set [PRT] as Default for [Render Scene Particles] mode…
• Switch to “Save Particles To File Sequence” mode.
• Uncheck the >PRT Loaders option and check the >PFlow Geometry and >PFlow Phantom options.
• Click the [>>] Presets button and select Set [PF Geometry, PF Phantom] as Default for [Save Particles To File Sequence] mode…
• Click the [>>] Presets button and check the option Switch to User Defaults on ‘Particle Render Mode’ change.
• Now if you switch between Save Particles To File Sequence and Render Scene Particles modes, the specified source types will be checked automatically - PRT Loaders for rendering, PF Geometry and PF Phantom for saving.
• Of course, you can modify the settings additionally after switching the mode if the suggested defaults are not what you need.

Specifying a File Sequence Name

• The Krakatoa GUI provides a Save Particles rollout which can be used to specify the output file name of the particle file or sequence of files to be saved.
  • This rollout will expand automatically when the option is changed to “Save Particles To File Sequence”.
  • The file name controls let you specify a custom path by entering it in a text field or picking a file using a Save File dialog, but it also provides a set of detailed controls to change various portions of the full path like just the Project name, just the Sequence name, just the version or just the base file name.
• The file type can be also set to Krakatoa’s own PRT, CSV or the RealFlow 4 BIN file format.
  • The file controls provide an extensive set of History icons that let you select a portion or the whole file name based on previous Krakatoa sessions.
  • The right side of the rollout also features buttons to explore the different depth levels of the specified save path.

Specifying the Particle Data Channels To Save

• The Save Particles rollout also provides controls over the channels to be written to the particle file.
  • Krakatoa supports arbitrary channel names, but some names have been defined as Krakatoa-internal data channels to provide renderer-specific information like Position, Velocity, Density, Color etc.
  • Where possible, the channel names of other systems like Particle Flow, RealFlow and FumeFX have been used.
  • In addition, arbitrary user-defined channels can be created using the controls in the rollout, for example if a custom channel is being created by a Krakatoa Channels Modifier and has to be saved to disk.
  • Krakatoa expects every particle file to have at least a Position channel, thus the Position channel is the only one that cannot be removed from the list of channels to save.

Saving a Single File

• Once you have specified the save file name and the particle sources to save, you can right-click the large SAVE PARTICLES button in the Main Controls rollout to open the Render Context Menu and ensure the output time is set to Single Frame.
• Pressing the SAVE PARTICLES button will save the particles to disk.

Saving a File Sequence

• To save a file sequence, all you have to change is the Output Time option in the Right-click Render Context Menu of the SAVE PARTICLES button, or in the Render Setup dialog.
  • Set the time to a range of frames like the Active Time Segment, Range or Frames list and press the SAVE PARTICLES button - one file per frame will be saved to disk.

Saving In Network Mode

• Since the Save Particles To File Sequence mode is a special mode of the renderer, it is fully supported for network rendering using any network manager like Backburner, Deadline etc.

Resaving PRT Loaders

• Particles saved using the above approach or coming from other applications like RealFlow can be loaded back into 3ds Max for rendering using Krakatoa via the PRT Loader object.
• The scene can contain an arbitrary number of PRT Loader objects and the same sequence can be loaded more than once by multiple PRT Loaders.
• The PRT Loader object allows you to
  • Load one frame, one file sequence, or multiple file sequences as a particle cloud
  • Offset the timing of the file sequence
  • Limit the range of frames to be loaded with the option to hold a frame or output blank outside the range
  • Completely retime the particle animation by loading frames specified by a function curve
  • Load a fraction of the particles and animate the percentage of particles to be loaded over time
  • Deform the particles using standard gizmo-based 3ds Max modifiers and Space Warps
  • Modify the content of the particle data channels using Krakatoa Channel Modifiers
  • Transform or animated the transformation of the particles in world space using the Position, Rotation and Scale tracks of the PRT Loader
  • Delete Particles using the Selection Channel including Soft-Selection and the Krakatoa Delete modifier
  • Cull Particles using arbitrary geometry volumes
  • Modify the Color, Emission, Absorption and Density of the particles using standard 3ds Max Materials and Maps
• As you can see, particles loaded from a pre-saved file sequence can be significantly modified when brought back into the 3ds Max scene.
• This opens the possibility to not only render these modified particles, but also re-process already saved particle file sequences to create new particle file sequences.
• By simply switching the Particle Sources option to >PRT Loaders and disabling all other sources, Krakatoa can input particles from file sequences loaded by the PRT Loader objects in the scene, apply all specified modifications to the particle data like retiming, transformations, deformations, materials and culling and output new particle file sequences to be loaded for rendering or yet another reprocessing cycle using other PRT Loaders.
• Note that it is not allowed to overwrite input particle files during output, so the file name specified for output must be new and unique.
• Using the same approach, you can also convert particles file sequences between formats, for example load a BIN sequence from RealFlow using a PRT Loader, then save as PRT file sequence which has better compression and can be set up to include only the channels relevant to Krakatoa.
• Also note that particle Position data is always saved in world space, but loaded in the object space of the PRT Loader.
  • Thus, if you want the positions loaded from a particle file sequence to be resaved in the same space, you should make sure the PRT Loader is either placed at and aligned to the world origin, or that the “Use Node Transform” checkbox in the PRT Loader is unchecked.
  • Otherwise, the transformations (Position, Rotation and Scale) of the PRT Loader will be applied to the particles and will affect the saved data - in some cases this might be desirable, for example if you want to offset, scale or rotate a particle cloud when resaving.

Reprocessing Particle File Sequences Using Particle Flow

Recreating a Saved File Sequence in Particle Flow

• Krakatoa provides several PFlow operators dedicated to the loading of particle file sequences into Particle Flow.
• The latest generation of PFlow operators shipping since Krakatoa v1.5.0 uses a PRT Loader object as the particle data source, exposing all possibilities described in the previous section (retiming, deformations, culling etc.) to Particle Flow particles.
• Particle Flow can be set to create particles based on the Krakatoa PRT Loader Birth operator which points at a PRT Loader object in the scene.
  • The particle file sequence should contain an ID channel for best results - the ID channel typically contains the Birth ID from a Particle Flow system which identifies a unique particle within the sequence.
  • If there is no ID channel, the order of particles in the stream will be used instead, but this might be a problem if the particle count changes over the duration of the sequence
  • When a particle is created by the Krakatoa PRT Loader Birth, its ID is copied into a FileID channel specific to the Krakatoa Operators.
  • The Birth Operator will only create a particle if it appears in the current particle file and a particle with the same ID does not appear in the Particle Flow system yet - this is determined by comparing the ID channel of the file with the FileID channel mentioned above.
• A Krakatoa PRT Loader Update operator pointing at the same PRT Loader as the Birth operator can refresh any of the particle data channels found in the file sequence.
  • The Update operator also looks for particles that exist in the Particle Flow system but are not in the current file anymore - these particles will be marked in the FileID channel with a value of -1 meaning that they are “death row candidates”
• A Krakatoa FileID Test can send out particles based on their FileID channel and by default it sends out particles with a FileID of -1 to another event, typically containing a Delete operator to remove the “death row candidates” from the system and thus replicate the reduction of particle count occurring in the file sequence.

Applying Forces and Collisions To File Sequences

• Since the Krakatoa PRT Loader Update can be set up to update only specific channels on each frame, it is possible to apply forces and even perform collisions with pre-saved particles as long as the Position channel has been unchecked.
  • If the Velocity channel is enabled in the Krakatoa PRT Loader Update and it exists in the file sequence, the particles will move according to the saved velocities and not according to the saved positions.
  • The Krakatoa PRT Birth operator will set the initial position of the particles when creating them, but the Krakatoa PRT Loader Update would not enforce the existing positions found in the file sequence after the birth, allowing for Force and Collision operators to apply data to the Velocity channel of the particles.

Deforming Particle Flow Systems

• Because the Particle Flow operators read the particle data from the top of the PRT Loader’s modifier stack, any modifications applied to the PRT Loader will be reflected by the Particle Flow.
  • This means that you can apply deformations to the PRT Loader and the Particle Flow particles will follow them,
  • You can add Krakatoa Channels Modifiers to the PRT Loader to change the content of any of the particle data channels or even create new ones and the Particle Flow will accept these changes
  • You can cull portions of the particles based on their inside/outside relationship with arbitrary geometry objects’ volumes and the Particle Flow particles will respect that, too
  • For details, see the following demonstration of using the Krakatoa Particle Flow operators.
• The resulting Particle Flow system can then be saved to a new particle file sequence which in turn could be reloaded later for rendering or yet another cycle of reprocessing via the methods described above.