Particle Partitioning Workflow

Overview

  • “Partitioning” is the term used in Krakatoa lingo to describe the generation of similar yet slightly different versions of the same particle system to produce a final particle count beyond the capabilities of the source particle system or to speed up the processing of single-threaded particle sources (e.g. Particle Flow).
  • The idea behind the Partitioning functionality of Krakatoa is that the same particle system can produce very similarly looking results of the random number generator used to modify particle data like Position, Velocity, Orientation etc. is initialized using a different Random Seed.
  • The Random Seed (also known as “Uniqueness” parameter in Particle Flow) causes the random number generator to produce the same sequence of pseudo-random numbers.
    • Changing the Random Seed produces a different unique and repeatable sequence of numbers, assuming the CPU is identical (note that mixing CPUs like Intel and AMD ones or even different modes from the same vendor could cause the same Random Seed to produce different results).
    • When the positioning of particles is constrained by Icon, Geometry Surface or Volume etc, the randomness of the particle data is at a very small scale - for example, particles would be placed on different points on the same Surface producing the same “macro” look, while differing from the exact placements produced with another random seed.
    • This means that merging two or more “partitions” together produces a coherent result where the statistical probability for two particles to share the same point in space is very low and almost all particles from one partition end up between the particles of the other partitions.
  • Krakatoa provides built-in tools for incrementing the random seeds of most supported particle sources, as well as a PRT Loader object which can be used to combine the saved partition file sequences into a single particle cloud for viewing and rendering.

Controlling Seed Incrementation

  • The Krakatoa Partitioning is set by default to affect all supported objects and their sub-components (see Supported Systems below), but also provide several levels of control to exclude specific objects or their sub-components.
  • The top level of control is the User Interface section of the Partitioning Rollout named “Increment Random Seeds Of”.
    • This group of controls provides checkboxes for the major types of randomizable parameters in the supported particle sources.
    • By default, all Particle Flow-related options are checked and the options to affect Modifiers of PRT Loader, PRT Volumes and Geometry objects are unchecked.
    • These controls can be used to affect or disable the effect on all objects or sub-components matching these categories.
  • The second level is the object / particle system level.
    • If an object like a PF_Source Emitter, Thinking Particles host object, 3ds Max Legacy Particle System, Geometry object whose vertices will be used as particles etc. contains the substring “noseed” in its name, it will be skipped by the seed-incrementing function of Krakatoa and will not be modified.
    • The case of the name substring or its location within the name does not matter, so objects like “NoSeedBox01”, “Sphere02_NOSEED”, “Circlenoseed03” would all be skipped.
  • The third level of seed incrementation control is at the operator group level (if applicable).
    • This includes the Events of Particle Flow and the DynamicSets of Thinking Particles.
    • In the case of Particle Flow, if an Event contains “noseed” in its name, the operators within that event will not be affected.
    • In the case of Thinking Particles, if a DynamicSet contains “noseed” in its Comments field, it will be skipped (adding “noseed” to the name of the DynamicSet will have no effect!)
  • The next level of seed incrementation controls is at the Operator level (where applicable).
    • This includes supported Particle Flow Operators and Thinking Particle Operators, as well as Geometry, PRT Loader and PRT Volume Modifiers with a random seed (like the Noise modifier)
    • Adding “noseed” to the name of a Particle Flow operator will prevent it from being changed by the Partitioning functions.
    • Adding “noseed” to the name of a Thinking Particles operator will prevent it from being changed by the Partitioning functions.
    • Adding “noseed” to the name of a Noise modifier applied to a PRT Loader, PRT Volume or Geometry object will prevent it from being changed by the Partitioning functions.
  • The last level of seed incrementation controls is at the Sub-Operator level (where applicable).
    • This includes supported sub-operators of the DataOperator and DataIcon operators implemented by Orbaz Particle Flow Tools Box #3.
    • When a sub-operator contains “noseed” as part of its name, the seed property will not be changed.

Supported Systems

  • While the Partitioning feature was initially developed primarily with Particle Flow in mind, it supports some other particle sources to various degree:

Particle Flow

  • As mentioned above, Partitioning works well with Particle Flow and all its operators and extensions that provide Random Seed values.
  • Krakatoa can modify the Random Seed of
    • Position Operators - affects the random distribution of particles on the emitter. This is the primary source of variation for Partitioning.
    • Spawn Operators - affects the “Variation %” and “Divergence” properties which control the number of particles to be spawned and their Velocity and Scale.
    • Speed Operators - affects the Velocity of particles when in “Random 3D” or “Random Horizontal” mode or when “Variation” or “Divergence” are used and are not 0.0.
    • Rotation Operators - affects the random Rotation when set to “Random 3D” or “Random Horizontal” or when “Divergence” is used and is greater than 0.0.
    • FumeFX Operators - affects the Vatiation parameters of the FumeFX Birth operator, thus creating particles in a different pattern.
    • PFlow Other Operators - affects any Particle Flow operators that might have a Uniqueness > Seed property but are not covered by the other categories.

Particle Flow Box #3

  • In addition to the out-of-the-box Particle Flow Operators, Krakatoa’s Partitioning can take into account Orbaz Particle Flow Tools Box #3 DataOperators and their Sub-Operators:
    • Box #3 Geometry Sub-Ops - affects the Seed of Geometry Sub-Ops set to “Random Surface Point” or “Random Volume Point”.
    • Box #3 Random Sub-Ops - affects the Seed and the output result of Random Sub-Ops in all modes.
    • Box #3 Scalar and Vector Sub-Ops - affects the Variation of the Scalar and Vector Sub-Ops.
    • Box #3 Icon Sub-Ops - affects the Seed of Icon Sub-Ops inside DataIcon operators set to “Random Surface Point” or “Random Volume Point”.

Thinking Particles

  • By default, all operators in all DynamicSets will be searched for a seed property and will be affected, unless the seed incrementation was disabled by adding the string “noseed” to the DynamicSet’s comments field or by renaming the whole system to contain “noseed” in its name.
  • Note that MatterWaves operators have a random seed property but are not exposed to MAXScript correctly. To work around this, Krakatoa v1.6.1 introduced support for a workaround workflow: Simply create a Helper Integer node, rename it to “RandomSeed”, wire to the Random Seed input of the MatterWaves operator and the value will be incremented automatically by Krakatoa when partitioning. You can use this method to cause ANY changes inside Thinking Particles, not just in MatterWaves.

3ds Max Legacy Particle Systems

  • Most 3ds Max Legacy Particle Systems (incl. Blizzard, SuperSpray, PArray and PCloud) provide a Seed property to control the random distribution and behavior of particles.
  • Since these systems are limited to only 65K particles, using Partitioning is the only way to produce higher particle counts for effective rendering.
  • The >Legacy Particles option is checked by default in the Partitioning rollout and you can exclude individual particle systems from changing the seed by adding “noseed” to their name.

Geometry Vertices

  • Normally, geometry vertices have fixed positions and no random seed options to produce random distributions.
  • It is possible to partition such objects with the help of a high-frequency Noise modifier as described here.
  • To disable the changing of a random seed of a geometry object, either add “noseed” to its name or add “noseed” to the modifier’s name.

Krakatoa PRT Objects

  • The Krakatoa PRT Loader and PRT Volume objects also support deformation modifiers with a random seed like the Noise modifier (see above).
  • There are dedicated >PRT Loader Modifiers and >PRT Volume Modifiers options in the Partitioning rollout that can be used to disable the changing of the random seed in these objects.
  • You can also add “noseed” to the object name or the modifier name to disable the whole object or specific modifiers from being affected by the random seed change.

Unsupported Systems

  • The following systems are not directly supported by Krakatoa Partitioning due to the lack of a Random Seed parameter.
  • FumeFX Systems - Krakatoa can directly save and render particles from FumeFX simulations, but cannot perform any direct Partitioning on them.
  • Rain and Snow Legacy Systems - the two oldest particle systems in 3ds Max do not provide a random seed and cannot be Partitioned.
  • A workaround for most cases above is to save the particles to PRT files and then use a high-frequency Noise on a PRT Loader to randomize the particles.

Step-By-Step Partitioning Workflow

Here are the steps to perform to partition supported particle systems:

  • Switch Krakatoa to “Save Particles To File Sequence” mode.
  • Specify a saving path in the Save Particles rollout - this will be the base name of the partition files, but a suffix describing the partition number and total count will be inserted automatically.
  • Open the the 3ds Max Render Setup dialog and set the time range you want to process, or use the right-click menu of the SAVE PARTICLES button in the Main Controls to switch between the time range modes.
  • Open the Partitioning rollout and make sure all the Operators you want to modify per partition are checked (they are by default).
  • Enter the total number of partitions you intend to produce - the default is 10 and the maximum is 100, but you can always produce less than that number by saving a sub-range or by canceling mid-saving.
  • You can specify a range to process at once, for example if the Partition Count is 10 and you enter Partition Range From 1 To 3, only the first 3 partitions will be processed and later you can continue with 4 to 10 if you like the results so far.
  • Press the “Generate Partition Range Locally…” or “Generate All Partitions Locally” to either process the specified range from the previous step or all 10 partitions. Remember that you can cancel at any time manually.
Note that if you requested to save a single frame, a prompt will appear warning you that at least two frames must be saved. This is caused by an optimization in the 3ds Max / Particle Flow notification handling where changing the random seed does not update the system if a single frame is requested for rendering, but it works if at least two frames are rendered. Thus, you will be asked to decide whether to save two frames per partition, switch to a range or exit without partitioning.
  • If you started partitioning, you will see Krakatoa calculating and saving the particles to disk, producing multiple versions of the same Particle Flow with incremented random seeds.

Maximizing The Local Workstation CPU Usage

  • When creating partitions from single-threaded sources like Particle Flow, the majority of the time only one CPU on a multi-core system will be fully loaded with the rest sitting idle.
  • If you have a good multi-core 64 bit machine with plenty of RAM, you could maximize the use of your processors by following these steps:
    • Save the scene just before launching the Partitioning
    • Launch one or more additional sessions of 3ds Max on the same workstation
    • Load the saved scene, open the Krakatoa User Interface
    • Set the Partition Range From … To to a different portion of the total number of partitions.
      • For example, the first copy of 3ds Max could be set to calculate partitions 1 to 2,
      • the second copy could be set to partition 3 to 4 and so on.
    • You can use as many instances as you have cores, or more or less depending on how much memory you have and how much memory will be used by 3ds Max. Note that Krakatoa does not use memory while saving particles.
    • Launch each of these instances of Krakatoa to partition using the Generate Partition Range Locally button.
  • This way, each of the Particle Flows will use one CPU you can get the final results many times faster…

Partitioning Using Deadline

  • Another way to increase the speed of Partitioning is to submit the Partitioning Job to the Deadline Network Manager.
    • If you have access to a larger Deadline-based network render farm, you could submit even more partitions to calculate in parallel.
  • Krakatoa supports two Deadline Submission Modes - “One Job Per Partition, Frames As Task” and “One Job Only, Partitions As Tasks”.
    • The latter method allows the same network machine to pick up more than one task of the job and process them concurrently by launching multiple instances of 3ds Max and thus loading the available CPUs more.
    • The “Deadline Submission Settings” exposed in the Partitioning rollout when Deadline has been detected on the system provide a control to define the Concurrent Tasks Per Machine and an option to limit them to the number of installed CPUs.
    • This way, you can set the number to 8 to leverage the power of 8-core machines, but if one of the machines has only 4 CPUs, it would dequeue only 4 partition tasks.
NOTE: Benchmarking the Partitioning of typical Particle Flow setups has shown the CPU load never to reach 100% due to the partition processing overhead. Also, the startup time of multiple 3ds Max sessions can add a significant overhead to the total processing time, thus launching 8 Concurrent Tasks on one machine might not perform 8 times faster but a lot less, typically about 3 times faster than processing all partitions locally and sequentially. Still, if the partitioning can be offset to a second machine while the workstation is being used to set up Krakatoa renders, it is a significant benefit.
  • See also the Using Krakatoa with Deadline topic.

Loading Partitions For Rendering

  • Once you are done saving all partitions, create a PRT Loader at the world origin by holding SHIFT while selecting the PRT icon from a toolbar or the “Create a PRT Loader…” menu item from the Krakatoa menu.
    • Alternatively, you can create the PRT Loader anywhere in the scene and uncheck the “Use Node Transforms” option to assume all particles at their original positions saved in the PRT files.
    • Add ANY of the saved sequences to the PRT Loader - because the file name contains the suffix “partXXofYY”, the PRT Loader will detect that you are loading a sub-set of a partition set with YY sequences and will ask you about loading all others - answer the prompt with Yes.
    • Confirm the loading of all or some selected partitions using the Select Partitions Dialog which will pop up automatically.
  • At this point you can disable the original particle sources, switch Krakatoa to “Render Scene Particles” and render the PRT Loader to get a denser version of the original particle system(s) combining all Partitions from disk.
  • See also the topic Loading Particles Using The PRT Loader > Loading_Partitions.

SEE ALSO

  • Partitioning Rollout
  • Creating Particle Partitions Tutorial
  • Using Krakatoa with Deadline