Particle Cache and Lighting Cache

Available in Krakatoa v1.0.0 and higher

Introduction

• In order to perform its volumetric lighting and rendering calculations, Krakatoa requires all particles to be loaded in memory in the beginning of the rendering process.
  • The calculation of the light attenuation and the drawing of the particles in the final image expects the particles to be sorted relatively to the light source and the camera.
  • Thus, the first step of rendering is always the loading of the particles from their respective sources (Particle Flow, Legacy Particle Systems, Thinking Particles, Geometry Vertices, PRT Loaders, PRT Volumes, FumeFX simulations) into the computer’s RAM.
  • This step can often account for half or more of the total render time of the scene and can be even slower if the particles are applied complex materials and maps or are being deformed or their channels modified by Krakatoa Channels Modifiers.
• In some cases, loading the particles into memory once and then performing several renders of the same data while changing parameters that do not affect the content of the allocated particle channels can be a big time saver.
• The Particle Cache and Lighting Cache features of Krakatoa allow you to do exactly that.

Feature Overview

• The Particle Cache and Lighting Cache main controls are located in, as you might suspect, the Main Controls rollout of the Krakatoa Renderer GUI.
  • Additional controls are available in a Context Menu which appears when either of the buttons is right-clicked.
  • Additional information about the content of the cached memory channels can be found in the Memory Channels rollout.
• The >PCache checkbutton controls the caching of all particle channels except for the Lighting channel.
• The >LCache checkbutton controls the caching only of the Lighting channel and can be enabled only if the >PCache option is also checked.
  • Thus, checking the >PCache alone provides a distinct mode that caches all particle channels but the Lighting one and allows you to keep the particles in memory but recalculate the light attenuation data in each iteration.
  • Checking the >LCache with the >PCache unchecked will automatically check both options.
  • Unchecking the >PCache with both options checked will also uncheck both since the >LCache option cannot be active without the >PCache being checked.
• These options are valid only in Render Scene Particles mode and have no effect on the Save Particles To File Sequence to avoid the accidental saving of the same particles over multiple frames which never really makes practical sense.
• A Cache Indicator in the form of a thin color swatch between the two checkbuttons provides information about the current state of the data in memory:
  • When gray, no particle data is found in memory.
  • When green, particle data was found in memory and all channels required by the currently enabled rendering features of Krakatoa have been allocated - if the cache is enabled, it will be used and will speed up the next frame’s rendering.
  • When orange, particle data was found in memory and all required channels have been allocated, but changed have been made to render settings that cannot be applied due to the cache being on - the cache will be used, but the changes will be ignored.
  • When yellow, particle data was found in memory, but at least one channel requested by a Krakatoa rendering feature was not found in memory - even if the cache is enabled, the particles will be reloaded to populate the missing channel and the next frame’s rendering will not be sped up.

Particle Cache Modes

• Krakatoa supports two Cache modes
  • Explicit mode - enable Cache first, then render)
  • Implicit mode - render, then enable Cache to lock and reuse data from memory.
  • The latter mode is the default one since it provides a better overall workflow.
  • You can change the Cache mode using the Keep Last Frame In Memory accessible via the Right-Click menu of the PCache/LCache checkbuttons.
    • When the option is checked (default), the particle data will be kept in memory after the rendering finishes and will allow you to lock the data after the fact by checking the >PCache and >LCache options after the fact.
    • When the option is unchecked, the particle data will be discarded after the rendering finishes unless the >PCache and optionally the >LCache checkboxes were checked before the rendering started.

Implicit Mode (Render First, Enable Cache Later)

Using the Implicit Mode

• If the >PCache option is not checked, Krakatoa will reload all particles into memory every time a frame is rendered.
  • After rendering finishes, the cache indicator will turn green and the Particle Cache Size item in the Right-click Context Menu will show a non-zero cache size.
  • In the Windows Task Manager, you will notice that the memory used by Krakatoa will not be released after the rendering finishes.
• At any point after rendering a frame, you can check the >PCache option to tell Krakatoa to lock and use the already loaded data for consecutive renders.
  • This will speed up test renderings significantly because it eliminates the acquiring particles portion of the process.
  • While using the PCache, changes made to the particle sources that would alter the particle channels in memory will have no effect on the rendering!
• If the cache is empty (the cache indicator is gray and not green), you can also check the >PCache option first, then render a frame.
  • Since the cache is enabled but empty, Krakatoa will load the particles and render them as usual, then lock them in memory.
  • Consecutive renders will use the cached data.

Using the LCache

• If the >Ignore Scene Lights option in the Main Controls rollout is unchecked, the >LCache checkbutton will be available.
  • By default, the PCache stores all requested channels, but when >LCache is unchecked, the lighting is recalculated dynamically and overwrites the content of the Lighting channel.
  • This is useful when testing the lighting of the scene by moving light sources around, changing their settings etc., but not changing the particle generation, positions, velocities, materials etc.
• If the >LCache button is checked, the content of the Lighting channel will be locked and reused in consecutive renders and the Lighting calculation phase will also be skipped.
  • Moving or changing lights will have no effect on the final rendering.
  • Also, changing the Lighting Pass Density or, if the Final Pass Density is used for the Lighting Pass, too, changing the Final Pass Density will have no effect on the Attenuation of light as it passes through the particles!
  • You can still manipulate the Final Pass Density of the particles as seen through the Camera and change the Camera to render the particles from various angles or with different Motion Blur and Depth Of Field settings.

Releasing Cache Memory

• To release the memory used up by the PCache (and the LCache, if active), you can use the Clear Both Caches option in the Right-click Context Menu.
  • This will reclaim the memory used by Krakatoa, for example if you need more memory for other applications at that point in time.
  • Note that unchecking the >PCache and/or >LCache checkbuttons will not clear the cache.
    • Thus, you can uncheck them, then change your mind and without performing a new rendering check them again - the last cache data will still be preserved and used!

Adding Uncached Channels

• If you introduce new memory channels into the rendering process by enabling features that were not active when the cache data was generated (i.e. Motion Blur adds the required Velocity channel, Phong Surface requires the Normals channel etc.), Krakatoa will have to rebuild the cache to include the requested channels.
  • In such cases, the cache indicator will turn yellow to inform you that while there is cached data in memory, its content does not match the required list of particle data channels and the cache has to be rebuilt.
  • Once a frame is rendered and the cache is rebuilt, the cache indicator will turn green and consecutive renders will use the new data.
  • In the Memory Channels rollout, you will notices that cached channels have a + next to their name.
    • If all channels shown on the right “Active Channels” list have a + sign, the cache is valid and no cache rebuilding will be performed.
    • If one or more channels on the right “Active Channels” list have a – sign, the cache must be rebuilt and the next render iteration will take longer.
• If any channel that has already been cached is not required anymore, the cache will NOT be rebuilt - the data will remain in memory and will simply be ignored.
  • The channel with still be shown with a + sign in the Memory Channels rollout, but it will appear on the left “Inactive Channels” list.
  • For example, caching with Motion Blur on will allocate a Velocity channel in memory.
  • Disabling Motion Blur with >PCache still checked will not remove the Velocity channel from memory causing a rebuild, it will simply ignore the cached Velocity data.
  • Thus, you can re-enable Motion Blur and start using the cached Velocity channel immediately without the need to rebuild the cache if it was already cached previously.

Explicit Mode (Enable Cache First, Render Later)

• This is a mode that you have to switch to yourself by unchecking the “Keep Last Frame In Cache” option in the right-click context menu of the >PCache and LCache checkbuttons.
• If >PCache is not checked, rendering the scene will load the particle channels data into memory, perform the rendering calculations and release the memory once the frame is done.
  • You can monitor this in Windows Task Manager - after rendering is finished, 3ds Max should return to the memory usage before the rendering started.
• To cache data in memory, you have to check the >PCache option first, then render.
  • Once the rendering is finished, the last frame data will be kept in memory until the PCache button is released.
  • The cache indicator will turn green and you can perform multiple render iterations using the same particle data.
• To release the cache, simply uncheck the >PCache button.
  • Thus, unchecking and checking the >PCache option will clear the cache and cause the next render iteration to reload all particles.
• The same applies to the >LCache option.

Why Two Modes?

• As you can see, the Implicit mode has huge advantages.
  • In production, 90% of the time the artist starts rendering a frame only to realize that he forgot to enable the cache.
  • If he were using the Explicit mode, this would mean stress and wasting a lot of time.
• The drawback of the Implicit mode is that, unlike other renderers, once the rendering is finished, the used memory is not reclaimed EVEN IF PCACHE IS NOT CHECKED!
  • This is because the option Keep Last Frame In Cache tells Krakatoa not to release memory, assuming that the artist might want to reuse the last frame as cache data at any time by checking the >PCache button.
• If you feel that Krakatoa is using up too much memory (potentially preventing other programs or even Windows from running well in the background) and you want to have full control over when memory is kept and when it is released, you can turn off the Keep Last Frame In Cache and thus switch to Explicit Caching mode.

Supported Changes To Krakatoa Settings In Cache Mode

• The main purpose of the Particle and Lighting Caches is to speed up render test iterations.
  • Obviously, any data that has been locked in the Cache cannot be modified though.
  • This means that there are several situations where the Cache can be used to speed up iterative tests, and many situations where it cannot.

Tweaking The View or Camera

• One of the main cases where the PCache and LCache options can be useful for speeding up render tests is the placement and set up of the Camera.
• The particle rendering uses the Camera directly without caching any of its parameters, thus changing the camera’s position, Field Of View, Krakatoa Camera Modifier parameters like f-stop or projection mode etc. are fully supported while using cached particles.
• The same applies to rendering the current view without a Camera - changing the position, orientation and FOV of the viewport will result in correctly rendered frames while using the cached particles.
• When using a Camera with a Krakatoa Camera Modifier and the Depth Of Field option in the Krakatoa Main Controls rollout is enabled, you can freely tweak the DOF parameters since they are also independent from the cached particle data.
• If the Camera has a Depth Of Field Multi-Pass Effect assigned, it can be used with cached particle data.
• If the Camera has a Motion Blur Multi-Pass Effect assigned, it cannot be used with the cached particles because it requires the reloading of particles at different times on each sub-step, but the cache would always return the same data, thus producing incorrect results.

Tweaking Global Particle Density

• Another major area supported by the Particle Cache is the Global Particle Density (Final Pass Density and Density Exponent in the Main Controls rollout of the Krakatoa Renderer GUI).
  • The Final Pass Density affects the drawing of particles as a multiplier and is applied on the fly during the rendering, so it is independent from the cached particle data.
  • If the >LCache option is unchecked and the Final Pass Density is used to define the Lighting Pass Density, both the lighting and the final pass will use the changed Density values.
  • If the >LCache option is unchecked and the Lighting Pass Density is used to define the attenuation of light as it passes through particles, you can use the Lighting Density and Exponent values to tweak the self-shadowing of particles while reusing all other cached particle channels.
  • If the >LCache option is checked, the attenuation data will be locked and reused and changing the lighting pass density will have no effect on the final appearance of the particles.

Tweaking Draw Point Filters

• The Draw Point Filter mode can be switched between Nearest, Bilinear and Bicubic as it is applied at render time and does not depend on the Particle Cache.
• The Self-Shadow Filter mode can only be tweaked if the Lighting Cache is not enabled and Attenuation is recalculated dynamically in each iteration.

Tweaking Environment Reflections

• The Environment Reflection Map is applied in camera space at render time and requires the Normals channel to be present in memory.
  • Thus, if the Normals channel has been already cached, you can tweak the Environment Reflection with Particle Cache on by moving the Camera around the object or changing the reflection map or its intensity.

Tweaking Phase Functions (Shaders)

• You can change the Phase Functions (Isotropic, Phong Surface, Henyey-Greenstein, Schlick) with Particle Cache enabled if the Lighting Cache is turned off, since they are calculated during the Lighting pass.
• If the Phase Function depends on per-particle channels for its parameters (SpecularPower, SpecularLevel, Eccentricity channels), these must have been cached already.
  • Since the Henyey-Greenstein and Schlick shaders use the same Eccentricity channel, if it was allocated for the one, you can switch to the other without recaching.
  • The per-particle data in these channels cannot be changed if the Particle Cache is on though.
  • If the per-particle data channels are not enabled for allocation, the shader parameters will be taken as static values and can thus be tweaked freely to produce different Phong or Eccentric shading with the Particle Cache on as long as the Lighting Cache is off.
  • Switching from any shader to Isotropic while Particle Cache is on is allowed since Isotropic mode uses no parameters or channels, but the Lighting Cache must be off.

Tweaking Motion Blur Settings

• If the Velocity channel has been cached already, tweaking the Krakatoa Motion Blur parameters like Motion Blur Segments, Motion Blur Shutter and Matte Objects Motion Blur Bias will produce the desired results in consecutive renders.
• Note that enabling >Jittered Motion Blur will request the allocation of an additional MBlurTime particle data channel.
  • Thus, if you intend to use Jittered Motion Blur in your tests, you should make sure the MBlurTime channel has been already cached.
  • If you enable >Jittered Motion Blur after the cache was built, a recaching will occur to populate the MBlurTime channel.

Tweaking Matte Objects

• The Matte Objects are evaluated on the fly during rendering and are independent from the Particle and Lighting caches.
  • This also applies to Deformation Motion Blur which simply re-evaluates the Matte Objects at the correct sub-frame time before each sample is rendered.
• You can make changes to the Matte Objects count, position, opacity mapping etc. while using cached particle data.

Tweaking Lighting

• As already mentioned, when the >LCache option is unchecked, the Lighting channel will not be cached and will be rebuilt dynamically in each iteration.
• If you want to tweak the light source placement, color, shadow map size or the Lighting Pass Density, you can use the Particle Cache with Lighting Cache disabled, thus saving time on reloading all other particle channels except for the Lighting channel.

Tweaking Voxel Renderer Parameters

• When using Voxel rendering mode, the Voxel Size and Filter Radius can be tweaked freely if the Particle Cache is on.
  • Note that Voxel mode does not use the Lighting Cache at all and will always calculate the lighting dynamically, so you can make any changes to light sources, lighting pass density and shading functions with Particle Cache on.

Unsupported Changes When Using Cached Data

• When the PCache is enabled, all particle data channels provided by the particle sources will be locked.
• Thus, changing any aspect of the particle sources or Krakatoa Settings that are evaluated during the process of loading particles and stuffing their data into the particle channels in memory will not be reflected in the final rendering if the Particle Cache is on.
• These aspects include:
  • Particle Sources - changing the particle source options (block of checkbuttons to the left of the cache options in the Main Controls rollout) or adding, deleting, hiding, unhiding, enabling and disabling particle sources in the scene will have no effect on the final rendering if the Particle Cache is on.
  • Particle Count - changing the Birth operator in Particle Flow, the Render Percentage or Culling settings in PRT Loaders, the parameters of a PRT Volume, the topology of geometry objects used as vertex sources etc. will not be reflected if the Particle Cache is on.
  • Particle Position - moving the particle source emitter or changing the Position Operators in Particle Flow, transforming a PRT Loader, changing the Deformation Modifiers of a PRT Loader or PRT Volume etc. will have no effect on the final rendering if the Particle Cache is on.
  • Particle Data Channels - changing any already cached particle data channels using local or global Krakatoa Channel Modifiers will have no effect if Particle Cache is on since the KCMs won’t be evaluated at all.
  • Color Sources - changing Materials, Maps or the Global Color Override in the Krakatoa Global Render Values rollout, as well as changing the object color of particle sources with no materials will have no effect on the final rendering if the Particle Cache is on. (Note: In Krakatoa v1.1.x, it was possible to change the Color Override with PCache on because it was used as static value at render time and no Color Channel was allocated. This has changed since v1.5.0 because the Color Override can now include a map tree and has to be evaluated during particle loading.)
  • Shading Channels - changing Emission, Absorption and Density Overrides will have no effect on the final rendering if the Particle Cache is on.
  • Lighting Channel - if the Lighing Cache is on, any changes to lights, shaders (phase functions) or the lighting pass density will be ignored.
• Krakatoa will monitor the UI for changes to controls that would have no effect due to the PCache being on and change the Cache Indicator to orange as a warning.
  • In addition, you can find an explanation of the color coding in the Memory Channels rollout.

Rendering Animations With Cached Data

• Both the PCache and the LCache are available when rendering a single frame or a range of frames, but in the latter case the same particles from the frame that was cached will be rendered throughout the whole animation!
  • In other words, once a frame is cached, changing the scene time will NOT render a different frame unless the PCache is rebuilt again using one of the methods described above.

Rendering With Cache Data On The Network

• By default, when a scene is submitted to render on the network, Krakatoa will ignore the state of the PCache and LCache options and will not populate the cache and use it on consecutive frames.
  • This is done to avoid user errors where the caching options were turned on for local tests and were forgotten enabled when submitting the scene for network rendering.
• In a very small percentage of cases though, the user might want to submit a job to the network and set it to run on a single machine and render the first frame using a moving Camera (for example to create a fly-by animation around a static particle cloud).
  • In such cases, it would be useful to let the network machine cache the first frame and then render much faster without the need to reprocess the otherwise static particles.
• For this special case, the option Use Cache In Slave Mode was added to the right-click context menu of the >PCache and >LCache buttons
  • This option is unchecked by default to ensure the Cache is not used when Krakatoa is running in Network mode.
  • You can check the option to allow Krakatoa to build and use the Cache when running in Network mode.
  • Be advised though that submitting the job to multiple network machines might result in unexpected results if the particle systems being rendered do contain animation - each machine would start rendering a different frame and would cache that frame for all consecutive frames, potentially causing particles to skip around from frame to frame!