3ds Max - Integrated Submitter - Scripting

Overview

The Submit Max To Deadline (SMTD) integrated submitter for 3ds Max was implemented using MAXScript in a manner that allows both Thinkbox and 3rd party developers to:

  • Use parts of its backend functionality to create new submitters
  • Expand its functionality by adding new Workflows
  • Access functionality that is not exposed in the SMTD User Interface

SMTD consists of the following major parts:

  • The SMTD Backend which implement the settings and functions related to the submission of a job and integration with Deadline, fully decoupled from any User Interface.
  • The SMTD User Interface which is just one possible access point to the SMTD Backend functionality, but could be replaced by alternative implementations
  • The SMTD Launcher which is the client script installed in the 3ds Max UI (Menu, Toolbar etc.) and is responsible for loading the SMTD Backend and User Interface scripts.
  • Various additional script and INI files used by parts of SMTD.

Creating New 3ds Max to Deadline Submitters

When creating a new submitter for a specific purpose, a developer can elect one of the following major approaches:

  • Write the submitter entirely from scratch.
  • Use the SMTD Backend functions and develop a completely new User Interface to match the end users’ needs.
  • Use both the SMTD Backend functions and SMTD UI and implement new UI and logic components using SMTD Workflows.

Developing A Submitter From Scratch

  • This is obviously the most difficult and time consuming approach approach, as every single feature and function would have to be implemented by the developer.

Pros

  • The main benefit would be that all functionality would be completely purpose-built for the specific task.

Cons

  • Slowest development turnaround.
  • Requires deep knowledge of both 3ds Max and Deadline.
  • Bug fixes made by Thinkbox to SMTD would not apply automatically to such custom submitters.
  • Resulting workflows and UI could appear alien to an SMTD user.

Developing A New Submitter Using the SMTD Backend Functions

  • Both Thinkbox and 3rd party developers can use the SMTDFunctions and SMTDSettings implemented by the SubmitMaxToDeadline_Functions.ms file to create new submitters.
  • Several Thinkbox plugins like Krakatoa MX, Stoke MX, and XMesh Save MX use this approach and offer custom Deadline submission panels within their own UIs, but reuse the functionality offered by SMTD’s Backend.
  • A few studios and individual users have also written their own custom submitters based on the SMTD Backend.

Pros

  • Fast development turnaround.
  • Will benefit from bug fixes, functionality and performance improvements rolled out with Deadline updates.

Cons

  • Requires familiarity with the SMTD properties and functions.
  • Resulting User Interface and workflow could differ from SMTD.

Developing New Submission Functionality Using Workflows

Pros

  • Very fast development (often under an hour) to implement a completely new submitter.
  • User Interface will be very familiar to the end user.

Cons

  • Not optimal for deep integration with plugins, e.g. Krakatoa Partitioning on Deadline or XMesh Saver submission are better integrated within the Krakatoa UI than they would be as external Workflows.

Features Accessible Only Via MAXScript

  • A few features have been added to SMTD specifically for 3rd party developers using the SMTD Backend for custom submitter development.
  • These features are not exposed to the SMTD User Interface shipping with Deadline and make sense only in the context of custom submission scripting.
  • They include:
    • Custom Frame List Definition - allows the developer to specify the exact frame order beyond the pre-defined custom sequences available in the SMTD UI.
    • Extra Output Filenames - allows the developer to specify a list of paths to be exposed in the Monitor’s Job right-click menus beyond the paths acquired from render output and render elements in the 3ds Max Render Setup dialog.
    • Extra Job Info Keys - allows the developer to specify arbitrary Job Info Key=Value pairs to be included with the job.
    • Extra Plugin Info Keys - allows the developer to specify arbitrary Plugin Info Key=Value pairs to be included with the job.

Custom Frame List Definition

  • The SMTD UI offers a number of possible pre-defined frame sequence orders, including straight and reversed, every Nth frame with different fill modes, etc.
  • However, in production some more elaborate frame sequences might be required to ensure the most important frames are processed first to catch any issues before finishing the rest of the job.
  • For that reason, the option to define a custom frame sequence entirely via MAXScript was added to SMTD in Deadline 10.
  • Two new SMTDSettings properties have been introduced:
    • SMTDSettings.CustomFrameList - an array defaulting to empty array #(). It can be used to define the exact explicit order to process the frames.
    • SMTDSettings.UseCustomFrameList - a boolean value which defaults to false. When set to true, the custom frame sequence defined by SMTDSettings.CustomFrameList will be used.

For example:

An actual customer asked for the following frame sequence on the Thinkbox Support Forum:

  1. First, Middle, and Last frames first
  2. First ten consecutive frames
  3. The remaining frames in the range, optionally supporting every Nth frame

Using the new custom frame list definition, the solution in a custom SMTD-based submission script would look like this:

SMTDSettings.UseCustomFrameList = true
SMTDSettings.CustomFrameList=#(rendStart.frame as integer, (rendEnd.frame as integer - rendStart.frame as integer )/2, rendEnd.frame as integer) --first, middle, last
for i = rendStart+1 to rendStart+10 do appendIfUnique SMTDSettings.CustomFrameList (i.frame as integer)  --first 10
for i = rendStart to rendEnd by 5 do appendIfUnique SMTDSettings.CustomFrameList (i.frame as integer)  -- every 5th
for i = rendStart to rendEnd do appendIfUnique SMTDSettings.CustomFrameList (i.frame as integer) --all others
with PrintAllElements on format "%\n" SMTDSettings.CustomFrameList --show me the result

The output from the last line of the script when submitting frame range from 0 to 100 would look like:

--> #(0, 50, 100, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 15, 20, 25, 30, 35, 40, 45, 55, 60, 65, 70, 75, 80, 85, 90, 95, 11, 12, 13, 14, 16, 17, 18, 19, 21, 22, 23, 24, 26, 27, 28, 29, 31, 32, 33, 34, 36, 37, 38, 39, 41, 42, 43, 44, 46, 47, 48, 49, 51, 52, 53, 54, 56, 57, 58, 59, 61, 62, 63, 64, 66, 67, 68, 69, 71, 72, 73, 74, 76, 77, 78, 79, 81, 82, 83, 84, 86, 87, 88, 89, 91, 92, 93, 94, 96, 97, 98, 99)

Extra Output Filenames

  • Normally, the function SMTDFunctions.CreateSubmitInfoFile() will write any output paths referenced by the renderer to the Job Info file as OutputFilename%= entries, where % will be replaced with an incremental index.
  • These paths automatically become visible in the Job’s right-click menu in the Monitor, allowing the user to explore them or open the files.
  • However, in some cases the job in question might contain output paths that are not directly discoverable as renderer properties.
  • For example, a script could be submitting a MAXScript job to perform some custom exporting, simulation, or data file creation unrelated to rendering - it would be useful to be able to add the output filename to the JOB file.
  • In versions prior to Deadline 10, this would be done by appending to the already written JOB file.
  • Starting with Deadline 10, it is now possible to simply populate the new property SMTDSettings.ExtraOutputFilenames with an array of paths to be included.
  • Note that once set, the property will keep its value until it set explicitly to an empty array #(), or SMTD is restarted - be sure to set to #() when done submitting!

Extra Job Info Keys

  • In many cases, submitting a custom job using the SMTD Backend functionality involves calling SMTDFunctions.CreateSubmitInfoFile() to create a Job Info file.
  • However, the content of the file is relatively rigid and oriented at render submissions.
  • In many cases, especially when submitting custom MAXScript jobs to perform non-rendering tasks, additional keys need to be added to the Job Info file.
  • Prior to Deadline 10, this could be done either by appending to the already created Job Info file, or by writing the whole file from scratch using completely custom logic.
  • Starting with Deadline 10, it is very easy to add arbitrary additional Key=Value Pairs by populating the property SMTDSettings.ExtraJobInfoKeys with an array of arrays
  • Each array element must be a sub-array containing exactly two elements; anything else will be skipped automatically.

For example:

SMTDSettings.ExtraJobInfoKeys = #( #("MyNewKey", "MyNewValue"), #("AnotherKey", 10.0) ) --define two key=value pairs
append SMTDSettings.ExtraJobInfoKeys #("TheAnswer", 42) --add another key=value pair
append SMTDSettings.ExtraJobInfoKeys 123.456 --this is of course invalid because not an array, but will be handled safely and will be skipped
append SMTDSettings.ExtraJobInfoKeys #(234.567) --this is also invalid, because the array has no second value, but it is safe and will be skipped
append SMTDSettings.ExtraJobInfoKeys #(1,2,3) --this is also invalid, because the array has more than two values, it will also be skipped

Extra Plugin Info Keys

  • Similar to the Extra Job Info Keys, the Extra Plugin Info Keys can be defined via the property SMTDSettings.ExtraPluginInfoKeys.
  • If the property is set to an array of two-element sub-arrays, extra Key=Value pairs will be included in the Plugin Info file when the function SMTDFunctions.CreateJobInfoFile() is called.
  • The same rules outlined in the previous section apply.