Application Plugins

Overview

All of Deadline’s plugins are written in Python, which means that it’s easy to create your own plugins or customize the existing ones. See the Scripting Overview documentation for more information, and links to the Deadline Scripting reference.

Note that because the Python scripts for application plugins will be executed in a non-interactive way, it is important that your scripts do not contain any blocking operations like infinite loops, or interfaces that require user input.

When a plugin is loaded the log will show where the plugin is being loaded from.

General Plugin Information

There are two types of plugins that can be created:

  • Simple
  • Advanced

Simple plugins provide the basics to wrap a command line application, and is typically used to build up command line arguments to pass to the application. Advanced plugins provide more control, and are typically used when running a simple command line application isn’t enough. Other than the plugin Python script itself though, Simple and Advanced plugins are very similar.

Creating a New Plugin

This section covers the the areas that Simple and Advanced plugins have in common. Specifics for Simple and Advanced plugins are covered later on.

To create a new plugin, start by creating a folder in the Repository’s custom\plugins folder and give it the name of your plugin. See the Scripting Overview documentation for more information on the ‘custom’ folder in the Repository and how it’s used.

For the sake of this document, we will call our new plugin MyPlugin. All relative script and configuration files for this plugin are to be placed in this plugin’s folder (some are required and some are optional).

.py File

Required

The MyPlugin.py is the main plugin script file. It defines the main DeadlinePlugin class that contains the necessary code that Deadline uses to render a job. This is where Simple and Advanced plugins will differ, and the specifics for each can be found later on, but the template for this script file might look like this:

from Deadline.Plugins import *

######################################################################
## This is the function that Deadline calls to get an instance of the
## main DeadlinePlugin class.
######################################################################
def GetDeadlinePlugin():
  return MyPlugin()

######################################################################
## This is the function that Deadline calls when the plugin is no
## longer in use so that it can get cleaned up.
######################################################################
def CleanupDeadlinePlugin( deadlinePlugin ):
  deadlinePlugin.Cleanup()

######################################################################
## This is the main DeadlinePlugin class for MyPlugin.
######################################################################
class MyPlugin (DeadlinePlugin):

  # TODO: Place code here instead of "pass"
  pass

The first thing to note is that we’re importing the Deadline.Plugins namespace so that we can access the DeadlinePlugin class.

The GetDeadlinePlugin() function is important, as it allows the Slave to get an instance of our MyPlugin class (which is extending the abstract DeadlinePlugin class). In Deadline 6.2 and later, the GetDeadlinePluginWithJob( job ) function can be defined as an alternative. It works the same as GetDeadlinePlugin(), except that it accepts an instance of the Job object that the plugin is being loaded for. If either of these functions are not defined, the Slave will report an error when it tries to render the job.

The MyPlugin class will need to implement certain callbacks based on the type of plugin it is, and these callbacks must be hooked up in the MyPlugin constructor. One callback that all plugins should implement is the InitializeProcess function. There are many other callbacks that can be implemented, which are covered in the Events section for the DeadlinePlugin class in the Deadline Scripting reference.

The CleanupDeadlinePlugin() function is also important, as it is necessary to clean up the plugin when it is no longer in use. Typically, this is used to clean up any callbacks that were created when the plugin was initialized.

To start off, the InitializeProcess callback is typically used to set some general plugin settings:

from Deadline.Plugins import *

######################################################################
## This is the function that Deadline calls to get an instance of the
## main DeadlinePlugin class.
######################################################################
def GetDeadlinePlugin():
    return MyPlugin()

######################################################################
## This is the function that Deadline calls when the plugin is no
## longer in use so that it can get cleaned up.
######################################################################
def CleanupDeadlinePlugin( deadlinePlugin ):
    deadlinePlugin.Cleanup()

######################################################################
## This is the main DeadlinePlugin class for MyPlugin.
######################################################################
class MyPlugin (DeadlinePlugin):

    ## Hook up the callbacks in the constructor.
    def __init__( self ):
        self.InitializeProcessCallback += self.InitializeProcess

    ## Clean up the plugin.
    def Cleanup():
        del self.InitializeProcessCallback

    ## Called by Deadline to initialize the plugin.
    def InitializeProcess( self ):
        # Set the plugin specific settings.
        self.SingleFramesOnly = False
        self.PluginType = PluginType.Simple

These are the common plugin properties that can be set in InitializeProcess callback. See the DeadlinePlugin class in the Deadline Scripting reference for additional properties.

Property Description
PluginType The type of plugin this is (PluginType.Simple/PluginType.Advanced).
SingleFramesOnly Set to True or False. Set to True if your plugin can only work on one frame at a time, rather than a frame sequence.

.param File

Required

The MyPlugin.param file is a required file that is used to declare the custom settings for the plugin and by the Plugin Configuration dialog in the Monitor to set these custom settings. It declares properties that the Monitor uses to generate a user interface for modifying custom settings in the database.

Each control is identified in the MyPlugin.param file by a unique name between square brackets, and a set of key=value pairs that determine its default value and how it appears in Deadline’s user interface. For example, a typical plugin defines a read-only “About Plugin” section, which contains a description of the plugin and indicates whether or not the plugin supports rendering concurrent tasks.

[About]
Type=label
Label=About
Category=About Plugin
CategoryOrder=-1
Default=An example of a custom Plugin
Description=Not configurable

[ConcurrentTasks]
Type=label
Label=ConcurrentTasks
Category=About Plugin
CategoryOrder=-1
Default=True
Description=Not configurable

We can add a new custom option to identify a render executable for MyPlugin. The property name between the square brackets will correspond to the custom setting to be defined in the database. This means that the control defined below will change the MyPluginRenderExecutable setting. Here, we specify Type=filename to indicate that this setting should be treated by Deadline as a path to a file.

[MyPluginRenderExecutable]
Type=filename
Label=My Plugin Render Executable
Default=c:\path\to\my\executable.exe
Description=The path to the executable file used for rendering.

Note that in previous versions of Deadline, Plugin settings were stored in dlinit files, which have been removed as of Deadline 8.0. All data that was previously stored in the dlinit file is now stored in the database. In order to declare custom settings for a plugin you must do so in the param file.

After you’ve created the MyPlugin.param file, open the Monitor and enter Super User mode. Then select Tools -> Configure Plugins and look for your plugin in the list on the left.

../_images/configure_plugin_myplugin.png

You can use this area to modify the values of the plugin’s parameters. Any changes you make to values here will be stored in the database when you press the OK button. You may also change the plugin’s icon and globally Set Limits for the application plugin using the options at the bottom. Clicking the Restore Default Icon button will change the icon back to the plugin’s ico file. You can also Enable or Disable a Plugin using the Enabled checkbox. Note that Plugins that are not Enabled will not be able to render.

Comment lines are supported in the param file, and must start with either ‘;’ or ‘#’. For example:

# This is the file name picker control to set the executable for this plugin.
[MyPluginRenderExecutable]
Type=filename
Label=My Plugin Render Executable
Default=c:\path\to\my\executable.exe
Description=The path to the executable file used for rendering.

The available key=value pairs for the properties defined here are:

Key Name Description
Category The category the control should go under.
CategoryIndex This determines the control’s order under its category. This does the same thing as Index.
CategoryOrder This determines the category’s order among other categories. If more than one CategoryOrder is defined for the same category, the lowest value is used.
Default The default value to be used if this property is not defined in the database. This does the same thing as DefaultValue.
DefaultValue The default value to be used if this property is not defined in the database file. This does the same thing as Default.
Description A short description of the property the control is for (displayed as a tooltip in the UI).
DisableIfBlank If True, a control will not be shown if this property is not defined in the param/options file (True/False). This does the same thing as IgnoreIfBlank.
IgnoreIfBlank If True, a control will not be shown if this property is not defined in the param/options file (True/False). This does the same thing as DisableIfBlank.
Index This determines the control’s order under its category. This does the same thing as CategoryIndex.
Label The control label.
Required If True, a control will be shown for this property even if it’s not defined in the database (True/False).
Type The type of control (see table below).

These are the available controls.

Control Type Description
Boolean A drop-down control that allows the selection of True or False.
Color Allows the selection of a color.
Enum A drop-down control that allows the selection of an item from a list.
Enumeration Same as Enum above.
Filename Allows the selection of an existing file.
FilenameSave Allows the selection of a new or existing file.
Float An floating point spinner control.
Folder Allows the selection of an existing folder.
Integer An integer spinner control.
Label A read-only text field.
MultiFilename Allows the selection of multiple existing files, which are then separated by semicolons in the text field.
MultiLineMultiFilename Allows the selection of multiple existing files, which are then placed on multiple lines in the text field.
MultiLineMultiFolder Allows the selection of multiple existing folders, which are then placed on multiple lines in the text field.
MultiLineString A text field with multiple lines.
Password A text field that masks the text. (Note: this is not usable in .options files)
SlaveList Allows the selection of existing Slaves, which are then separated by commas in the text field.
String A text field.

There are also key/value pairs for specific controls:

Key Name Description
DecimalPlaces The number of decimal places for the Float controls.
Filter The filter string for the Color, Filename, FilenameSave, or MultiFilename controls. Examples: Filter=Windows Bitmap (*.bmp);;Targa (*.tga);;JPEG (*.jpg *.jpeg);;PCX (*.pcx);;PNG (*.png);;TIFF (*.tif *.tiff);;All files (*) or Filter=(color %R %G %B).
Increment The value to increment the Integer or Float controls by.
Items The semicolon separated list of items for the Enum control. This does the same thing as Values.
Maximum The maximum value for the Integer or Float controls.
Minimum The minimum value for the Integer or Float controls.
Validator A regular expression for the String control that is used to ensure the value is valid. Examples: Validator=\w* matches any word character; equal to [a-zA-Z0-9_] or Validator=\d* matches any digit; equal to [0-9].
Values The semicolon separated list of items for the Enum control. This does the same thing as Items.

.options File

Optional

The MyPlugin.options file is an optional file that is used by the Job Properties dialog in the Monitor. It declares properties that the Monitor uses to generate a user interface for modifying plugin specific options as they appear in the plugin info file that was submitted with the job. After you’ve created this file, you can right-click on a job in the Monitor that uses this plugin and select Modify Properties. You should then see a MyPlugin page at the bottom of the list on the left which you can select to view these properties.

../_images/options_file_myplugin.png

Often, these plugin specific options are used to build up the arguments to be passed to the rendering application. Let’s assume that our render executable takes a “-verbose” argument that accepts a boolean parameter, and that the plugin info file submitted with the job contains the following:

Verbose=True

Now we would like to be able to change this value from the Job Properties dialog in the Monitor, so our MyPlugin.options file might look like this:

[Verbose]
Type=boolean
Label=Verbose Logging
Description=If verbose logging is enabled.
Required=true
DisableIfBlank=false
DefaultValue=True

You’ll notice that the property name between the square brackets matches the Verbose setting in our plugin info file. This means that this control will change the Verbose setting. The available key=value pairs for the properties defined here are the same as those defined for the param file above (with the exception of the Password control which is param file only). Comment lines are also supported in the options file in the same way they are supported in the param file.

.ico File

Optional

The MyPlugin.ico file is an optional 16x16 icon file that can be used to easily identify jobs that use this plugin in the Monitor. Typically, it is the plugin application’s logo, or something else that represents the plugin. If a plugin does not have an icon file, a generic icon will be shown in the jobs list in the Monitor.

JobPreLoad.py

Optional

The JobPreLoad.py file is an optional script that will be executed by the Slave prior to loading a job that uses this plugin. It is placed in the <your_repo>/plugins/MyPlugin/ directory of your Repository, where “MyPlugin” is the name of the specific app plugin you wish the script to execute against. Note that in this case, the file does not share its name with the plugin folder. This script can be used to do things like synchronize plugins prior to starting the render job or load a set of environment variables which should be inherited by the render process when it is executed by the DeadlinePlugin -> ManagedProcess class.

The only requirement for the JobPreLoad.py script is that you define a __main__ function, which is called by the Slave when it executes the script. It must accept a single parameter, which is the current instance of the DeadlinePlugin class. Here is an example script that returns the JobId and JobName, copies a couple of files from a server to the local machine, and sets some environment variables:

from System import *
from System.IO import *

def __main__( deadlinePlugin ):
    job = deadlinePlugin.GetJob()
    deadlinePlugin.LogInfo( "JobName: %s" % job.JobName )
    deadlinePlugin.LogInfo( "JobId: %s" % job.JobId )

    deadlinePlugin.LogInfo( "Copying some files" )
    File.Copy( r"\\server\files\file1.ext", r"C:\local\files\file1.ext", True )
    File.Copy( r"\\server\files\file2.ext", r"C:\local\files\file2.ext", True )

    deadlinePlugin.LogInfo( "Setting EnvVar1 to True" )
    deadlinePlugin.SetProcessEnvironmentVariable( "EnvVar1", "True" )

    deadlinePlugin.LogInfo( "Setting EnvVar2 to False" )
    deadlinePlugin.SetProcessEnvironmentVariable( "EnvVar2", "False" )

PluginPreLoad.py

Optional

The PluginPreLoad.py file is an optional script that will be executed by the Slave prior to executing any Python script for the plugin (MyPlugin.py or JobPreLoad.py), and any pre or post job or task script for the current job. It is placed in the <your_repo>/plugins/MyPlugin/ directory of your Repository, where “MyPlugin” is the name of the specific app plugin you wish the script to execute against. Note that in this case, the file does not share its name with the plugin folder. This script can be used to set up the Python environment prior to running any other Python script, including setting sys.path to control where additional modules will be loaded from.

The only requirement for the PluginPreLoad.py script is that you define a __main__ function, which is called by the Slave when it executes the script. It does not accept any parameters. Here is an example script that updates sys.path with custom paths:

import sys

def __main__():
    path = r"\\server\python"
    if path not in sys.path:
        sys.path.append( path )

GlobalJobPreLoad.py

Optional

The GlobalJobPreLoad.py file is a functionally identical script to a JobPreLoad.py script except that it is placed in the <your_repo>/plugins directory of your Repository and will execute for ALL Deadline app plugin types. Additionally, a GlobalJobPreLoad.py script file placed in <your_repo>/custom/plugins will override the same named script file if one is present in <your_repo>/plugins. Note, if present this GlobalJobPreLoad.py script will execute prior to any application plugin specific JobPreLoad.py script that might be present. Additionally, using this script entry point allows you to bootstrap your pipeline if required.

GlobalPluginPreLoad.py

Optional

The GlobalPluginPreLoad.py file is a functionally identical script to a PluginPreLoad.py script except that it is placed in the <your_repo>/plugins directory of your Repository and will execute for ALL Deadline app plugin types. Additionally, a GlobalPluginPreLoad.py script file placed in <your_repo>/custom/plugins will override the same named script file if one is present in <your_repo>/plugins. Note, if present this GlobalPluginPreLoad.py script will execute prior to any application plugin specific PluginPreLoad.py script that might be present.

Pipeline Bootstrapping

WARNING - This topic covers an advanced use of the Scripting API to bootstrap an existing studio pipeline.

In Deadline 9.0 an additional Deadline plugin callback: ModifyCommandLineCallback is available to allow studios with an existing pipeline system to prefix and modify inline all existing dispatcher commands to Deadline at either a per plugin level via a JobPreLoad.py file or for all jobs via a GlobalJobPreLoad.py file. To illustrate how this can work, in the next section below, we will create a GlobalJobPreLoad.py script which in-turn can call the studio’s theoretical pipeline_launch.csh or pipeline_launch.sh shell script (depending on OS) which will set up the current, working environment for a user or job in the context of the renderfarm. We will assume this is purely a Linux renderfarm setup. The ‘pipeline_launch’ shell script should be considered the ‘bootstrap’ entry point for a machine to resolve into a valid context before rendering via a Deadline plugin should commence. For example, this ‘pipeline_launch’ script could query a production tracking system or pipeline toolkit, returning the requested ‘project/shot’ context, which might set environment variables, control application executable paths, etc.

So, we will use the Deadline Scripting API and SHELL scripting to modify a standard Deadline render command when it picks up a task from a job from this:

"exe + args" #Simple Deadline Plugin (std basic render command involving an executable file)
"batch-process-exe + args" #Advanced Deadline Plugin (where application is held open between tasks)

to always be executed such as this:

# Load my pipeline, in context for the given project number and optional, shot number
source "/mnt/nfs/pipeline.csh" $proj_args:q #where $proj_args are env vars: $PROJECT=abcdef and optionally, $SHOT=123456

# Now execute the original render command given the now loaded project/shot context on the Deadline Slave
"exe + args" or "batch-process-exe + args"

Modify Cmd Line

Create a GlobalJobPreLoad.py file in <your_repo>/custom/plugins with the following contents:

GlobalJobPreLoad.py
import sys

from Deadline.Scripting import *
from FranticX.Diagnostics import *

def AlterCommandLine( executable, arguments, workingDirectory ):
    print "Entering Command Line Hook"

    scriptPrefix = None

    # depending on OS, we return global valid file path to the pipeline launch bootstrap script
    if sys.platform.startswith( 'linux' ):
        scriptPrefix = '/mnt/nfs/pipeline_launch.csh'
    elif sys.platform == 'darwin':
        scriptPrefix = '/Volumes/nfs/pipeline_launch.sh'
    else:
        Trace2.WriteLine( "Unrecognized platform '{}'. No prefix will be added.", sys.platform )

    if scriptPrefix:
        # Quotes the executable if it wasn't already quoted, to ensure it gets passed as a single arg
        if not executable.startswith( '"' ) and not executable.startswith( "'" ):
            executable = '"{}"'.format( executable )

        if arguments:
            arguments = executable + " " + arguments
        else:
            arguments = executable

        executable = scriptPrefix

        print "-----Modified Command Line-----"
        print " Executable: '{}'".format( executable )
        print "  Arguments: '{}'".format( arguments )
        print "Working Dir: '{}'".format( workingDirectory )
        print "-------------------------------"

    # return the original or modified exe, args, workingDir even if pipeline not available
    return (executable, arguments, workingDirectory)

def __main__( deadlinePlugin ):
    deadlinePlugin.DebugLogging = True # optional, can be commented out
    deadlinePlugin.ModifyCommandLineCallback += AlterCommandLine

Pipeline Launch

We will assume that ALL jobs submitted to your Deadline farm already contain the environment variable: PROJECT and optionally, SHOT.

Here we create a pipeline_launch.csh shell script at /mnt/nfs/pipeline_launch.csh to be executed when a Slave renders a task of any job it dequeues. This scipt will in turn, call your main Studio pipeline.csh shell script on Linux which controls the context for all your users for any project/job/shot they might be working on:

pipeline_launch.csh
#!/bin/tcsh

# Do any pre-job processing here (e.g. sourcing VFX studio environment)

# Debug Code
#echo "Environment prior to sourcing Pipeline-----------------"
#env
#echo "-------------------------------------------------------"

# Sanity checks to make sure expected environment is present.
if ( $?PROJECT ) then

    # Check if SHOT is present, and if so, append to project in args
    if ( $?SHOT ) then
        set proj_args = ( $proj_args:q "$SHOT" )
    else
        echo "WARNING: SHOT was not found in the environment."
    endif

    # Source the Project's environment.
    source "/mnt/nfs/pipeline.csh" $proj_args:q
    echo "Done sourcing environment!"
else
    echo "WARNING: PROJECT was not found in the environment."
endif

# Debug Code
#echo "Environment after sourcing Pipeline--------------------"
#env
#echo "-------------------------------------------------------"

echo "-----Running Command-----------------------------------"
# Run the original command -- note that $argv:q will quote the
# executable as well, which would be problematic for aliases
# and shell built-ins (but required for EXEs with spaces).
$argv:q
set EXIT_CODE=$?
echo "-------------------------------------------------------"

# Do any post-job processing here.
# Note that this might be after several Tasks have run,
# depending on the plugin.

# Exit with the code provided by the original command
exit $EXIT_CODE

Simple Plugins

A render job goes through three stages:

  • StartJob: A job enters this stage when it is first picked up by a Slave.
  • RenderTasks: A job can enter this stage many times (once for each task a Slave dequeues while it has the current job loaded).
  • EndJob: A job enters this stage when a Slave is unloading the job.

Simple plugins only covers the RenderTasks stage, and are pretty straight forward. They are commonly used to render with applications that support simple command line rendering (running a command line executable and waiting for it to complete). For example, After Effects has a command line renderer called aerender.exe, which can be executed by the Slave to render specific frames of an After Effects project file.

Initialization

By default, a plugin is considered to be a Simple plugin, but you can explicitly set this in the InitializeProcess() callback (as explained above). You can also define settings specific to the simple plugin, as well as any popup or stdout handlers that you need. These additional settings are covered in the ManagedProcess class in the Deadline Scripting reference (note that the DeadlinePlugin class inherits from the ManagedProcess class). For example:

from Deadline.Plugins import *

from System.Diagnostics import *

######################################################################
## This is the function that Deadline calls to get an instance of the
## main DeadlinePlugin class.
######################################################################
def GetDeadlinePlugin():
    return MyPlugin()

######################################################################
## This is the function that Deadline calls when the plugin is no
## longer in use so that it can get cleaned up.
######################################################################
def CleanupDeadlinePlugin( deadlinePlugin ):
    deadlinePlugin.Cleanup()

######################################################################
## This is the main DeadlinePlugin class for MyPlugin.
######################################################################
class MyPlugin (DeadlinePlugin):

    ## Hook up the callbacks in the constructor.
    def __init__( self ):
        self.InitializeProcessCallback += self.InitializeProcess

    ## Clean up the plugin.
    def Cleanup():
        # Clean up stdout handler callbacks.
        for stdoutHandler in self.StdoutHandlers:
            del stdoutHandler.HandleCallback

        del self.InitializeProcessCallback

    ## Called by Deadline to initialize the process.
    def InitializeProcess( self ):
        # Set the plugin specific settings.
        self.SingleFramesOnly = False
        self.PluginType = PluginType.Simple

        # Set the ManagedProcess specific settings.
        self.ProcessPriority = ProcessPriorityClass.BelowNormal
        self.UseProcessTree = True

        #StdoutHandling should be enabled if required in your plugin
        self.StdoutHandling = True

        #PopupHandling should be enabled if required in your plugin
        self.PopupHandling = True

        # Set the stdout handlers.
        self.AddStdoutHandlerCallback(
            "WARNING:.*" ).HandleCallback += self.HandleStdoutWarning
        self.AddStdoutHandlerCallback(
            "ERROR:(.*)" ).HandleCallback += self.HandleStdoutError

        # Set the popup ignorers.
        self.AddPopupIgnorer( "Popup 1" )
        self.AddPopupIgnorer( "Popup 2" )

        # Set the popup handlers.
        self.AddPopupHandler( "Popup 3", "OK" )
        self.AddPopupHandler( "Popup 4", "Do not ask me this again;Continue" )

    ## Callback for when a line of stdout contains a WARNING message.
    def HandleStdoutWarning( self ):
        self.LogWarning( self.GetRegexMatch(0) )

    ## Callback for when a line of stdout contains an ERROR message.
    def HandleStdoutError( self ):
        self.FailRender( "Detected an error: " + self.GetRegexMatch(1) )

Stdout Handlers

The AddStdoutHandlerCallback() function accepts a string parameter, which is a POSIX compliant regular expression used to match against lines of stdout from the command line process. This function also returns a RegexHandlerCallback instance, which you can hook up a callback to that is called when a line of stdout is matched. This can all be done on one line, which is shown in the example above.

Examples of handler callback functions are also shown in the example above. Within these handler functions, the GetRegexMatch() function can be used to get a specific match from the regular expression. The parameter passed to GetRegexMatch() is the index for the matches that were found. 0 returns the entire matched string, and 1, 2, etc returns the matched substrings (matches that are surrounded by round brackets). If there isn’t a corresponding substring, you’ll get an error (note that 0 is always a valid index).

In HandleStdoutWarning(), 0 is the only valid index because there is no substring in round brackets in the regular expression. In HandleStdoutError(), 0 and 1 are valid. 0 will return the entire matched string, whereas 1 will return the substring in the round brackets.

For further examples, please open up any of our application plugin Python script files and inspect them. An example of comprehensive Stdout handlers can be found in the MayaBatch plugin.

  • ../plugins/MayaBatch/MayaBatch.py

Note, that Deadline’s default shipping StdoutHandlers require the Slave’s Operating System to be using ENGLISH as it’s language.

Render Executable and Arguments

The RenderExecutable() callback is used to get the path to the executable that will be used for rendering. This callback must be implemented in a Simple plugin, or an error will occur. Continuing our example from above, we’ll use the path specified in the database, and we can access it using the global GetConfigEntry() function.

Another important (but optional) callback is the RenderArgument() callback. This callback should return the arguments you want to pass to the render executable. Typically, these arguments are built from values that are pulled from the DeadlinePlugin class (like the scene file name, or the start and end frame for the task), or from the plugin info file that was submitted with the job using the GetPluginInfoEntry() function. If this callback is not implemented, then no arguments will be passed to the executable.

After adding these callbacks, our example plugin script now looks like this:

from Deadline.Plugins import *

from System.Diagnostics import *

######################################################################
## This is the function that Deadline calls to get an instance of the
## main DeadlinePlugin class.
######################################################################
def GetDeadlinePlugin():
    return MyPlugin()

######################################################################
## This is the function that Deadline calls when the plugin is no
## longer in use so that it can get cleaned up.
######################################################################
def CleanupDeadlinePlugin( deadlinePlugin ):
    deadlinePlugin.Cleanup()

######################################################################
## This is the main DeadlinePlugin class for MyPlugin.
######################################################################
class MyPlugin (DeadlinePlugin):

    ## Hook up the callbacks in the constructor.
    def __init__( self ):
        self.InitializeProcessCallback += self.InitializeProcess
        self.RenderExecutableCallback += self.RenderExecutable
        self.RenderArgumentCallback += self.RenderArgument

    ## Clean up the plugin.
    def Cleanup():
        # Clean up stdout handler callbacks.
        for stdoutHandler in self.StdoutHandlers:
            del stdoutHandler.HandleCallback

        del self.InitializeProcessCallback
        del self.RenderExecutableCallback
        del self.RenderArgumentCallback

    ## Called by Deadline to initialize the process.
    def InitializeProcess( self ):
        # Set the plugin specific settings.
        self.SingleFramesOnly = False
        self.PluginType = PluginType.Simple

        # Set the ManagedProcess specific settings.
        self.ProcessPriority = ProcessPriorityClass.BelowNormal
        self.UseProcessTree = True
        self.StdoutHandling = True
        self.PopupHandling = True

        # Set the stdout handlers.
        self.AddStdoutHandlerCallback(
            "WARNING:.*" ).HandleCallback += self.HandleStdoutWarning
        self.AddStdoutHandlerCallback(
            "ERROR:(.*)" ).HandleCallback += self.HandleStdoutError

        # Set the popup ignorers.
        self.AddPopupIgnorer( "Popup 1" )
        self.AddPopupIgnorer( "Popup 2" )

        # Set the popup handlers.
        self.AddPopupHandler( "Popup 3", "OK" )
        self.AddPopupHandler( "Popup 4", "Do not ask me this again;Continue" )

    ## Callback for when a line of stdout contains a WARNING message.
    def HandleStdoutWarning( self ):
        self.LogWarning( self.GetRegexMatch(0) )

    ## Callback for when a line of stdout contains an ERROR message.
    def HandleStdoutError( self ):
        self.FailRender( "Detected an error: " + self.GetRegexMatch(1) )

    ## Callback to get the executable used for rendering.
    def RenderExecutable( self ):
        return self.GetConfigEntry( "MyPluginRenderExecutable" )

    ## Callback to get the arguments that will be passed to the executable.
    def RenderArgument( self ):
        arguments = " -continueOnError"
        arguments += " -verbose " + self.GetPluginInfoEntry( "Verbose" )
        arguments += " -start " + str(self.GetStartFrame())
        arguments += " -end " + str(self.GetEndFrame())
        arguments += " -scene \"" + self.GetDataFilename() + "\""
        return arguments

There are many other callbacks that can be implemented for Simple plugins, which are covered in the Events section for the ManagedProcess class in the Deadline Scripting reference. The best place to find examples of Simple plugins is to look at some of the plugins that are shipped with Deadline. These range from the very basic (Blender), to the more complex (MayaCmd).

Advanced Plugins

To reiterate, a render job goes through three stages:

  • StartJob: A job enters this stage when it is first picked up by a Slave.
  • RenderTasks: A job can enter this stage many times (once for each task a Slave dequeues while it has the current job loaded).
  • EndJob: A job enters this stage when a Slave is unloading the job.

Advanced plugins are more complex, as they control all three of these job stages. They are commonly used to render with applications that support some sort of slave/server mode that Deadline can interact with. Usually, this requires the application to be started during the StartJob phase, fed commands during the RenderTasks stage(s), and finally shut down during the EndJob stage. For example, the 3ds Max plugin starts up 3dsmax in slave mode and forces it to load our Lightning plugin. The Lightning plugin listens for commands from Deadline and executes them as necessary. After rendering is complete, 3ds Max is shut down.

Initialization

To indicate that your plugin is an Advanced plugin, you need to set the PluginType property in the InitializeProcess() callback.

from Deadline.Plugins import *

######################################################################
## This is the function that Deadline calls to get an instance of the
## main DeadlinePlugin class.
######################################################################
def GetDeadlinePlugin():
    return MyPlugin()

######################################################################
## This is the function that Deadline calls when the plugin is no
## longer in use so that it can get cleaned up.
######################################################################
def CleanupDeadlinePlugin( deadlinePlugin ):
    deadlinePlugin.Cleanup()

######################################################################
## This is the main DeadlinePlugin class for MyPlugin.
######################################################################
class MyPlugin (DeadlinePlugin):

    ## Hook up the callbacks in the constructor.
    def __init__( self ):
        self.InitializeProcessCallback += self.InitializeProcess

    ## Clean up the plugin.
    def Cleanup():
        del self.InitializeProcessCallback

    ## Called by Deadline to initialize the process.
    def InitializeProcess( self ):
        # Set the plugin specific settings.
        self.SingleFramesOnly = False
        self.PluginType = PluginType.Advanced

Render Tasks

The RenderTasks() callback is the only required callback for Advanced plugins. If it is not implemented, an error will occur. It contains the code to be executed for each task that a Slave renders. This could involve launching applications, communicating with already running applications, or simply running a script to automate a particular task (like backing up a group of files).

Other common callbacks for Advanced plugins are the StartJob() and EndJob() callbacks. The StartJob() callback can be used to start up an application, or to set some local variables that will be used in other callbacks. If the StartJob() callback is not implemented, then nothing is done during the StartJob phase. The EndJob() callback can be used to shut down a running application, or to clean up temporary files. If the EndJob() callback is not implemented, then nothing is done during the EndJob phase.

In the example below, we will be launching our application during the StartJob phase. The benefit to this is that the application can be left running during the duration of the job, which eliminates the overhead of having to launch the application for each task. To launch and monitor the application, we will be implementing a ManagedProcess class, and calling it MyPluginProcess .This ManagedProcess class will define the render executable and command line arguments for launching the process we will be monitoring. Note that we aren’t passing it any frame information, as this needs to be handled in the RenderTasks() callback when it interacts with the process.

After adding these three callbacks, and the MyPluginProcess class, our example code looks like this. Note that the RenderTasks() callback still needs code to allow it to interact with the running process launched in the StartJob() callback.

from Deadline.Plugins import *

######################################################################
## This is the function that Deadline calls to get an instance of the
## main DeadlinePlugin class.
######################################################################
def GetDeadlinePlugin():
    return MyPlugin()

######################################################################
## This is the function that Deadline calls when the plugin is no
## longer in use so that it can get cleaned up.
######################################################################
def CleanupDeadlinePlugin( deadlinePlugin ):
    deadlinePlugin.Cleanup()

######################################################################
## This is the main DeadlinePlugin class for MyPlugin.
######################################################################
class MyPlugin (DeadlinePlugin):

    ## Variable to hold the Managed Process object.
    Process = None

    ## Hook up the callbacks in the constructor.
    def __init__( self ):
        self.InitializeProcessCallback += self.InitializeProcess
        self.StartJobCallback += self.StartJob
        self.RenderTasksCallback += self.RenderTasks
        self.EndJobCallback += self.EndJob

    ## Clean up the plugin.
    def Cleanup():
        del self.InitializeProcessCallback
        del self.StartJobCallback
        del self.RenderTasksCallback
        del self.EndJobCallback

        # Clean up the managed process object.
        if self.Process:
            self.Process.Cleanup()
            del self.Process

    ## Called by Deadline to initialize the process.
    def InitializeProcess( self ):
        # Set the plugin specific settings.
        self.SingleFramesOnly = False
        self.PluginType = PluginType.Advanced

    ## Called by Deadline when the job starts.
    def StartJob( self ):
        myProcess = MyPluginProcess()
        StartMonitoredManagedProcess( "My Process", myProcess )

    ## Called by Deadline for each task the Slave renders.
    def RenderTasks( self ):
        # Do something to interact with the running process.
        pass

    ## Called by Deadline when the job ends.
    def EndJob( self ):
        ShutdownMonitoredManagedProcess( "My Process" )

######################################################################
## This is the ManagedProcess class that is launched above.
######################################################################
class MyPluginProcess (ManagedProcess):
    deadlinePlugin = None

    ## Hook up the callbacks in the constructor.
    def __init__( self, deadlinePlugin ):
        self.InitializeProcessCallback += self.InitializeProcess
        self.RenderExecutableCallback += self.RenderExecutable
        self.RenderArgumentCallback += self.RenderArgument

    ## Clean up the managed process.
    def Cleanup():
        # Clean up stdout handler callbacks.
        for stdoutHandler in self.StdoutHandlers:
            del stdoutHandler.HandleCallback

        del self.InitializeProcessCallback
        del self.RenderExecutableCallback
        del self.RenderArgumentCallback

    ## Called by Deadline to initialize the process.
    def InitializeProcess( self ):
        # Set the ManagedProcess specific settings.
        self.ProcessPriority = ProcessPriorityClass.BelowNormal
        self.UseProcessTree = True
        self.StdoutHandling = True
        self.PopupHandling = True

        # Set the stdout handlers.
        self.AddStdoutHandlerCallback(
            "WARNING:.*" ).HandleCallback += self.HandleStdoutWarning
        self.AddStdoutHandlerCallback(
            "ERROR:(.*)" ).HandleCallback += self.HandleStdoutError

        # Set the popup ignorers.
        self.AddPopupIgnorer( "Popup 1" )
        self.AddPopupIgnorer( "Popup 2" )

        # Set the popup handlers.
        self.AddPopupHandler( "Popup 3", "OK" )
        self.AddPopupHandler( "Popup 4", "Do not ask me this again;Continue" )

    ## Callback for when a line of stdout contains a WARNING message.
    def HandleStdoutWarning( self ):
        self.deadlinePlugin.LogWarning( self.GetRegexMatch(0) )

    ## Callback for when a line of stdout contains an ERROR message.
    def HandleStdoutError( self ):
        self.deadlinePlugin.FailRender( "Detected an error: " + self.GetRegexMatch(1) )

    ## Callback to get the executable used for rendering.
    def RenderExecutable( self ):
        return self.deadlinePlugin.GetConfigEntry( "MyPluginRenderExecutable" )

    ## Callback to get the arguments that will be passed to the executable.
    def RenderArgument( self ):
        arguments = " -verbose " + self.deadlinePlugin.GetPluginInfoEntry( "Verbose" )
        arguments += " -scene \"" + self.deadlinePlugin.GetDataFilename() + "\""
        return arguments

Because the Advanced plugins are much more complex than the Simple plugins, we recommend taking a look at the following plugins that are shipped with Deadline for examples:

  • 3dsmax
  • Fusion
  • Lightwave
  • MayaBatch
  • Modo
  • Nuke
  • SoftimageBatch

Migrating Plugins from Deadline 5

Some changes were made to the Scripting API in Deadline 6, which means that Deadline 6 and later are NOT backward compatible with plugin scripts written for Deadline 5. However, migrating your scripts over is relatively straightforward, and this guide will walk you through the API changes so that you can update your scripts as necessary.

Global Functions

In Deadline 6, all global API functions were removed, and replaced with DeadlinePlugin member functions, or with static utility functions. See the Migrating Scripts From Deadline 5 section in the Scripting Overview documentation for more information, including replacement functions.

Almost all plugin-specific global functions are now DeadlinePlugin member functions. For example, the global ‘LogInfo( message )’ function has been replaced with a member function for the DeadlinePlugin class, which you created in your event Python file. So instead of:

LogInfo( "this is a test message" )

You would use this code:

self.LogInfo( "this is a test message" )

The only functions that aren’t DeadlinePlugin member functions are listed below, along with their replacement utility functions.

Original Global Function Replacement Function
CheckPathMapping( path ) RepositoryUtils.CheckPathMapping( path )
CheckPathMappingInFile( inFileName, outFileName ) RepositoryUtils.CheckPathMappingInFile( inFileName, outFileName )
CheckPathMappingInFileAndReplaceSeparator( inFileName, outFileName , separatorToReplace, newSeparator ) RepositoryUtils.CheckPathMappingInFileAndReplaceSeparator( inFileName, outFileName, separatorToReplace, newSeparator )
PathMappingRequired( path ) RepositoryUtils.PathMappingRequired( path )

Callbacks

You need to set up callbacks in the constructor of your DeadlinePlugin class that you created in your plugin Python file. Examples are shown in the documentation above, and you can look at the plugins that ship with Deadline for references as well. For example:

def __init__( self ):
    self.InitializeProcessCallback += self.InitializeProcess
    self.RenderExecutableCallback += self.RenderExecutable
    self.RenderArgumentCallback += self.RenderArgument
    self.PreRenderTasksCallback += self.PreRenderTasks
    self.PostRenderTasksCallback += self.PostRenderTasks

Note that these callbacks need to be manually cleaned up when the plugin is no longer in use. See the documentation regarding the CleanupDeadlinePlugin function above for more information.