Application Plugins¶

Overview¶

All of Deadline’s plugins are written in Python, which means that it’s easy to create your own plugins or customize the existing ones. See the Scripting Overview documentation for more information, and links to the Deadline Scripting reference.

Note that because the Python scripts for application plugins will be executed in a non-interactive way, it is important that your scripts do not contain any blocking operations like infinite loops, or interfaces that require user input.

When a plugin is loaded the log will show where the plugin is being loaded from.

General Plugin Information¶

There are two types of plugins that can be created:

Simple

Advanced

Simple plugins provide the basics to wrap a command line application, and is typically used to build up command line arguments to pass to the application. Advanced plugins provide more control, and are typically used when running a simple command line application isn’t enough. Other than the plugin Python script itself though, Simple and Advanced plugins are very similar.

Creating a New Plugin¶

This section covers the the areas that Simple and Advanced plugins have in common. Specifics for Simple and Advanced plugins are covered later on.

To create a new plugin, start by creating a folder in the Repository’s custom\plugins folder and give it the name of your plugin. See the Scripting Overview documentation for more information on the ‘custom’ folder in the Repository and how it’s used.

For the sake of this document, we will call our new plugin MyPlugin. All relative script and configuration files for this plugin are to be placed in this plugin’s folder (some are required and some are optional).

.py File¶

Required

The MyPlugin.py is the main plugin script file. It defines the main DeadlinePlugin class that contains the necessary code that Deadline uses to render a job. This is where Simple and Advanced plugins will differ, and the specifics for each can be found later on, but the template for this script file might look like this:

#!/usr/bin/env python3
from Deadline.Plugins import DeadlinePlugin

def GetDeadlinePlugin():
    """This is the function that Deadline calls to get an instance of the
    main DeadlinePlugin class.
    """
    return MyPlugin()

def CleanupDeadlinePlugin(deadlinePlugin):
    """This is the function that Deadline calls when the plugin is no
    longer in use so that it can get cleaned up.
    """
  deadlinePlugin.Cleanup()


class MyPlugin(DeadlinePlugin):
    """This is the main DeadlinePlugin class for MyPlugin."""
    # TODO: Place code here instead of "pass"
    pass

The first thing to note is that we’re importing the Deadline.Plugins namespace so that we can access the DeadlinePlugin class.

The GetDeadlinePlugin() function is important, as it allows the Worker to get an instance of our MyPlugin class (which is extending the abstract DeadlinePlugin class). In Deadline 6.2 and later, the GetDeadlinePluginWithJob(job) function can be defined as an alternative. It works the same as GetDeadlinePlugin(), except that it accepts an instance of the Job object that the plugin is being loaded for. If either of these functions are not defined, the Worker will report an error when it tries to render the job.

The MyPlugin class will need to implement certain callbacks based on the type of plugin it is, and these callbacks must be hooked up in the MyPlugin constructor. One callback that all plugins should implement is the InitializeProcess function. There are many other callbacks that can be implemented, which are covered in the Events section for the DeadlinePlugin class in the Deadline Scripting reference.

The CleanupDeadlinePlugin() function is also important, as it is necessary to clean up the plugin when it is no longer in use. Typically, this is used to clean up any callbacks that were created when the plugin was initialized.

To start off, the InitializeProcess callback is typically used to set some general plugin settings:

#!/usr/bin/env python3
from Deadline.Plugins import DeadlinePlugin

def GetDeadlinePlugin():
    """This is the function that Deadline calls to get an instance of the
    main DeadlinePlugin class.
    """
    return MyPlugin()

def CleanupDeadlinePlugin(deadlinePlugin):
    """This is the function that Deadline calls when the plugin is no
    longer in use so that it can get cleaned up.
    """
    deadlinePlugin.Cleanup()

class MyPlugin(DeadlinePlugin):
    """This is the main DeadlinePlugin class for MyPlugin."""
    def __init__(self):
        """Hook up the callbacks in the constructor."""
        self.InitializeProcessCallback += self.InitializeProcess

    def Cleanup(self):
        """Clean up the plugin."""
        del self.InitializeProcessCallback

    def InitializeProcess(self):
        """Called by Deadline to initialize the plugin."""
        # Set the plugin specific settings.
        self.SingleFramesOnly = False
        self.PluginType = PluginType.Simple

These are the common plugin properties that can be set in InitializeProcess callback. See the DeadlinePlugin class in the Deadline Scripting reference for additional properties.

Property	Description
PluginType	The type of plugin this is (PluginType.Simple/PluginType.Advanced).
SingleFramesOnly	Set to True or False. Set to True if your plugin can only work on one frame at a time, rather than a frame sequence.

.param File¶

Required

The MyPlugin.param file is a required file that is used to declare the custom settings for the plugin and by the Plugin Configuration dialog in the Monitor to set these custom settings. It declares properties that the Monitor uses to generate a user interface for modifying custom settings in the database.

Each control is identified in the MyPlugin.param file by a unique name between square brackets, and a set of key=value pairs that determine its default value and how it appears in Deadline’s user interface. For example, a typical plugin defines a read-only “About Plugin” section, which contains a description of the plugin and indicates whether or not the plugin supports rendering concurrent tasks.

[About]
Type=label
Label=About
Category=About Plugin
CategoryOrder=-1
Default=An example of a custom Plugin
Description=Not configurable

[ConcurrentTasks]
Type=label
Label=ConcurrentTasks
Category=About Plugin
CategoryOrder=-1
Default=True
Description=Not configurable

We can add a new custom option to identify a render executable for MyPlugin. The property name between the square brackets will correspond to the custom setting to be defined in the database. This means that the control defined below will change the MyPluginRenderExecutable setting. Here, we specify Type=filename to indicate that this setting should be treated by Deadline as a path to a file.

[MyPluginRenderExecutable]
Type=filename
Label=My Plugin Render Executable
Default=c:\path\to\my\executable.exe
Description=The path to the executable file used for rendering.

Note that in previous versions of Deadline, Plugin settings were stored in dlinit files, which have been removed as of Deadline 8.0. All data that was previously stored in the dlinit file is now stored in the database. In order to declare custom settings for a plugin you must do so in the param file.

After you’ve created the MyPlugin.param file, open the Monitor and enter Super User mode. Then select Tools -> Configure Plugins and look for your plugin in the list on the left.

../_images/configure_plugin_myplugin.png

You can use this area to modify the values of the plugin’s parameters. Any changes you make to values here will be stored in the database when you press the OK button. You may also change the plugin’s icon and globally Set Limits for the application plugin using the options at the bottom. Clicking the Restore Default Icon button will change the icon back to the plugin’s ico file. You can also Enable or Disable a Plugin using the Enabled checkbox. Note that Plugins that are not Enabled will not be able to render.

Comment lines are supported in the param file, and must start with either ‘;’ or ‘#’. For example:

# This is the file name picker control to set the executable for this plugin.
[MyPluginRenderExecutable]
Type=filename
Label=My Plugin Render Executable
Default=c:\path\to\my\executable.exe
Description=The path to the executable file used for rendering.

The available key=value pairs for the properties defined here are:

Key Name	Description
Category	The category the control should go under.
CategoryIndex	This determines the control’s order under its category. This does the same thing as Index.
CategoryOrder	This determines the category’s order among other categories. If more than one CategoryOrder is defined for the same category, the lowest value is used.
Default	The default value to be used if this property is not defined in the database. This does the same thing as DefaultValue.
DefaultValue	The default value to be used if this property is not defined in the database file. This does the same thing as Default.
Description	A short description of the property the control is for (displayed as a tooltip in the UI).
DisableIfBlank	If True, a control will not be shown if this property is not defined in the param/options file (True/False). This does the same thing as IgnoreIfBlank.
IgnoreIfBlank	If True, a control will not be shown if this property is not defined in the param/options file (True/False). This does the same thing as DisableIfBlank.
Index	This determines the control’s order under its category. This does the same thing as CategoryIndex.
Label	The control label.
Required	If True, a control will be shown for this property even if it’s not defined in the database (True/False).
Type	The type of control (see table below).
IsServerData	Whether or not the value of this section should be stored as server data.

These are the available controls.

Control Type	Description
Boolean	A drop-down control that allows the selection of True or False.
Color	Allows the selection of a color.
Enum	A drop-down control that allows the selection of an item from a list.
Enumeration	Same as Enum above.
Filename	Allows the selection of an existing file.
FilenameSave	Allows the selection of a new or existing file.
Float	An floating point spinner control.
Folder	Allows the selection of an existing folder.
Integer	An integer spinner control.
Label	A read-only text field.
MultiFilename	Allows the selection of multiple existing files, which are then separated by semicolons in the text field.
MultiLineMultiFilename	Allows the selection of multiple existing files, which are then placed on multiple lines in the text field.
MultiLineMultiFolder	Allows the selection of multiple existing folders, which are then placed on multiple lines in the text field.
MultiLineString	A text field with multiple lines.
Password	A text field that masks the text (Note: this is not usable in .options files). If using the Deadline Secrets Management feature, any parameter with the Control Type of Password will be saved and retrieved based on the Application Plugin Secrets Access Level.
SlaveList	Allows the selection of existing Workers, which are then separated by commas in the text field.
String	A text field.

There are also key/value pairs for specific controls:

Key Name	Description
DecimalPlaces	The number of decimal places for the Float controls.
Filter	The filter string for the Color, Filename, FilenameSave, or MultiFilename controls. Examples: `Filter=Windows Bitmap (.bmp);;Targa (.tga);;JPEG (.jpg .jpeg);;PCX (.pcx);;PNG (.png);;TIFF (.tif .tiff);;All files (*)` or `Filter=(color %R %G %B)`.
Increment	The value to increment the Integer or Float controls by.
Items	The semicolon separated list of items for the Enum control. This does the same thing as Values.
Maximum	The maximum value for the Integer or Float controls.
Minimum	The minimum value for the Integer or Float controls.
Validator	A regular expression for the String control that is used to ensure the value is valid. Examples: `Validator=\w` matches any word character; equal to `[a-zA-Z0-9_]` or `Validator=\d` matches any digit; equal to `[0-9]`.
Values	The semicolon separated list of items for the Enum control. This does the same thing as Items.

Server Data¶

When an item is marked as server data, it is stored differently than other data. The purpose of this is to reduce data transfer in cases where a lot of data needs to be stored as configuration data, but needs to be accessed less frequently than the rest of the config. In this case the data can be retrieved as follows: 1) Call var myServerData = RepositoryUtils.GetServerData("plugin.mypluginname") 2) Call myServerData["sectionname"]

.options File¶

Optional

The MyPlugin.options file is an optional file that is used by the Job Properties dialog in the Monitor. It declares properties that the Monitor uses to generate a user interface for modifying plugin specific options as they appear in the plugin info file that was submitted with the job. After you’ve created this file, you can right-click on a job in the Monitor that uses this plugin and select Modify Properties. You should then see a MyPlugin page at the bottom of the list on the left which you can select to view these properties.

Often, these plugin specific options are used to build up the arguments to be passed to the rendering application. Let’s assume that our render executable takes a “-verbose” argument that accepts a boolean parameter, and that the plugin info file submitted with the job contains the following:

Verbose=True

Now we would like to be able to change this value from the Job Properties dialog in the Monitor, so our MyPlugin.options file might look like this:

[Verbose]
Type=boolean
Label=Verbose Logging
Description=If verbose logging is enabled.
Required=true
DisableIfBlank=false
DefaultValue=True

You’ll notice that the property name between the square brackets matches the Verbose setting in our plugin info file. This means that this control will change the Verbose setting. The available key=value pairs for the properties defined here are the same as those defined for the param file above (with the exception of the Password control which is param file only). Comment lines are also supported in the options file in the same way they are supported in the param file.

.ico File¶

Optional

The MyPlugin.ico file is an optional 16x16 icon file that can be used to easily identify jobs that use this plugin in the Monitor. Typically, it is the plugin application’s logo, or something else that represents the plugin. If a plugin does not have an icon file, a generic icon will be shown in the jobs list in the Monitor.

JobPreLoad.py¶

Optional

The JobPreLoad.py file is an optional script that will be executed by the Worker prior to loading a job that uses this plugin. It is placed in the <your_repo>/plugins/MyPlugin/ directory of your Repository, where “MyPlugin” is the name of the specific app plugin you wish the script to execute against. Note that in this case, the file does not share its name with the plugin folder. This script can be used to do things like synchronize plugins prior to starting the render job or load a set of environment variables which should be inherited by the render process when it is executed by the DeadlinePlugin -> ManagedProcess class.

The only requirement for the JobPreLoad.py script is that you define a __main__ function, which is called by the Worker when it executes the script. It must accept a single parameter, which is the current instance of the DeadlinePlugin class. Here is an example script that returns the JobId and JobName, copies a couple of files from a server to the local machine, and sets some environment variables:

#!/usr/bin/env python3
from System import *
from System.IO import *

def __main__(deadlinePlugin):
    job = deadlinePlugin.GetJob()
    deadlinePlugin.LogInfo("JobName: %s" % job.JobName)
    deadlinePlugin.LogInfo("JobId: %s" % job.JobId)

    deadlinePlugin.LogInfo("Copying some files")
    File.Copy(r"\\server\files\file1.ext", r"C:\local\files\file1.ext", True)
    File.Copy(r"\\server\files\file2.ext", r"C:\local\files\file2.ext", True)

    deadlinePlugin.LogInfo("Setting EnvVar1 to True")
    deadlinePlugin.SetProcessEnvironmentVariable("EnvVar1", "True")

    deadlinePlugin.LogInfo("Setting EnvVar2 to False")
    deadlinePlugin.SetProcessEnvironmentVariable("EnvVar2", "False")

PluginPreLoad.py¶

Optional

The PluginPreLoad.py file is an optional script that will be executed by the Worker prior to executing any Python script for the plugin (MyPlugin.py or JobPreLoad.py), and any pre or post job or task script for the current job. It is placed in the <your_repo>/plugins/MyPlugin/ directory of your Repository, where “MyPlugin” is the name of the specific app plugin you wish the script to execute against. Note that in this case, the file does not share its name with the plugin folder. This script can be used to set up the Python environment prior to running any other Python script, including setting sys.path to control where additional modules will be loaded from.

The only requirement for the PluginPreLoad.py script is that you define a __main__ function, which is called by the Worker when it executes the script. It does not accept any parameters. Here is an example script that updates sys.path with custom paths:

#!/usr/bin/env python3
import sys

def __main__():
    path = r"\\server\python"
    if path not in sys.path:
        sys.path.append(path)

GlobalJobPreLoad.py¶

Optional

The GlobalJobPreLoad.py file is a functionally identical script to a JobPreLoad.py script except that it is placed in the <your_repo>/plugins directory of your Repository and will execute for ALL Deadline app plugin types. Additionally, a GlobalJobPreLoad.py script file placed in <your_repo>/custom/plugins will override the same named script file if one is present in <your_repo>/plugins. Note, if present this GlobalJobPreLoad.py script will execute prior to any application plugin specific JobPreLoad.py script that might be present. Additionally, using this script entry point allows you to bootstrap your pipeline if required.

GlobalPluginPreLoad.py¶

Optional

The GlobalPluginPreLoad.py file is a functionally identical script to a PluginPreLoad.py script except that it is placed in the <your_repo>/plugins directory of your Repository and will execute for ALL Deadline app plugin types. Additionally, a GlobalPluginPreLoad.py script file placed in <your_repo>/custom/plugins will override the same named script file if one is present in <your_repo>/plugins. Note, if present this GlobalPluginPreLoad.py script will execute prior to any application plugin specific PluginPreLoad.py script that might be present.

Pipeline Bootstrapping¶

Note

This topic covers an advanced use of the Scripting API to bootstrap an existing studio pipeline.

In Deadline 9.0 an additional Deadline plugin callback: ModifyCommandLineCallback is available to allow studios with an existing pipeline system to prefix and modify inline all existing dispatcher commands to Deadline at either a per plugin level via a JobPreLoad.py file or for all jobs via a GlobalJobPreLoad.py file. To illustrate how this can work, in the next section below, we will create a GlobalJobPreLoad.py script which in-turn can call the studio’s theoretical pipeline_launch.csh or pipeline_launch.sh shell script (depending on OS) which will set up the current, working environment for a user or job in the context of the renderfarm. We will assume this is purely a Linux renderfarm setup. The ‘pipeline_launch’ shell script should be considered the ‘bootstrap’ entry point for a machine to resolve into a valid context before rendering via a Deadline plugin should commence. For example, this ‘pipeline_launch’ script could query a production tracking system or pipeline toolkit, returning the requested ‘project/shot’ context, which might set environment variables, control application executable paths, etc.

So, we will use the Deadline Scripting API and SHELL scripting to modify a standard Deadline render command when it picks up a task from a job from this:

"exe + args" #Simple Deadline Plugin (std basic render command involving an executable file)
"batch-process-exe + args" #Advanced Deadline Plugin (where application is held open between tasks)

to always be executed such as this:

# Load my pipeline, in context for the given project number and optional, shot number
source "/mnt/nfs/pipeline.csh" $proj_args:q #where $proj_args are env vars: $PROJECT=abcdef and optionally, $SHOT=123456

# Now execute the original render command given the now loaded project/shot context on the Deadline Worker
"exe + args" or "batch-process-exe + args"

Modify Cmd Line¶

Create a GlobalJobPreLoad.py file in <your_repo>/custom/plugins with the following contents:

GlobalJobPreLoad.py¶

#!/usr/bin/env python3
import sys

from Deadline.Scripting import *
from FranticX.Diagnostics import *

def AlterCommandLine(executable, arguments, workingDirectory):
    print("Entering Command Line Hook")

    scriptPrefix = None

    # depending on OS, we return global valid file path to the pipeline launch bootstrap script
    if sys.platform.startswith('linux'):
        scriptPrefix = '/mnt/nfs/pipeline_launch.csh'
    elif sys.platform == 'darwin':
        scriptPrefix = '/Volumes/nfs/pipeline_launch.sh'
    else:
        Trace2.WriteLine("Unrecognized platform '{}'. No prefix will be added.", sys.platform)

    if scriptPrefix:
        # Quotes the executable if it wasn't already quoted, to ensure it gets passed as a single arg
        if not executable.startswith('"') and not executable.startswith("'"):
            executable = '"{}"'.format(executable)

        if arguments:
            arguments = executable + " " + arguments
        else:
            arguments = executable

        executable = scriptPrefix

        print("-----Modified Command Line-----")
        print(" Executable: '{}'".format(executable))
        print("  Arguments: '{}'".format(arguments))
        print("Working Dir: '{}'".format(workingDirectory))
        print("-------------------------------")

    # return the original or modified exe, args, workingDir even if pipeline not available
    return (executable, arguments, workingDirectory)

def __main__(deadlinePlugin):
    deadlinePlugin.DebugLogging = True # optional, can be commented out
    deadlinePlugin.ModifyCommandLineCallback += AlterCommandLine

Pipeline Launch¶

We will assume that ALL jobs submitted to your Deadline farm already contain the environment variable: PROJECT and optionally, SHOT.

Here we create a pipeline_launch.csh shell script at /mnt/nfs/pipeline_launch.csh to be executed when a Worker renders a task of any job it dequeues. This scipt will in turn, call your main Studio pipeline.csh shell script on Linux which controls the context for all your users for any project/job/shot they might be working on:

pipeline_launch.csh¶

#!/bin/tcsh

# Do any pre-job processing here (e.g. sourcing VFX studio environment)

# Debug Code
#echo "Environment prior to sourcing Pipeline-----------------"
#env
#echo "-------------------------------------------------------"

# Sanity checks to make sure expected environment is present.
if ( $?PROJECT ) then

    # Check if SHOT is present, and if so, append to project in args
    if ( $?SHOT ) then
        set proj_args = ( $proj_args:q "$SHOT" )
    else
        echo "WARNING: SHOT was not found in the environment."
    endif

    # Source the Project's environment.
    source "/mnt/nfs/pipeline.csh" $proj_args:q
    echo "Done sourcing environment!"
else
    echo "WARNING: PROJECT was not found in the environment."
endif

# Debug Code
#echo "Environment after sourcing Pipeline--------------------"
#env
#echo "-------------------------------------------------------"

echo "-----Running Command-----------------------------------"
# Run the original command -- note that $argv:q will quote the
# executable as well, which would be problematic for aliases
# and shell built-ins (but required for EXEs with spaces).
$argv:q
set EXIT_CODE=$?
echo "-------------------------------------------------------"

# Do any post-job processing here.
# Note that this might be after several Tasks have run,
# depending on the plugin.

# Exit with the code provided by the original command
exit $EXIT_CODE

Simple Plugins¶

A render job goes through three stages:

StartJob: A job enters this stage when it is first picked up by a Worker.

RenderTasks: A job can enter this stage many times (once for each task a Worker dequeues while it has the current job loaded).

EndJob: A job enters this stage when a Worker is unloading the job.

Simple plugins only covers the RenderTasks stage, and are pretty straight forward. They are commonly used to render with applications that support simple command line rendering (running a command line executable and waiting for it to complete). For example, After Effects has a command line renderer called aerender.exe, which can be executed by the Worker to render specific frames of an After Effects project file.

Initialization¶

By default, a plugin is considered to be a Simple plugin, but you can explicitly set this in the InitializeProcess() callback (as explained above). You can also define settings specific to the simple plugin, as well as any popup or stdout handlers that you need. These additional settings are covered in the ManagedProcess class in the Deadline Scripting reference (note that the DeadlinePlugin class inherits from the ManagedProcess class). For example:

#!/usr/bin/env python3
from Deadline.Plugins import *

from System.Diagnostics import *

def GetDeadlinePlugin():
    """This is the function that Deadline calls to get an instance of the
    main DeadlinePlugin class.
    """
    return MyPlugin()

def CleanupDeadlinePlugin(deadlinePlugin):
    """This is the function that Deadline calls when the plugin is no
    longer in use so that it can get cleaned up.
    """
    deadlinePlugin.Cleanup()

class MyPlugin(DeadlinePlugin):
    """This is the main DeadlinePlugin class for MyPlugin."""
    def __init__(self):
        """Hook up the callbacks in the constructor."""
        self.InitializeProcessCallback += self.InitializeProcess

    def Cleanup(self):
        """Clean up the plugin."""
        # Clean up stdout handler callbacks.
        for stdoutHandler in self.StdoutHandlers:
            del stdoutHandler.HandleCallback

        del self.InitializeProcessCallback

    def InitializeProcess(self):
        """Called by Deadline to initialize the process."""
        # Set the plugin specific settings.
        self.SingleFramesOnly = False
        self.PluginType = PluginType.Simple

        # Set the ManagedProcess specific settings.
        self.ProcessPriority = ProcessPriorityClass.BelowNormal
        self.UseProcessTree = True

        #StdoutHandling should be enabled if required in your plugin
        self.StdoutHandling = True

        #PopupHandling should be enabled if required in your plugin
        self.PopupHandling = True

        # Set the stdout handlers.
        self.AddStdoutHandlerCallback(
            "WARNING:.*").HandleCallback += self.HandleStdoutWarning
        self.AddStdoutHandlerCallback(
            "ERROR:(.*)").HandleCallback += self.HandleStdoutError

        # Set the popup ignorers.
        self.AddPopupIgnorer("Popup 1")
        self.AddPopupIgnorer("Popup 2")

        # Set the popup handlers.
        self.AddPopupHandler("Popup 3", "OK")
        self.AddPopupHandler("Popup 4", "Do not ask me this again;Continue")

    def HandleStdoutWarning(self):
        """Callback for when a line of stdout contains a WARNING message."""
        self.LogWarning(self.GetRegexMatch(0))

    def HandleStdoutError(self):
        """Callback for when a line of stdout contains an ERROR message."""
        self.FailRender("Detected an error: " + self.GetRegexMatch(1))

Stdout Handlers¶

The AddStdoutHandlerCallback() function accepts a string parameter, which is a POSIX compliant regular expression used to match against lines of stdout from the command line process. This function also returns a RegexHandlerCallback instance, which you can hook up a callback to that is called when a line of stdout is matched. This can all be done on one line, which is shown in the example above.

Examples of handler callback functions are also shown in the example above. Within these handler functions, the GetRegexMatch() function can be used to get a specific match from the regular expression. The parameter passed to GetRegexMatch() is the index for the matches that were found. 0 returns the entire matched string, and 1, 2, etc returns the matched substrings (matches that are surrounded by round brackets). If there isn’t a corresponding substring, you’ll get an error (note that 0 is always a valid index).

In HandleStdoutWarning(), 0 is the only valid index because there is no substring in round brackets in the regular expression. In HandleStdoutError(), 0 and 1 are valid. 0 will return the entire matched string, whereas 1 will return the substring in the round brackets.

For further examples, please open up any of our application plugin Python script files and inspect them. An example of comprehensive Stdout handlers can be found in the MayaBatch plugin.

../plugins/MayaBatch/MayaBatch.py

Note, that Deadline’s default shipping StdoutHandlers require the Worker’s Operating System to be using ENGLISH as it’s language.

Popup Ignorers and Handlers¶

The AddPopupIgnorer() function accepts a string parameter, which is a POSIX compliant regular expression. If a popup is displayed with a title that matches the given regular expression, the popup is simply ignored. Popup ignorers should only be used if the popup doesn’t halt the rendering because it is waiting for a button to be pressed. In the case where a button needs to be pressed to continue, popup handlers should be used instead. The AddPopupHandler() function takes two parameters: a regular expression string, and the button(s) to press (multiple buttons can be separated with semicolons).

Note, that Deadline’s default shipping PopupIgnorers and PopupHandlers require the Worker’s Operating System to be using ENGLISH as it’s language.

Here is an example using “.*” at the beginning and end of the title search string which acts as a wildcard. The dialog also has a “Adopt the File’s Unit Scale” checkbox that needs to be checked ON and then the “OK” button should be pressed in that order.

self.PopupHandling = True
self.AddPopupHandler(".*File Load: Units Mismatch.*", "Adopt the File's Unit Scale?;OK")

In this example, the Optical Flares license popup uses a “wxWindowClassNR” control for its “OK” button, so we need to add this special class type to our built-in list of possible button classes, just for the After Effects plugin. Once this class is added, we can search for it and react by pressing the “OK” button in it’s dialog. Although, in this case, visually the button displays the word “OK”, but actually the name of the button is “panel”.

self.PopupHandling = True
self.PopupButtonClasses = ("Button", "wxWindowClassNR")
# Handle Optical Flares License popup (the "OK" button is actually called "panel")
self.AddPopupHandler(".*Optical Flares License.*", "panel")

For users without access to a recent (2012+) version of Visual Studio which includes the Spy++ utility, then the free application WinSpy++ is very useful to help identify the correct syntax for a dialog’s title or button.

In this example, we force all Qt based widgets to be native instead of alien based widgets, set our HandleQtPopups variable to True and then we are able to handle V-Ray Qt based alien widget based dialogs whilst rendering in Rhino by pressing the [X] symbol in the top right corner of the Rhino Qt dialog:

self.PopupHandling = True
self.HandleQtPopups = True
self.SetEnvironmentVariable("QT_USE_NATIVE_WINDOWS","1")
self.AddPopupHandler(r"Rhino", "[X]")

In this final example, we need to handle Windows 8 Mobile / Windows 10 based popup dialogs and ensure we react correctly depending on the name of the dialog title. A sometimes tricky task if you have multiple, very similar named popup title dialogs in the application. We use the “.*” characters as a wildcard, the “^” character to ensure the text appears at the start of the string and the “$” character to ensure the text appears at the end of the string we are searching for.

self.PopupHandling = True
self.HandleWindows10Popups = True
self.AddPopupIgnorer("SAFE 12.*")
self.AddPopupIgnorer("^SAFE$")
self.AddPopupHandler("^$", "[X]")
self.AddPopupHandler("Tip of the Day", "[X]")

For further examples, please open up any of our application plugin Python script files and inspect them. Good examples are to be found in:

../plugins/3dsmax/3dsmax.py

../plugins/AfterEffects/AfterEffects.py

../plugins/CSiSAFE/CSiSAFE.py

../plugins/Rhino/Rhino.py

Further information on Regular Expressions can be found on Wikipedia and many online POSIX compliant RegEx testers are available to help you develop and test your RegEx before testing your code in Deadline:

regex101

regexr

regexpal

regextester

Finally, the Deadline “FranticX.Processes.ManagedProcess” class has a number of functions to further assist with Popup Handling and it is recommended to review our Scripting API docs for further information on these functions:

PopupButtonClasses

PopupMaxChildWindows

PopupTextClasses

PressEnterDuringRender

Render Executable and Arguments¶

The RenderExecutable() callback is used to get the path to the executable that will be used for rendering. This callback must be implemented in a Simple plugin, or an error will occur. Continuing our example from above, we’ll use the path specified in the database, and we can access it using the global GetConfigEntry() function.

Another important (but optional) callback is the RenderArgument() callback. This callback should return the arguments you want to pass to the render executable. Typically, these arguments are built from values that are pulled from the DeadlinePlugin class (like the scene file name, or the start and end frame for the task), or from the plugin info file that was submitted with the job using the GetPluginInfoEntry() function. If this callback is not implemented, then no arguments will be passed to the executable.

After adding these callbacks, our example plugin script now looks like this:

#!/usr/bin/env python3
from Deadline.Plugins import *

from System.Diagnostics import *

def GetDeadlinePlugin():
    """This is the function that Deadline calls to get an instance of the
    main DeadlinePlugin class.
    """
    return MyPlugin()

def CleanupDeadlinePlugin(deadlinePlugin):
    """This is the function that Deadline calls when the plugin is no
    longer in use so that it can get cleaned up.
    """
    deadlinePlugin.Cleanup()

class MyPlugin(DeadlinePlugin):
    """This is the main DeadlinePlugin class for MyPlugin."""
    def __init__(self):
        """Hook up the callbacks in the constructor."""
        self.InitializeProcessCallback += self.InitializeProcess
        self.RenderExecutableCallback += self.RenderExecutable
        self.RenderArgumentCallback += self.RenderArgument

    def Cleanup(self):
        """Clean up the plugin."""
        # Clean up stdout handler callbacks.
        for stdoutHandler in self.StdoutHandlers:
            del stdoutHandler.HandleCallback

        del self.InitializeProcessCallback
        del self.RenderExecutableCallback
        del self.RenderArgumentCallback

    ## Called by Deadline to initialize the process.
    def InitializeProcess(self):
        # Set the plugin specific settings.
        self.SingleFramesOnly = False
        self.PluginType = PluginType.Simple

        # Set the ManagedProcess specific settings.
        self.ProcessPriority = ProcessPriorityClass.BelowNormal
        self.UseProcessTree = True
        self.StdoutHandling = True
        self.PopupHandling = True

        # Set the stdout handlers.
        self.AddStdoutHandlerCallback(
            "WARNING:.*").HandleCallback += self.HandleStdoutWarning
        self.AddStdoutHandlerCallback(
            "ERROR:(.*)").HandleCallback += self.HandleStdoutError

        # Set the popup ignorers.
        self.AddPopupIgnorer("Popup 1")
        self.AddPopupIgnorer("Popup 2")

        # Set the popup handlers.
        self.AddPopupHandler("Popup 3", "OK")
        self.AddPopupHandler("Popup 4", "Do not ask me this again;Continue")

    def HandleStdoutWarning(self):
        """Callback for when a line of stdout contains a WARNING message."""
        self.LogWarning(self.GetRegexMatch(0))

    def HandleStdoutError(self):
        """Callback for when a line of stdout contains an ERROR message."""
        self.FailRender("Detected an error: " + self.GetRegexMatch(1))

    def RenderExecutable(self):
        """Callback to get the executable used for rendering."""
        return self.GetConfigEntry("MyPluginRenderExecutable")

    def RenderArgument(self):
        """Callback to get the arguments that will be passed to the executable."""
        arguments = " -continueOnError"
        arguments += " -verbose " + self.GetPluginInfoEntry("Verbose")
        arguments += " -start " + str(self.GetStartFrame())
        arguments += " -end " + str(self.GetEndFrame())
        arguments += ' -scene "{}"'.format(self.GetDataFilename())
        return arguments

There are many other callbacks that can be implemented for Simple plugins, which are covered in the Events section for the ManagedProcess class in the Deadline Scripting reference. The best place to find examples of Simple plugins is to look at some of the plugins that are shipped with Deadline. These range from the very basic (Blender), to the more complex (MayaCmd).

Advanced Plugins¶

To reiterate, a render job goes through three stages:

StartJob: A job enters this stage when it is first picked up by a Worker.

RenderTasks: A job can enter this stage many times (once for each task a Worker dequeues while it has the current job loaded).

EndJob: A job enters this stage when a Worker is unloading the job.

Advanced plugins are more complex, as they control all three of these job stages. They are commonly used to render with applications that support some sort of worker/server mode that Deadline can interact with. Usually, this requires the application to be started during the StartJob phase, fed commands during the RenderTasks stage(s), and finally shut down during the EndJob stage. For example, the 3ds Max plugin starts up 3dsmax in Worker mode and forces it to load our Lightning plugin. The Lightning plugin listens for commands from Deadline and executes them as necessary. After rendering is complete, 3ds Max is shut down.

Initialization¶

To indicate that your plugin is an Advanced plugin, you need to set the PluginType property in the InitializeProcess() callback.

#!/usr/bin/env python3
from Deadline.Plugins import *

def GetDeadlinePlugin():
    """This is the function that Deadline calls to get an instance of the
    main DeadlinePlugin class.
    """
    return MyPlugin()

def CleanupDeadlinePlugin(deadlinePlugin):
    """This is the function that Deadline calls when the plugin is no
    longer in use so that it can get cleaned up.
    """
    deadlinePlugin.Cleanup()

class MyPlugin (DeadlinePlugin):
    """This is the main DeadlinePlugin class for MyPlugin."""
    def __init__(self):
        """Hook up the callbacks in the constructor."""
        self.InitializeProcessCallback += self.InitializeProcess

    def Cleanup(self):
        """Clean up the plugin."""
        del self.InitializeProcessCallback

    def InitializeProcess(self):
        """Called by Deadline to initialize the process."""
        # Set the plugin specific settings.
        self.SingleFramesOnly = False
        self.PluginType = PluginType.Advanced

Render Tasks¶

The RenderTasks() callback is the only required callback for Advanced plugins. If it is not implemented, an error will occur. It contains the code to be executed for each task that a Worker renders. This could involve launching applications, communicating with already running applications, or simply running a script to automate a particular task (like backing up a group of files).

Other common callbacks for Advanced plugins are the StartJob() and EndJob() callbacks. The StartJob() callback can be used to start up an application, or to set some local variables that will be used in other callbacks. If the StartJob() callback is not implemented, then nothing is done during the StartJob phase. The EndJob() callback can be used to shut down a running application, or to clean up temporary files. If the EndJob() callback is not implemented, then nothing is done during the EndJob phase.

In the example below, we will be launching our application during the StartJob phase. The benefit to this is that the application can be left running during the duration of the job, which eliminates the overhead of having to launch the application for each task. To launch and monitor the application, we will be implementing a ManagedProcess class, and calling it MyPluginProcess .This ManagedProcess class will define the render executable and command line arguments for launching the process we will be monitoring. Note that we aren’t passing it any frame information, as this needs to be handled in the RenderTasks() callback when it interacts with the process.

After adding these three callbacks, and the MyPluginProcess class, our example code looks like this. Note that the RenderTasks() callback still needs code to allow it to interact with the running process launched in the StartJob() callback.

#!/usr/bin/env python3
from Deadline.Plugins import *

def GetDeadlinePlugin():
    """This is the function that Deadline calls to get an instance of the
    main DeadlinePlugin class.
    """
    return MyPlugin()

def CleanupDeadlinePlugin(deadlinePlugin):
    """This is the function that Deadline calls when the plugin is no
    longer in use so that it can get cleaned up.
    """
    deadlinePlugin.Cleanup()

class MyPlugin(DeadlinePlugin):
    """This is the main DeadlinePlugin class for MyPlugin."""
    Process = None  # Variable to hold the Managed Process object.

    def __init__(self):
        """Hook up the callbacks in the constructor."""
        self.InitializeProcessCallback += self.InitializeProcess
        self.StartJobCallback += self.StartJob
        self.RenderTasksCallback += self.RenderTasks
        self.EndJobCallback += self.EndJob

    def Cleanup(self):
        """Clean up the plugin."""
        del self.InitializeProcessCallback
        del self.StartJobCallback
        del self.RenderTasksCallback
        del self.EndJobCallback

        # Clean up the managed process object.
        if self.Process:
            self.Process.Cleanup()
            del self.Process

    def InitializeProcess(self):
        """Called by Deadline to initialize the process."""
        # Set the plugin specific settings.
        self.SingleFramesOnly = False
        self.PluginType = PluginType.Advanced

    def StartJob(self):
        """Called by Deadline when the job starts."""
        myProcess = MyPluginProcess()
        StartMonitoredManagedProcess("My Process", myProcess)

    def RenderTasks(self):
        """Called by Deadline for each task the Worker renders."""
        # Do something to interact with the running process.
        pass

    def EndJob(self):
        """Called by Deadline when the job ends."""
        ShutdownMonitoredManagedProcess("My Process")

class MyPluginProcess(ManagedProcess):
    """This is the ManagedProcess class that is launched above."""
    def __init__(self, deadlinePlugin):
        """Hook up the callbacks in the constructor."""
        self.deadlinePlugin = deadlinePlugin

        self.InitializeProcessCallback += self.InitializeProcess
        self.RenderExecutableCallback += self.RenderExecutable
        self.RenderArgumentCallback += self.RenderArgument

    def Cleanup(self):
        """Clean up the managed process."""
        # Clean up stdout handler callbacks.
        for stdoutHandler in self.StdoutHandlers:
            del stdoutHandler.HandleCallback

        del self.InitializeProcessCallback
        del self.RenderExecutableCallback
        del self.RenderArgumentCallback

    def InitializeProcess(self):
        """Called by Deadline to initialize the process."""
        # Set the ManagedProcess specific settings.
        self.ProcessPriority = ProcessPriorityClass.BelowNormal
        self.UseProcessTree = True
        self.StdoutHandling = True
        self.PopupHandling = True

        # Set the stdout handlers.
        self.AddStdoutHandlerCallback(
            "WARNING:.*").HandleCallback += self.HandleStdoutWarning
        self.AddStdoutHandlerCallback(
            "ERROR:(.*)").HandleCallback += self.HandleStdoutError

        # Set the popup ignorers.
        self.AddPopupIgnorer("Popup 1")
        self.AddPopupIgnorer("Popup 2")

        # Set the popup handlers.
        self.AddPopupHandler("Popup 3", "OK")
        self.AddPopupHandler("Popup 4", "Do not ask me this again;Continue")

    def HandleStdoutWarning(self):
        """Callback for when a line of stdout contains a WARNING message."""
        self.deadlinePlugin.LogWarning(self.GetRegexMatch(0))

    def HandleStdoutError(self):
        """Callback for when a line of stdout contains an ERROR message."""
        self.deadlinePlugin.FailRender("Detected an error: " + self.GetRegexMatch(1))

    def RenderExecutable(self):
        """Callback to get the executable used for rendering."""
        return self.deadlinePlugin.GetConfigEntry("MyPluginRenderExecutable")

    def RenderArgument(self):
        """Callback to get the arguments that will be passed to the executable."""
        arguments = ' -verbose ' + self.deadlinePlugin.GetPluginInfoEntry('Verbose')
        arguments += ' -scene "{}"'.format(self.deadlinePlugin.GetDataFilename())
        return arguments

Because the Advanced plugins are much more complex than the Simple plugins, we recommend taking a look at the following plugins that are shipped with Deadline for examples:

3dsmax

Fusion

Lightwave

MayaBatch

Modo

Nuke

SoftimageBatch

Migrating Plugins from Deadline 5¶

Some changes were made to the Scripting API in Deadline 6, which means that Deadline 6 and later are NOT backward compatible with plugin scripts written for Deadline 5. However, migrating your scripts over is relatively straightforward, and this guide will walk you through the API changes so that you can update your scripts as necessary.

Global Functions¶

In Deadline 6, all global API functions were removed, and replaced with DeadlinePlugin member functions, or with static utility functions. See the Code Completion for the Deadline Scripting API section in the Scripting Overview documentation for more information, including replacement functions.

Almost all plugin-specific global functions are now DeadlinePlugin member functions. For example, the global ‘LogInfo(message)’ function has been replaced with a member function for the DeadlinePlugin class, which you created in your event Python file. So instead of:

LogInfo("this is a test message")

You would use this code:

self.LogInfo("this is a test message")

The only functions that aren’t DeadlinePlugin member functions are listed below, along with their replacement utility functions.

Original Global Function	Replacement Function
CheckPathMapping(path)	RepositoryUtils.CheckPathMapping(path)
CheckPathMappingInFile(inFileName, outFileName)	RepositoryUtils.CheckPathMappingInFile(inFileName, outFileName)
CheckPathMappingInFileAndReplaceSeparator(inFileName, outFileName , separatorToReplace, newSeparator)	RepositoryUtils.CheckPathMappingInFileAndReplaceSeparator(inFileName, outFileName, separatorToReplace, newSeparator)
PathMappingRequired(path)	RepositoryUtils.PathMappingRequired(path)

Callbacks¶

You need to set up callbacks in the constructor of your DeadlinePlugin class that you created in your plugin Python file. Examples are shown in the documentation above, and you can look at the plugins that ship with Deadline for references as well. For example:

def __init__(self):
    self.InitializeProcessCallback += self.InitializeProcess
    self.RenderExecutableCallback += self.RenderExecutable
    self.RenderArgumentCallback += self.RenderArgument
    self.PreRenderTasksCallback += self.PreRenderTasks
    self.PostRenderTasksCallback += self.PostRenderTasks

Note that these callbacks need to be manually cleaned up when the plugin is no longer in use. See the documentation regarding the CleanupDeadlinePlugin function above for more information.

Application Plugins¶

Overview¶

General Plugin Information¶

Creating a New Plugin¶

.py File¶

.param File¶

Server Data¶

.options File¶

.ico File¶

JobPreLoad.py¶

PluginPreLoad.py¶

GlobalJobPreLoad.py¶

GlobalPluginPreLoad.py¶

Pipeline Bootstrapping¶

Modify Cmd Line¶

Pipeline Launch¶

Simple Plugins¶

Initialization¶

Stdout Handlers¶

Render Executable and Arguments¶

Advanced Plugins¶

Initialization¶

Render Tasks¶

Migrating Plugins from Deadline 5¶

Global Functions¶

Callbacks¶

Table of Contents

Previous topic

Next topic