Event Plugins

Overview

Event plug-ins can be created to execute specific tasks in response to specific events in Deadline (like when a job is submitted or when it finishes). For example, event plug-ins can be used to communicate with in-house pipeline tools to update the state of shots or tasks, or they can be used to submit a post-processing job when another job finishes. All of Deadline’s event plug-ins are written in Python, which means that it’s easy to create your own plug-ins or customize the existing ones. See the Scripting Overview documentation for more information, and links to the Deadline Scripting reference.

Note that because the Python scripts for event plug-ins will be executed in a non-interactive way, it is important that your scripts do not contain any blocking operations like infinite loops, or interfaces that require user input.

When an event is executed the log will show where the script is being loaded from.

Triggering Events

An event plug-in can respond to one or more of the following DeadlineEventListener events:

  • When a job is submitted OnJobSubmittedCallback
  • When a job starts rendering OnJobStartedCallback
  • When a job finishes rendering OnJobFinishedCallback
  • When a job is requeued OnJobRequeuedCallback
  • When a job fails OnJobFailedCallback
  • When a job is suspended OnJobSuspendedCallback
  • When a suspended or failed job is resumed OnJobResumedCallback
  • When a job is placed in the pending state OnJobPendedCallback
  • When a job is released from a pending state OnJobReleasedCallback
  • When a job is deleted OnJobDeletedCallback
  • When a job error occurs during rendering OnJobErrorCallback
  • When a job is about to be purged from the database OnJobPurgedCallback
  • When a house cleaning operation finishes OnHouseCleaningCallback
  • When a repository repair operation finishes OnRepositoryRepairCallback
  • When a slave starts OnSlaveStartedCallback
  • When a slave stops OnSlaveStoppedCallback
  • When a slave becomes idle OnSlaveIdleCallback
  • When a slave starts rendering OnSlaveRenderingCallback
  • When a slave starts a job OnSlaveStartingJobCallback
  • When a slave is marked as stalled OnSlaveStalledCallback
  • When power management’s Idle Shutdown feature shuts down slaves OnIdleShutdownCallback
  • When power management’s Machine Startup feature starts up slaves OnMachineStartupCallback
  • When power management’s Thermal Shutdown feature shuts down slaves OnThermalShutdownCallback
  • When power management’s Machine Restart feature restarts slaves OnMachineRestartCallback

The corresponding Event Callbacks for these events can be found in the ‘Deadline.Events.DeadlineEventListener Class Reference’ section of the Deadline Scripting Reference documentation. The full Deadline Scripting Reference can be found on the Thinkbox Software Documentation Website. Offline PDF and HTML versions can be downloaded from here as well.

By default, all jobs will trigger event plug-ins when they are submitted or change state. However, there is a job property that can be enabled to suppress events. In the Monitor, you can set the Suppress Events property under the Advanced tab in the Job Properties dialog. If you have a custom submission tool or script, you can specify the following in the job info file:

SuppressEvents=True

Note that events will be executed by different Deadline applications, depending on the context of the event. For example, the job submission event is processed by the Command application after the job has been submitted, while the job finished event is normally processed by the Slave that finishes the last task for the job. However, the job finished event could also be processed by the Monitor if manually marking a job as complete.

Creating an Event Plug-in

To create a custom event plug-in, you start by creating a folder in the Repository’s custom\events folder and give it the name of your event plug-in. See the Scripting Overview documentation for more information on the ‘custom’ folder in the Repository and how it’s used.

For the sake of this document, we will call our new event plug-in MyEvent. All relative script and configuration files for this event plug-in are to be placed in this folder (some are required and some are optional).

The dlinit File - Required

The first required file is MyEvent.dlinit, which is the main configuration file for this event plug-in. It is a plain text file that defines a few general key=value event plug-in properties, which include:

Key Name Description
Enabled Set to True or False (default is False). Only enabled event plug-ins will respond to events.
DeprecatedMode Set to True or False (default is False). Only set to True if you want a custom Python.NET event plug-in from Deadline 5.1 or 5.2 to work with Deadline 6 or later. More information on DeprecatedMode can be found later on.

It can also define key=value custom settings to be used by the event plug-in. For example, if you are connecting to an in-house pipeline tool, you may want the URL and credentials to be configurable, in which case our MyEvent.dlinit file might look like this:

Enabled=True
PipelineURL=http://[myserver]/pipeline
PipelineUserName=myuser
PipelinePassword=mypassword

The py File - Required

The other required file is MyEvent.py, which is the main event plug-in script file. It defines the main DeadlineEventListener class that contains the necessary callbacks that will respond to specific events. The template for this script file might look like this:

from Deadline.Events import *

######################################################################
## This is the function that Deadline calls to get an instance of the
## main DeadlineEventListener class.
######################################################################
def GetDeadlineEventListener():
    return MyEvent()

######################################################################
## This is the function that Deadline calls when the event plugin is
## no longer in use so that it can get cleaned up.
######################################################################
def CleanupDeadlineEventListener( deadlinePlugin ):
    deadlinePlugin.Cleanup()

######################################################################
## This is the main DeadlineEventListener class for MyEvent.
######################################################################
class MyEvent (DeadlineEventListener):

    # TODO: Place code here to replace "pass"
    pass

The first thing to note is that we’re importing the Deadline.Events namespace so that we can access the DeadlineEventListener class.

The GetDeadlineEventListener() function is important, as it allows Deadline to get an instance of our MyEvent class (which is extending the abstract DeadlineEventListener class). In Deadline 6.2 and later, the GetDeadlineEventListenerWithJobs( jobs ) function can be defined as an alternative. It works the same as GetDeadlineEventListener(), except that it accepts a list of the Job objects that the event plug-in is being loaded for. If either of these functions are not defined, Deadline will report an error when it tries to load the event plug-in.

The MyEvent class will need to implement certain callbacks based on the events you want to respond to, and these callbacks must be hooked up in the MyEvent constructor. All callbacks are optional, but make sure to include at least one so that your event plug-in actually does something. For a list of all available callbacks, refer to the DeadlineEventListener class in the Deadline Scripting reference.

The CleanupDeadlineEventListener() function is also important, as it is necessary to clean up the event plug-in when it is no longer in use. Typically, this is used to clean up any callbacks that were created when the event plug-in was initialized.

After implementing a few functions, your MyEvent.py script file might look something like this:

from Deadline.Events import *

######################################################################
## This is the function that Deadline calls to get an instance of the
## main DeadlineEventListener class.
######################################################################
def GetDeadlineEventListener():
    return MyEvent()

######################################################################
## This is the function that Deadline calls when the event plugin is
## no longer in use so that it can get cleaned up.
######################################################################
def CleanupDeadlineEventListener( deadlinePlugin ):
    deadlinePlugin.Cleanup()

######################################################################
## This is the main DeadlineEventListener class for MyEvent.
######################################################################
class MyEvent (DeadlineEventListener):

    def __init__( self ):
        # Set up the event callbacks here
        self.OnJobSubmittedCallback += self.OnJobSubmitted
        self.OnJobFinishedCallback += self.OnJobFinished

    def Cleanup( self ):
        del self.OnJobSubmittedCallback
        del self.OnJobFinishedCallback

    def OnJobSubmitted( self, job ):
        # TODO: Connect to pipeline site to notify it that a job has been submitted
        # for a particular shot or task.
        pass

    def OnJobFinished( self, job ):
        # TODO: Connect to pipeline site to notify it that the job for a particular
        # shot or task is complete.
        pass

The param File - Optional

The MyEvent.param file is an optional file that is used by the Event Configuration dialog in the Monitor. It declares properties that the Monitor uses to generate a user interface for modifying custom settings in the MyEvent.dlinit file. After you’ve created this file, open the Monitor and enter Super User mode. Then select Tools -> Configure Events and look for your event plug-in in the list on the left.

../_images/configure_event_myevent.png

The file might look something like:

[Enabled]
Type=boolean
Label=Enabled
Default=True
Description=If this event plug-in should respond to events.

[PipelineURL]
Type=string
Label=Pipeline URL
Default=http://[myserver]/pipeline
Description=The URL for our pipeline website.

[PipelineUserName]
Type=string
Label=Pipeline User Name
Default=
Description=The user name for our pipeline website.

[PipelinePassword]
Type=string
Label=Pipeline Password
Default=
Description=The password for our pipeline website.

Comment lines are supported in the param file, and must start with either ‘;’ or ‘#’. For example:

# This is a comment about this PipelineURL property.
[PipelineURL]
Type=string
Label=Pipeline URL
Default=http://[myserver]/pipeline
Description=The URL for our pipeline website.

You’ll notice that the property names between the square brackets matches the custom keys we defined in our MyEvent.dlinit file. This means that these control will change the corresponding settings. The available key=value pairs for the properties defined here are:

Key Name Description
Category The category the control should go under.
CategoryIndex This determines the control’s order under its category. This does the same thing as Index.
CategoryOrder This determines the category’s order among other categories. If more than one CategoryOrder is defined for the same category, the lowest value is used.
Default The default value to be used if this property is not defined in the dlinit file. This does the same thing as DefaultValue.
DefaultValue The default value to be used if this property is not defined in the dlinit file. This does the same thing as Default.
Description A short description of the property the control is for (displayed as a tooltip in the UI).
DisableIfBlank If True, a control will not be shown if this property is not defined in the dinit file (True/False). This does the same thing as IgnoreIfBlank.
IgnoreIfBlank If True, a control will not be shown if this property is not defined in the dinit file (True/False). This does the same thing as DisableIfBlank.
Index This determines the control’s order under its category. This does the same thing as CategoryIndex.
Label The control label.
Required If True, a control will be shown for this property even if it’s not defined in the dlinit file (True/False).
Type The type of control (see table below).

These are the available controls.

Control Type Description
Boolean A drop-down control that allows the selection of True or False.
Color Allows the selection of a color.
Enum A drop-down control that allows the selection of an item from a list.
Enumeration Same as Enum above.
Filename Allows the selection of an existing file.
FilenameSave Allows the selection of a new or existing file.
Float An floating point spinner control.
Folder Allows the selection of an existing folder.
Integer An integer spinner control.
Label A read-only text field.
MultiFilename Allows the selection of multiple existing files, which are then separated by semicolons in the text field.
MultiLineMultiFilename Allows the selection of multiple existing files, which are then placed on multiple lines in the text field.
MultiLineMultiFolder Allows the selection of multiple existing folders, which are then placed on multiple lines in the text field.
MultiLineString A text field with multiple lines.
Password A text field that masks the text.
SlaveList Allows the selection of existing Slaves, when are then separated by commas in the text field.
String A text field.

There are also key/value pairs for specific controls:

Key Name Description
DecimalPlaces The number of decimal places for the Float controls.
Filter The filter string for the Filename, FilenameSave, or MultiFilename controls.
Increment The value to increment the Integer or Float controls by.
Items The semicolon separated list of items for the Enum control. This does the same thing as Values.
Maximum The maximum value for the Integer or Float controls.
Minimum The minimum value for the Integer or Float controls.
Validator A regular expression for the String control that is used to ensure the value is valid.
Values The semicolon separated list of items for the Enum control. This does the same thing as Items.

Event Plug-in and Error Reports

Logs and reports can be stored with the job or the slave, depending on the event type.

Job Event Reports

Event types that start with “OnJob...” will save reports with the corresponding job.

When an event plug-in that uses the LogInfo or LogWarning functions finishes executing, its log will be stored with the job’s other render logs, which you can view in the Monitor by right-clicking on the job and selecting View Job Reports.

When an error occurs in an event-plugin, an error report will also be stored with the job’s other render errors, which you can view in the Monitor by right-clicking on the job and selecting View Job Reports.

Slave Event Reports

Event types that start with “OnSlave...” will save reports with the corresponding slave.

When an event plug-in that uses the LogInfo or LogWarning functions finishes executing, its log will be stored with the slave’s other render logs, which you can view in the Monitor by right-clicking on the slave and selecting View Slave Reports.

When an error occurs in an event-plugin, an error report will also be stored with the slave’s other render errors, which you can view in the Monitor by right-clicking on the slave and selecting View Slave Reports.

Quicktime Generation Example

An event plug-in can be used to automatically submit a Quicktime job to create a movie from the rendered images of a job that just finished. An example of an event plug-in like this can be downloaded from the Miscellaneous Deadline Downloads Page. To install the event plugin, just unzip the downloaded file to your Repository’s custom/events folder.

Configuration Files

The QuicktimeGen.dlinit and QuicktimeGen.param files define a couple of settings that can be configured from the Monitor. Here you can specify a path to the Quicktime settings XML file you want to use. This settings file can be generated from the Submit Quicktime Job To Deadline submitter in the Monitor.

The QuicktimeGen.dlinit file:

Enabled=True
QTSettings=\\ws-wpg-026\share\quicktime_export_settings.xml

The QuicktimeGen.param file:

[Enabled]
Type=boolean
Label=Enabled
Default=True
Description=If this event plug-in should respond to events.

[QTSettings]
Type=filename
Label=QT Settings XML File
Default=
Description=The QT settings xml file.

Cron / Scheduled Event

Zabbix Zenoss Nagios Opennms Solarwinds

A ‘regular time interval’ based event plugin can be called via listening for the House Cleaning event in Deadline to be completed. This is ideal for the execution of a Deadline event plugin, at a regular time interval when the Deadline database is as up to date as possible. The time interval of the House Cleaning operation is controlled in the repository options.

Deadline provides the possibility of integration with IT monitoring systems such as Zabbix, Zenoss, Nagios, Opennms, SolarWinds or indeed any other monitoring software via the house cleaning event callback. As an example, this event could be used to regularly inject Deadline data based on it’s job, slave, pulse, balancer statistics or info/settings into another database thereby providing integration and consistency between separate information systems in different departments in a company.

Building your own scheduled event script file might look something like this:

from Deadline.Events import *

######################################################################
## This is the function that Deadline calls to get an instance of the
## main DeadlineEventListener class.
######################################################################
def GetDeadlineEventListener():
    return ScheduledEvent()

######################################################################
## This is the function that Deadline calls when the event plugin is
## no longer in use so that it can get cleaned up.
######################################################################
def CleanupDeadlineEventListener( deadlinePlugin ):
    deadlinePlugin.Cleanup()

######################################################################
## This is the main DeadlineEventListener class for ScheduledEvent.
######################################################################
class ScheduledEvent (DeadlineEventListener):

    def __init__( self ):
        # Set up the event callbacks here
        self.OnHouseCleaningCallback += self.OnHouseCleaning

    def Cleanup( self ):
        del self.OnHouseCleaningCallback

    def OnHouseCleaning( self ):
        # TODO: Execute generic pipeline duties here such as
        # reporting to an external studio database or injecting
        # Deadline Farm Stats into Zabbix, Zenoss, Nagios for IT
        pass

Software Configuration Management Integration

CFEngine Puppet Saltstack Chef SCCM

Deadline provides the possibility of integration with Software Configuration Management (SCM) systems such as CFEngine, Puppet, Saltstack, Chef, SCCM or indeed any SCM software via the slave event callbacks. Deadline ships with Puppet and Salt Maintenance Jobs which can be submitted to Deadline via their monitor submission scripts and also via Puppet and Salt slave centric event plugins.

Building your own SCM event plugin might look something like this:

from Deadline.Events import *

######################################################################
## This is the function that Deadline calls to get an instance of the
## main DeadlineEventListener class.
######################################################################
def GetDeadlineEventListener():
    return SoftwareEvent()

######################################################################
## This is the function that Deadline calls when the event plugin is
## no longer in use so that it can get cleaned up.
######################################################################
def CleanupDeadlineEventListener( deadlinePlugin ):
    deadlinePlugin.Cleanup()

######################################################################
## This is the main DeadlineEventListener class for SoftwareEvent.
######################################################################
class SoftwareEvent (DeadlineEventListener):

    def __init__( self ):
        # Set up the event callbacks here
        self.OnSlaveIdleCallback += self.OnSlaveIdle
        self.OnSlaveStartedCallback += self.OnSlaveStarted
        self.OnSlaveStartingJobCallback += self.OnSlaveStartingJob

    def Cleanup( self ):
        del self.OnSlaveIdleCallback
        del self.OnSlaveStartedCallback
        del self.OnSlaveStartingJob

    # This is called when a slave becomes idle.
    def OnSlaveIdle(self, string):
        # If a slave is IDLE, then it is not processing,
        # which might be an optimal time to check for
        # system updates.
        self.SoftwareUpdate()

    # This is called when a slave is started.
    def OnSlaveStarted(self, string):
        # If a slave has just started on a rendernode,
        # this can typically be a reliable and safe time
        # to carry out config/software deployment.
        self.SoftwareUpdate()

    # This is called when a slave starts a job.
    def OnSlaveStartingJob(self, string, job):
        # You could query the returned job object when a
        # slave first starts a job. Correct version of
        # renderer installed?
        self.SoftwareUpdate()

    def SoftwareUpdate(self):
        ClientUtils.LogText("Preparing for Software Update")
        # TODO: Execute command here to query your in-house
        # software deployment tool (SCM) to see if any new
        # software/sys env variables are required to be updated.
        pass

Migrating Event Plug-ins from Deadline 5

Some changes were made to the Scripting API in Deadline 6, which means that Deadline 6 and later are NOT backward compatible with event plugin scripts written for Deadline 5. However, migrating your scripts over is relatively straightforward, and this guide will walk you through the API changes so that you can update your scripts as necessary.

Global Functions

In Deadline 6, all global API functions were removed, and replaced with DeadlineEventListener member functions, or with static utility functions. See the Migrating Scripts From Deadline 5 section in the Scripting Overview documentation for more information, including replacement functions.

Almost all event plugin-specific global functions are now DeadlineEventListener member functions. For example, the global ‘LogInfo( message )’ function has been replaced with a member function for the DeadlineEventListener class, which you created in your event python file. So instead of:

LogInfo( "this is a test message" )

You would use this code:

self.LogInfo( "this is a test message" )

The only functions that aren’t DeadlineEventListener member functions are listed below, along with their replacement utility functions.

Original Global Function Replacement Function
CheckPathMapping( path ) RepositoryUtils.CheckPathMapping( path )
CheckPathMappingInFile( inFileName, outFileName ) RepositoryUtils.CheckPathMappingInFile( inFileName, outFileName )
CheckPathMappingInFileAndReplaceSeparator( inFileName, outFileName , separatorToReplace, newSeparator ) RepositoryUtils.CheckPathMappingInFileAndReplaceSeparator( inFileName, outFileName, separatorToReplace, newSeparator )
PathMappingRequired( path ) RepositoryUtils.PathMappingRequired( path )

Callbacks

You need to set up callbacks in the constructor of your DeadlineEventListener class that you created in your event python file. Examples are shown in the documentation above, and you can look at the event plug-ins that ship with Deadline for references as well. For example:

def __init__( self ):
    self.OnJobSubmittedCallback += self.OnJobSubmitted
    self.OnJobStartedCallback += self.OnJobStarted
    self.OnJobFinishedCallback += self.OnJobFinished
    self.OnJobRequeuedCallback += self.OnJobRequeued
    self.OnJobFailedCallback += self.OnJobFailed

Note that these callbacks need to be manually cleaned up when the event plug-in is no longer in use. See the documentation regarding the CleanupDeadlineEventListener function above for more information.

Deprecated Mode

As mentioned above, you can set the DeprecatedMode property in your dlinit file to True. This mode allows Python.NET event plug-ins written for Deadline 5.1 or 5.2 to work with Deadline 6 and later, which can make the transition to Deadline 6 easier if you have custom event plug-ins.

Note that when DeprecatedMode is enabled, all global functions will still be available, so if you have custom Python.NET event plug-ins, you just need to drop them in the ‘custom/events’ folder in the Repository, and add “DeprecatedMode=True” to your dlinit file.

If you have custom IronPython event plug-ins from Deadline 5.2 or earlier, they will not work with Deadline 6 and later.