Cloud Plugins

Overview

Cloud plug-ins can be created to allow Deadline to communicate with different cloud providers. All of Deadline’s cloud plug-ins are written in Python, which means that it’s easy to create your own plug-ins or customize the existing ones. You can also refer to them in the Repository’s cloud folder for examples of how they work. See the Scripting Overview documentation for more information, and links to the Deadline Scripting reference.

Note that because the Python scripts for cloud plug-ins will be executed in a non-interactive way, it is important that your scripts do not contain any blocking operations like infinite loops, or interfaces that require user input.

When a cloud script is executed the log will show where the script is being loaded from.

Creating a Cloud Plug-in

To create a custom cloud plug-in, you start by creating a folder in the Repository’s custom\cloud folder and give it the name of your cloud plug-in. See the Scripting Overview documentation for more information on the ‘custom’ folder in the Repository and how it’s used.

For the sake of this document, we will call our new cloud plug-in MyCloud. All relative script and configuration files for this cloud plug-in are to be placed in this folder.

.py File

Required

The first required file is MyCloud.py, which is the main cloud plugin script file. It defines the main CloudPluginWrapper class that contains the necessary callbacks that will respond to specific commands. The template for this script file might look like this:

from Deadline.Cloud import *

######################################################################
## This is the function that Deadline calls to get an instance of the
## main CloudPluginWrapper class.
######################################################################
def GetCloudPluginWrapper():
    return MyCloudPlugin()

######################################################################
## This is the function that Deadline calls when the cloud plugin is
## no longer in use so that it can get cleaned up.
######################################################################
def CleanupCloudPlugin( deadlinePlugin ):
    deadlinePlugin.Cleanup()

######################################################################
## This is the main DeadlineCloudListener class for MyCloud.
######################################################################
class MyCloud (CloudPluginWrapper):

    # TODO: Place code here instead of "pass"
    pass

The GetCloudPluginWrapper() function is important, as it allows Deadline to get an instance of our MyCloud class (which is extending the abstract CloudPluginWrapper class). If this function isn’t defined, Deadline will report an error when it tries to load the cloud plug-in. Notice that we’re importing the Deadline.Cloud namespace so that we can access the CloudPluginWrapper class.

The MyCloud class will need to implement certain callbacks so that Deadline can get information from the cloud provider, and these callbacks must be hooked up in the MyCloud constructor. For a list of all available callbacks, refer to the CloudPluginWrapper class in the Deadline Scripting reference.

After implementing a few functions, your MyCloud.py script file might look something like this:

from Deadline.Cloud import *

######################################################################
## This is the function that Deadline calls to get an instance of the
## main CloudPluginWrapper class.
######################################################################
def GetCloudPluginWrapper():
    return MyCloudPlugin()

######################################################################
## This is the function that Deadline calls when the cloud plugin is
## no longer in use so that it can get cleaned up.
######################################################################
def CleanupCloudPlugin( deadlinePlugin ):
    deadlinePlugin.Cleanup()

######################################################################
## This is the main DeadlineCloudListener class for MyCloud.
######################################################################
class MyCloud (CloudPluginWrapper):

    def __init__( self ):
        #Set up our callbacks for cloud control
        self.VerifyAccessCallback += self.VerifyAccess
        self.AvailableHardwareTypesCallback += self.GetAvailableHardwareTypes
        self.AvailableOSImagesCallback += self.GetAvailableOSImages
        self.CreateInstancesCallback += self.CreateInstances
        self.TerminateInstancesCallback += self.TerminateInstances
        self.CloneInstanceCallback += self.CloneInstance
        self.GetActiveInstancesCallback += self.GetActiveInstances
        self.StopInstancesCallback += self.StopInstances
        self.StartInstancesCallback += self.StartInstances
        self.RebootInstancesCallback += self.RebootInstances

    def Cleanup( self ):
        #Clean up our callbacks for cloud control
        del self.VerifyAccessCallback
        del self.AvailableHardwareTypesCallback
        del self.AvailableOSImagesCallback
        del self.CreateInstancesCallback
        del self.TerminateInstancesCallback
        del self.CloneInstanceCallback
        del self.GetActiveInstancesCallback
        del self.StopInstancesCallback
        del self.StartInstancesCallback
        del self.RebootInstancesCallback

    def VerifyAccess( self ):
        #TODO: Return True if connection to cloud provider can be verified.
        pass

    def GetAvailableHardwareTypes( self ):
        #TODO: Return list of HardwareType objects representing the hardware
        #types supported by this provider.
        #Must be implemented for the Balancer to work.
        pass

    def GetAvailableOSImages( self ):
        #TODO: Return list of OSImage objects representing the OS images
        #supported by this provider.
        #Must be implemented for the Balancer to work.
        pass

    def GetActiveInstances( self ):
        #TODO: Return list of CloudInstance objects that are currently active.
        pass

    def CreateInstances( self, hardwareID, imageID, count ):
        #TODO: Start instances and return list of CloudInstance objects that
        #have been started.
        #Must be implemented for the Balancer to work.
        pass

    def TerminateInstances( self, instanceIDs ):
        #TODO: Return list of boolean values indicating which instances
        #terminated successfully.
        #Must be implemented for the Balancer to work.
        pass

    def StopInstances( self, instanceIDs ):
        #TODO: Return list of boolean values indicating which instances
        #stopped successfully.
        pass

    def StartInstances( self, instanceIDs ):
        #TODO: Return list of boolean values indicating which instances
        #started successfully.
        pass

    def RebootInstances( self, instanceIDs ):
        #TODO: Return list of boolean values indicating which instances
        #rebooted successfully.
        pass

.param File

Required

The MyCloud.param file is a required file that is used by the Cloud Provider Configuration dialog in the Monitor. It declares properties that the Monitor uses to generate a user interface for modifying settings for this provider, which are then stored in the database. After you’ve created this file, open the Monitor and enter Super User mode. Then select Tools -> Configure Cloud Providers and click the Add button under the Cloud Region box to see your cloud plugin.

The file might look something like:

[Enabled]
Type=boolean
Label=Enabled
Default=True
Description=If this cloud plug-in should be enabled.

[AccessID]
Type=string
Category=Options
CategoryOrder=0
Index=1
Label=Access ID
Default=
Description=Your Cloud Provider Access ID.
[SecretKey]
Type=password
Category=Options
CategoryOrder=0
Index=2
Label=Secret Key
Default=
Description=Your Cloud Provider Secret Key.

Comment lines are supported in the param file, and must start with either ‘;’ or ‘#’. For example:

# This is a comment about this Enabled property.
[Enabled]
Type=boolean
Label=Enabled
Default=True
Description=If this cloud plug-in should be enabled.

The available key=value pairs for the properties defined here are:

Key Name	Description
Category	The category the control should go under.
CategoryIndex	This determines the control’s order under its category. This does the same thing as Index.
CategoryOrder	This determines the category’s order among other categories. If more than one CategoryOrder is defined for the same category, the lowest value is used.
Default	The default value to be used if this property is not defined in the database. This does the same thing as DefaultValue.
DefaultValue	The default value to be used if this property is not defined in the database. This does the same thing as Default.
Description	A short description of the property the control is for (displayed as a tooltip in the UI).
DisableIfBlank	If True, a control will not be shown if this property is not defined in the database (True/False). This does the same thing as IgnoreIfBlank.
IgnoreIfBlank	If True, a control will not be shown if this property is not defined in the database (True/False). This does the same thing as DisableIfBlank.
Index	This determines the control’s order under its category. This does the same thing as CategoryIndex.
Label	The control label.
Required	If True, a control will be shown for this property even if it’s not defined in the database (True/False).
Type	The type of control (see table below).

These are the available controls.

Control Type	Description
Boolean	A drop-down control that allows the selection of True or False.
Color	Allows the selection of a color.
Enum	A drop-down control that allows the selection of an item from a list.
Enumeration	Same as Enum above.
Filename	Allows the selection of an existing file.
FilenameSave	Allows the selection of a new or existing file.
Float	An floating point spinner control.
Folder	Allows the selection of an existing folder.
Integer	An integer spinner control.
Label	A read-only text field.
MultiFilename	Allows the selection of multiple existing files, which are then separated by semicolons in the text field.
MultiLineMultiFilename	Allows the selection of multiple existing files, which are then placed on multiple lines in the text field.
MultiLineMultiFolder	Allows the selection of multiple existing folders, which are then placed on multiple lines in the text field.
MultiLineString	A text field with multiple lines.
Password	A text field that masks the text. If using the Deadline Secrets Management feature, any parameter with the Control Type of Password will be saved and retrieved based on the Cloud Plugin Secrets Access Level.
SlaveList	Allows the selection of existing Workers, when are then separated by commas in the text field.
String	A text field.

There are also key/value pairs for specific controls:

Key Name	Description
DecimalPlaces	The number of decimal places for the Float controls.
Filter	The filter string for the Filename, FilenameSave, or MultiFilename controls.
Increment	The value to increment the Integer or Float controls by.
Items	The semicolon separated list of items for the Enum control. This does the same thing as Values.
Maximum	The maximum value for the Integer or Float controls.
Minimum	The minimum value for the Integer or Float controls.
Validator	A regular expression for the String control that is used to ensure the value is valid.
Values	The semicolon separated list of items for the Enum control. This does the same thing as Items.