# Manual Job Submission¶

## Overview¶

Manual job submission is useful if you want more control over the submission process. For example, if you’re writing a custom submission script, or you are integrating the submission process into an internal pipeline tool, you will probably want full control over which job settings are being set.

If you are just looking to submit jobs from one of the many scripts that are shipped with Deadline, you should refer to the Submitting Jobs documentation.

## Arbitrary Command Line Jobs¶

To manually submit arbitrary command line jobs, you can use the -SubmitCommandLineJob option with the Command application. The key parameters that you need to specify are:

• -executable: The executable we wish to use.

• -arguments: The arguments we wish to pass to the executable. In the arguments string, there are a few key words that will be replaced with the appropriate text when rendering the job:

• <STARTFRAME> will be replaced with the current start frame for each task.
• <ENDFRAME> will be replaced by the current end frame for each task.
• <STARTFRAME%#> will be replaced with the current start frame for each task, and will be padded with 0’s to match the length specified for #. For example, <STARTFRAME%4> will ensure a start frame padding of 4.
• <ENDFRAME%#> will be replaced by the current end frame for each task, and will be padded with 0’s to match the length specified for #. For example, <ENDFRAME%6> will ensure an end frame padding of 6.
• <QUOTE> will be replaced with an actual quote character (“).
• -frames: The frames we wish to render.

The following parameters can also be included, but are optional:

• -startupdirectory: The directory that the command line will start up in.
• -chunksize: The number of frames per task (defaults to 1).
• -pool: The pool we wish to submit to (defaults to none).
• -group: The group we wish to submit to (defaults to none).
• -priority: The job’s priority (defaults to 50).
• -name: The job’s name (defaults to “Untitled”).
• -department: The job’s department (defaults to “”).
• -initialstatus: Specify “Active” or “Suspended” (defaults to “Active”).
• -prop: Specify additional job properties in the form KEY=VALUE, where KEY is any of the property names that can be specified in the Job Info File.

For example, say we want to submit a job that uses 3dsmaxcmd.exe to render frames in the scene file “\shared\path\scene.max”. We want to render frames 1 to 10, and we want an image resolution of 480x320. The command line to do this from a command prompt would look like:

3dsmaxcmd.exe
-start:1
-end:10
-width:480
-height:320
"\\shared\path\scene.max"


For the job, we want a task chunk size of 2, we want to submit to the 3dsmax group, we want a priority of 50, and we want a machine limit of 5. Finally, we want to call the job “3dsmax command line job”. The command line to submit this job would look like this:

deadlinecommand.exe
-SubmitCommandLineJob
-executable "c:\Program Files\Autodesk\3dsmax8\3dsmaxcmd.exe"
-arguments "-start:<STARTFRAME> -end:<ENDFRAME>
-width:480 -height:320 <QUOTE>\\shared\path\scene.max<QUOTE>"
-frames 1-10
-chunksize 2
-group 3dsmax
-priority 50
-name "3dsmax command line job"
-prop MachineLimit=5


## Maintenance Jobs¶

Maintenance jobs are special jobs where each task for the job will render on a different machine in your farm. This is useful for performing benchmark tests, installing new software, synchronizing files to each machine, etc. When a maintenance job is submitted, a task will automatically be created for each slave, and once a slave has finished a task, it will no longer pick up the job.

One way to submit a Maintenance job is to manually submit a job to Deadline by creating the the necessary job submission files as documented below. In the job info file, you must set MaintenanceJob to True:

MaintenanceJob=True


By default, a Maintenance job will render frame 0 on every machine. To render a different frame, or a sequence of frames, you can specify the MaintenanceJobStartFrame and MaintenanceJobEndFrame options in the job info file:

MaintenanceJob=True
MaintenanceJobStartFrame=1
MaintenanceJobEndFrame=5


Note that if you specify a whitelist or blacklist in the job info file, the number of tasks that are created for the Maintenance job will equal the number of valid slaves that the job could render on.

Another way to submit a Maintenance job is to right-click on an existing job in the Monitor and choose the Resubmit Job option. See the Resubmitting Jobs section of the Controlling Jobs documentation for more information.

## Creating Job Submission Files¶

This is the method that our submission scripts use to submit jobs. This method is far more flexible, but requires more work to setup the job. It also uses the Command application to submit the job.

Before the job can be submitted though, a Job Info File and a Plug-in Info File must be created. These are the first two files that should always be submitted with the job.

You can also submit additional auxiliary files with the job, such as the scene file you want to render, or any other files the job will need. Any number of auxiliary files can be specified after the job info and plugin info file. These auxiliary file are copied to the Repository, and are then copied locally to each machine during rendering. Because these files will be copied to the same folder, it is necessary that every file name be unique.

Once these files are ready to go, you can submit the job using the command line:

deadlinecommand.exe
[Job Info File] [Plug-in Info File] [Auxiliary File 1] [Auxiliary File 2]


### Job Info File¶

The Job Info File is a plain text file that uses Key/Value pairs (key=value) to define all the generic job options used to render the job. A couple options are required, but most are optional. All jobs can use these options, regardless of the plug-in that they use. Some examples have been provided further down.

Required Options

These options must be specified in the job info file, or the job will fail to submit. The rest of the options are optional.

• Plugin=<plugin name> : Specifies the plugin to use. Must match an existing plugin in the repository.

General Options

• Frames=<1,2,3-10,20> : Specifies the frame range of the render job. See the Frame List Formatting Options in the Job Submission documentation for more information (default = 0).
• Name=<job name> : Specifies the name of the job (default = Untitled).
• BatchName=<batch name> : Specifies an optional name to logically group jobs together (default = blank).
• MachineName=<machineName> : Specifies the machine the job was submitted from (default = current machine).
• Department=<department name> : Specifies the department that the job belongs to. This is simply a way to group jobs together, and does not affect rendering in any way (default = blank).
• Comment=<comment> : Specifies a comment for the job (default = blank).
• Group=<groupName> : Specifies the group that the job is being submitted to (default = none).
• Pool=<poolName> : Specifies the pool that the job is being submitted to (default = none).
• SecondaryPool=<poolName> : Specifies the secondary pool that the job can spread to if machines are available. If not specified, the job will not use a secondary pool.
• Priority=<0 or greater> : Specifies the priority of a job with 0 being the lowest (default = 50). The maximum priority can be configured in the Job Settings of the Repository Options, and defaults to 100.
• ChunkSize=<1 or greater> : Specifies how many frames to render per task (default = 1).
• ForceReloadPlugin=<true/false> : Specifies whether or not to reload the plugin between subsequent frames of a job (default = false). This deals with memory leaks or applications that do not unload all job aspects properly.
• SynchronizeAllAuxiliaryFiles=<true/false> : If enabled, all job files (as opposed to just the job info and plugin info files) will be synchronized by the Slave between tasks for this job (default = false). Note that this can add significant network overhead, and should only be used if you plan on manually editing any of the files that are being submitted with the job.
• InitialStatus=<Active/Suspended> : Specifies what status the job should be in immediately after submission (default = Active).
• LimitGroups=<limitGroup,limitGroup,limitGroup> : Specifies the limit groups that this job is a member of (default = blank).
• MachineLimit=<0 or greater> : Specifies the maximum number of machines this job can be rendered on at the same time (default = 0, which means unlimited).
• MachineLimitProgress=<0.1 or greater> : If set, the slave rendering the job will give up its current machine limit lock when the current task reaches the specified progress. If negative, this feature is disabled (default = -1.0). The usefulness of this feature is directly related to the progress reporting capabilities of the individual plugins.
• Whitelist=<slaveName,slaveName,slaveName> : Specifies which slaves are on the job’s whitelist (default = blank). If both a whitelist and a blacklist are specified, only the whitelist is used.
• Blacklist=<slaveName,slaveName,slaveName> : Specifies which slaves are on the job’s blacklist (default = blank). If both a whitelist and a blacklist are specified, only the whitelist is used.
• ConcurrentTasks=<1-16> : Specifies the maximum number of tasks that a slave can render at a time (default = 1). This is useful for script plugins that support multithreading.
• LimitTasksToNumberOfCpus=<true/false> : If ConcurrentTasks is greater than 1, setting this to true will ensure that a slave will not dequeue more tasks than it has processors (default = true).
• Sequential=<true/false> : Sequential rendering forces a slave to render the tasks of a job in order. If an earlier task is ever requeued, the slave won’t go back to that task until it has finished the remaining tasks in order (default = false).
• Interruptible=<true/false> : Specifies if tasks for a job can be interrupted by a higher priority job during rendering (default = false).
• InterruptiblePercentage=<0 to 100>: A task for this job will only be interrupted if the task progress is less than or equal to this value.
• SuppressEvents=<true/false> : If true, the job will not trigger any event plugins while in the queue (default = false).
• NetworkRoot=<repositoryUNCPath> : Specifies the repository that the job will be submitted to. This is required if you are using more than one repository (default = current default repository for the machine from which submission is occurring).

Cleanup Options

• Protected=<true/false> : If enabled, the job can only be deleted by the job’s user, a super user, or a user that belongs to a user group that has permissions to handle protected jobs. Other users will not be able to delete the job, and the job will also not be cleaned up by Deadline’s automatic house cleaning.
• OnJobComplete=<Nothing/Delete/Archive> : Specifies what should happen to a job after it completes (default = Nothing).
• DeleteOnComplete=<true/false> : Specifies whether or not the job should be automatically deleted after it completes (default = false).
• ArchiveOnComplete=<true/false> : Specifies whether or not the job should be automatically archived after it completes (default = false).
• OverrideAutoJobCleanup=<true/false> : If true, the job will ignore the global Job Cleanup settings and instead use its own (default = false).
• OverrideJobCleanup=<true/false> : If OverrideAutoJobCleanup is true, this will determine if the job should be automatically cleaned up or not.
• JobCleanupDays=<true/false> : If OverrideAutoJobCleanup and OverrideJobCleanup are both true, this is the number of days to keep the job before cleaning it up.
• OverrideJobCleanupType=<ArchiveJobs/DeleteJobs> : If OverrideAutoJobCleanup and OverrideJobCleanup are both true, this is the job cleanup mode.

Environment Options

• EnvironmentKeyValue#=<key=value> : Specifies environment variables to set when the job renders. This option is numbered, starting with 0 (EnvironmentKeyValue0), to handle multiple environment variables. For each additional variable, just increase the number (EnvironmentKeyValue1, EnvironmentKeyValue2, etc). Note that these variables are only applied to the rendering process, so they do not persist between jobs.
• IncludeEnvironment=<true/false> : If true, the submission process will automatically grab all the environment variables from the submitter’s current environment and set them in the job’s environment variables (default = false). Note that these variables are only applied to the rendering process, so they do not persist between jobs.
• UseJobEnvironmentOnly=<true/false> : If true, only the job’s environment variables will be used at render time (default = false). If False, the job’s environment variables will be merged with the slave’s current environment, with the job’s variables overwriting any existing ones with the same name.
• CustomPluginDirectory=<directoryName> : If specified, the job will look for for the plugin it needs to render in this location. If it does not exist in this location, it will fall back on the Repository plugin directory. For example, if you are rendering with a plugin called MyPlugin, and it exists in \server\development\plugins\MyPlugin, you would set CustomPluginDirectory=\server\development\plugins.

Failure Detection Options

• OverrideJobFailureDetection=<true/false> : If true, the job will ignore the global Job Failure Detection settings and instead use its own (default = false).
• FailureDetectionJobErrors=<0 or greater> : If OverrideJobFailureDetection is true, this sets the number of errors before the job fails. If set to 0, job failure detection will be disabled.
• OverrideTaskFailureDetection=<true/false> : If true, the job will ignore the global Task Failure Detection settings and instead use its own (default = false).
• FailureDetectionTaskErrors=<0 or greater> : If OverrideTaskFailureDetection is true, this sets the number of errors before a task for the job fails. If set to 0, task failure detection will be disabled.
• IgnoreBadJobDetection=<true/false> : If true, slaves will never mark the job as bad for themselves. This means that they will continue to make attempts at jobs that often report errors until the job is complete, or until it fails (default = false).
• SendJobErrorWarning=<true/false> : If the job should send warning notifications when it reaches a certain number of errors (default = false).

Timeout Options

• MinRenderTimeSeconds=<0 or greater> : Specifies the minimum time, in seconds, a slave should render a task for, otherwise an error will be reported (default = 0, which means no minimum). Note that if MinRenderTimeSeconds and MinRenderTimeMinutes are both specified, MinRenderTimeSeconds will be ignored.
• MinRenderTimeMinutes=<0 or greater> : Specifies the minimum time, in minutes, a slave should render a task for, otherwise an error will be reported (default = 0, which means no minimum). Note that if MinRenderTimeSeconds and MinRenderTimeMinutes are both specified, MinRenderTimeSeconds will be ignored.
• TaskTimeoutSeconds=<0 or greater> : Specifies the time, in seconds, a slave has to render a task before it times out (default = 0, which means unlimited). Note that if TaskTimeoutSeconds and TaskTimeoutMinutes are both specified, TaskTimeoutSeconds will be ignored.
• TaskTimeoutMinutes=<0 or greater> : Specifies the time, in minutes, a slave has to render a task before it times out (default = 0, which means unlimited). Note that if TaskTimeoutSeconds and TaskTimeoutMinutes are both specified, TaskTimeoutSeconds will be ignored.
• StartJobTimeoutSeconds=<0 or greater> : Specifies the time, in seconds, a slave has to start a render job before it times out (default = 0, which means unlimited). Note that if StartJobTimeoutSeconds and StartJobTimeoutMinutes are both specified, StartJobTimeoutSeconds will be ignored.
• StartJobTimeoutMinutes=<0 or greater> : Specifies the time, in minutes, a slave has to start a render job before it times out (default = 0, which means unlimited). Note that if StartJobTimeoutSeconds and StartJobTimeoutMinutes are both specified, StartJobTimeoutSeconds will be ignored.
• OnTaskTimeout=<Error/Notify/ErrorAndNotify/Complete> : Specifies what should occur if a task times out (default = Error).
• EnableAutoTimeout=<true/false> : If true, a slave will automatically figure out if it has been rendering too long based on some Repository Configuration settings and the render times of previously completed tasks (default = false).
• EnableTimeoutsForScriptTasks=<true/false> : If true, then the timeouts for this job will also affect its pre/post job scripts, if any are defined (default = false).

Dependency Options

• JobDependencies=<jobID,jobID,jobID> : Specifies what jobs must finish before this job will resume (default = blank). These dependency jobs must be identified using their unique job ID, which is outputted after the job is submitted, and can be found in the Monitor in the “Job ID” column.
• JobDependencyPercentage=<-1, or 0 to 100> : If between 0 and 100, this job will resume when all of its job dependencies have completed the specified percentage number of tasks. If -1, this feature will be disabled (default = -1).
• IsFrameDependent=<true/false> : Specifies whether or not the job is frame dependent (default = false).
• FrameDependencyOffsetStart=<-100000 to 100000> : If the job is frame dependent, this is the start frame offset (default = 0).
• FrameDependencyOffsetEnd=<-100000 to 100000> : If the job is frame dependent, this is the end frame offset (default = 0).
• ResumeOnCompleteDependencies=<true/false> : Specifies whether or not the dependent job should resume when its dependencies are complete (default = true).
• ResumeOnDeletedDependencies=<true/false> : Specifies whether or not the dependent job should resume when its dependencies have been deleted (default = false).
• ResumeOnFailedDependencies=<true/false> : Specifies whether or not the dependent job should resume when its dependencies have failed (default = false).
• RequiredAssets=<assetPath,assetPath,assetPath> : Specifies what asset files must exist before this job will resume (default = blank). These asset paths must be identified using full paths, and multiple paths can be separated with commas. If using frame dependencies, you can replace padding in a sequence with the ‘#’ characters, and a task for the job will only be resumed when the required assets for the task’s frame) exist.
• ScriptDependencies=<scriptPath,scriptPath,scriptPath> : Specifies what Python script files will be executed to determine if a job can resume (default = blank). These script paths must be identified using full paths, and multiple paths can be separated with commas. See the Scripting section of the documentation for more information on script dependencies.

Scheduling Options

• ScheduledType=<None/Once/Daily> : Specifies whether or not you want to schedule the job (default = None).
• ScheduledStartDateTime=<dd/MM/yyyy HH:mm> : The date/time at which the job will run. The start date/time must match the specified format. Here’s an explanation:
• dd: The day of the month. Single-digit days must have a leading zero.
• MM: The numeric month. Single-digit months must have a leading zero.
• yyyy: The year in four digits, including the century.
• HH: The hour in a 24-hour clock. Single-digit hours must have a leading zero.
• mm: The minute. Single-digit minutes must have a leading zero.
• ScheduledDays=<day interval> : If scheduling a Daily job, this is the day interval for when the job runs (default = 1).
• JobDelay=<dd:hh:mm:ss> : A start time delay. If there is no ScheduledStartDateTime this delay will be applied to the submission date. The delay value is represented by the number of days, hours, minutes, and seconds, all separated by colons.

Output Options

• OutputFilename#=<fileName> : Specifies the output image filenames for each frame (default = blank). This allows the Monitor to display the “View Output Image” context menu option in the task list. There is no minimum or maximum limit to padding length supported. A padding of 4 x #### is very common in many applications. If the filename is a full path, then the OutputDirectory# option is not needed. This option is numbered, starting with 0 (OutputFilename0), to handle multiple output file names per frame. For each additional file name, just increase the number (OutputFilename1, OutputFilename2, etc).

• OutputFilename#Tile?=<fileName> : Specifies the output image filenames for each task of a Tile job (default = blank). This allows the Monitor to display the “View Output Image” context menu option in the task list for Tile jobs. The ‘#’ is used to support multiple outputs per frame (see OutputFilename# above), and the ‘?’ is used to specify the output for each task in the Tile job. For example, a Tile job with 2 outputs and 2 tiles would specify OutputFilename0Tile0, OutputFilename0Tile1, OutputFilename1Tile0, and OutputFilename1Tile1.

• OutputDirectory#=<directoryName> : Specifies the output image directory for the job (default = blank). This allows the Monitor to display the “Explore Output” context menu option in the job list. If the filename is a full path, then the OutputDirectory# option is not needed. This option is numbered, starting with 0 (OutputDirectory0), to handle multiple output directories per frame. For each additional directory, just increase the number (OutputDirectory1, OutputDirectory2, etc).

OutputDirectory0=\\fileserver\Project\Renders\OutputFolder\
OutputFilename0=o_HDP_010_BG_v01.####.exr
OutputDirectory1=\\fileserver\Project\Renders\OutputFolder\
OutputFilename1=o_HDP_010_SPEC_v01####.dpx
OutputDirectory2=\\fileserver\Project\Renders\OutputFolder\
OutputFilename2=o_HDP_010_RAW_v01_####.png


• ClearNotificationTargets=<true/false> : If enabled, all of the job’s notification targets will be removed (default = false).

• OverrideNotificationMethod=<true/false> : If the job user’s notification method should be ignored (default = false).

• EmailNotification=<true/false> : If overriding the job user’s notification method, whether to use email notification (default = false).

• PopupNotification=<true/false> : If overriding the job user’s notification method, whether to use popup notification (default = false).

• NotificationNote=<note> : A note to append to the notification email sent out when the job is complete (default = blank). Separate multiple lines with [EOL], for example:

This is a line[EOL]This is another line[EOL]This is the last line


Script Options

• PreJobScript=<path to script> : Specifies a full path to a python script to execute when the job initially starts rendering (default = blank).
• PostJobScript=<path to script> : Specifies a full path to a python script to execute when the job completes (default = blank).
• PreTaskScript=<path to script> : Specifies a full path to a python script to execute before each task starts rendering (default = blank).
• PostTaskScript=<path to script> : Specifies a full path to a python script to execute after each task completes (default = blank).

Tile Job Options

• TileJob=<true/false> : If this job is a tile job (default = false).
• TileJobFrame=<frameNumber> : The frame that the tile job is rendering (default = 0).
• TileJobTilesInX=<xCount> : The number of tiles in X for a tile job (default = 0). This should be specified with the TileJobTilesInY option below.
• TileJobTilesInY=<yCount> : The number of tiles in Y for a tile job (default = 0). This should be specified with the TileJobTilesInX option above.
• TileJobTileCount=<count> : The number of tiles for a tile job (default = 0). This is an alternative to specifying the TileJobTilesInX and TileJobTilesInY options above.

Maintenance Job Options

• MaintenanceJob=<true/false> : If this job is a maintenance job (default = false).
• MaintenanceJobStartFrame=<frameNumber> : The first frame for the maintenance job (default = 0).
• MaintenanceJobEndFrame=<frameNumber> : The last frame for the maintenance job (default = 0).

Extra Info Options

These are extra arbitrary properties that have corresponding columns in the Monitor that can be sorted on. There are a total of 10 Extra Info properties that can be specified.

• ExtraInfo0=<arbitrary value>
• ExtraInfo1=<arbitrary value>
• ExtraInfo2=<arbitrary value>
• ExtraInfo3=<arbitrary value>
• ExtraInfo4=<arbitrary value>
• ExtraInfo5=<arbitrary value>
• ExtraInfo6=<arbitrary value>
• ExtraInfo7=<arbitrary value>
• ExtraInfo8=<arbitrary value>
• ExtraInfo9=<arbitrary value>

These are additional arbitrary properties. There is no limit on the number that are specified, but they do not have corresponding columns in the Monitor.

• ExtraInfoKeyValue0=<key=value>
• ExtraInfoKeyValue1=<key=value>
• ExtraInfoKeyValue2=<key=value>
• ExtraInfoKeyValue3=<key=value>

Job Info File Examples

3ds Max Job Info File:

Plugin=3dsmax
Frames=0-400
Priority=50
Pool=3dsmax
Name=IslandWaveScene_lighted01
Comment=Testing
OutputDirectory0=\\fileserver\Renders\OutputFolder\
OutputFilename0=islandWaveBreak_Std####.png


Lightwave Job Info File:

Plugin=Lightwave
Frames=1-10,21-30
ChunkSize=10
Priority=99
Pool=LightwavePool
Group=NiceShot
Name=Lightwave Test
OutputFilename0=\\fileserver\Renders\OutputFolder\test####.tif
DeleteOnComplete=true
MachineLimit=5
SlaveTimeout=3600


Fusion Job Info File:

Plugin=Fusion
Frames=1-100
Priority=50
Group=Fusion
Name=Fusion Dependency Test
OutputFilename0=\\fileserver\Renders\OutputFolder\dfusion_test####.tif
JobDependencies=546cc87357dbb04344a5c6b5,53d27c9257dbb027b8a4ccd2
InitialStatus=Suspended
LimitGroups=DFRNode
ExtraInfo0=Regression Testing
ExtraInfoKeyValue0=TestID=344
ExtraInfoKeyValue1=DeveloperID=12


### Plug-in Info File¶

The Plug-in Info File is a plain text file that uses Key/Value pairs (key=value) to define the plug-in specific options that are used by the individual plug-ins to render the job. Often, these options are used to build up the command line arguments that are to be passed on to the rendering application.

The plug-ins read in the settings from the Plug-in Info File using the script functions GetPluginInfoEntry(…) and GetPluginInfoEntryWithDefault(…), which are discussed in more detail in the Plug-in Scripting documentation (Application Plugins).

## Batch Job Submission¶

To submit a high-volume of jobs into Deadline with optimal performance via DeadlineCommand, it’s best to carry out an additional step of creating the JobInfo/PluginInfo files for each submission (use unique filenames if they all reside in the same root directory) and then reference them all in one temporary file, and instantiate the DeadlineCommand just once which will submit all these jobs in one session. Note, some operating systems have a limit on the number of characters that an individual command line can consist of, so this method is also recommended to resolve this issue. Here is an example demonstrating this approach. Batch Job Submission by creating a temporary file called args.txt containing:

-SubmitMultipleJobs
-job
\\path\to\job_1_info_file.txt
\\path\to\job_1_plugin_file.txt
-job
\\path\to\job_2_info_file.txt
\\path\to\job_2_plugin_file.txt
-job
\\path\to\job_3_info_file.txt
\\path\to\job_3_plugin_file.txt


You would then instantiate DeadlineCommand just once and pass it the arguments like this:

deadlinecommand args.txt