Client Deployment - Centrally Managed

Note

Some of the topics covered in this page require you to have administrator/root access to your infrastructure. Please consult your local IT staff if in doubt.

Here are instructions to install Deadline client software locally on Linux & OS X. If you wish to centrally manage your install of the Deadline client software on Linux & OS X, please read on (Windows is unsupported with this configuration).

Network Install

A new option in Deadline 9.0 is the ability to deploy Deadline client software on a network located NFS share and clients run the software without needing a local installation. This approach delivers the option to have a centrally managed software installation. This is ONLY supported on Linux/OS X platforms. This document outlines the process to deploy Deadline in this manner. Zero to hero in the following steps:

Create NFS Shares

To follow along with these example guidelines, create the following directories under a NFS share (you may already have a ‘software’ file server in your pipeline):

/[MOUNT]/nfs/sw/thinkbox/deadlinerepo/[VERSION] # Deadline Repository (DB runs locally)
/mnt/nfs/sw/thinkbox/deadline/linux/[VERSION] # Deadline Linux client software
/Volumes/nfs/sw/thinkbox/deadline/osx/[VERSION] # Deadline OS X client software

# where [MOUNT] might be '/mnt/' or '/Volumes/' (depending on OS) and [VERSION] would be the version of Deadline, such as 9.0.0.0

We shall assume you have already installed the Deadline DB onto your Database machine’s local fast disk and the Repository onto the above NFS share /[MOUNT]/nfs/sw/thinkbox/deadlinerepo/[VERSION]/ which ideally will be tier 0 or 1 fast network disk.

Run the Client Installer

Note

Deadline’s Automatic Upgrades / Downgrades option is NOT supported and should be disabled if you choose to deploy and run Deadline from a network located NFS share.

We will take advantage of a new client installer option in Deadline 9.0 to extract just the binary files from the client installer.

  • --binariesonly <binariesonly>: If enabled, the installer will only install files to the installation directory. The installer will not perform any additional configuration to run Deadline on this machine. Default: false.

Initially, you will either elect a machine (could be a disposable VM or your local machine’s hard-disk) which you will use to carry out a dummy install and then copy the Deadline application files onto the NFS share (if you don’t have direct access) or you can create the applicable NFS share and tell the client installer to extract and directly copy the binary files to this NFS location. Run the Client Installer on that machine. The installer will prompt you for a Repository path, and if you are installing Deadline 10.1.22 or earlier, a license server. Setting these now avoids having to change these settings manually later, but neither the Repository nor the license server need to be reachable by this machine at this point. Alternatively, use the --mode unattended option to silently execute and copy the binaries only to your directed location. Here are Linux and OS X examples for your NFS share:

# Linux
>>> ./DeadlineClient-9.0.0.0-linux-x64-installer.run --mode unattended --prefix "/mnt/nfs/sw/thinkbox/deadline/linux/9.0.0.0" --binariesonly true
# OS X
>>> ./DeadlineClient-9.0.0.0-osx-installer.app/Contents/MacOS/installbuilder.sh --mode unattended --prefix "/Volumes/nfs/sw/thinkbox/deadline/osx/9.0.0.0" --binariesonly true

Rsync Files to NFS

OPTIONAL - This section can be skipped if the client installer extracted it’s binary files directly to your NFS share.

If you elected to extract the client installer files locally, then you will need to use Rsync to copy the files over to your NFS share. It is important to copy the root installation directory, rather than just the binaries folder. By default, the root installation directory is /opt/Thinkbox/Deadline[VERSION] where [VERSION] is the MAJOR version number of Deadline. It is important to use the --links flag to preserve symbolic links when performing the rsync transfer (-a (archive) already includes this follow --links argument) and ensure you elevate by using su -s or sudo to ensure permissions are carried over for all the files. Any file permission errors during the transfer will be reported by Rsync. An example Rsync command that accomplishes this is the following:

# Linux
>>> sudo rsync -avH /opt/Thinkbox/Deadline<VERSION> user@hostname:/mnt/nfs/sw/thinkbox/deadline/linux/9.0.0.0/
# OS X
>>> sudo rsync -aVH /Applications/Thinkbox/Deadline<VERSION> user@hostname:/Volumes/nfs/sw/thinkbox/deadline/osx/9.0.0.0/

Note

<VERSION> is the Deadline major version.

You may also wish to copy the deadline.ini Client Configuration file to a network location as well. This file contains settings that you may want to apply to all render nodes on your farm, such as connection information for the Repository and license server. By default, this file is located in /var/lib/Thinkbox/Deadline[VERSION]/deadline.ini.

Control Environment Variables

To run successfully, Deadline client applications need to have read and write access to a handful of locations which store settings, system and temporary files. It is possible in Deadline 9.0 and above to customize these locations by means of environment variables. Please see the next section for an overview of these environment variables and how they can then be used to create *.config files.

At the bare minimum, Deadline must have read access to a deadline.ini configuration file. The location of this file can be set via the DEADLINE_CONFIG_FILE environment variable using your preferred shell and shell configuration file such as bash and ~/.bashrc. To change the environment variable “permanently” you’ll need to consider at least these situations:

  • Higher level ‘pipeline’ tool controlling ‘project’/context switching in your studio

  • Login/Non-login shell

  • Interactive/Non-interactive shell

Here are some examples:

# bash
>>> export DEADLINE_CONFIG_FILE=/path/to/dir/deadline.ini

# csh or tcsh
>>> setenv DEADLINE_CONFIG_FILE /path/to/dir/deadline.ini

If the machines running Deadline are diskless, it may be necessary to redirect other paths as well. The full set of environment variables which can be modified are described in the Environment Variables section below. You can use these to configure Deadline to avoid writing to local directories, protected directories, or to paths which aren’t mapped to persistent storage.

Lastly, if you are going to be using any Integrated Submitters, you will need to set the DEADLINE_PATH environment variable to point to the directory that contains the application binaries.

Environment Variables

The DEADLINE_PATH environment variable is used by the Integrated Submitters to determine the location of the Deadline client applications. It should be configured to point to the directory which contains the application binaries. On Linux, this is the path to Deadline’s bin directory (/opt/Thinkbox/Deadline[VERSION]/bin by default). On OS X, it is instead a file located by default at /Users/Shared/Thinkbox which contains the path to Deadline’s resources directory. In both cases, it should be re-directed to your NFS share.

All of the following environment variables are available in Deadline 9.0 and onwards. These can be optionally used to override where the Deadline client applications store various settings and files. ALL of these environment variables should be configured to locate all client files onto a network share.

  • DEADLINE_SYSTEM_PATH: This is where Deadline stores system-level information. This is where process information is stored while the applications are running, and is also the default location for the system deadline.ini file. The default locations for this directory on each operating system in a typical Deadline local install are:

    • Linux: /var/lib/Thinkbox

    • OS X: /Users/Shared/Thinkbox

  • DEADLINE_LOCAL_USER_PATH: This is where Deadline stores user-level information. This is where files for job submission are stored, as well as Repository cache folders. It is also where the user-level deadline.ini file is stored. Ideally, for Worker performance reasons, DEADLINE_LOCAL_USER_PATH should be located on local storage to the machine such as a ~tmp scratch disk. The default locations for this directory on each operating system in a typical Deadline local install are:

    • Linux: ~/Thinkbox

    • OS X: ~/Library/Application Support/Thinkbox

Note

The job and plugin files used for rendering used to be stored in the DEADLINE_LOCAL_USER_PATH. As of Deadline 10.1.11, they are stored in the Worker Local Data Storage

  • DEADLINE_ROAMING_USER_PATH: This is where Deadline stores roaming user-level information. This is where UI state files for the GUI applications are stored. The default locations for this directory on each operating system in a typical Deadline local install are:

    • Linux: ~/.config/Thinkbox

    • OS X: ~/.config/Thinkbox

  • DEADLINE_LOG_PATH: This is where application log files are stored. The default locations for this directory on each operating system in a typical Deadline local install are:

    • Linux: /var/log/Thinkbox

    • OS X: /Library/Logs/Thinkbox

  • DEADLINE_CONFIG_FILE: This variable can be used to override where Deadline looks for the system deadline.ini file. By default, Deadline expects it to be found in a typical Deadline local install at:

    • Linux / OS X: $DEADLINE_SYSTEM_PATH/deadline[VERSION]/deadline.ini

Create Config Files

In Deadline 9 onwards, the shell scripts responsible for launching the Deadline client applications on Linux and OS X will source optional deadline.config and <appname>.config files within the bin (Linux) or Resources (OS X) directory on startup, if present. The <appname> would be named as follows for Worker: deadlineworker.config. Taking into account the environment variables described above, users can create one or more of these config files to control client software pathing.

Linux

Here’s an example deadline.config to configure ALL apps, created in /mnt/nfs/sw/thinkbox/deadline/linux/[VERSION]/bin/ on CentOS 6 with a user running the default bash shell. We will also highlight the recommended NFS storage configuration for each of the Deadline path variables:

Linux ‘deadline.config’ or ‘appname.config’
##########################################################
# This is read by root to run Deadline Launcher
# [deadline.config] file is sourced to set the
# root environment to find Deadline central install
##########################################################

# bash as login shell will load /etc/profile, ~/.bash_profile, ~/.bash_login, ~/.profile in that order
# bash as non-login interactive shell will load ~/.bashrc
# bash as non-login non-interactive shell will load the configuration specified in environment variable $BASH_ENV

DEADLINE_VERSION="9.0.0.0"
DEADLINE_MAJOR_VERSION="9"
DEADLINE_INSTALL_PATH="/nfs/sw/thinkbox/deadline/linux"

# [NFS] /nfs/sw/ - static mount, software repository, read-only
export DEADLINE_PATH="${DEADLINE_INSTALL_PATH}/${DEADLINE_VERSION}/bin"
export DEADLINE_CONFIG_FILE="${DEADLINE_INSTALL_PATH}/${DEADLINE_MAJOR_VERSION}/deadline.ini"

# [NFS] /nfs - auto-mount, user directory, read-write access
export DEADLINE_ROAMING_USER_PATH="/usr/people/${USER}/.config/thinkbox/deadline/${DEADLINE_MAJOR_VERSION}"

# [LOCAL] /mnt/local/disk - local caching disk (could be network located), read-write access
export DEADLINE_SYSTEM_PATH="/mnt/local/disk/sw/thinkbox/deadline/${DEADLINE_MAJOR_VERSION}/var/lib"
export DEADLINE_LOCAL_USER_PATH="/mnt/local/disk/sw/thinkbox/deadline/${DEADLINE_MAJOR_VERSION}/${USER}"
export DEADLINE_LOG_PATH="/mnt/local/disk/sw/thinkbox/deadline/${DEADLINE_MAJOR_VERSION}/var/log"

OS X

Here’s an example deadline.config to configure ALL apps, created in /Volumes/nfs/sw/thinkbox/deadline/osx/[VERSION]/Resources/ on OS X. We will also highlight the recommended NFS storage configuration for each of the Deadline path variables:

OSX ‘deadline.config’ or ‘appname.config’
##########################################################
# This is read by root to run Deadline Launcher
# [deadline.config] file is sourced to set the
# root environment to find Deadline central install
##########################################################

DEADLINE_VERSION="9.0.0.0"
DEADLINE_MAJOR_VERSION="9"
DEADLINE_INSTALL_PATH="/nfs/sw/thinkbox/deadline/osx"

# [NFS] /nfs/sw/ - static mount, software repository, read-only
export DEADLINE_PATH="${DEADLINE_INSTALL_PATH}/${DEADLINE_VERSION}/Resources"
export DEADLINE_CONFIG_FILE="${DEADLINE_INSTALL_PATH}/${DEADLINE_MAJOR_VERSION}/deadline.ini"

# [NFS] /nfs - auto-mount, user directory, read-write access
export DEADLINE_ROAMING_USER_PATH="/usr/people/${USER}/.config/thinkbox/deadline/${DEADLINE_MAJOR_VERSION}"

# [LOCAL] /Volumes/local/disk - local caching disk (could be network located), read-write access
export DEADLINE_SYSTEM_PATH="/Volumes/local/disk/sw/thinkbox/deadline/${DEADLINE_MAJOR_VERSION}/Users/Shared/Thinkbox"
export DEADLINE_LOCAL_USER_PATH="/Volumes/local/disk/sw/thinkbox/deadline/${DEADLINE_MAJOR_VERSION}/${USER}"
export DEADLINE_LOG_PATH="/Volumes/local/disk/sw/thinkbox/deadline/${DEADLINE_MAJOR_VERSION}/Library/Logs/Thinkbox"

Update ‘deadline.ini’ File

Make sure you update the declared deadline.ini file (in the example above, this would be: "$DEADLINE_INSTALL_PATH/$DEADLINE_MAJOR_VERSION/deadline.ini") so that it is pointing to the correct Deadline Repository path that you wish the applications to connect to.

Create Aliases

As a final step, you may wish to define aliases to the various common Deadline client applications in your local or studio wide SHELL configuration. You could reuse your configuration from earlier to define the location of the client application executables. Here’s a snippet for each platform (missing the environment variables from above which would need to be declared).

Linux

alias dlm "${DEADLINE_INSTALL_PATH}/${DEADLINE_VERSION}/bin/deadlinemonitor"
alias dlw "${DEADLINE_INSTALL_PATH}/${DEADLINE_VERSION}/bin/deadlineworker"
alias dlc "${DEADLINE_INSTALL_PATH}/${DEADLINE_VERSION}/bin/deadlinecommand"
alias dll "${DEADLINE_INSTALL_PATH}/${DEADLINE_VERSION}/bin/deadlinelauncher"

OS X

alias dlm "${DEADLINE_INSTALL_PATH}/${DEADLINE_VERSION}/Resources/deadlinemonitor"
alias dlw "${DEADLINE_INSTALL_PATH}/${DEADLINE_VERSION}/Resources/deadlineworker"
alias dlc "${DEADLINE_INSTALL_PATH}/${DEADLINE_VERSION}/Resources/deadlinecommand"
alias dll "${DEADLINE_INSTALL_PATH}/${DEADLINE_VERSION}/Resources/deadlinelauncher"

Client Upgrades

In the future, as Deadline’s Automatic Upgrades / Downgrades is not supported, you will download and run the client installer to install the updated client software (9.0.0.1 in this example) to a new location on your NFS share as configured by your directory structure such as:

# Linux
>>> ./DeadlineClient-9.0.0.1-linux-x64-installer.run --mode unattended --prefix "/mnt/nfs/sw/linux/thinkbox/deadline/9.0.0.1" --binariesonly true
# OS X
>>> ./DeadlineClient-9.0.0.1-osx-installer.app/Contents/MacOS/installbuilder.sh --mode unattended --prefix "/Volumes/nfs/sw/osx/thinkbox/deadline/9.0.0.1" --binariesonly true

To assist in updating the local init.d (Linux) or plist (OS X) startup scripts on the local client machines, we will take advantage of another new client installer option in Deadline 9.0 to NOT extract any of the client files but rather reconfigure the existing init.d/plist script file to point to the new client software version.

  • --nobinaries <nobinaries>: If enabled, the installer will not install any binaries. The installation directory should be pointed to a directory that already contains a Deadline client installation. Default: false.