If you’re new to AWS Portal we recommend starting here. If you’re new to Deadline we recommend starting here.

AWS Account Considerations

Here you will find some guidelines for using your AWS account with AWS Portal.

Tagging and Tracking Costs

AWS Portal supports tagging of all of its resources. For example you could tag your Spot Fleet Requests with different departments. You can easily track costs in the AWS Billing Dashboard. All AWS Portal resources are also tagged with DeadlineRole, so you know which AWS components are being used by Deadline. You can see how much each tag is costing you in the Billing service in the AWS Console. You can enable Cost Allocation tags by going to Services > Billing > Cost Allocation Tags and checking the tags you would like to track then clicking Activate. You can read more about cost allocation tagging here.

Switching AWS Accounts

Use the following steps if you would like to switch which AWS Account you are using with your Deadline Repository.

  1. Stop any current running AWS Portal Infrastructures.
  2. Log out of the AWS Portal panel and input the credentials of the new account.
  3. Find the account number of your old account.

a. Log into the AWS Web Console on your old account.

b. In the top right click the drop down menu that contains you username.

c. Copy the number under the Account header and remove the hyphens.

  1. In the AWS Portal Panel, open the AWS Portal Settings. Click Remove Orphaned AWS Entities. Enter the old account number.
  2. In the monitor, go to Tools > Asset Server Settings. Click Create New Bucket.
  3. Navigate to the install directory of the AWS Portal Asset Server in a command shell. Below are the default paths.

a. On Windows this is in C:\Program Files (x86)\Thinkbox\AWSPortalAssetServer\.

b. On Linux this is in /opt/Thinkbox/AWSPortalAssetServer/.

  1. Next run update_credentials as an Administrator/Root.

a. On Windows run

"%DEADLINE_PATH%\dpython.exe" update_credentials.py

b. On Linux run

"$DEADLINE_PATH/dpython" update_credentials.py
  1. Input your new AWS Access Key ID and hit enter.
  2. Input your new AWS Secret Key and hit enter.

You’ve now successfully switched AWS Accounts.

Can I use one account for more than one Deadline repository?

No, right now AWS Portal only supports a one AWS account per one Deadline Repository.

AWS Service Limits

Note

Please ensure you allow adequate time to submit your service limit request prior to large scale rendering via AWS Portal!

By default, all new AWS accounts have default limits applied to various components of each AWS service, including EC2 specific limits. Studios using EC2 Spot for low-cost rendering at scale will quickly find themselves hitting one or more of these AWS limits. There are many limits as explained in the AWS Service Limits documentation.

AWS limits should be considered useful as they protect your AWS account from scaling too large too quickly without valid AWS experience. The AWS limits by default are fine for all render tests up to 20 instances via a Spot Fleet Request (SFR). This document explains the specific AWS Service and EC2 limits to request an increase for larger scale distributed compute jobs (>20 instances). Actual recommended limit values are provided at the end of this document to assist you.

You can review your current AWS account limits in your AWS Console [Check AWS Region] -> EC2 -> Limits as well as request individual limit increases for your account.

Service Limits

AWS Trusted Advisor offers a Service Limits check (in the Performance category) that displays your usage and limits for some aspects of some services per region. For more information, see Service Limits Check Questions in the Trusted Advisor FAQs.

You can take one of the following steps to request an increase for limits. These increases are not granted immediately, so it may take a couple of days for your increase to become effective.

To request via support case

  1. Open the AWS Support Center page, sign in if necessary, and choose Create Case.
  2. For Regarding, choose Service Limit Increase.
  3. Complete Limit Type, Use Case Description, and Contact method. If this request is urgent, choose Phone as the method of contact instead of Web. Also note that Spot limits are seperate from on-demand limits.
  4. Choose Submit.

Alternatively: To request via EC2 Dashboard

  1. Open your EC2 Dashboard in your AWS Console and choose Limits.
  2. Click on Request limit increase and complete the support case page.

Elastic Block Store (EBS) Limits

Depending on the number of instances you intend to run, the following EBS Limits should be increased from the default values. Typically, General Purpose SSD (gp2) volumes will be used. However, you may need to replace this with the EBS Volume Type you are using such as Provisioned IOPS SSD (io1).

For reference, General Purpose SSD (gp2) provide a baseline performance of 3 IOPS per GiB, with a minimum of 100 IOPS and a maximum of 10,000 IOPS. General Purpose (SSD) volumes under 1000 GiB can burst up to 3000 IOPS. Provisioned IOPS (SSD) can provision up to 50 IOPS per GiB.

If you have multiple volumes attached per AMI instance, you may need to increase the total number of EBS snapshots allowed per region as well.

AWS Service Limit Description Default Limit
IOPS Total provisioned EBS IOPS (io1 only) 40,000
GP2 Total storage of General Purpose SSD (gp2) volumes 100 TiB
EBS snapshots Number of EBS snapshots 10,000

Virtual Private Cloud (VPC) Limits

Depending on the number of instances you intend to run, you may need to increase the total number of network interfaces allowed per AWS region in the VPC limits. A request can be submitted to have this limit increased.

AWS Service Limit Description Default Limit
Network interfaces per region This limit is the greater of either the default limit (350) or your On-Demand Instance limit multiplied by 5. The default limit for On-Demand Instances is 20. If your On-Demand Instance limit is below 70, the default limit of 350 applies. You can increase the number of network interfaces per region by contacting AWS Support, or by increasing your On-Demand Instance limit. 350

EC2 Service Limits

You can check EC2 limits for your user account via AWS Console [Check AWS Region] -> EC2 -> Limits.

An overall limit is applied to the total number of Spot instances per region that is allowed. Don’t forget to request a limit increase if you intend to run greater than 20 Spot instances via the Amazon EC2 instance request form.

AWS Service Limit Description Default Limit
Spot instance requests Total limit of Spot instances per region 20

Spot Instance Limits

Finally, EC2 Spot Fleet requests have a number of specific Spot Limits you should just be aware of. It is unlikely that you will be hindered by these limits. However, we will provide them below for reference.

Spot Fleet Limits

The following limits apply:

  • The number of active Spot Fleets per region: 1,000
  • The number of launch specifications per fleet: 50
  • The size of the user data in a launch specification: 16 KB
  • The target capacity per Spot Fleet: 3,000
  • The target capacity across all Spot Fleets in a region: 5,000
  • A Spot Fleet request can’t span regions.
  • A Spot Fleet request can’t span different subnets from the same Availability Zone.

Example Limits

Here are some recommended initial limits for you based on the maximum number of Spot instances, which would need to be applied per AWS region.

20 instances

AWS Service Limit Default Limit NEW Limit
IOPS (io1 only) 40,000 n/a
GP2 100 TiB n/a
Network interfaces per region 350 n/a
Spot instance requests 20 n/a

50 instances

AWS Service Limit Default Limit NEW Limit
IOPS (io1 only) 40,000 n/a
GP2 100 TiB n/a
Network interfaces per region 350 n/a
Spot instance requests 20 50

100 instances

AWS Service Limit Default Limit NEW Limit
IOPS (io1 only) 40,000 50,000
GP2 100 TiB n/a
Network interfaces per region 350 n/a
Spot instance requests 20 100

250 instances

AWS Service Limit Default Limit NEW Limit
IOPS (io1 only) 40,000 60,000
GP2 100 TiB n/a
Network interfaces per region 350 n/a
Spot instance requests 20 250

500 instances

AWS Service Limit Default Limit NEW Limit
IOPS (io1 only) 40,000 80,000
GP2 100 TiB n/a
Network interfaces per region 350 500
Spot instance requests 20 500

1000 instances

AWS Service Limit Default Limit NEW Limit
IOPS (io1 only) 40,000 100,000
GP2 100 TiB 200 TiB
Network interfaces per region 350 1000
Spot instance requests 20 1000

4000 instances

AWS Service Limit Default Limit NEW Limit
IOPS (io1 only) 40,000 1,000,000
GP2 100 TiB 800 TiB
Network interfaces per region 350 4000
Spot instance requests 20 4000