Farm Statistics¶

Overview¶

Deadline can keep track of some basic statistics. It can keep track of all of your completed Jobs so that you refer to them later. It stores the User that submitted the Job, when the Job was submitted, the error count, as well as some useful rendering metrics like render time, CPU usage, and memory usage. This data will be stored at job completion time on a best-effort basis. If the Worker exits before it can report back or changes are made to a job after it has completed, that data will not propagate to its statistics. This data will not match the duration of an AWS instance running the Worker and should not be relied on for estimating costs.

Statistical information is also gathered for individual Workers, including the Worker’s running time, rendering time, and idle time. It also includes information about the number of tasks the Worker has completed, the number of errors it has reported, and its average Memory and CPU usage. You can use all of this information to figure out if there are any Workers that aren’t being utilized to their full potential.

Note that some statistics can only be gathered if Pulse is running.

Enabling Statistics Gathering¶

You must first make sure Statistics Gathering has been enabled before Deadline will start logging information, which can be done in the Statistics Gathering section of the Repository Options.

../_images/statistics_enable_stats_gathering.png

Note that if Pulse is not running, only statistics for completed Jobs, User usage and Worker Statistics will be recorded. You must run Pulse to keep track of Worker Resource Usage and overall Repository statistics. When running, Pulse will periodically gather information about Workers Resource Usage and the general state of the repository, and record them in the Database.

Viewing Farm Reports¶

To view Statistics, open the Monitor and select ‘Tools’ -> ‘View Farm Reports’. This must be done in Power User mode, unless you have the proper User Privileges to do so.

From this window, you can specify which type of report(s) to generate, and a date range to filter the statistics. You can also specify a region to filter the statistics, but only the Active Worker Stats and Workers Overview reports will use it.

There are five default Reports that will always be available, but custom reports can also be created and saved for later use (see the ‘Custom Reports’ section below for more info).

Active Worker Stats¶

The Active Worker Stats report displays Worker usage statistics for the farm, which are logged by Workers as they are running. The statistics displayed by this report are generated by each individual Worker at regular intervals and do not require Pulse to be running.

Completed Job Stats¶

The Completed Job Stats report consists of a list of completed Jobs with detailed statistics. Pulse does not need to be running to gather these statistics.

Farm Overview¶

The Farm Overview report displays statistics about the Farm using graphs. The statistics displayed by this report are assembled by Pulse, and will therefore only be gathered if Pulse is running.

The State Counts section displays the statistics in terms of counts.

The State Totals gives a visual representation of the statistics in terms of percentages.

Workers Overview¶

The Workers Overview report displays the statistics for each Worker on the farm with graphs to help display the statistics. The statistics displayed by this report are assembled by Pulse, and will therefore only be gathered if Pulse is running.

The Workers Overview chart shows now many Workers were in each state (starting job, rendering, idle, offline, stalled, and disabled).

The Available/Active Workers charts show the number of Workers that are available, and the number of available Workers that are active.

The Individual Workers list and charts show the average CPU and Memory usage for individual Workers, as well as average time each Worker spends in each state.

User Farm Time Report¶

The User Farm Time Report displays the farm usage statistics for each User. Pulse does not need to be running to gather these statistics.

Custom Reports¶

Users can create their own custom Reports to control how the gathered statistics are aggregated and presented. By doing this, users can create their own arsenal of specialized reports that help to drill down and expose potential problems with the farm.

In order to create or edit Custom Reports you first need to be in Power User mode. Right-click in the Reports to Generate section to create or modify an existing custom report saved in your database. You can also import and export a custom farm report (*.json file) to share with others!

New: Create a new custom report, based on one of the existing stats objects.
Import Farm Report: Browse file system to import a custom farm report, stored as a *.json file.
Modify Farm Report: Modify an existing custom farm report saved in your database.
Delete Farm Report: Delete an existing custom farm report from your database.
Archive Farm Report: Export a custom farm report as an externally stored *.json file, so you can share with others.

By clicking the ‘New’ button, you will be prompted to specify a name for your new report and select the type of statistics which this report will display.

Once you’ve done that, you’ll be brought to the Edit view for your new Report. You’ll note that this is very similar to generating a report under normal circumstances, but with the addition of several buttons that allow further customization of your Report.

../_images/farm_reports_preview_custom.png

Chief among these new buttons is the ‘Edit Data Columns’ button, which will allow you to select which columns are displayed. You can also specify if you want to aggregate row information by selecting a Group By column, and a Group Op for each other column.

The way the aggregation works is similar to a SQL query with a “group by” statement. Data rows will be combined based on identical values of the Group By column, while the values of other columns will be determined by performing the Group Ops on the combined rows.

As a simple example to demonstrate how this works in practice, let us consider a case where you might want to view the error information on a per-plugin basis. We don’t have a built-in report to do this, but all this information is contained in Completed Job Stats. With that in mind, you can create a Custom Report based on Completed Job Stats to group by Plugin, and aggregate Error Counts and Wasted Error Time, as illustrated below.

Once you’ve specified which columns are displayed, and whether/how rows are aggregated, you can also add simple Graphs to your report. Simply click the ‘Add Graph’ button, and specify the type of graph you want along with the columns on which the graph should be based. Graphs are always based on all of the data presented the list view, and currently cannot be based on selection or a different data model. You can right-click on the graph once generated to save it as an image for your report to management!

../_images/farm_reports_graph_custom.png

Once you’re done customizing your new report, simply click the ‘OK’ button on the Farm Status Reports window, and your changes will be committed to the Database. Now, every time anyone brings up this dialog, they should be able to generate the report you’ve just created!

Custom Statistics¶

If you need to keep track of more information, we suggest writing your own tool that uses Deadline Command. Deadline Command can be used to query the repository for all sorts of information, like the current state of all the Jobs and all the Workers. You can have it print these out in an ini file format and use any ini file parser to extract the information (Python has a module for this). This is also handy if you want to post stats to a web page, or insert entries into a separate database.