An auto-configuring Render Farm (Feb 16, 2017)

Version: Deadline 8.0

INTRODUCTION

There is a VFX shop running a small render farm consisting of 10 artist workstations. This works fine since their artists work in shifts and some workstations become render nodes when not in use. However, they just landed a big contract which means they would need more resources to get the work done on time. Learn more about virtual workstations here.

After some discussion, they have decided to purchase 20 new machines to use as dedicated render nodes. Their systems administrator mentioned that since they are getting a new Deadline license, they can use this as an opportunity to migrate their license server to a dedicated machine.

To facilitate this migration, they could send an email to all artists asking them to update their Deadline license server settings on their machines. But since they currently work in shifts, not all machines will be updated at once. Which means if they switch license servers, some machines may fail to find a license because they are still pointing to the old license server.

Another problem they face is, since the new machines will be dedicated render nodes, their administrator would have to go to every machine and update the Deadline license server settings. This is not a huge problem since he can run the client installer in "Unattended" mode on each machine, and include the license server information:

DeadlineClient-8.0.13.3-windows-installer.exe --mode unattended --repositorydir /mnt/dlrepo --licenseserver @my-licenseserver

However, the administrator would like to future proof this so that if the license server must change later on, he would not have to remote in to all the render nodes and change them.

CAN WE MAKE THIS EASIER?

Deadline has a feature called Auto Configuration that can be used to make this easier. It will allow the administrator to configure the render nodes and the workstations from one central location. He can use this feature to automatically set the Deadline License Server settings across all the machines.

The Auto Configuration settings can be opened from the Deadline Monitor, while in Super User Mode. From the main menu, select Tools -> Configure Repository Options, and then select the Auto Configuration page from the list on the left.

Here are the steps to auto-configure Deadline's licensing:

• Click the Add button at the bottom to create a new Ruleset. To keep things organized, name the Ruleset "Licensing".
• Change the Client Filter Type to "Host Regex", and set the Client Filter to "*". This will ensure all machines are matched.
• Check the Licensing check box. Keep the License Mode standard (because floating licenses are being used in this case), and put the host name or IP address of the license server in the License Server text box, prefixed with the @ symbol. In this case, the License Server setting will be "@my-licenseserver".
• Hit OK to save the changes.

While the Deadline Launcher is running on the render nodes and workstations, it will periodically check the Auto Configuration settings and update any setting that has been changed. Other applications like the Deadline Worker and Monitor will check these settings on startup as well. The next time the License Server host name changes, all the administrator has to do is update the "Licensing" Ruleset and the render nodes and workstations will automatically configure themselves to use it!

While he was configuring the "License" Ruleset, the administrator used the Client Filter option to match all machines. By using more restrictive filters, it makes it possible to apply a Ruleset to only a subset of machines.

There are three different options for the Client Filter Type:

• IPv4 Match: This can be used to match machines based on IP address, and can even support an IP range. For example "192.168.0.1-150" will match any machine in the subnet of 192.168.0.1" to "192.168.0.150" but not "192.168.0.151".
• IP Regex: This uses regular expressions to match machines based on their IP Address. For example "192.168..*" will match any machine with an IP address begining with "192.168" such as "192.168.2.224".
• Host Regex: This uses regular expressions to match machines based on their host name. For example "mobile-.*" will match any machine with a host name that begins with "mobile-" such as "mobile-bob" or "mobile-010".

This is perfect! The administrator would like to configure the render nodes to launch the Deadline Worker on startup, and let the artists choose what to do with their workstations. He'll be able to do this by adding another Ruleset.

Luckily, they already have a standard in place for naming their machines. All workstations have the form "ws-01", "ws-02", etc, and all render nodes have the form "rn-01", "rn-02", etc. This will allow the administrator to use the Host Regex filter type again.

Here are the steps to auto-configure the render nodes to launch the Worker at startup:

• Click the Add button at the bottom to create another new Ruleset. To keep things organized, name the Ruleset "Render Nodes".
• Change the Client Filter Type to "Host Regex", and set the Client Filter to "rn-.*". This will ensure only the render nodes are matched.
• Check the Launch Worker At Startup check box.
• Hit OK to save the changes.

Now the render nodes will automatically configure themselves to launch the Worker at startup. If any new render nodes are added to the farm in the future, they will also automatically configure themselves, providing their machine names starts with "rn-".

There is an important thing to note when multiple Rulesets are defined. They are processed in a top down manner, in the order that they are listed in the Auto Configuration settings. Typically, the top Rulesets should be generalized and the lower ones can be more specific, as the lower ones can override settings in the previous Rulesets.

THIS IS GREAT! WHAT ARE THE REQUIREMENTS?

There is only one additional requirement, and it's only applicable if Auto Configuration is being used to automatically configure the Repository Path that the machines connect to. This can be useful if the Repository ever needs to be migrated to a new machine.

The requirement in this case is to run Deadline Pulse. It has an auto-discovery feature that allows the Deadline applications to connect to it before they connect to a Repository, and pull down the Repository path they should be using.

Note that this is only a requirement for the Repository Path setting. If this setting is not being configured automatically, then there are no additional requirements!

WRAPPING UP?

This is just an example of how Auto Configuration can be used to simplify the maintenance of a growing render farm, and additional Rulesets can be added as necessary. More information about Auto Configuration can be found in the Auto Configuration Documentation.