Using Distributed Processing with Compressor

You can increase speed and productivity by distributing processing across multiple computers. The Apple Qmaster features of Compressor subdivide the work for speed, route the work to the computers with the most available computing power, and direct the processing across designated computers.

See the Distributed Processing Setup Guide, available in Compressor Help, for complete instructions on setting up and managing an Apple Qmaster distributed processing network.

This section covers two specific topics that you should be aware of when you are using the Apple Qmaster distributed processing system with Compressor.

Job Segmenting and Two-Pass or Multi-Pass Encoding

If you choose the two-pass or the multi-pass mode, and you have distributed processing enabled, you may have to make a choice between speedier processing and ensuring the highest possible quality.

The Apple Qmaster distributed processing system speeds up processing by distributing work to multiple processing nodes (computers). One way it does this is by dividing up the total number of frames in a job into smaller segments. Each of the processing computers then works on a different segment. Since the nodes are working in parallel, the job is finished sooner than it would have been on a single computer. But with two-pass VBR and multi-pass encoding, each segment is treated individually so the bit-rate allocation generated in the first pass for any one segment does not include information from the segments processed on other computers.

First, evaluate the encoding difficulty (complexity) of your source media. Then, decide whether or not to allow job segmenting (with the Allow Job Segmenting checkbox at the top of the Encoder pane). If the distribution of simple and complex areas of the media is similar throughout the whole source media file, then you can get the same quality whether segmenting is turned on or not. In that case, it makes sense to allow segmenting to speed up the processing time.

However, you may have a source media file with an uneven distribution of complex scenes. For example, suppose you have a 2-hour sports program in which the first hour is the pregame show with relatively static talking heads, and the second hour is high-action sports footage. If this source media were evenly split into two segments, the bit-rate allocation plan for the first segment would not be able to “donate” some of its bits to the second segment because the segments would be processed on separate computers. The quality of the more complex action footage in the second segment would suffer. In this case, if your goal were ensuring the highest possible quality over the entire 2-hour program, it would make sense to not allow job segmenting by deselecting the checkbox at the top of the Encoder pane. This forces the job (and, therefore, the bit-rate allocation) to be processed on a single computer.

Note: The Allow Job Segmenting checkbox affects only the segmenting of individual jobs (source files). If you are submitting batches with multiple jobs, the distributed processing system will continue to speed up processing by distributing (unsegmented) jobs, even with job segmenting turned off.

For more information on variable bit-rate encoding (VBR), see Quality Tab. For more information on the Apple Qmaster distributed processing system, see the Distributed Processing Setup Guide, available in Compressor Help.

About AutoCluster and Unmanaged Services

Compressor includes an AutoCluster feature that makes it easy to take advantage of the distributed processing capabilities offered by Apple Qmaster, without requiring a lot of knowledge about how clusters are configured, setting up file sharing, and so on.

Using AutoCluster is a two-step process:

  • Automatically creating Apple Qmaster service nodes as you install Final Cut Studio or Apple Qmaster

  • Selecting the “Include unmanaged services on other computers” checkbox when you submit a Compressor batch for processing

These two steps let you harness the processing power of any number of computers on your network without any additional effort or knowledge on your part.

See the Distributed Processing Setup Guide, available in Compressor Help, for additional information.