blenderdist

Distribute Blender jobs across multiple hosts

When doing some heavy 3D rendering with Blender, I realized that one of my animation was going to take 53 hours to render. Existing distributed rendering systems such as DrQueue were fine but require that some software other than Blender or basic interpreters (such as Python or Perl) is installed on the contributing machines.

So I wrote a simple Python script called blenderdist.py which only needs blender and python to run. A server is launched with:

 python blenderdist.py --server PORT JOBDIR RENDERDIR

and will monitor the status of job files (three lines each, the blender file, the first frame to render and the last one to render) in JOBDIR. Resulting frames are placed under RENDERDIR/jobname. Job names have to end with .job and if a file named JOBNAME.job.suspend is present, its rendering is suspended to allow urgent jobs to be rendered first.

Clients are launched with:

 python blenderdist.py —client HOST PORT

The server constantly monitors its source code. Whenever the Python script changes, the server relaunches itself (without loosing its state saved in a checkpoint file) and the next time the clients connect to it they will receive the new version of the program and relaunch themselves too.

I have currently a dozen machines working as I type, most of them out of my control. Some friends of mine have agreed to run the script and are contributing CPU cycles for my rendering. This proves to be very helpful. The program is much less powerful than generic ones such as DrQueue, but it does not require that disk space is shared between machines or setting up complex scripts. It just gets the job done.

Note: as this script has been written for a one-time shot need, I place it under the public domain, do whatever you want with it.

Getting blenderdist

You can get the current development version of blenderdist using git:

git clone https://github.com/samueltardieu/blenderdist.git

This will create a blenderdist directory in which you will be able to record your own changes. You can also browse the blenderdist repository on GitHub.

Contributing to blenderdist

Reporting bugs and asking for features

If you find a bug or have an idea for a new feature, you might consider adding a new issue. The more precise you will be in your description, the more useful it will be.

Submitting patches

Patches are gladly accepted from their original author. Along with any patches, please state that the patch is your original work and that you license the work to the blenderdist project under a license compatible with the current one (public domain).

To propose a patch, you may fork the blenderdist repository on GitHub, and issue a pull request. You may also send patches and pull requests by email.