How to run a Python script on multiple computers

First, make the work a function of its inputs

The pattern every tool on this page needs:

# sim.py
def run(alpha: float, seed: int) -> dict:
    ...                     # read its own inputs, compute
    return {"alpha": alpha, "seed": seed, "score": score}

One call reads only its arguments and files it can reach, returns or writes its own result, and never waits on another call. A loop over run() on one computer becomes many calls on many computers. Work in which each step needs the one before, or processes exchange data constantly, does not split this way and needs a real cluster.

Make each call long enough to be worth sending: a call that takes 50 milliseconds spends more time on the network than computing. Batch small items into one call.

The quickest way: GNU parallel over SSH

If the computers are Macs and Linux machines with SSH and the same Python, a command-line wrapper around run() is all you need:

parallel --sshlogin 8/alice@build-box,4/alice@studio,: \
  --basefile sim.py --results out/ \
  python3 sim.py --alpha {1} --seed {2} ::: 0.1 0.2 0.5 ::: 1 2 3 4

8/ and 4/ are jobs per computer, : is this one, and --basefile copies the script first. How to use the idle computers you own for batch jobs covers its job log, retries and limits.

The Python-native way: Dask distributed

Dask runs a scheduler on one computer and a worker on each of the others, and your script hands it function calls. Install the same dask[distributed] on every computer, then:

# on build-box: the scheduler (port 8786; a dashboard on 8787)
dask scheduler

# on every computer that should compute, the scheduler's own included
dask worker tcp://build-box:8786 --nworkers 1 --nthreads 4

From your laptop:

from dask.distributed import Client
from sim import run

client = Client("tcp://build-box:8786")
client.upload_file("sim.py")                 # send the module to every worker

alphas, seeds = zip(*[(a, s) for a in (0.1, 0.2, 0.5) for s in range(1, 5)])
futures = client.map(run, alphas, seeds)
results = client.gather(futures)

Workers take calls as they free up, so a fast computer simply does more of them, and a worker that disappears has its calls run elsewhere. Dask workers run on macOS, Linux and Windows.

The errors you will meet first:

• ModuleNotFoundError: No module named 'sim' on a worker: the worker can’t import your code. client.upload_file() sends one file; a package needs to be installed on every worker.
• Mismatched versions found, followed by a table of packages per machine: Dask checks the client, scheduler and workers when they connect. Its documentation says it is “critical that each of your dask workers uses the same set of python packages and modules”. Differences in dask, distributed, cloudpickle, numpy or pandas can break unpickling at run time, not just warn.
• Nothing connects: the scheduler’s port is blocked by a firewall, or a worker resolved build-box to the wrong address. Use an IP address to rule that out.

Out of the box the scheduler accepts any connection that reaches its port, and anyone who can reach it can run code on your workers. Keep it on a private network, or set up Dask’s TLS security.

Ray

Ray runs Python functions as tasks across machines (@ray.remote, then ray.get), with ray start --head on one computer and ray start --address=… on the others. On a mix of Macs and Windows PCs, the first thing you see is:

Multi-node Ray clusters are not supported on Windows and OSX.
Restart the Ray cluster with the environment variable `RAY_ENABLE_WINDOWS_OR_OSX_CLUSTER=1` to proceed anyway.

The variable lets you continue at your own risk; Ray’s own documentation supports multi-node clusters on Linux only. On a set of Linux machines, Ray is a strong choice, especially for work that is more than independent calls.

A task queue

If jobs arrive over days rather than in one sweep, a queue such as Celery or RQ with Redis keeps them until a worker is free. The idle computers guide has an RQ example. RQ’s workers need fork(), so not on native Windows.

The same environment everywhere

This is where most multi-computer Python fails: an import that works on one computer and not the next, a wheel that exists for macOS arm64 but not Windows, python3 on one machine and python on another. What helps:

• One Python version on every computer, and one lock file (uv lock, or pip freeze into requirements.txt from one machine). Install from it everywhere, every time it changes.
• Packages with wheels for every platform you use; anything that builds from source will build differently, or not at all, somewhere.
• Run one call by hand on each kind of computer before the batch.
• Portable file paths: pathlib, not hand-joined strings with / or \.
• The same results: floating-point output can differ slightly between chips and maths libraries. Decide what counts as equal before you compare.

The fleet way: Oarbank

Oarbank is a self-hosted job queue for the Macs, Linux machines and Windows PCs you own. One computer runs the coordinator; every computer that computes runs a node. You package the work once as a module with the open-source Oarbank SDK (Apache-2.0), and each set of jobs you start is a campaign.

For Python, the module has two small files besides its manifest: a coordinator side that turns parameters into jobs (one per alpha and seed here), and a runner, a plain Python script that reads one job’s payload, calls run() and writes its result. The parts that are hard by hand:

• One environment on every node. With runtime = { kind = "uv" } and a uv.lock, each node builds the same locked environment offline from wheels in the bundle; oarbank-sdk bundle wheels downloads the pinned wheels for every platform the module declares.
• Mixed operating systems on purpose. The manifest lists the platforms the runner supports, and jobs go only to nodes that match and have the CPU and memory a stage asks for.
• Proof before results count. A node runs the module’s doctor and its golden jobs, which have known answers, before it gets real work; replicas on other nodes catch one that drifts.
• The owner’s work first. A memory guard, per-node caps and fewer jobs while someone uses the computer.
• A sandbox. Every job runs in the operating system’s own sandbox and reaches only the network hosts you approved for the module.

Equal inputs are one job: a campaign that asks for a parameter set already computed gets the stored result instead of a second run. oarbank campaign retry-failed reruns only failures, and oarbank campaign download fetches the results. Build a module in a day walks through a complete module.

It is more work up front than client.map(): you write a manifest, a runner and golden jobs, and pass the SDK’s conformance kit before installing it. It suits work you run again and again across computers people also use; for an interactive notebook session on one cluster, Dask or Ray fit better. Oarbank is source-available: free for personal and noncommercial use, with a commercial licence from Codonic for commercial use.

Python on several computers

Can multiprocessing use more than one computer?

No. multiprocessing and concurrent.futures start processes on the computer the program runs on. To use other computers you need something that sends work over the network, such as GNU parallel over SSH, Dask distributed, Ray or a task queue.

What is the simplest way to run Python on several machines at home?

For one sweep on Macs and Linux machines with SSH, GNU parallel with --sshlogin runs one command per parameter set across them. For Python code you call as functions, Dask distributed needs a scheduler on one computer and a worker on each of the others.

Does Ray work with Mac and Windows machines in one cluster?

Ray supports multi-node clusters on Linux only. On macOS and Windows it stops with “Multi-node Ray clusters are not supported on Windows and OSX” unless you set RAY_ENABLE_WINDOWS_OR_OSX_CLUSTER=1, which lets you proceed at your own risk.

Why do my Dask workers fail with ModuleNotFoundError?

The worker cannot import your code or a package it needs. client.upload_file() sends one file to every worker; packages must be installed on every computer, at the same versions, or Dask warns “Mismatched versions found” and results can fail to unpickle.

Do all the computers need the same Python?

In practice, yes: the same Python version and the same locked package versions, with wheels for every platform you use. Run one job by hand on each kind of computer before the whole batch.