Three deliberately selected Mac configurations ran native MLX models as distributed inference nodes and independent workers. For a four-page audit, local processing followed by cloud review used 51% fewer measured cloud API tokens, while taking longer. The useful workflow combined local assignments, exact evidence handling and selective cloud review.
I started with a practical question: how could Macs with different chip generations and memory capacities work together on a meaningful business backlog?
We already had access to capable cloud models. I wanted to understand where local inference could fit: whether multiple machines could contribute, whether larger models would improve the work, and whether we could reduce cloud usage without creating more work for ourselves.
The project took us through EXO, native Apple MLX, distributed inference over Thunderbolt and IP, and finally independent workers reviewing actual Arcalea website content.
The most useful result came when we changed how we divided the work.
We selected three Apple Silicon configurations to compare how chip generation and memory capacity affected local inference and distributed workloads:
| Name | Hardware | Unified memory |
|---|---|---|
| R2-D2 | M2 Pro | 32 GB |
| C-3PO | M1 Pro | 32 GB |
| BB-8 | M3 Pro | 18 GB |
The initial setup combined Open WebUI as the chat interface, LiteLLM and a Python router for directing requests, and EXO for local inference.
Some requests worked well. Others were slow enough to make the system frustrating. Earlier attempts to split a model across two machines had been particularly disappointing.
The challenge was figuring out why. Was it the model, the network, the interface, or the interaction among them?
I didn’t want to keep adding workarounds. We needed a clean baseline.
We moved to native MLX-LM while preserving the existing setup for comparison.
MLX is Apple’s machine-learning framework. MLX-LM provides the tools to load and run language models, including an API server that our existing interface could use. We started with an already downloaded Qwen3.5-9B-8bit model.
Apple Silicon’s unified memory makes these machines useful for local inference: the CPU and GPU share memory within each Mac. Connecting several Macs, however, does not create one seamless memory pool. Distributing a model still requires coordination across the network.
We began on one machine, measuring how quickly the answer started, how long it took to finish, and how much memory it used.
That immediately made the system easier to understand. The chat interface could add substantial context to a request. A model that responded quickly on its own could appear slow once the surrounding application added more material.
Separating those layers gave us a working baseline. Then we could ask whether a larger model or another machine actually improved it.
We tried larger models, including Qwen3.5-27B-4bit, on a task relevant to our work: extracting evidence from Arcalea pages and explaining what the sources supported.
One problem appeared repeatedly. Models would alter punctuation, combine passages, or present an edited sentence as an exact quotation.
Increasing model size did not reliably fix it.
We changed the workflow instead. The model selected a passage identifier, and code copied the corresponding text directly from the saved source.
That was an important design lesson. Exact copying is a job code can perform reliably. Asking a language model to regenerate the same text adds an unnecessary opportunity for error.
Interpretation remained a separate challenge. A perfectly copied quotation could still be attached to a weak recommendation.
With single-machine inference working, we returned to distribution, this time using MLX’s distributed backend.
We first connected R2-D2 and C-3PO through Thunderbolt networking.
Getting the connection right took some work. USB-C describes a connector; it does not tell you whether the cable supports Thunderbolt networking. We checked the detected connections and resolved a Local Network permission issue before testing.
The results were encouraging:
| Model and workload | R2-D2 alone | Two Macs over Thunderbolt |
|---|---|---|
| 9B, short answer | 4.39 sec | 4.39 sec |
| 9B, moderate context | 15.84 sec | 11.74 sec |
| 27B, moderate context | 50.12 sec | 34.97 sec |
The larger model required some tuning. An initial run produced a GPU timeout; reducing the prompt-processing batch size allowed it to complete.
Distribution helped with the longer input and reduced memory use per machine. Short answers benefited less. Processing the prompt and generating the answer were different parts of the workload, and adding a machine affected them differently.
That became clearer when we added the third Mac.
BB-8 joined through an existing IP connection using Tailscale. We installed MLX, transferred the models, and checked the files.
The first distributed connection attempts stalled. Allowing the new Python runtime through the Mac’s firewall resolved the issue, and all three machines could participate.
We compared two machines with three over IP:
| Workload | Two Macs | Three Macs |
|---|---|---|
| 9B, short answer | 5.5 sec | 7.1 sec |
| 9B, moderate context | 14.3 sec | 14.9 sec |
| 27B, short answer | 8.4 sec | 9.9 sec |
| 27B, moderate context | 37.2 sec | 32.1 sec |
For the 27B model, peak MLX memory fell from approximately 8.8 GB per machine to 6.4–6.6 GB.
The third Mac improved the longer 27B task by about 14%, while the short answer took longer. More machines gave us additional capacity, but they also introduced communication overhead.
We had established that distribution worked. The next question was how to make the machines useful.
At this point, I wanted to finish a test with something we could actually use.
We changed the assignment. Instead of asking the Macs to cooperate on one answer, we gave each machine its own pages to review.
Each worker ran a complete model and processed its assignments independently. We began with 12 saved Arcalea website pages, divided between C-3PO and BB-8.
The task was straightforward: identify specific issues, attach supporting evidence, and propose a change that could be verified.
After review, we had a seven-item backlog.
One confirmed finding was a guide advertised as both 42 and 45 pages. Other items identified claims needing support, a saved 404 to investigate, and a demo page that needed further inspection.
We also rejected recommendations that would have wasted time. A model alleged wording absent from the source. Another miscounted a framework. Several proposed adding explanations or qualifications already on the page.
The workers were useful at generating candidates. Reviewing those candidates was essential to making the output usable, a lesson we also explore in Your AI Will Cheat on Its Own Test.
We then compared two ways of auditing the same four pages.
In the first, a cloud model received the full saved text and performed the audit directly.
In the second, all three Macs performed the first pass. The same cloud model then reviewed a smaller packet: candidate findings, surrounding evidence, simple code-based checks, and one full-page spot check.
The direct cloud audit used 26,737 API tokens and took 29 seconds. Local workers plus cloud review used 12,993 tokens and took 151 seconds. That is approximately 51% fewer audit API tokens, at about five times the elapsed time. Both produced eight accepted findings and found three of four known issues, with different misses.
For background work, that can be a reasonable exchange. Two and a half minutes is acceptable for a task that does not need an immediate answer.
But the review packet matters. Both approaches missed an issue, and they missed different ones. The local-first reviewer missed a contradiction because the relevant passages were outside the evidence it received.
Reducing input saves tokens. It also reduces what the reviewer can see. That is why we kept full-page checks in the workflow.
This was a small comparison, and the savings cover the measured audit API calls rather than all setup and coordination. It was enough to demonstrate a useful tradeoff and identify what needed improvement.
We ended up with a practical division of work:
For the page audit, independent workers were the more useful default. They could move through separate assignments concurrently, while the cloud reviewer focused on a smaller body of evidence.
The next step is reliable continuous operation: saving progress, detecting changes, limiting retries, and escalating decisions that need an owner. We have a working workflow; making it dependable around the clock is the next stage.
We started by trying to make several Macs run a model together. We finished with something more useful: an approach to assigning work across local models, code, and cloud reasoning.
The Macs produced an actionable backlog. The hybrid approach reduced measured cloud usage. And the mistakes showed us exactly where review needed to remain part of the process.
That is the measure I would keep: how much usable work the complete system produces, including the effort required to check and correct it.