Paused

Local AI harness

My attempt to have frontier AI do the planning and free, local models do the building.

Benchmark results for local models on my machine
One small, bounded work package and its checks
The review that challenged the plan

Where it's at: paused. It's the project I can't quite quit. I'll be back.

How it started

When I started building with AI, I was on the $20 plans for Claude and ChatGPT, and I kept running out of room and waiting. That pushed me into open models for the first time. I rebuilt my whole PC for development first: four drives reorganized, a proper dev setup, models downloaded, and my own benchmarks to see which models were good at what on my hardware.

How it works

A frontier model plans the work the way a real engineering team would, breaking it into the smallest pieces a weaker model could be trusted with. Local models write the code. Other models check it, and the frontier model gives the final sign-off. It was built safety-first, so nothing would wreck my projects.

Where it's worked, and where it hasn't

The setup was the fun part. The rest humbled me. Safety-first turned into an elaborate system that never actually shipped code. When I sent the local models' work back to a frontier model for review, it called the code bad and risky for the project. Then better tools came out, and my upgraded coding subscription now covers most of what I build. So I paused it.

What it's teaching me

  • If I don't understand what the AI is building, I shouldn't assume it'll come back around. It won't. It goes off on tangents and takes everything to extremes.
  • Infrastructure eats the time you meant to spend creating. I now understand firsthand why tools that handle it for you are so popular.
  • Build versus buy. I used to tell that story to customers. Now I've lived it, and I think it's made me a better salesperson.

What I'm working toward

Treating local models like interns: give them small, well-defined work, check it carefully, and promote them as they earn it. That's especially useful for small tools where the code doesn't need to be perfect, and for some of the agent work in CareerHQ.

Questions I'm still chewing on

  • Would the planning and reviewing burn just as many tokens as building it directly?
  • Where do open models actually belong in a workflow, now that the paid tools are this strong?
  • How are other people using local models day to day?

How are you using local or open models? Tell me.