rncz.net

Pi Coding Agent + Llama.cpp: The Best Local AI Setup

llmllama-cpppicoding-agentlocal-aiskillsgit

I’ve tried a lot of AI coding harnesses — OpenCode, Hermes, OpenGSD — but I keep coming back to one lightweight agent by Mario Zechner: Pi. It runs really smooth with local AI models, and it doesn’t get in the way.

Pi is a lightweight wrapper around your AI model with just four simple tools: it can read files, edit files, write files, and use bash to run commands. That minimalism is exactly what makes it so powerful — and it’s why it works so well with local models, even small ones.

This post walks through installing Pi, connecting it to a llama.cpp server, building a small app, and using skills and git diffs to stay in control of what the LLM writes.

1. Install Pi

  1. Go to pi.dev and copy the install command.
  2. Open a terminal (CMD on Windows) and paste it, then press Enter to install.
  3. Clean the screen with cls and start the agent:
pi

That’s it — the Pi agent is running.

2. Start the llama.cpp server

If you don’t have llama.cpp set up yet, see my previous post on installing Llama.cpp on Windows.

Open a second terminal tab, cd into your llama folder, and start the server. This time we use the --models-dir parameter and point it at the folder containing your GGUF files (e.g. your downloads folder) instead of a single model:

llama-server --models-dir C:\Users\<username>\Downloads

Control-click the URL the server prints to open the built-in web chat. Say “hi” — if you get a response, everything is working.

3. Connect Pi to your model

Back in the Pi terminal, there are no models loaded yet. Use the /login command and pick llama.cpp:

  1. URL — the default already matches the address the server is listening on. Just hit Enter.
  2. API key — a local server doesn’t need one, so hit Enter again.
  3. Run /model and select your model.

Say “hi” and you should get an answer. That’s the entire setup — from here on, everything you type in Pi runs on your local model.

4. Build something

Let’s build a tic-tac-toe game:

Can you create a tic-tac-toe game in HTML in index.html?

Pi writes the file, and the game works. It’s pretty basic, so let’s prompt it again to make the game a bit fancier:

Can you make a score tracker of player one wins, player two wins, and the total of draws?

If a bug shows up, that’s possible with small models — you can try a larger LLM to get better code, or just ask the model to fix it. Let’s do the latter with a bigger 27B parameter model (with MTP):

I get errors when I want to reset the game. Can you fix it?

5. Use skills

A skill is an instruction set the LLM can load when it thinks it’s relevant to what you’re asking.

To create one, go to ~/.pi/agent/ and create a skills directory (if it doesn’t exist yet), then a directory for your skill with a SKILL.md inside. For example, an HTML separator skill that tells Pi: when a web app is written into a single index.html, split it into separate CSS, JS, and HTML files.

Back in Pi:

  1. Run /reload to pick up the new skill.
  2. Run /skill to see it listed.
  3. Select it with /skill:html-separator — it will be applied to your next prompt.

Then ask Pi:

Can you check index.html, which has CSS and JS, and separate it?

Everything gets split into three files, and the game still works. The skill did its job.

Another skill worth recommending is Grill Me by Matt Pocock. Describe what you want to build, and instead of jumping straight to code, the LLM asks you questions about it — a conversation that ends with you both understanding the same thing. When you finally let it execute, the code is much more likely to be what you actually wanted.

6. Stay in control of your code

Skills give you control over what gets implemented, but you still don’t know what kind of code was written. The fix: put the project in a git repository (mine is on a locally hosted Forgejo) and watch the diffs.

I use the Zed editor, which has a git panel that shows you the differences. Ask Pi to change the background color — it goes straight to style.css and edits the gradient. Refresh the browser, and there it is. In Zed, you see exactly what changed: one gradient color, replaced by another.

And because Pi has bash, you can just say:

Can you commit it and push it to origin?

The commit (“change background to dark gradient”) shows up in Forgejo, with the diff right there. That’s the way to stay in control of your code.

Next steps

  • Try Pi with different model sizes — larger models write better code in general.
  • Let the Pi harness build your skills in a SKILL.md that the Pi harness itself can use.
  • Use the Grill Me skill to come to a point where the LLM and the user of the coding agent are on the same page — all the things are clear and understood by both the user and the LLM.
  • Keep a git repo (local or hosted) and review diffs — it’s the most important part.
← All posts