Running an agentic coding assistant entirely on your own machine is possible with a local model stack built on Ollama, Gemma 4, and Claude Code. This setup gives you a private, no-subscription workflow for terminal-based coding tasks, including generating and editing project files from natural language prompts.
If you want Claude Code to use a local model instead of a hosted Anthropic model, the key is pointing it at Ollama’s local API and choosing a Gemma 4 model your hardware can handle.
🎥 Video Overview
What this setup does
This workflow combines three pieces:
- Ollama to run local language models on your computer
- Gemma 4 as the model that handles prompts and code generation
- Claude Code as the terminal-based coding agent interface
With the right environment variables, Claude Code can send requests to Ollama on localhost instead of using a remote API.
Why use Claude Code with Ollama and Gemma 4?
- Privacy: prompts and code stay local
- No rate limits from hosted APIs: you are limited by your own hardware
- No monthly subscription required for inference: the model runs on your machine
- Useful for coding workflows: file generation, edits, and iterative prompts from the terminal
This approach is especially appealing for developers who want a local-first AI coding setup or who are experimenting with smaller open models.
What you need before you start
Before installing anything, make sure you have:
- A Windows, Mac, or Linux machine
- Enough GPU and system memory for the Gemma 4 model you plan to use
- Node.js installed if your system needs it for the Claude Code package installation
- Terminal access
The process is very similar across operating systems, but exact commands for environment variables can differ.
Step 1: Pick a Gemma 4 model your computer can run
Gemma 4 comes in multiple sizes. The most important early decision is choosing a model that fits your hardware well enough to be usable.
A practical way to do this is to use a hardware compatibility tool such as canirun.ai. Select your GPU and search for Gemma 4. The tool can help estimate whether a model will run smoothly, barely run, or be a poor fit.
In the demonstrated setup, a smaller Gemma 4 variant such as e4b was chosen as a balance between speed and capability.
Why model size matters
- Smaller models are faster and easier to run locally
- Larger models usually provide better coding performance, but require more VRAM and RAM
- If the model is too large, responses may be slow or unstable
If your machine supports a larger Gemma 4 model, you will generally get a better coding experience.
Step 2: Install Ollama
Go to the Ollama website and install the version for your operating system. On Windows, you can use the installer directly. Once installed, open a terminal and verify Ollama is available.
ollama ps
ollama lsIf Ollama is installed correctly, the commands should run even if no models are currently loaded.
Step 3: Download the Gemma 4 model in Ollama
After choosing a Gemma 4 model from the Ollama model library, pull it from the terminal.
ollama pull gemma4:e4bThen confirm the model is installed:
ollama lsYou should see your Gemma 4 model in the list.
Step 4: Test that Gemma 4 runs locally
Before bringing Claude Code into the workflow, confirm the model launches correctly in Ollama.
ollama run gemma4:e4bOnce the session starts, enter a simple prompt to confirm the model responds normally.
If it works here, the local model layer is ready.
Step 5: Install Node.js if needed
Claude Code installation may require Node.js, especially on Windows. If npm is not available in your terminal, install Node.js and restart the terminal before continuing.
If you try to install Claude Code and get an error saying npm is not recognized, Node.js is either not installed properly or your terminal session needs to be restarted.
Step 6: Install Claude Code
Install the Claude Code package globally:
npm install -g @anthropic-ai/claude-codeAfter installation, open a new terminal session.
Step 7: Point Claude Code to Ollama instead of Anthropic
This is the part that makes the local setup work. You need to set environment variables so Claude Code uses your local Ollama API endpoint.
The demonstrated configuration uses:
- Auth token: a placeholder value such as
ollama - Base URL:
http://localhost:11434
Windows
$env:ANTHROPIC_AUTH_TOKEN="ollama"
$env:ANTHROPIC_BASE_URL="http://localhost:11434"Mac/Linux
export ANTHROPIC_AUTH_TOKEN=ollama
export ANTHROPIC_BASE_URL=http://localhost:11434The important detail is the base URL. Claude Code needs to look at Ollama’s local API on port 11434.
One practical warning from the setup: using a /v1 suffix on the Ollama base URL did not work in that configuration. If your setup fails with a /v1 path, try the plain localhost endpoint instead.
The exact environment variable commands depend on your operating system and shell.
Step 8: Launch Claude Code with your Gemma 4 model
Navigate to the folder where you want Claude Code to work, then start it with your selected model.
claude --model gemma4:e4bIf prompted to trust the folder, approve it only if you are comfortable letting the agent work there.
At this point, Claude Code should be running against the local Gemma 4 model through Ollama.
What you can do with this setup
Once connected, Claude Code can handle common terminal-based coding tasks such as:
- Generating a new HTML file
- Creating a landing page structure
- Editing existing files based on prompts
- Iterating on content such as pricing tiers, sections, or styles
A simple example is asking it to create a single-page SaaS landing page using Tailwind CSS via CDN, including a hero section, features list, and pricing table. In the tested workflow, the model generated an index.html file and later updated the pricing tiers when prompted.
Example prompt for a local coding task
You can use natural language prompts with clear requirements. For example:
Build a modern SaaS landing page for a tool called 'CloudSync'. Use a dark 'midnight' theme with high-contrast accents. Create an index.html file with a hero section, features list, and a pricing table using Tailwind CSS via CDN. The features and tools I ask for are requirements for this page. Please follow the directions.Being specific helps smaller local models follow instructions more reliably.
What to expect from a smaller Gemma 4 model
A smaller Gemma 4 model can work for coding tasks, but there are tradeoffs.
Likely strengths
- Fast enough for basic prompts
- Usable for simple HTML, structure, and iterative edits
- Good for experimenting with local-first AI coding
Likely limitations
- May need more explicit prompts
- Can struggle with longer context windows
- May forget earlier details during extended sessions
- Can be less reliable than larger models for complex coding tasks
If performance feels weak, the first thing to evaluate is whether your hardware can support a larger Gemma 4 model.
Common problems and fixes
Claude Code is not using the local model
Check that your environment variables point to localhost:11434. If Claude Code is still trying to use a hosted endpoint, the local API configuration is likely missing or incorrect.
Ollama command not found
Ollama may not be installed correctly, or your terminal session may not have refreshed after installation. Restart the terminal and try again.
npm is not recognized
Install Node.js, then restart your terminal or computer if necessary.
The model responds, but coding quality is poor
This is often a model size issue. Smaller local models can work, but they may need:
- More detailed prompts
- Shorter sessions
- Simpler tasks
- A larger model if your hardware supports it
Claude Code says it cannot access the file system
That can happen depending on how the session is launched and what permissions are available. In the tested workflow, file access behavior was inconsistent at first, then later allowed file creation after confirmation. If file operations fail, review folder trust and permission prompts carefully.
The generated page looks generic
That is normal for a first-pass prompt. Ask for specific branding, stronger layout direction, feature copy, pricing language, or visual hierarchy improvements. Local models often produce better results when requirements are spelled out clearly.
Tips for better results with local coding agents
- Choose the largest model your hardware can comfortably run
- Keep prompts explicit, especially with required sections and output files
- Review edits before approving them
- Use shorter iterative requests instead of one huge instruction block
- Restart sessions when context quality degrades
For example, asking for a landing page first and then separately asking for pricing tier updates can work better than trying to pack everything into one prompt.
Is this actually free?
From a software usage standpoint, this local setup avoids hosted API billing for inference because the model runs on your own hardware. However, there is still a real hardware cost in terms of compute, memory, electricity, and storage.
So while there may be no monthly AI API fee in this workflow, the practical limit is your machine’s capability.
Who should use this setup?
This approach is a good fit for:
- Developers who want local AI coding tools
- People who care about privacy
- Anyone testing open models for code generation
- Users who want to avoid cloud subscriptions for simple coding assistance
It is less ideal if you need the strongest possible coding performance on large codebases and your hardware cannot run a sufficiently capable local model.
Quick setup checklist
- Check which Gemma 4 model your hardware can run
- Install Ollama
- Pull the Gemma 4 model in Ollama
- Test the model with
ollama run - Install Node.js if required
- Install Claude Code with npm
- Set environment variables to point Claude Code at
localhost:11434 - Launch Claude Code with your local Gemma 4 model
- Start with small, clear coding prompts
Bottom line
Yes, you can run Claude Code locally with Ollama and Gemma 4, and it can handle practical coding tasks such as generating and editing project files from the terminal. The most important factors are choosing a model that fits your hardware, configuring Claude Code to use Ollama’s local API, and keeping prompts clear enough for the model size you selected.
If you want a private, local-first AI coding workflow without relying on hosted inference, this is a strong starting point.