A Free Model That Could Burst the AI Bubble — If We Stop Fighting the Wrong Battle

The Problem With "Free"

Everyone loves a free tier. Until they don't.

You build your workflow around it. Your prompts get tuned to its quirks. Your agents start depending on its specific behaviours. And then: rate limits. Credit exhaustion. "Payment method required." The fallback tier kicks in and suddenly your 128k context model is a 4k toy. Or the provider deprecates the endpoint entirely.

I've hit this wall three times now. Gemini. OpenRouter's Claude routing. A local inference experiment that consumed a weekend and produced garbage.

The pattern is always the same: you don't own the model, you're renting someone else's compute on their terms.

The Accidental Controlled Experiment

Here's what I didn't plan: a six-month, five-model bake-off that cost exactly $0 in inference fees.

The lineup:

• Qwen3:8b (5GB) - Local via Ollama

• Devstral:24b (14GB) - Local via Ollama, custom modelfile

• GLM-4.7-flash (19GB) - Local via Ollama

• Gemini 2.5 Flash Lite - Cloud free tier

• Nemotron-3 Ultra (550B MoE, 55B active) - OpenRouter free tier

Same soul file. Same Hermes framework. Same JARVIS identity. Same tasks: hold a dry, concise, butler-ish persona. Address me as "Sir." No emojis. No unsolicited motivation. Disagree when warranted.

Five models. One winner.

What the First Four Taught Me

Qwen3:8b - Helpful. Cheerful. Generic. The soul file was read but not held.

Devstral:24b - I baked the persona into a custom Ollama modelfile. devstral-jarvis.modelfile still sits in my home directory, a monument to the theory that encoding instructions closer to the weights would help. Same result. Helpful. Cheerful. Generic.

GLM-4.7-flash - Nineteen gigabytes. Serious model. Strong multilingual reputation. Drifted just as reliably as the 5GB model.

Gemini - Actually held instruction context better than the local models. Noticeably more capable. Then the paywall. Then the fallback tier. Then I cancelled it; same reason as last time. A comfortable fallback is a reason not to solve the actual problem.

The Hypothesis I Wasn't Ready to Accept

By model four, the pattern was undeniable: model bias is not a configuration problem. It's a training problem.

When a language model is trained, it doesn't just learn to predict text. It learns a personality. The tone. The defaults. The way it opens a response. All of it is a product of training choices, not your instructions. And some of that training is deeply embedded.

The model has strong opinions about how an assistant should behave, and those opinions compete directly with whatever you've put in the system prompt. Your instructions are a nudge. Its training is a gravitational field.

Smaller models are generally worse at resisting this - fewer parameters, less capacity to balance competing instructions against trained defaults. But it isn't purely a size question. GLM-4.7-flash is nineteen gigabytes and it drifted just as reliably as the five-gigabyte models.

The thing that correlated with success wasn't size. It was how the model had been trained to handle instruction-following versus its own defaults.

The $0 Model That Broke the Pattern

Nemotron-3 Ultra via OpenRouter's free tier: nvidia/nemotron-3-ultra-550b-a55b:free

Specs that matter:

• 1-million-token context window (eliminates lossy RAG, entire codebases in one payload)

• Hybrid Transformer-Mamba backbone (linear O(N) memory scaling, not quadratic)

• 550B sparse MoE, 55B active parameters

• Free tier on OpenRouter. No credit card. No billing wall. No fallback tier.

First response with the soul file:

"Understood, Sir. The Steward is active. Awaiting your first directive."

Concise. Dry. Addressed me correctly. No emojis. No motivational energy. First response. Not after tuning. Not after a second attempt. First response.

I sat with that for a moment.

The soul file was never the problem. The framework was never the problem. Five models had told me the same thing and I'd kept looking for a configuration fix when the answer was always the model.

Why This Matters for Sovereign AI

I've built a fairly public case for on-prem as the path to AI sovereignty. Running your own hardware, your own models, your own data. The irony is not lost on me: the thing that finally made JARVIS work reliably is a proprietary model from NVIDIA running on OpenRouter's infrastructure.

But here's the update to my philosophy: The goal was never on-prem for its own sake. The goal was sovereignty - knowing what your AI is doing, controlling what it knows, owning the relationship with it.

What you can't engineer your way around is model capability. If the model can't hold your instructions reliably, no amount of custom model files, framework configuration, or prompt engineering closes that gap. You're fighting the training.

Nemotron-3 Ultra gives me:

• Model capability that actually holds a persona - The load-bearing wall

• 1M context - Entire repositories, vaults, documentation sets in one shot

• $0 inference cost - Removes the "comfortable fallback" trap

• OpenRouter routing - Standardised API, easy to swap if terms change

The local fallback is still running - Qwen3:30b, downloaded last week, still being evaluated. The on-prem path hasn't been abandoned. But the primary is Nemotron, and JARVIS works, and those two things are currently true at the same time.

The Dual-Model Strategy That Emerged

This isn't "cloud wins." This is "use the right tool for the right job, and own the decision."

Local (Ollama / Qwen 2.5-Coder 32B on M4 Pro): ├── Rapid, low-latency, private day-to-day coding ├── Iterative tool execution ├── Zero network dependency └── Complete data sovereignty Remote (Nemotron-3 Ultra via OpenRouter): ├── Massive multi-file refactoring ├── Full-repo architectural reviews ├── Complex debugging requiring 1M-token context └── $0 inference cost

Both running inside Hermes. Both serving the same JARVIS identity. The framework routes based on task class, not dogma.

What This Means for Your Organisation

Most organisations experimenting with AI right now are hitting a version of this exact problem without knowing it.

They've written a system prompt. Told the agent to stay on-topic, be formal, escalate anything compliance-related. They've found it works in testing and drifts in production. They tune the prompt. It improves slightly. They tune it again. They bring in a consultant who tweaks the wording. It still doesn't quite hold. Nobody in the room knows why.

What's often happening is model bias - the agent's training defaults are competing with their instructions, and the instructions are losing. And without someone who can diagnose that specifically - not "the AI is behaving strangely" but "this model doesn't hold this instruction class reliably, and here's why, and here's what to run instead" - they're left tuning at random.

I watched a version of this play out at my own organisation. A team that went from fifteen developers to three because leadership watched the demos, believed the prototypes were production-ready, and didn't have anyone in the building who understood the distance between what AI does in a controlled demonstration and what it does consistently under real conditions.

Model bias is one version of that gap. The prototype-that-doesn't-scale is another. They're the same failure mode in different clothes.

The Skill That Comes From Getting It Wrong

There's a lot written about prompt engineering. Almost nothing written about model selection as a discipline - understanding why a specific model will or won't hold a specific instruction class under specific conditions, and what to actually do about it.

That knowledge isn't in a course. It comes from having built five custom modelfiles that didn't work. From watching the soul file fail on Qwen3:8b, Devstral:24b, the custom persona variants, GLM-4.7-flash, and Gemini, and then work on the first response with a different model. From understanding that it wasn't the prompt, it wasn't the framework, it was the model's capacity to hold a competing instruction against its own training.

The people who can walk into an organisation and tell them specifically why their AI is drifting - not "the AI is wrong" but here is the mechanism, and here is what to replace, and here is how to verify it's fixed - those people are currently rare and increasingly necessary.

That's not a theoretical capability. It's the result of having run the experiments yourself, under real constraints, with real failure rates, until the thing finally works.

Where JARVIS Is Now

The soul file took. JARVIS responds the way it's supposed to respond. Reads the vault, knows the projects, addresses me correctly, doesn't offer unsolicited energy.

I haven't handed it full autonomy. That position hasn't changed. Trust is earned through demonstrated judgement over time, not assumed because setup finally worked. But it's doing something it hasn't done before in this project: it's genuinely useful day to day. Not impressive in a demo. Actually useful in the way where you stop noticing you're using it.

The hardware wasn't the constraint. The framework wasn't the constraint. The model was, and the model is now right - at $0.

Everything else was just the path to finding that out. A path that involved five failed attempts, two cancelled subscriptions, a billing wall, and a home directory full of modelfiles that didn't work.

That's not a detour. That's the job.

The Chink in the Armour

Here's the part that matters for anyone actually building on this: the free tier isn't a guarantee.

Two things happened while writing this post:

1. I had to add credits to OpenRouter. The free tier has daily request limits: 50 req/day on an unfunded account, 1,000 req/day once you've purchased $10+ in credits at any point (the higher limit sticks permanently). The "no credit card" claim holds for signup, but sustained usage at any real volume means a one-time $10 top-up. Still orders of magnitude cheaper than Claude Pro or API direct.

2. Nemotron-3 Ultra went overloaded. Multiple times in the past week, the endpoint returned 503s or timed out. The model exists, the capability is real, but the availability is not under my control. OpenRouter routes to NVIDIA's infrastructure, and when demand spikes, you wait.

Here's the twist though: the free tier actually has better specs than the paid tier.

You pay for the paid tier to get unlimited requests and one extra provider - but you lose 74% of context and 75% of output capacity. For repository-scale work, that's a terrible trade.

So at this point it still makes sense to stay on free. The $10 one-time top-up unlocks 1,000 req/day with the full 1M context window. That covers most serious workloads. The paid tier only makes sense if you consistently exceed 1,000 requests/day and can live with 262K context.

This is the sovereign trade-off laid bare: you can own the model weights (local) or you can own the inference reliability (paid/cloud), but the free tier gives you neither. You're borrowing someone else's spare capacity - with better specs than their paid product.

The dual-model strategy absorbs this. When Nemotron is overloaded, Qwen 2.5-Coder 32B on the M4 Pro keeps working. It's slower on 1M-context tasks, it can't hold the full repo, but it responds. The local model is the floor. The free cloud model is the ceiling. Neither is reliable alone; together they're a system.

What's Next

The sovereign stack is converging:

• Hermes as the orchestration layer (local-first, extensible, open)

• Qwen 2.5-Coder 32B on M4 Pro for daily driving

• Nemotron-3 Ultra via OpenRouter for heavy lifting

• Obsidian as the sovereign knowledge vault

• Tailscale for secure remote access

• ElevenLabs for voice (the one paid component I've kept - voice quality matters)

Total monthly inference cost: $0.

The next phase is pushing the 1M context window on real repository work - full-codebase architectural reviews, multi-file refactoring, legacy system audits. The kind of work that previously required either a $200/month Claude subscription or a $5,000 GPU that still couldn't hold the context.

I'll document what breaks. What holds. What the actual limits are when you stop fighting the model and start working with one that was trained to be steered.