Same answers, half the tokens: our first two models are on Hugging Face

GreenPT is now on Hugging Face. We’ve published our first two open models:

• GreenPT/Qwen3.5-9B-honey: LoRA adapter on Qwen3.5-9B
• GreenPT/Qwen3.8-27B-honey: LoRA adapter on Qwen3.8-27B

Neither one knows anything the base model doesn’t. They’re trained for a single behaviour: give the same answer in fewer tokens. It is the same rule behind Honey for Mac, moved from the tool into the weights.

This is an early experiment. We’re publishing it now, warts included, because the early numbers changed how we think about efficiency, and because we’d rather have people poke at it than polish it in private.

The problem we’re chasing

Every token a model produces costs compute, money, and energy. Most of the tokens in a typical answer are not the answer. They’re the preamble, the restated question, the “Certainly! Here’s a breakdown”, the closing summary of the summary.

For a chatbot that’s mildly annoying. For agents, models calling models thousands of times a day, it’s pure overhead, paid by both the sender and the receiver.

You can prompt this away. We tried: a detailed style instruction in the system prompt cut output by 74%. It also made every request 8.3× more expensive in total, because the instruction itself is re-sent and re-processed on every call. The brevity was real, but the bill went the wrong way.

So the question became: can you train the brevity in, so it costs nothing at inference?

What a LoRA adapter is (30-second version)

A LoRA adapter is a small set of extra weights trained on top of a frozen base model.

Instead of updating a large weight matrix W, you learn two thin matrices A and B and compute W + BA. Because A and B are low-rank, hence Low-Rank Adaptation, the trainable part is a few hundred megabytes, not tens of gigabytes.

That buys you three things:

• Cheap to train: one GPU, hours, not weeks.
• Portable: ship the adapter alone; users bring the base model they already have.
• Non-destructive: toggle it off and you’re back to stock.

It also means the change is small enough to reason about. Which makes the results below more interesting, not less.

The numbers

We ran paired A/B evaluations: same prompts, same greedy decoding, thinking off, adapter toggled on and off. 26 prompts with a checkable answer, across Q&A, explanations, and code.

	Output tokens	Correct (base / adapter)
9B honey	−52%	25/26 · 25/26
27B honey	−46%	26/26 · 26/26

Per category, the 9B adapter cut Q&A by 58%, explanations by 56%, and code by 27%. The 27B went further on Q&A (−78%) and held −47% on explanations. Same pass rate in every case. Where the two models failed, they failed on the same prompt.

Half the output for identical correctness, from an adapter that fits on a USB stick.

What didn’t work

Publishing early means publishing the bad parts too.

The 27B adapter is longer on bare code prompts. Ask it “write a function that does X” and it produces code that’s about 12% tighter than the base, then appends a “How it works” section the base model never writes. Net effect: +24% tokens on that prompt type. On realistic coding tasks (fix this bug, refactor this, extend this, with code in the prompt), the adapter saves 5–12% as expected. So the regression is narrow, and we know where it comes from: our style corpus has explanation sections on its code examples. That’s a training-data fix, and it’s next on the list.

Two traps that bit us, so they don’t bite you:

1. Adapter keys need to match the model tree you load. Our trainer saved weights under model.language_model.layers…; AutoModelForCausalLM builds model.layers…. PEFT silently ignores every mismatched tensor, so you get a “working” model that is just the base. The published adapters carry remapped keys, but if you retrain, check this first.
2. The base repo’s stop token is wrong for chat. Qwen/Qwen3.5-9B ships an eos_token_id of <|endoftext|>, while its chat template ends turns with <|im_end|>. A harness inheriting that config never sees the adapter stop, and it looks exactly like a broken adapter. Generate with eos_token_id=[248046, 248044]. Our adapter repos ship a correct generation_config.json.

One serving note. Running an unmerged LoRA costs roughly 37% extra per token in our setup, because every adapted layer does an extra pair of small matmuls. For production, merge the adapter into the base weights once and serve a plain model. Keep it unmerged only while you’re iterating or hot-swapping adapters.

Try it

from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import PeftModel

adapter = "GreenPT/Qwen3.5-9B-honey"
tok = AutoTokenizer.from_pretrained(adapter)
model = AutoModelForCausalLM.from_pretrained("Qwen/Qwen3.5-9B", torch_dtype="auto", device_map="auto")
model = PeftModel.from_pretrained(model, adapter)

msgs = [{"role": "user", "content": "Explain what a mutex is."}]
ids = tok.apply_chat_template(msgs, add_generation_prompt=True, return_tensors="pt").to(model.device)
out = model.generate(ids, max_new_tokens=512, do_sample=False, eos_token_id=[248046, 248044])
print(tok.decode(out[0][ids.shape[-1]:], skip_special_tokens=True))

Toggle the last PeftModel line off and run the same prompt to see the difference yourself.

Where this goes

Two beliefs came out of this work.

Efficiency is something you can train. We didn’t make a new model. We took a model people already run and made it cheaper to run, with a few hundred MB and a few GPU-hours. That recipe shouldn’t be specific to Qwen. We’re building open-source pipelines to fine-tune any open model for efficiency, covering training data, training code, the eval harness, and the traps, and releasing all of it. Not one model on one benchmark; a repeatable process anyone can run on the model they’ve already chosen.

The bigger win is between agents. When a model’s output is another model’s input, every polite preamble is paid for twice: once to generate, once to read. We think more efficient agent-to-agent protocols will matter more than any single-model optimisation. The adapters are our first step: a model that already speaks tersely is the easiest peer to build a lean protocol on.

Both models are live on Hugging Face. Run them, break them, and tell us what you find.