BTL-4

A 35B agentic reasoning model from Bad Theory Labs, fine-tuned from Ornith-1.0-35B on an execution-gated reasoning corpus.

Built for tool use, software engineering and long-horizon agent work.

---

Benchmarks

Benchmark	BTL-4	Base Ornith-1.0-35B	Harness
BFCL v4 (AST)	73.5%	69.2%	official ast_checker, all 1240 cases
LiveCodeBench v6	66.1%	—	official, 442 problems, 2024-08 → 2025-05
SWE-bench Verified	78.4%	—	official harness

BFCL and LiveCodeBench were run in-house with the official scorers, full splits, no subsetting. The BFCL number is a paired comparison: identical harness, identical decoding, only the weights differ.

LiveCodeBench by difficulty

	pass@1
easy	99.1%
medium	86.7%
hard	60.5%

The set is 45% hard problems, which is what pulls the aggregate down.

---

Usage

from transformers import AutoModelForCausalLM, AutoTokenizer

model_id = "badtheorylabs/BTL-4"
tok = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(model_id, dtype="bfloat16",
                                             device_map="auto")

messages = [{"role": "user", "content": "Refactor this function to be pure."}]
inputs = tok.apply_chat_template(messages, add_generation_prompt=True,
                                 return_tensors="pt").to(model.device)
out = model.generate(inputs, max_new_tokens=2048)
print(tok.decode(out[0][inputs.shape[-1]:], skip_special_tokens=True))

Serving

vllm serve badtheorylabs/BTL-4 \
  --max-model-len 131072 \
  --enable-auto-tool-choice --tool-call-parser qwen3_xml \
  --reasoning-parser qwen3 \
  --trust-remote-code

Generation settings

Ornith's published settings, used for every number above:

temperature	1.0
top_p	0.95
context	262144 native

Give it room to think. LiveCodeBench improved 60.9% → 66.1% purely by raising the output budget from 16K to 32K. At 16K, 23.5% of problems were truncated mid-solution and scored zero. Hard problems reason longer; cutting them off costs real points.

---

What it is good at

• Tool calling — 73.5% BFCL v4 AST, +4.3 points over base
• Competitive programming — 99.1% easy / 86.7% medium on LiveCodeBench v6
• Long context — 262K native, and it uses it

What it is not

• Not a chat model. It reasons before answering and is verbose by default.
• Reasoning accumulates across agent turns. The chat template strips prior reasoning from older turns, but this only works if your harness separates it into reasoning_content. With vLLM, that means --reasoning-parser qwen3. Without it, thinking lands in content, accumulates every turn, and long agent runs degrade.
• Token-hungry on hard problems. Budget accordingly.

---

Training

Fine-tuned from Ornith-1.0-35B on an execution-gated reasoning corpus: candidate trajectories were kept only where the resulting code actually ran and passed its tests, so the reasoning that survived is reasoning that led somewhere.

Citation

@misc{btl4-2026,
  title  = {BTL-4: An Execution-Gated Agentic Reasoning Model},
  author = {Bad Theory Labs},
  year   = {2026},
  url    = {https://huggingface.co/badtheorylabs/BTL-4}
}