PrismML Releases Ternary Bonsai 2 27B: A 5.9 GB Apache 2.0 Model Retaining 98.2% of Qwen3.8 27B Performance

PrismML has released Ternary Bonsai 2 27B, a ternary-weight version of Qwen3.8 27B that compresses the model from 53.80 GB in FP16 to just 5.93 GB while retaining 98.2% of the parent model’s average performance across 20 benchmarks. The Apache 2.0 licensed weights can run on a 16 GB laptop or a single 24 GB GPU, making a 27-billion-parameter model accessible on consumer hardware.

The release comes two months after the first Bonsai 27B, whose ternary variant retained about 95% of the parent model’s performance. Ternary Bonsai 2 improves on that retention while adding multimodal support and a 262,000-token context window.

How ternary compression works

Ternary Bonsai 2 27B keeps the Qwen3.8 27B architecture unchanged. The model has 27.36 billion parameters, split into a 24.35 billion language backbone, 2.54 billion in embeddings and LM head, and a 0.47 billion vision tower. The backbone uses hybrid attention, with about 75% linear-attention and 25% full-attention layers.

Each weight takes one of three values: -1, 0 or +1. Every group of 128 weights shares one FP16 scale. A ternary value carries about 1.585 bits, and adding 16 scale bits per 128 weights gives 1.71 bits per weight. Counting high-precision tensors brings the model to 1.72 bits per weight on average.

PrismML offers two GGUF packing formats. PTQ1_0 packs trits densely at 1.76 bits per weight, resulting in the 5.93 GB file size. PQ2_0 stores each trit in a 2-bit slot at 7.25 GB, which is cheaper to unpack during inference. The weights are also stored in a rotated basis using a blockwise Hadamard rotation with block size 1,024, an idea the whitepaper credits to SpinQuant.

Only 26.2 million parameters, or 0.0976% of the total, stay in higher precision. Those are the recurrent state path and normalization weights. The vision tower ships separately as a 0.63 GB GGUF file, loaded only when processing image input.

Benchmark performance and trade-offs

PrismML evaluated all models in thinking mode using EvalScope and vLLM on H100 GPUs. The comparison with conventional quantization shows sharper results. An IQ2_XXS build of Qwen3.8 27B averages 75.2 at 7.3 GB. On AIME26, Bonsai 2 scores 95.83 against 78.6 for the IQ2_XXS version. On LiveCodeBench v6, the gap is 90.07 versus 70.05.

The 98.2% figure is an average, and losses are uneven across tasks. Vision retains 96.3% of the parent model’s performance, while knowledge and reasoning retains 96.9%. Long-horizon agent work drops further. Bonsai 2 scores 52.8 on Terminal-Bench 2.1, against 69.7 for Qwen3.8 27B. On SWE-bench Verified, it scores 60.8 against 80.6, about 75% retention. Both tasks sit outside the 20-benchmark average.

Reasoning effort also affects results. At medium effort, the model averages 79.3 against 82.6 for the FP16 baseline. Low effort is not supported. All results are PrismML’s own and have not been independently reproduced.

Datos clave

Metric	Ternary Bonsai 2 27B	Qwen3.8 27B FP16
Model size	93 GB	80 GB
Bits per weight	72	16
Average benchmark retention	2%	100%
Hardware requirement	16 GB laptop or 24 GB GPU	Enterprise GPU

Deployment and hardware requirements

The model runs today on PrismML’s llama.cpp fork or its MLX runtime. Stock llama.cpp does not support the PTQ1_0 and PQ2_0 types. The Bonsai-demo repository provides the supported path with a setup script and a server launch script for chat, vision and tools at localhost:8080. Mac users can use the MLX pack with its bundled loader. A WebGPU demo runs the model inside a browser.

Performance figures, measured September 16, 2026, at batch size 1 decode on PrismML’s custom kernels, show an RTX 5090 reaching 142.5 tokens per second at 0.582 mWh per token. An RTX 4090 reaches 96.7 with PTQ1_0, and a 72 W L4 reaches 32.1. On Apple laptops, an M5 Max reaches 46.8 tokens per second and an M5 Pro reaches 27.7.

Neither packing format wins everywhere. PTQ1_0 is faster on Ada-generation cards and the L4. PQ2_0 is faster on Blackwell, Hopper, Ampere and Apple silicon, and at prompt processing everywhere. PrismML also claims 40% better energy efficiency than a full-precision 8B model.

What this means for Indian AI developers

For Indian developers and researchers working with limited hardware budgets, Ternary Bonsai 2 27B opens up a 27-billion-parameter model that runs on a laptop with 16 GB of RAM. The Apache 2.0 license allows commercial use, modification and redistribution without restrictions. The model supports text and image inputs, which makes it suitable for multimodal applications in document processing, visual question answering and content analysis.

The 262K-token context window enables processing of long documents, code repositories or multi-turn conversations without truncation. PrismML demos the model driving Cline coding agents and computer use on an RTX 5090, suggesting potential for local AI assistants and automation workflows.

However, developers should note the uneven performance retention. Tasks requiring long-horizon agent reasoning show larger drops, at about 75% retention on SWE-bench Verified and Terminal-Bench 2.1. The model also requires PrismML’s custom runtime, which is not yet compatible with standard llama.cpp deployments.

Source: MarkTechPost – https://www.marktechpost.com/2026/09/18/prismml-releases-ternary-bonsai-2-27b-a-5-9-gb-apache-2-0-model-retaining-98-2-of-qwen3-8-27b-performance/