Prism ML Inc. announced Thursday the launch of Bonsai 2 27B, the second generation of its ultra-compact multimodal generative artificial intelligence small enough to fit on PCs and some high-end mobile devices.
The company said it used ternary, which uses three parts, to scale down its Qwen3.8 27B-based model. Qwen3.8 weighs in at about 56 gigabytes at its full 16-bit uncompressed size, and Bonsai 2 reduces it to about 5.9 gigabytes while retaining around 98.2% of its capabilities.
Although it is possible to shrink AI models using other compression techniques called quantization, these methods usually strip away accuracy, knowledge and other systematic capabilities. Qwen3.8’s minimal memory footprint is 9.4 gigabytes.
Ternary provides an interesting compression method when shrinking the “weights,” or parameters that make up the model. Weights are the model’s numerical dials that control how it processes information and generates outputs. In full-size models, these are represented by 16 bits; with PrismML’s approach, these are simplified down to ternary, or three bits, represented by +1, 0, and -1. This lets the company store information in a much smaller memory footprint while still holding onto reasonably high intelligence.
In essence, this allows Bonsai 2 to punch well above its weight class at a very small size.
On benchmarks, Bonsai 2 showed close performance on agentic and tool calling compared to Qwen3.8 within three points, at 77.6 and 79.8 respectively; with aggregate scores of 81.6 and 82.2 for coding across HumanEval+, LiveCodeBench v6, MBPP+ and BigCodeBench; and 82.7 and 81.3 for knowledge and reasoning across MMLU-Redux, GPQA Diamond and AA-LCR.
The model can run on an Nvidia GeForce GTX 5090 card without quantization, reaching 143 tokens per second and 46.8 tokens per second on Apple Inc.’s M5 Max chip. The company said the model consumes extremely low power per token at 0.714 megawatt-hours, making it 40% more energy-efficient than other 8B models running at full precision, meaning uncompressed.
Ultra-small models let users run AI locally on their own machines without sending inference to the cloud. Anytime data is sent across the internet, there can be a delay in receiving a response, or sensitive information might be sent to a third party. Bringing intelligence onto a local machine eliminates third-party data sharing, keeps prompts and responses local, helps meet strict privacy regulations and can improve security.
For example, simple translation, summarization and search organization could run on-device, while long-horizon task comprehension and research might need to be sent to an expensive cloud model. For an everyday user, or even an enterprise use case, running a local model that is far less expensive and respects privacy when a task is simple and involves sensitive information, while scaling up to highly intelligent, cloud-based models to handle complex, high-touch, goal-oriented work.
The new model runs on Nvidia graphics processing units via CUDA and on Apple devices, including Mac, iPhone and iPad, via MLX, through low-bit kernels. Model weights are available today under Apache 2.0 licenses.
