Smaug-Decisions: State-of-the-Art Audio Decisions in a 4B Model That Also Sees and Reads

By Balajee Sambasivam and Arvind Sundararajan

We are happy to welcome two new members to the Smaug family: Smaug-Decisions, a 4B model built on Google’s Gemma 4 E4B, and Smaug-Audio-Decisions-0.7B, a 744M audio model built on EmbeddingGemma 2. Both are open weights under Apache 2.0, and both ship inside one iOS and Android app.

Many of the things people want from a language model are decisions rather than conversations. Has the speaker finished their turn? Which sounds are in this room? Does this receipt total more than $50? Should this message go to billing or support? Each has a fixed set of answers, and what the application needs back is a probability per answer that it can compare against a threshold. We call these typed decisions.

Three properties matter for a model that makes them. It has to be accurate on the input the decision is about, which is often audio. It has to be calibrated, so that 0.8 means right about 80% of the time and a threshold means what it says. And it has to be cheap enough to run on every event, ideally on the device where the sound or picture is captured.

Results: ahead of the frontier APIs on audio

We measure audio decisions on Earshot, our benchmark of 6,671 items and 17.2 hours of audio in four tracks: conversation dynamics (turn completion, next speaker, speaker count, speech activity), external turn completion (the Smart Turn v3.2 test set), sound and music (genre, instrument, tempo, scene, sound events) and a small pilot track. Every model went through the same harness, and the APIs received the audio and returned JSON probabilities.

Scores are skill, accuracy rescaled so that guessing is 0 and a perfect score is 100, averaged per track and then over the tracks. Calibration is the expected calibration error (ECE): split the n answers into bins B_m by their top probability and compare each bin’s accuracy with its average confidence.

skill = 100 · (acc − chance) / (1 − chance)

ECE = Σ_m (|B_m| / n) · | acc(B_m) − conf(B_m) |

Model	Audio skill ↑	ECE ↓	Conversation	Ext. turn	Sound & music	Pilot
Smaug-Decisions (4B)	60.1	0.069	48.2	80.8	66.1	45.2
Gemini 4 Argon (API)	59.4	0.123	49.6	76.5	64.7	47.0
Smaug-Audio-Decisions-0.7B	53.0	0.086	42.4	65.8	58.0	45.8
Gemini 3.8 Flash (API)	50.5	0.204	48.5	81.8	42.4	29.1
AudioJev (Qwen2.5-Omni-3B)	43.6	0.191	26.7	62.0	57.6	28.2
Gemma 4 E4B-it, zero-shot	5.0	0.464	3.6	11.2	5.2	0.1

Smaug-Decisions has the highest audio skill of any model we tested, ahead of Gemini 4 Argon, and its probabilities are much better calibrated (0.069 against 0.123). Smaug-Audio-Decisions-0.7B, at about a fifth of the size, is ahead of Gemini 3.8 Flash on skill and on calibration.

It also sees and reads

Most audio models stop at audio, and most small decision models stop at text. Smaug-Decisions answers audio, image and text questions with the same prompt and the same readout. On the public Jev Decision Index it scores 56.6 on text and 57.9 on images, the best results for a Gemma 4 E4B model on both. As far as we know it is the only 4B decision model with results on all three input types.

Reading a decision instead of generating it

A decision is rendered as one prompt: the input, optional rules and context, the question, and the options labelled A, B, C and so on, up to T. The model reads the prompt once. Instead of sampling an answer, we read the language-model logits of the option letters at the answer position. Let z_k be the logit of the letter for option k. After the model’s own tanh soft cap, a one-answer question takes a softmax over the options:

p_k = softmax_k( 30 · tanh(z_k / 30) )

A select-all question treats each option on its own, with one learned threshold b (8.01 for Smaug-Decisions):

p_k = σ( 30 · tanh(z_k / 30) − b )

So a decision costs one prefill and no decoding: about 200 ms per audio decision on an H100. Because options are shuffled during fine-tuning, the model learns what the options say rather than where they sit. And because calibration is an explicit objective, the probabilities can be used directly as thresholds, which is what makes a cheap on-device gate possible.

Smaug-Audio-Decisions-0.7B reads its decisions differently. Its base model is an embedding model, so each option line starts with a marker token, and a small linear head scores the hidden state at every marker: softmax over the markers for one answer, a sigmoid per marker for select-all. It answers in 9.7 ms on an H100.

Fitting a 4B multimodal model into 3.45 GB

The full-precision checkpoint is about 16 GB. To run it on a phone we packaged it for Google’s on-device runtime, LiteRT-LM, as one 3.45 GB file with the audio and vision encoders inside. Three facts about the model decide where the bytes go.

Most of the stored weights are lookup tables. Gemma 4 E4B has 4B effective parameters but 8.6B stored ones, and 3.5B of those are embedding tables: the token embeddings and the per-layer embeddings. They are looked up, not multiplied, so they tolerate very low precision. We store them at 2 bits.

The decoder carries the decisions. The transformer layers are stored at 4 bits. Plain post-training rounding to 4 bits cost several points, so the 4-bit decoder and the 2-bit tables are quantization-aware, fine-tuned to match the full-precision model in the format they ship in.

A decision only needs 20 rows of the output layer. The vocabulary has 262,144 tokens, but a decision reads only the letters A to T, so the phone build keeps an output head with just those 20 rows. The audio encoder is stored at 8 bits.

The result keeps almost all of the model’s quality at about a fifth of the size:

	Full precision, 16 GB	Phone build, 3.45 GB	Change
Earshot audio skill	60.1	59.4	−0.7
Earshot calibration error (ECE)	0.069	0.074	+0.005
Decision Index, text	56.6	53.9	−2.7
Decision Index, images (560 image tokens)*	57.2	55.1	−2.1

The phone build matches Gemini 4 Argon on Earshot (59.4) while running offline on a phone. The same weights are also packaged without the audio encoder (3.15 GB) or without the vision encoder (3.28 GB), and these component files give bit-identical letter logits on the inputs they support.

One detail from getting it onto the phone GPU

Our first compressed build picked the 20 letter rows out of the full output layer with a gather. That works on the CPU, but the phone GPU delegate rejects the boolean and gather operations it needs, so the model fell back to the CPU. The shipping build computes the 20 letter logits with one small matrix, 2,560 × 20, and maps them back to vocabulary positions with a second fully connected layer whose weights are a fixed one-hot matrix, plus a bias of −30 on every other vocabulary position. Every operation is now a plain matrix multiply the GPU accepts, and on the CPU the letter logits are bit-identical to the gather version.

On an iPhone

The Cascade app runs both models offline on iPhone and Android. Smaug-Audio-Decisions-0.7B ships inside the app (Core ML with int8 weights on iOS, ONNX Runtime on Android), and its int8 weights score the same as float: 53.0 on Earshot, with 99.6% of answers unchanged. On first launch the app asks how much to download and recommends a setup for the phone’s memory: the full model for audio, text and photos, a smaller file for text and photos only, or the audio model alone.

iPhone 14 Pro Max (6 GB), phone GPU	Measured
Smaug-Audio-Decisions-0.7B, audio decision	0.27–0.37 s
Smaug-Decisions, text decision	0.6–0.9 s
Smaug-Decisions, audio decision	~1.6 s
Smaug-Decisions, image decision	5.6–7.4 s
Smaug-Decisions load, first / later	13–21 s / 0.48 s
Peak app memory, both models loaded	1.35 GB
App download (iOS / Android)	602 MB / ~582 MB

Phones with 8 GB of memory or more get the build that reads images at 560 image tokens, as the model was trained; 6 GB phones get the same file with a 280-token vision encoder, which needs about 1 GB less memory per image; smaller phones run the audio model alone. Audio questions go to the 0.7B audio model first and move up to Smaug-Decisions only when its confidence is below 0.9; on a server GPU that keeps Smaug-Decisions’ accuracy (60.1) at 130 ms per decision instead of 210 ms.

Try it

# pip install "transformers>=5.20" torch soundfile pillow huggingface_hub
import sys
from huggingface_hub import snapshot_download
sys.path.insert(0, snapshot_download("abacusai/Smaug-Decisions", allow_patterns=["smaug_decide.py"]))
from smaug_decide import SmaugDecide

model = SmaugDecide.from_pretrained("abacusai/Smaug-Decisions")    # CUDA, Apple Silicon or CPU
model("call.wav", "Has the speaker finished their turn?")          # {'Yes': 0.93, 'No': 0.07}
model("room.wav", "Which sounds are present?", ["speech", "music", "alarm"], multi=True)
model("receipt.jpg", "Does the total exceed $50?")
model("The order never arrived.", "How upset is the customer?", ["calm", "annoyed", "angry"])

• Smaug-Decisions: the 4B model for Transformers (audio, images, text)
• Smaug-Audio-Decisions-0.7B: the audio model, with int8 Core ML units for iPhone
• Smaug-Decisions-LiteRT: the 3.45 GB phone build for LiteRT-LM
• Smaug-Decisions-Mobile: the iOS and Android app, helper scripts and an HTTP server
• The technical report, with the full tables

Both models are released under Apache 2.0, the license of their base models. We would love to hear what you decide with them.