Introducing Qwen3.5 4B on LLM Gateway—optimized by AssemblyAI for the fast rewrite tasks at the center of voice products

The right model for voice rewrite

A small model's value doesn't show up on a general leaderboard, which averages across tasks it was never designed for. It shows up on the specific task it was built to do. The qwen3.5-4b-32k-fast model hosted by AssemblyAI is purpose-built for one class of work: fast rewrites over voice data.

• Dictation cleanup—filler words out, punctuation in, self-corrections resolved
• Transcript rewrite—raw speech-to-text output reshaped into emails, notes, or messages
• Live formatting—applying formatting rules to streaming output while the user is still talking
• Turn summarization—compressing a conversational turn into a line the next system can act on

Frontier models keep getting better, but not every task needs one. For voice tasks, customers are increasingly choosing small, fast, low-cost models instead—and choosing a small model means being clear about the trade: qwen3.5-4b-32k-fast supports max_tokens, temperature, and stream only.

What a rewrite looks like

Hi, Marissa. Um, quick recap on where we landed, uh, with the Q3 vendor review. So, uh, we got through 6 of the 8— oh no, 5 of the 8 assessments. Um, and the last 3, they’re waiting on security sign-off. Um, the big thing is that pricing came back higher than, than we modeled, something like, uh, between 12— no, 15% over. So I think we need to revisit The budget line before we commit to anything. Can you pull the original forecast for me? And I’ll put 30 minutes on the calendar.

Notice what the rewrite handled: filler words dropped, both mid-sentence corrections resolved to the speaker's final intent—"6 of the 8— oh no, 5 of the 8" became five of the eight, and "between 12— no, 15%" became 15%—and the recap reformatted into clean paragraphs. The numbers in the corner are real: this exact audio ran through the exact pipeline below, and the rewrite came back in 622 ms at a cost of about a hundredth of a cent.

How Qwen3.5 4B benchmarks on voice rewrite tasks

We benchmark every model on our LLM Gateway on the task they're built for, and for qwen3.5-4b-32k-fast, that task is voice rewrite.¹ Dictation cleanup, transcript rewrite, formatting, and turn summarization, measured on representative voice data rather than general-purpose benchmark prompts.

For building dictation features

If you're building dictation—voice typing, dictated notes and messages, spoken commands that become clean text—the pattern we recommend is a two-step loop: our Sync API for turning audio into raw words fast at flagship accuracy—1.59% word error rate on short-form audio in ~134 ms at the median—then a rewrite pass to turn raw words into clean, formatted text with qwen3.5-4b-32k-fast.

Here's the whole pipeline—the same two-step loop we run ourselves—in about 30 lines. One API key covers both calls. Speech in, clean formatted text out.

import requests

API_KEY = "YOUR_API_KEY"

# Step 1: speech in — one call to the Sync API, finished
# transcript back in the same response (~134 ms p50)
with open("dictation.wav", "rb") as f:
    sync = requests.post(
        "https://sync.assemblyai.com/transcribe",
        headers={
            "Authorization": API_KEY,
            "X-AAI-Model": "universal-3-5-pro",
        },
        files={"audio": ("dictation.wav", f, "audio/wav")},
    )
raw_text = sync.json()["text"]

# Step 2: clean text out — one rewrite pass through
# Qwen3.5 4B on LLM Gateway
rewrite = requests.post(
    "https://llm-gateway.assemblyai.com/v1/chat/completions",
    headers={"Authorization": API_KEY, "Content-Type": "application/json"},
    json={
        "model": "qwen3.5-4b-32k-fast",
        "max_tokens": 500,
        "messages": [
            {
                "role": "system",
                "content": "Rewrite raw dictation as clean, formatted text. "
                           "Fix punctuation, drop filler words, and resolve "
                           "self-corrections to the speaker's final intent.",
            },
            {"role": "user", "content": raw_text},
        ],
    },
)

print(rewrite.json()["choices"][0]["message"]["content"])

For live formatting, stream the rewrite and render tokens as they arrive:

curl https://llm-gateway.assemblyai.com/v1/chat/completions \
  -H "Authorization: YOUR_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "qwen3.5-4b-32k-fast",
    "stream": true,
    "messages": [
      {"role": "system", "content": "Format this dictation as a bulleted list of tasks."},
      {"role": "user", "content": "first thing is uh review the contract then ping legal about the uh indemnity clause and last book the offsite venue"}
    ]
  }'

For building voice agents

Voice agents make a different demand of their model: tool calling, multi-step reasoning, and accuracy under a live conversation's latency budget. Because Qwen3.5 4B doesn't support tool calling, it isn't the right choice here. For agent workloads, we recommend qwen3-next-80b-a3b, which is strong in a different dimension: accuracy on agent tasks relative to the mid-tier proprietary models, at a fraction of their cost.

Using it is the same integration as Qwen3.5 4B: point your chat-completions client at llm-gateway.assemblyai.com and set the model to qwen3-next-80b-a3b. Tool calling and the full agent loop work through the standard OpenAI-compatible interface, so existing agent frameworks connect without special handling.

Pricing and free credits

LLM Gateway usage draws down your credits at each model's list price, so those credits go a long way on a 4B model: at $0.10/$0.50 per M tokens, a dictation cleanup costs a fraction of a cent.

LLM Gateway is the inference layer for Voice AI, and the Qwen family sits alongside the frontier models your product already depends on—Claude, GPT, and Gemini are all available through the same endpoint and the same API key. Fast open models for the high-volume rewrite path, frontier models for the reasoning path, one integration for both. The full model list and pricing is public, no auth required.

Get started

Create a free account, grab your API key, and the LLM Gateway quickstart defaults to qwen3.5-4b-32k-fast—your first rewrite is one curl command away. If you'd rather start without writing code, you can test speech-to-text and rewrite together on your own audio in the Playground.

Frequently asked questions