How to analyze a call recording with AI (no code required)

Analyzing a conversation is harder than analyzing a document, and the reason isn't the audio. It's that you usually don't know what you're looking for yet.

A support call has three or four things happening at once: a customer with a goal, an agent with a script, a mood that shifts halfway through, and a topic that nobody labeled when the recording was saved. Pull a transcript and you have a wall of text. Pull a thousand transcripts and you have a wall of text you'll never read.

So before anyone writes a line of code, it's worth answering a smaller question: what can AI actually tell me about this call? You can find out in about two minutes, on one real recording, in the AssemblyAI Playground — no integration, no SDK, no script.

This walkthrough follows the same example used in the video above: a customer calling a financial services company to activate a credit card, change their mailing address, and set a new PIN. One call, three intents, and a mood that moves. Perfect test material.

What "analyzing a conversation" actually means

"Conversation analysis" is a vague phrase covering a handful of very specific models, each answering a different question. It helps to see them separated out before you start flipping switches:

These are all part of the Speech Understanding family, and they run alongside transcription rather than as a separate pass. You get the transcript and the analysis from the same request. That's the part that makes this practical at scale later — but first, the two-minute version.

Step 1: Open the Playground and load your call

The Playground lives inside your AssemblyAI dashboard. Sign up free — new accounts come with $50 in credits — then log in and click Playground.

One detail worth knowing: this isn't a sandboxed demo running a trimmed-down model. The Playground uses your actual account, your API key, and the same production models the API serves, so what you see is what you'd get in production. Usage shows up in your dashboard's cost tab, which makes it a genuinely useful way to forecast spend before you commit.

Load your audio one of two ways: upload a file (wav, mp3, m4a, mp4, webm, and friends) or point it at a public URL. Below the upload area there's a language dropdown. Automatic Language Detection is selected by default, so you can leave it alone — but if you already know the call is in Spanish or German, setting it explicitly is the more reliable choice.

Step 2: Turn on the analysis you actually need

Here's where most people either under- or over-select. The toggles are cheap to flip, but each one answers a different question, so it's worth being deliberate.

For the credit card call in the video, the reasoning went like this:

• Speaker Labels — non-negotiable for a two-party call. A conversation transcript without speaker attribution is just a monologue that contradicts itself.
• Summarization — the call is long enough that nobody wants to read it end to end.
• Topic Detection — nobody tagged this recording when it was saved, so let the model categorize it.
• Key Phrases — surfaces the specific vocabulary. "Credit card" and "student loan" live in the same category but mean very different things for routing.
• Sentiment Analysis — the question behind the question: did this customer get agitated?

Depending on your use case you may also want Entity Detection (names, addresses, card numbers, dates) or PII Redaction, which strips sensitive values out of the transcript. On a financial services call that mentions card numbers and a home address, redaction stops being optional fairly quickly.

Then hit run.

Step 3: Read the results

The results view is where the abstract toggles turn into something you can act on. On the credit card call, here's what came back.

The transcript, split by speaker. Agent lines and customer lines, separated. This is the foundation for everything else — talk-time ratios, agent scorecards, and "who raised the objection" all depend on getting this right.

Key phrases. The highest-ranked phrases were credit card, minimum credit card payment, and activating the card. Notice what that gives you: not just "this is a banking call" but the specific tasks the customer was trying to complete. Each phrase comes back with a relevance rank, an occurrence count, and timestamps, so you can jump straight to the moment it was said.

Sentiment. 51 positive sentences and 10 negative ones. On its own that ratio is a blunt instrument — but 10 negative sentences in a 61-sentence call is a very different story than 10 in a 400-sentence call, and it's enough to decide whether this recording deserves a human listen.

A summary. A short read of what was discussed, so you can triage without pressing play.

Topics. Student loans, personal loans, credit cards, personal debt. This is the IAB Content Taxonomy at work — a standardized set of roughly 698 topics, which matters more than it sounds. Standardized labels mean two calls tagged "personal debt" in different months are actually comparable. Free-form tags never are.

That's a real analysis of a real call, in under two minutes, with zero code written. It's also a fair bit more specific than what a general-purpose sandbox gives you — if you're comparing options, we've written up how the major AI playgrounds stack up, and the short version is that audio evaluation needs an audio-shaped tool.

What's changed since this video was recorded

The video above is from late 2023, and the Playground has moved on. If you're following along on screen, expect differences — here are the ones that matter.

It's not a standalone demo page anymore. The Playground moved into the dashboard and now runs on your real API key and production models. In 2023 it was closer to a public preview; today it's a production-parity testing surface.

The Playground is no longer the "lite" version. The old caveat was that the Playground gave you sentiment counts while only the API gave you sentence-level detail. That gap has closed — you can view sentence-level sentiment right in the Playground, export transcripts with speaker labels, and share results with a generated link.

It generates the code for you. This is the biggest change in workflow terms. After a successful run you can copy a production-ready API request directly out of the Playground. The old path was "play in the Playground, then go read the docs and rebuild it." Now the Playground is step one of the integration, not a detour from it.

"Important phrases" is now Key Phrases. The feature is the same; the name in the UI changed. Same for the category itself — what used to be called Audio Intelligence is now Speech Understanding.

Summarization got a rebuild, and Action Items showed up. Summaries are now configurable by shape (bullets or paragraph) and by effort, and there's a separate Action Items model that extracts commitments with a supporting quote and a timestamp. For a support call, "what did we promise this customer" is often more valuable than the summary itself. Neither existed when this video was made.

There's more than one Playground now. Alongside pre-recorded audio, there are sections for streaming transcription and for the LLM Gateway, so you can test live captioning or run a prompt across multiple LLM providers without leaving the page.

Step 4: Move the same analysis to the API

The Playground answers "is this signal useful?" The API answers "can I do this to 40,000 calls a night?" The good news is that the second question is mostly a copy-paste away, because every toggle you flipped maps to one parameter.

Here's the same set of features as a single request:

import requests
import time

base_url = "https://api.assemblyai.com"
headers = {"authorization": "<YOUR_API_KEY>"}

data = {
    "audio_url": "YOUR_AUDIO_URL",
    "language_detection": True,
    "speaker_labels": True,
    "sentiment_analysis": True,
    "iab_categories": True,
    "auto_highlights": True,
    "speech_understanding": {
        "request": {
            "summarization": {
                "summary_type": "paragraph"
            }
        }
    }
}

response = requests.post(base_url + "/v2/transcript", json=data, headers=headers)
transcript_id = response.json()["id"]
polling_endpoint = base_url + "/v2/transcript/" + transcript_id

while True:
    result = requests.get(polling_endpoint, headers=headers).json()
    if result["status"] == "completed":
        break
    elif result["status"] == "error":
        raise RuntimeError(f"Transcription failed: {result['error']}")
    time.sleep(3)

for item in result["sentiment_analysis_results"]:
    print(item["speaker"], item["sentiment"], item["text"])

A few notes on that payload:

• Notice there's no model specified. Omitting speech_models means you always get the latest Universal Pro flagship — currently Universal-3.5 Pro, at $0.21/hr. Pin a specific model only when you need a frozen snapshot.
• Turning on speaker_labels alongside sentiment_analysis adds a speaker field to every sentiment result. That's the combination that lets you ask "was the customer negative, or the agent?" — which is the question that actually matters for QA.
• auto_highlights is the parameter name for Key Phrases, and results come back in a separate auto_highlights_result object rather than inline in the transcript text.
• iab_categories returns both per-segment topic labels with timestamps and a whole-file relevance summary. The summary object is what you'd aggregate across a call library.

Each of these is documented on its own page — Speech Understanding is the index — and if you'd rather not hand-write the payload at all, generate it from a Playground run and edit from there. Full per-feature pricing is on the pricing page.

What teams do with this in production

One call is a demo. The value shows up at volume, when you stop sampling and start analyzing every conversation — which is the core promise of conversation intelligence.

In contact centers, that usually means routing negative-sentiment calls to a QA queue automatically instead of spot-checking 2% of them. In marketing, it means call analytics that attribute topics back to campaigns. In product, it means counting how often "minimum payment" comes up before you rewrite the billing page.

Calabrio, which builds workforce optimization and conversation intelligence for enterprise contact centers, put the dependency plainly:

"The transcription accuracy, reliability, and speed of AssemblyAI's API have greatly enhanced our operations, reinforcing our trust in their technology and solidifying our partnership."

— Raj Shankar, SVP Product, Calabrio

That order is deliberate, and it's the thing the Playground makes visible faster than any benchmark table: every downstream signal inherits the transcript's errors. If "minimum payment" comes through as "minimum plan," your key phrase extraction is confidently wrong and your topic labels drift. Sentiment on a garbled sentence is worse than no sentiment, because it looks like data.

The thing worth testing next

Most people run one call through the Playground, see that it works, and move on to the integration. The more useful experiment is to run your worst call — the one with the crosstalk, the hold music, the caller on speakerphone in a car. That's the call that decides whether your analysis pipeline survives contact with your real audio, and it's the one you'll never discover from a clean sample file.

And when you're ready for the harder version of this problem: everything above happens after the call ends. The current frontier is running the same analysis live, while the agent still has the customer on the line and can do something about it.

Frequently asked questions

Can AI analyze a recorded phone call?

Yes. AI can transcribe a call recording and then extract structured signals from it: which speaker said each line, the sentiment of each sentence, the topics discussed, the key phrases mentioned, and a summary. AssemblyAI's Speech Understanding models return all of these alongside the transcript from a single API request, and you can test them on your own recording in the Playground without writing code.

How do I analyze a call recording with AI for free?

Create a free AssemblyAI account, open the Playground from your dashboard, upload the recording, and enable the analysis features you want. New accounts include $50 in credits, which covers a substantial amount of testing — pre-recorded transcription with the current flagship model runs $0.21/hr. No integration or code is required to see results.

What's the difference between sentiment analysis and topic detection?

Sentiment analysis judges how something was said, labeling each sentence POSITIVE, NEUTRAL, or NEGATIVE with a confidence score. Topic detection judges what was discussed, mapping the conversation to the IAB Content Taxonomy's roughly 698 standardized categories. You typically want both: sentiment tells you which calls need attention, topics tell you which part of the business they belong to.

Can ChatGPT analyze a call recording?

General-purpose chat models can reason about a conversation once you paste in a transcript, but they don't produce the transcript, and they don't give you speaker attribution, timestamps, per-sentence confidence scores, or standardized topic labels. For call analysis you generally want a purpose-built speech pipeline for the structured layer — transcription, diarization, sentiment, topics — and an LLM on top of it for open-ended questions. AssemblyAI's LLM Gateway covers that second half through the same platform.

How accurate is AI sentiment analysis on customer service calls?

Sentiment models interpret the transcript, not the tone of voice, so they can miss sarcasm and read polite frustration as neutral. Two things improve results materially: transcription accuracy, since every downstream model inherits its errors, and enabling speaker labels so you can separate customer sentiment from agent sentiment. For decisions with consequences — agent scorecards, escalations — treat sentiment as a way to prioritize human review rather than replace it.

Do I need code to test speech-to-text and conversation analysis?

No. The Playground covers pre-recorded transcription, streaming transcription, and the LLM Gateway with no code at all, and it runs the same production models the API serves. When you're ready to integrate, you can copy a production-ready API request straight out of a successful Playground run rather than building the payload from scratch.

‍