Overview

Upload a media file to AssemblyAI's servers.

An error occurred while processing the request

Upload a media file

Files

Create a transcript from a media file that is accessible via a URL.

Transcribe audio

Get the transcript resource. The transcript is ready when the "status" is "completed".

Get transcript

Get the transcript split by sentences. The API will attempt to semantically segment the transcript into sentences to create more reader-friendly transcripts.

Get sentences in transcript

Get the transcript split by paragraphs. The API will attempt to semantically segment your transcript into paragraphs to create more reader-friendly transcripts.

Get paragraphs in transcript

Export your transcript in SRT or VTT format to use with a video player for subtitles and closed captions.

The maximum number of characters per caption

Get subtitles for transcript

Retrieve the redacted audio object containing the status and URL to the redacted audio.

Get redacted audio

Search through the transcript for keywords. You can search for individual words, numbers, or phrases containing up to five words or numbers.

Search words in transcript

Retrieve a list of transcripts you created.
Transcripts are sorted from newest to oldest. The previous URL always points to a page with older transcripts.

Maximum amount of transcripts to retrieve

Only get transcripts created on this date

Get transcripts that were created before this transcript ID

Get transcripts that were created after this transcript ID

Only get throttled transcripts, overrides the status filter

List transcripts

Remove the data from the transcript and mark it as deleted.

Delete transcript

Transcripts

Use the LeMUR task endpoint to input your own LLM prompt.

Run a task using LeMUR

Custom Summary allows you to distill a piece of audio into a few impactful sentences.
You can give the model context to obtain more targeted results while outputting the results in a variety of formats described in human language.

Summarize a transcript using LeMUR

Question & Answer allows you to ask free-form questions about a single transcript or a group of transcripts.
The questions can be any whose answers you find useful, such as judging whether a caller is likely to become a customer or whether all items on a meeting's agenda were covered.

Ask questions using LeMUR

Use LeMUR to generate a list of action items from a transcript

Extract action items

Retrieve a LeMUR response that was previously generated.

The ID of the LeMUR request you previously made.
This would be found in the response of the original request.

Retrieve LeMUR response

Delete the data for a previously submitted LeMUR request.
The LLM response data, as well as any context provided in the original request will be removed.

The ID of the LeMUR request whose data you want to delete. This would be found in the response of the original request.

Purge LeMUR request data

LeMUR

Streaming

Create a temporary authentication token for Streaming Speech-to-Text

Create temporary authentication token for Streaming STT

Add up to 2500 characters of custom vocabulary.
The parameter value must be a JSON encoded array of strings. The JSON must be URL encoded like other query string parameters.

Set to true to not receive partial transcripts. Defaults to false.

Set to true to receive the SessionInformation message before the session ends. Defaults to false.

Streaming Speech-to-Text

Error

UploadedFile

A URL that points to your audio file, accessible only by AssemblyAI's servers

LemurActionItemsResponse

LemurBaseParams

Context to provide the model. This can be a string or a free-form JSON value.

The model that is used for the final prompt after compression is performed.

The temperature to use for the model.
Higher values result in answers that are more creative, lower values are more conservative.
Can be any value between 0.0 and 1.0 inclusive.

LemurBaseParamsContext

LemurBaseResponse

LemurModel

Claude 3.5 Sonnet is Anthropic's most intelligent model to date, outperforming Claude 3 Opus on a wide range of evaluations, with the speed and cost of Claude 3 Sonnet.

Claude 3 Opus is good at handling complex analysis, longer tasks with many steps, and higher-order math and coding tasks.

Claude 3 Haiku is the fastest model that can execute lightweight actions.

Claude 3 Sonnet is a legacy model with a balanced combination of performance and speed for efficient, high-throughput tasks.

Claude 2.1 is a legacy model similar to Claude 2.0. The key difference is that it minimizes model hallucination and system prompts, has a larger context window, and performs better in citations.

Claude Instant is a legacy model that is optimized for speed and cost. Claude Instant can complete requests up to 20% faster than Claude 2.0.

Legacy model. The same as Claude Instant.

LeMUR Mistral 7B is an LLM self-hosted by AssemblyAI. It's the fastest and cheapest of the LLM options. We recommend it for use cases like basic summaries and factual Q&A.

LemurQuestion

The question you wish to ask. For more complex questions use default model.

Any context about the transcripts you wish to provide. This can be a string or any object.

An answer generated by LeMUR and its question

LemurQuestionAnswer

LemurQuestionAnswerResponse

The answers generated by LeMUR and their questions

LemurQuestionContext

LemurResponse

LemurStringResponse

LemurSummaryResponse

LemurTaskResponse

LemurUsage

The number of input tokens used by the model

The number of output tokens generated by the model

PurgeLemurRequestDataResponse

The ID of the deletion request of the LeMUR request

The ID of the LeMUR request to purge the data for

AudioData

AudioEncoding

Configure the threshold for how long to wait before ending an utterance. Default is 700ms.

ConfigureEndUtteranceSilenceThreshold

FinalTranscript

Whether the text is formatted, for example Dollar -> $

ForceEndUtterance

A boolean value to communicate that you wish to force the end of the utterance

MessageType

PartialTranscript

Realtime

RealtimeBaseMessage

RealtimeBaseTranscript

Start time of audio sample relative to session start, in milliseconds

End time of audio sample relative to session start, in milliseconds

The confidence score of the entire transcription, between 0 and 1

An array of objects, with the information for each word in the transcription text.
Includes the start and end time of the word in milliseconds, the confidence score of the word, and the text, which is the word itself.

RealtimeError

RealtimeMessage

RealtimeTemporaryTokenResponse

The temporary authentication token for Streaming Speech-to-Text

RealtimeTranscript

RealtimeTranscriptType

SessionBegins

Unique identifier for the established session

Information about the session that is concluding.
This message is sent at the end of the session, before the SessionTerminated message.

SessionInformation

The total duration of the audio in seconds

SessionTerminated

TerminateSession

Set to true to end your streaming session forever

Word

receiveMessage

sendMessage

Either success, or unavailable in the rare case that the model failed

AudioIntelligenceModelStatus

AutoHighlightResult

The total number of times the key phrase appears in the audio file

The total relevancy to the overall audio file of this key phrase - a greater number means more relevant

AutoHighlightsResult

The status of the Key Phrases model. Either success, or unavailable in the rare case that the model failed.

A temporally-sequential array of Key Phrases

Chapter

An ultra-short summary (just a few words) of the content spoken in the chapter

A single sentence summary of the content spoken during the chapter

A one paragraph summary of the content spoken during the chapter

The starting time, in milliseconds, for the chapter

ContentSafetyLabel

The confidence score for the topic being discussed, from 0 to 1

How severely the topic is discussed in the section, from 0 to 1

ContentSafetyLabelResult

The transcript of the section flagged by the Content Moderation model

An array of safety labels, one per sensitive topic that was detected in the section

The sentence index at which the section begins

The sentence index at which the section ends

ContentSafetyLabelsResult

The status of the Content Moderation model. Either success, or unavailable in the rare case that the model failed.

A summary of the Content Moderation confidence results for the entire audio file

A summary of the Content Moderation severity results for the entire audio file

Entity

The type of entity for the detected entity

The starting time, in milliseconds, at which the detected entity appears in the audio file

The ending time, in milliseconds, for the detected entity in the audio file

EntityType

Customer account or membership identification number (e.g., Policy No. 10042992, Member ID: HZ-5235-001)

Banking information, including account and routing numbers

Credit card verification code (e.g., CVV: 080)

Specific calendar date (e.g., December 18)

Broader time periods, including date ranges, months, seasons, years, and decades (e.g., 2020-2021, 5-9 May, January 1984)

Date of birth (e.g., Date of Birth: March 7,1961)

Driver's license number. (e.g., DL# 356933-540)

Medications, vitamins, or supplements (e.g., Advil, Acetaminophen, Panadol)

Periods of time, specified as a number and a unit of time (e.g., 8 months, 2 years)

Email address (e.g., support@assemblyai.com)

Name of an event or holiday (e.g., Olympics, Yom Kippur)

Names of computer files, including the extension or filepath (e.g., Taxes/2012/brad-tax-returns.pdf)

Terms indicating gender identity or sexual orientation, including slang terms (e.g., female, bisexual, trans)

Healthcare numbers and health plan beneficiary numbers (e.g., Policy No.: 5584-486-674-YM)

Bodily injury (e.g., I broke my arm, I have a sprained wrist)

Internet IP address, including IPv4 and IPv6 formats (e.g., 192.168.0.1)

Name of a natural language (e.g., Spanish, French)

Any Location reference including mailing address, postal code, city, state, province, country, or coordinates. (e.g., Lake Victoria, 145 Windsor St., 90210)

Terms indicating marital status (e.g., Single, common-law, ex-wife, married)

Name of a medical condition, disease, syndrome, deficit, or disorder (e.g., chronic fatigue syndrome, arrhythmia, depression)

Medical process, including treatments, procedures, and tests (e.g., heart surgery, CT scan)

Name and/or amount of currency (e.g., 15 pesos, $94.50)

Terms indicating nationality, ethnicity, or race (e.g., American, Asian, Caucasian)

Numerical PII (including alphanumeric strings) that doesn't fall under other categories

Job title or profession (e.g., professor, actors, engineer, CPA)

Name of an organization (e.g., CNN, McDonalds, University of Alaska, Northwest General Hospital)

Passport numbers, issued by any country (e.g., PA4568332, NU3C6L86S12)

Account passwords, PINs, access keys, or verification answers (e.g., 27%alfalfa, temp1234, My mother's maiden name is Smith)

Number associated with an age (e.g., 27, 75)

Name of a person (e.g., Bob, Doug Jones, Dr. Kay Martinez, MD)

Distinctive bodily attributes, including terms indicating race (e.g., I'm 190cm tall, He belongs to the Black students' association)

Terms referring to a political party, movement, or ideology (e.g., Republican, Liberal)

Terms indicating religious affiliation (e.g., Hindu, Catholic)

Medical statistics (e.g., 18%, 18 percent)

Expressions indicating clock times (e.g., 19:37:28, 10pm EST)

Usernames, login names, or handles (e.g., @AssemblyAI)

Vehicle identification numbers (VINs), vehicle serial numbers, and license plate numbers (e.g., 5FNRL38918B111818, BIF7547)

Names of Zodiac signs (e.g., Aries, Taurus)

Details of the transcript page. Transcripts are sorted from newest to oldest. The previous URL always points to a page with older transcripts.

PageDetails

The number of results this page is limited to

The URL used to retrieve the current page of transcripts

The URL to the next page of transcripts. The previous URL always points to a page with older transcripts.

The URL to the next page of transcripts. The next URL always points to a page with newer transcripts.

1	curl -X POST https://api.assemblyai.com/lemur/v3/generate/summary \
2	-H "Authorization: <apiKey>" \
3	-H "Content-Type: application/json" \
4	-d '{}'

1	{
2	"request_id": "5e1b27c2-691f-4414-8bc5-f14678442f9e",
3	"usage": {
4	"input_tokens": 27,
5	"output_tokens": 3
6	},
7	"response": "- Wildfires in Canada are sending smoke and air pollution across parts of the US, triggering air quality alerts from Maine to Minnesota. Concentrations of particulate matter have exceeded safety levels.\n\n- Weather systems are channeling the smoke through Pennsylvania into the Mid-Atlantic and Northeast regions. New York City has canceled outdoor activities to keep children and vulnerable groups indoors.\n\n- Very small particulate matter can enter the lungs and impact respiratory, cardiovascular and neurological health. Young children, the elderly and those with preexisting conditions are most at risk.\n\n- The conditions causing the poor air quality could get worse or shift to different areas in coming days depending on weather patterns. More wildfires may also contribute to higher concentrations.\n\n- Climate change is leading to longer and more severe fire seasons. Events of smoke traveling long distances and affecting air quality over wide areas will likely become more common in the future.\"\n"
8	}