Overview

Upload a media file to AssemblyAI's servers.

An error occurred while processing the request

Upload a media file

Files

Create a transcript from a media file that is accessible via a URL.

Transcribe audio

Get the transcript resource. The transcript is ready when the "status" is "completed".

Get transcript

Get the transcript split by sentences. The API will attempt to semantically segment the transcript into sentences to create more reader-friendly transcripts.

Get sentences in transcript

Get the transcript split by paragraphs. The API will attempt to semantically segment your transcript into paragraphs to create more reader-friendly transcripts.

Get paragraphs in transcript

Export your transcript in SRT or VTT format to use with a video player for subtitles and closed captions.

Get subtitles for transcript

The maximum number of characters per caption

Retrieve the redacted audio object containing the status and URL to the redacted audio.

Get redacted audio

Search through the transcript for keywords. You can search for individual words, numbers, or phrases containing up to five words or numbers.

Search words in transcript

Retrieve a list of transcripts you created.
Transcripts are sorted from newest to oldest. The previous URL always points to a page with older transcripts.

List transcripts

Maximum amount of transcripts to retrieve

Only get transcripts created on this date

Get transcripts that were created before this transcript ID

Get transcripts that were created after this transcript ID

Only get throttled transcripts, overrides the status filter

Delete the transcript.
Deleting does not delete the resource itself, but removes the data from the resource and marks it as deleted.

Delete transcript

Transcripts

Use the LeMUR task endpoint to input your own LLM prompt.

Run a task using LeMUR

Custom Summary allows you to distill a piece of audio into a few impactful sentences.
You can give the model context to obtain more targeted results while outputting the results in a variety of formats described in human language.

Summarize a transcript using LeMUR

Question & Answer allows you to ask free-form questions about a single transcript or a group of transcripts.
The questions can be any whose answers you find useful, such as judging whether a caller is likely to become a customer or whether all items on a meeting's agenda were covered.

Ask questions using LeMUR

Use LeMUR to generate a list of action items from a transcript

Extract action items

Delete the data for a previously submitted LeMUR request.
The LLM response data, as well as any context provided in the original request will be removed.

Purge LeMUR request data

The ID of the LeMUR request whose data you want to delete. This would be found in the response of the original request.

LeMUR

Create a temporary authentication token for Streaming Speech-to-Text

Create temporary authentication token for Streaming STT

Streaming

Add up to 2500 characters of custom vocabulary.
The parameter value must be a JSON encoded array of strings.

Set to true to not receive partial transcripts. Defaults to false.

Set to true to receive the SessionInformation message before the session ends. Defaults to false.

Error

UploadedFile

A URL that points to your audio file, accessible only by AssemblyAI's servers

LemurActionItemsResponse

LemurBaseParams

Context to provide the model. This can be a string or a free-form JSON value.

The model that is used for the final prompt after compression is performed.
Defaults to "default".

The temperature to use for the model.
Higher values result in answers that are more creative, lower values are more conservative.
Can be any value between 0.0 and 1.0 inclusive.

LemurBaseParamsContext

LemurBaseResponse

The model that is used for the final prompt after compression is performed.

LemurModel

LeMUR Default is best at complex reasoning. It offers more nuanced responses and improved contextual comprehension.

LeMUR Basic is a simplified model optimized for speed and cost. LeMUR Basic can complete requests up to 20% faster than Default.

LeMUR Mistral 7B is an LLM self-hosted by AssemblyAI. It's the fastest and cheapest of the LLM options. We recommend it for use cases like basic summaries and factual Q&A.

Claude 2.1 is similar to Default, with key improvements: it minimizes model hallucination and system prompts, has a larger context window, and performs better in citations.

LemurQuestion

The question you wish to ask. For more complex questions use default model.

Any context about the transcripts you wish to provide. This can be a string or any object.

An answer generated by LeMUR and its question

LemurQuestionAnswer

LemurQuestionAnswerResponse

The answers generated by LeMUR and their questions

LemurQuestionContext

LemurSummaryResponse

LemurTaskResponse

PurgeLemurRequestDataResponse

The ID of the deletion request of the LeMUR request

The ID of the LeMUR request to purge the data for

AudioData

AudioEncoding

Configure the threshold for how long to wait before ending an utterance. Default is 700ms.

ConfigureEndUtteranceSilenceThreshold

FinalTranscript

Whether the text is formatted, for example Dollar -> $

ForceEndUtterance

A boolean value to communicate that you wish to force the end of the utterance

MessageType

PartialTranscript

RealtimeBaseMessage

RealtimeBaseTranscript

Start time of audio sample relative to session start, in milliseconds

End time of audio sample relative to session start, in milliseconds

The confidence score of the entire transcription, between 0 and 1

An array of objects, with the information for each word in the transcription text.
Includes the start and end time of the word in milliseconds, the confidence score of the word, and the text, which is the word itself.

RealtimeError

RealtimeMessage

RealtimeTemporaryTokenResponse

The temporary authentication token for Streaming Speech-to-Text

RealtimeTranscript

RealtimeTranscriptType

SessionBegins

Unique identifier for the established session

Information about the session that is concluding.
This message is sent at the end of the session, before the SessionTerminated message.

SessionInformation

The total duration of the audio in seconds

SessionTerminated

TerminateSession

Set to true to end your streaming session forever

Word

receiveMessage

sendMessage

Either success, or unavailable in the rare case that the model failed

AudioIntelligenceModelStatus

AutoHighlightResult

The total number of times the key phrase appears in the audio file

The total relevancy to the overall audio file of this key phrase - a greater number means more relevant

AutoHighlightsResult

The status of the Key Phrases model. Either success, or unavailable in the rare case that the model failed.

A temporally-sequential array of Key Phrases

Chapter

An ultra-short summary (just a few words) of the content spoken in the chapter

A single sentence summary of the content spoken during the chapter

A one paragraph summary of the content spoken during the chapter

The starting time, in milliseconds, for the chapter

ContentSafetyLabel

The confidence score for the topic being discussed, from 0 to 1

How severely the topic is discussed in the section, from 0 to 1

ContentSafetyLabelResult

The transcript of the section flagged by the Content Moderation model

An array of safety labels, one per sensitive topic that was detected in the section

The sentence index at which the section begins

The sentence index at which the section ends

ContentSafetyLabelsResult

The status of the Content Moderation model. Either success, or unavailable in the rare case that the model failed.

A summary of the Content Moderation confidence results for the entire audio file

A summary of the Content Moderation severity results for the entire audio file

Entity

The type of entity for the detected entity

The starting time, in milliseconds, at which the detected entity appears in the audio file

The ending time, in milliseconds, for the detected entity in the audio file

EntityType

Banking information, including account and routing numbers

Credit card verification code (e.g., CVV: 080)

Specific calendar date (e.g., December 18)

Date of Birth (e.g., Date of Birth: March 7, 1961)

Driver's license number (e.g., DL #356933-540)

Medications, vitamins, or supplements (e.g., Advil, Acetaminophen, Panadol)

Email address (e.g., support@assemblyai.com)

Name of an event or holiday (e.g., Olympics, Yom Kippur)

Bodily injury (e.g., I broke my arm, I have a sprained wrist)

Name of a natural language (e.g., Spanish, French)

Any location reference including mailing address, postal code, city, state, province, or country

Name of a medical condition, disease, syndrome, deficit, or disorder (e.g., chronic fatigue syndrome, arrhythmia, depression)

Medical process, including treatments, procedures, and tests (e.g., heart surgery, CT scan)

Name and/or amount of currency (e.g., 15 pesos, $94.50)

Terms indicating nationality, ethnicity, or race (e.g., American, Asian, Caucasian)

Job title or profession (e.g., professor, actors, engineer, CPA)

Name of an organization (e.g., CNN, McDonalds, University of Alaska)

Account passwords, PINs, access keys, or verification answers (e.g., 27%alfalfa, temp1234, My mother's maiden name is Smith)

Number associated with an age (e.g., 27, 75)

Terms referring to a political party, movement, or ideology (e.g., Republican, Liberal)

Terms indicating religious affiliation (e.g., Hindu, Catholic)

Expressions indicating clock times (e.g., 19:37:28, 10pm EST)

Internet addresses (e.g., www.assemblyai.com)

Details of the transcript page. Transcripts are sorted from newest to oldest. The previous URL always points to a page with older transcripts.

PageDetails

The number of results this page is limited to

The URL used to retrieve the current page of transcripts

The URL to the next page of transcripts. The previous URL always points to a page with older transcripts.

The URL to the next page of transcripts. The next URL always points to a page with newer transcripts.

ParagraphsResponse

PiiPolicy

RedactPiiAudioQuality

MP3 audio format is lower quality and lower size than WAV.

WAV audio format is the highest quality (no compression) and larger size than MP3.

RedactedAudioResponse

RedactedAudioStatus

SentencesResponse

Sentiment

The result of the Sentiment Analysis model

SentimentAnalysisResult

The starting time, in milliseconds, of the sentence

The ending time, in milliseconds, of the sentence

The detected sentiment for the sentence, one of POSITIVE, NEUTRAL, NEGATIVE

The confidence score for the detected sentiment of the sentence, from 0 to 1

SeverityScoreSummary

The speech model to use for the transcription.

SpeechModel

A lightweight, lower cost model for a wide range of languages.

Deprecated: Conformer-2 is a heavy-duty model optimized for accuracy.
This option will stop working in the near future. Use best or nano instead.

SubstitutionPolicy

SubtitleFormat

SummaryModel

SummaryType

Timestamp containing a start and end property in milliseconds

Timestamp

TopicDetectionModelResult

The status of the Topic Detection model. Either success, or unavailable in the rare case that the model failed.

An array of results for the Topic Detection model

The overall relevance of topic to the entire audio file

TopicDetectionResult

The text in the transcript in which a detected topic occurs

1	curl -X POST https://api.assemblyai.com/lemur/v3/generate/summary \
2	-H "Authorization: <apiKey>" \
3	-H "Content-Type: application/json" \
4	-d '{
5	"context": "This is an interview about wildfires.",
6	"final_model": "default",
7	"max_output_size": 3000,
8	"temperature": 0,
9	"transcript_ids": [
10	"47b95ba5-8889-44d8-bc80-5de38306e582"
11	]
12	}'

1	{
2	"request_id": "5e1b27c2-691f-4414-8bc5-f14678442f9e",
3	"response": "- Wildfires in Canada are sending smoke and air pollution across parts of the US, triggering air quality alerts from Maine to Minnesota. Concentrations of particulate matter have exceeded safety levels.\n\n- Weather systems are channeling the smoke through Pennsylvania into the Mid-Atlantic and Northeast regions. New York City has canceled outdoor activities to keep children and vulnerable groups indoors.\n\n- Very small particulate matter can enter the lungs and impact respiratory, cardiovascular and neurological health. Young children, the elderly and those with preexisting conditions are most at risk.\n\n- The conditions causing the poor air quality could get worse or shift to different areas in coming days depending on weather patterns. More wildfires may also contribute to higher concentrations.\n\n- Climate change is leading to longer and more severe fire seasons. Events of smoke traveling long distances and affecting air quality over wide areas will likely become more common in the future.\"\n"
4	}