LeMUR API Endpoints and Parameters
AssemblyAI's LeMUR (Leveraging Large Language Models to Understand Recognized Speech) is a framework to process audio files with an LLM.
Endpoints
POST | /lemur/v3/generate/summary | Generate a custom summary from one or more transcripts. |
POST | /lemur/v3/generate/question-answer | Create answers to one or more questions about one or more transcripts. |
POST | /lemur/v3/generate/action-items | Extract action items from one or more meeting transcripts. |
POST | /lemur/v3/generate/task | Ask LeMUR to use one or more transcripts with a Custom Task to handle your specialized task. |
DELETE | /lemur/v3/{request_id} | Request deletion of a previously submitted LeMUR request. |
Custom Summary
Custom Summary allows you to distill a piece of audio into a few impactful sentences. You can give the model context to obtain more targeted results while outputting the results in a variety of formats described in human language.
transcript_ids | string[] | No | N/A | None | A list of completed transcripts with text. Up to a maximum of 100 files or 100 hours, whichever is lower. Use either transcript_ids or input_text as input into LeMUR. |
input_text | string | No | N/A | None | Custom formatted transcript data. Maximum size is the context limit of the selected model, which defaults to 100000. Use either transcript_ids or input_text as input into LeMUR. |
context | any | No | N/A | None | Context to provide the model. This can be a string or a free-form JSON value. |
answer_format | string | No | N/A | None | How you want the summary to be returned. This can be any text. Examples: "TLDR", "bullet points" |
final_model | string | No | default, basic | default | The model that is used for the final prompt after compression is performed (options: "basic" and "default"). |
max_output_size | int | No | N/A | 2000 | Max output size in tokens. Up to 4000 allowed. |
temperature | float | No | N/A | 0.0 | The temperature to use for the model. Higher values result in answers that are more creative, lower values are more conservative. Can be any value between 0.0 and 1.0 inclusive. |
Example JSON:
{
"transcript_ids": ["abc-def-123", "def-abc-123"],
"context": "these are sales calls",
"answer_format": "bullet points"
"final_model": "basic"
}
Question & Answer
Question & Answer allows you to ask free-form questions about a single transcript or a group of transcripts. The questions can be any whose answers you find useful, such as judging whether a caller is likely to become a customer or whether all items on a meeting's agenda were covered.
transcript_ids | string[] | No | N/A | None | A list of completed transcripts with text. Up to a maximum of 100 files or 100 hours, whichever is lower. Use either transcript_ids or input_text as input into LeMUR. |
input_text | string | No | N/A | None | Custom formatted transcript data. Maximum size is the context limit of the selected model, which defaults to 100000. Use either transcript_ids or input_text as input into LeMUR. |
questions | question[] | Yes | N/A | None | A list of questions to ask. Question format listed below. |
context | any | No | N/A | None | Context to provide the model. This can be a string or a free-form JSON value. |
final_model | string | No | default, basic | default | The model that is used for the final prompt after compression is performed (options: "basic" and "default"). |
max_output_size | int | No | N/A | 2000 | Max output size in tokens. Up to 4000 allowed. |
temperature | float | No | N/A | 0.0 | The temperature to use for the model. Higher values result in answers that are more creative, lower values are more conservative. Can be any value between 0.0 and 1.0 inclusive. |
Question
These are the fields to be used in the list of questions.
question | string | Yes | N/A | None | The question you wish to ask. For more complex questions use default model. |
context | any | No | N/A | None | Any context about the transcripts you wish to provide. This can be a string, or free-form JSON. |
answer_format | string | No | N/A | None | How you want the answer to be returned. This can be any text. Can't be used with answer_options. Examples: "short sentence", "bullet points" |
answer_options | string[] | No | N/A | None | What discrete options to return. Useful for precise responses. Can't be used with answer_format. Example: ["Yes", "No"] |
Example JSON:
{
"transcript_ids": ["abc-def-123", "def-abc-123"],
"questions": [
{
"question": "Were the transcripts about politics?"
},
{
"question": "Were the policies related to taxes?",
"answer_options": ["yes", "no"]
}
],
"context": "this is a political meeting"
}
Action Items
Use LeMUR to generate a list of Action Items from a transcript
transcript_ids | string[] | No | N/A | None | A list of completed transcripts with text. Up to a maximum of 100 files or 100 hours, whichever is lower. Use either transcript_ids or input_text as input into LeMUR. |
input_text | string | No | N/A | None | Custom formatted transcript data. Maximum size is the context limit of the selected model, which defaults to 100000. Use either transcript_ids or input_text as input into LeMUR. |
context | any | No | N/A | None | Context to provide the model. This can be a string or a free-form JSON value. |
final_model | string | No | default, basic | default | The model that is used for the final prompt after compression is performed (options: "basic" and "default"). |
max_output_size | int | No | N/A | 2000 | Max output size in tokens. Up to 4000 allowed. |
temperature | float | No | N/A | 0.0 | The temperature to use for the model. Higher values result in answers that are more creative, lower values are more conservative. Can be any value between 0.0 and 1.0 inclusive. |
Example JSON:
{
"transcript_ids": ["abc-def-123", "def-abc-123"],
"context": "these are customer support calls",
"final_model": "basic"
}
Custom Task
Use LeMUR to ask anything with Custom Task
transcript_ids | string[] | No | N/A | None | A list of completed transcripts with text. Up to a maximum of 100 files or 100 hours, whichever is lower. Use either transcript_ids or input_text as input into LeMUR. |
input_text | string | No | N/A | None | Custom formatted transcript data. Maximum size is the context limit of the selected model, which defaults to 100000. Use either transcript_ids or input_text as input into LeMUR. |
prompt | string | Yes | N/A | None | Your text to prompt the model to produce a desired output, including any context you want to pass into the model. |
final_model | string | No | default, basic | default | The model that is used for the final prompt after compression is performed (options: "basic" and "default"). |
max_output_size | int | No | N/A | 2000 | Max output size in tokens. Up to 4000 allowed. |
temperature | float | No | N/A | 0.0 | The temperature to use for the model. Higher values result in answers that are more creative, lower values are more conservative. Can be any value between 0.0 and 1.0 inclusive. |
Example JSON:
{
"transcript_ids": ["abc-def-123", "def-abc-123"],
"prompt": "You are a helpful coach. Provide an analysis of the transcripts and offer areas to improve with exact quotes. Include no preamble. Start with an overall summary then get into the examples with feedback.",
"final_model": "basic"
}
Request Deletion
Delete the data for a previously submitted LeMUR request. LLM response data, as well as any context provided in the original request will be removed.
request_id | string | Yes | N/A | None | The ID of the LeMUR request whose data you want to delete. This would be found in the response of the original request. |
Example usage:
curl -X DELETE "https://api.assemblyai.com/lemur/v3/{request_id}" \
-H "Authorization: API_TOKEN"
LeMUR Basic & Default
LeMUR features two model modes, Basic & Default, that allow you to configure your request to suit your needs. These options tell LeMUR whether to use the more advanced Default model or the cheaper, faster, but simplified Basic model. The implicit setting is Default when no option is explicitly passed in.
LeMUR Default
LeMUR Default is the standard model to use. It's capable of handling any task as well, or better than LeMUR Basic. Default is capable of more nuanced and complex questions, where Basic would provide poor results. The drawback of this expanded functionality and quality is execution speed and cost. Default is up to 20% slower than Basic and has an increased cost (see ).
LeMUR Basic
LeMUR Basic is a simplified model optimized for speed and cost allowing you to complete simple requests quickly, and cheaply. LeMUR Basic can complete requests up to 20% faster than Default.
The best use cases for Basic include summary and simple questions with factual answers. It isn't recommended to use Basic for complex/subjective questions or where the answer requires more nuance to be effective.
Some example Questions that work well with Basic.
{
"questions": [
{
"question": "Was a sale made?",
"answer_options": ["Yes", "No"]
},
{
"question": "What were the names of people in the transcript?",
"answer_format": "comma separated list"
},
{
"question": "Was the customer greeted in every call?",
"answer_options": ["Yes", "No"]
}
]
}
And here are some example questions that are too complex to be recommended with Basic. These questions ask for nuanced responses that require subjective analysis of the transcript. Basic excels at summarizing and finding facts.
{
"questions": [
{
"question": "How did the customer respond to the sale?",
"answer_format": "short sentence"
},
{
"question": "Could you describe each person's behavior in the call?"
},
{
"question": "Was the customer greeted positively and respectfully in every call?",
"answer_options": ["Yes", "No"]
}
]
}
Additional Resources
- Processing Audio with LeMUR: This guide helps you understand how to use LeMUR from the start of transcribing audio to interacting with LLMs.
- LeMUR Best Practices: Use the best practices to help figure out the best way to get the info you need from your transcriptions.