LeMUR
AssemblyAI's LeMUR (Leveraging Large Language Models to Understand Recognized Speech) framework to process audio files with an LLM.
LeMUR API Parameters
Question & Answer
Question & Answer allows you to ask free form questions about one or many transcripts. This can be any question you find useful, such as judging the outcome or determining facts about the audio. For instance, you can ask for action items from a meeting, did the customer respond positively, or count how many times a word or phrase was said.
transcript_ids | string[] | Yes | N/A | None | A list of completed transcripts with text. Up to 10 files max, or 10 hours max. Whichever is lower. |
questions | question[] | Yes | N/A | None | A list of questions to ask. Question format listed below. Maximum of 5. |
context | any | No | N/A | None | Context to provide the model. This can be a string or a free form json value. |
model | string | No | default, basic | default | The type of model to chose. Either the more robust default model, or the simplified basic model. |
Question
These are the fields to be used in the list of questions. We only allow a maximum of 5 questions at this time.
question | string | Yes | N/A | None | The question you wish to ask. For more complex questions use default model. |
context | any | No | N/A | None | Any context about the transcripts you wish to provide. This can be a string, or free form json. |
answer_format | string | No | N/A | None | How you want the answer to be returned. This can be any text. Cannot be used with answer_options. Examples: "short sentence", "bullet points" |
answer_options | string[] | No | N/A | None | What discrete options to return. Useful for precise responses. Cannot be used with answer_format. Example: ["Yes", "No"] |
Example JSON:
{
"transcript_ids": ["abc-def-123", "def-abc-123"],
"questions": [
{
"question": "Were the transcripts about politics?"
},
{
"question": "Were the policies related to taxes?",
"answer_options": ["yes", "no"]
}
],
"context": "this is a political meeting"
}
Summary
Summary allows you to distill a piece of audio into a few impactful sentences. You can give the model context to get more pinpoint results while outputting the results in a variety of formats described in human language.
transcript_ids | string[] | Yes | N/A | None | A list of completed transcripts with text. Up to 10 files max, or 10 hours max. Whichever is lower. |
context | any | No | N/A | None | Context to provide the model. This can be a string or a free form json value. |
answer_format | string | No | N/A | None | How you want the answer to be returned. This can be any text. Cannot be used with answer_options. Examples: "short sentence", "bullet points" |
model | string | No | default, basic | default | The type of model to chose. Either the more robust default model, or the simplified basic model. |
Example JSON:
{
"transcript_ids": ["abc-def-123", "def-abc-123"],
"context": "these are sales calls",
"answer_formt": "bullet points"
"model": "basic"
}
AI Coach
AI Coach allows you to gain feedback on the audio. For example, it can help the people in a customer call to improve their way of talking to achieve better results. You can provide context on the type of call/meeting to allow the model to determine the type of feedback to provide.
transcript_ids | string[] | Yes | N/A | None | A list of completed transcripts with text. Up to 10 files max, or 10 hours max. Whichever is lower. |
context | any | No | N/A | None | Context to provide the model. This can be a string or a free form json value. |
model | string | No | default, basic | default | The type of model to chose. Either the more robust default model, or the simplified basic model. |
Example JSON:
{
"transcript_ids": ["abc-def-123", "def-abc-123"],
"context": "these are product meetings"
}
LeMUR Basic & Default
LeMUR features two model modes, Basic & Default, that allow you to configure your request to suit your needs. These options tell LeMUR whether to use the more advanced Default model or the cheaper, faster, but simplified Basic model. The implicit setting is Default when no option is explicitly passed in.
LeMUR Default
LeMUR Default is the standard model to use. It is capable of handling any task as well, or better than LeMUR Basic. Default is capable of more nuanced and complex questions, where Basic would provide poor results. Additionally, our AI Coach feature provides more insightful responses with Default. The drawback of this expanded functionality and quality is execution speed and cost. Default is up to 20% slower than Basic and has an increased cost (see Pricing).
LeMUR Basic
LeMUR Basic is a simplified model optimized for speed and cost allowing you to complete simple requests quickly, and cheaply. LeMUR Basic can complete requests up to 20% faster than Default.
The best use cases for Basic include summary and simple questions with factual answers. It is not recommended to use Basic for complex/subjective questions, or AI Coach where the answer requires more nuance to be effective.
Some example Questions that work well with Basic.
{
"questions": [
{
"question": "Was a sale made?",
"answer_options": [
"Yes",
"No"
]
},
{
"question": "What were the names of people in the transcript?",
"answer_format": "comma separated list"
},
{
"question": "Was the customer greeted in every call?",
"answer_options": [
"Yes",
"No"
]
}
]
}
And here are some example questions that are too complex to be recommended with Basic. These questions ask for nuanced responses that require subjective analysis of the transcript. Basic excels at summarizing and finding facts.
{
"questions": [
{
"question": "How did the customer respond to the sale?",
"answer_format": "short sentence"
},
{
"question": "Could you describe each person's behavior in the call?"
},
{
"question": "Was the customer greeted positively and respectfully in every call?",
"answer_options": [
"Yes",
"No"
]
}
]
}