Skip to main content

LeMUR API Endpoints and Parameters

AssemblyAI's LeMUR (Leveraging Large Language Models to Understand Recognized Speech) is a framework to process audio files with an LLM.

Endpoints​

POST/lemur/v3/generate/summaryGenerate a custom summary from one or more transcripts.
POST/lemur/v3/generate/question-answerCreate answers to one or more questions about one or more transcripts.
POST/lemur/v3/generate/action-itemsExtract action items from one or more meeting transcripts.
POST/lemur/v3/generate/taskAsk LeMUR to use one or more transcripts with a Custom Task to handle your specialized task.
DELETE/lemur/v3/{request_id}Request deletion of a previously submitted LeMUR request.

Custom Summary​

Custom Summary allows you to distill a piece of audio into a few impactful sentences. You can give the model context to obtain more targeted results while outputting the results in a variety of formats described in human language.

transcript_idsstring[]NoN/ANoneA list of completed transcripts with text. Up to a maximum of 100 files or 100 hours, whichever is lower. Use either transcript_ids or input_text as input into LeMUR.
input_textstringNoN/ANoneCustom formatted transcript data. Maximum size is the context limit of the selected model, which defaults to 100000. Use either transcript_ids or input_text as input into LeMUR.
contextanyNoN/ANoneContext to provide the model. This can be a string or a free-form JSON value.
answer_formatstringNoN/ANoneHow you want the summary to be returned. This can be any text. Examples: "TLDR", "bullet points"
final_modelstringNodefault, basicdefaultThe model that is used for the final prompt after compression is performed (options: "basic" and "default").
max_output_sizeintNoN/A2000Max output size in tokens. Up to 4000 allowed.
temperaturefloatNoN/A0.0The temperature to use for the model. Higher values result in answers that are more creative, lower values are more conservative. Can be any value between 0.0 and 1.0 inclusive.

Example JSON:

{
  "transcript_ids": ["abc-def-123", "def-abc-123"],
  "context": "these are sales calls",
  "answer_format": "bullet points"
  "final_model": "basic"
}

Question & Answer​

Question & Answer allows you to ask free-form questions about a single transcript or a group of transcripts. The questions can be any whose answers you find useful, such as judging whether a caller is likely to become a customer or whether all items on a meeting's agenda were covered.

transcript_idsstring[]NoN/ANoneA list of completed transcripts with text. Up to a maximum of 100 files or 100 hours, whichever is lower. Use either transcript_ids or input_text as input into LeMUR.
input_textstringNoN/ANoneCustom formatted transcript data. Maximum size is the context limit of the selected model, which defaults to 100000. Use either transcript_ids or input_text as input into LeMUR.
questionsquestion[]YesN/ANoneA list of questions to ask. Question format listed below.
contextanyNoN/ANoneContext to provide the model. This can be a string or a free-form JSON value.
final_modelstringNodefault, basicdefaultThe model that is used for the final prompt after compression is performed (options: "basic" and "default").
max_output_sizeintNoN/A2000Max output size in tokens. Up to 4000 allowed.
temperaturefloatNoN/A0.0The temperature to use for the model. Higher values result in answers that are more creative, lower values are more conservative. Can be any value between 0.0 and 1.0 inclusive.

Question​

These are the fields to be used in the list of questions.

questionstringYesN/ANoneThe question you wish to ask. For more complex questions use default model.
contextanyNoN/ANoneAny context about the transcripts you wish to provide. This can be a string, or free-form JSON.
answer_formatstringNoN/ANoneHow you want the answer to be returned. This can be any text. Can't be used with answer_options. Examples: "short sentence", "bullet points"
answer_optionsstring[]NoN/ANoneWhat discrete options to return. Useful for precise responses. Can't be used with answer_format. Example: ["Yes", "No"]

Example JSON:

{
  "transcript_ids": ["abc-def-123", "def-abc-123"],
  "questions": [
    {
      "question": "Were the transcripts about politics?"
    },
    {
      "question": "Were the policies related to taxes?",
      "answer_options": ["yes", "no"]
    }
  ],
  "context": "this is a political meeting"
}

Action Items​

Use LeMUR to generate a list of Action Items from a transcript

transcript_idsstring[]NoN/ANoneA list of completed transcripts with text. Up to a maximum of 100 files or 100 hours, whichever is lower. Use either transcript_ids or input_text as input into LeMUR.
input_textstringNoN/ANoneCustom formatted transcript data. Maximum size is the context limit of the selected model, which defaults to 100000. Use either transcript_ids or input_text as input into LeMUR.
contextanyNoN/ANoneContext to provide the model. This can be a string or a free-form JSON value.
final_modelstringNodefault, basicdefaultThe model that is used for the final prompt after compression is performed (options: "basic" and "default").
max_output_sizeintNoN/A2000Max output size in tokens. Up to 4000 allowed.
temperaturefloatNoN/A0.0The temperature to use for the model. Higher values result in answers that are more creative, lower values are more conservative. Can be any value between 0.0 and 1.0 inclusive.

Example JSON:

{
  "transcript_ids": ["abc-def-123", "def-abc-123"],
  "context": "these are customer support calls",
  "final_model": "basic"
}

Custom Task​

Use LeMUR to ask anything with Custom Task

transcript_idsstring[]NoN/ANoneA list of completed transcripts with text. Up to a maximum of 100 files or 100 hours, whichever is lower. Use either transcript_ids or input_text as input into LeMUR.
input_textstringNoN/ANoneCustom formatted transcript data. Maximum size is the context limit of the selected model, which defaults to 100000. Use either transcript_ids or input_text as input into LeMUR.
promptstringYesN/ANoneYour text to prompt the model to produce a desired output, including any context you want to pass into the model.
final_modelstringNodefault, basicdefaultThe model that is used for the final prompt after compression is performed (options: "basic" and "default").
max_output_sizeintNoN/A2000Max output size in tokens. Up to 4000 allowed.
temperaturefloatNoN/A0.0The temperature to use for the model. Higher values result in answers that are more creative, lower values are more conservative. Can be any value between 0.0 and 1.0 inclusive.

Example JSON:

{
  "transcript_ids": ["abc-def-123", "def-abc-123"],
  "prompt": "You are a helpful coach. Provide an analysis of the transcripts and offer areas to improve with exact quotes. Include no preamble. Start with an overall summary then get into the examples with feedback.",
  "final_model": "basic"
}

Request Deletion​

Delete the data for a previously submitted LeMUR request. LLM response data, as well as any context provided in the original request will be removed.

request_idstringYesN/ANoneThe ID of the LeMUR request whose data you want to delete. This would be found in the response of the original request.

Example usage:

curl -X DELETE "https://api.assemblyai.com/lemur/v3/{request_id}" \
-H "Authorization: API_TOKEN"

LeMUR Basic & Default​

LeMUR features two model modes, Basic & Default, that allow you to configure your request to suit your needs. These options tell LeMUR whether to use the more advanced Default model or the cheaper, faster, but simplified Basic model. The implicit setting is Default when no option is explicitly passed in.

LeMUR Default​

LeMUR Default is the standard model to use. It's capable of handling any task as well, or better than LeMUR Basic. Default is capable of more nuanced and complex questions, where Basic would provide poor results. The drawback of this expanded functionality and quality is execution speed and cost. Default is up to 20% slower than Basic and has an increased cost (see ).

LeMUR Basic​

LeMUR Basic is a simplified model optimized for speed and cost allowing you to complete simple requests quickly, and cheaply. LeMUR Basic can complete requests up to 20% faster than Default.

The best use cases for Basic include summary and simple questions with factual answers. It isn't recommended to use Basic for complex/subjective questions or where the answer requires more nuance to be effective.

Some example Questions that work well with Basic.

{
  "questions": [
    {
      "question": "Was a sale made?",
      "answer_options": ["Yes", "No"]
    },
    {
      "question": "What were the names of people in the transcript?",
      "answer_format": "comma separated list"
    },
    {
      "question": "Was the customer greeted in every call?",
      "answer_options": ["Yes", "No"]
    }
  ]
}

And here are some example questions that are too complex to be recommended with Basic. These questions ask for nuanced responses that require subjective analysis of the transcript. Basic excels at summarizing and finding facts.

{
  "questions": [
    {
      "question": "How did the customer respond to the sale?",
      "answer_format": "short sentence"
    },
    {
      "question": "Could you describe each person's behavior in the call?"
    },
    {
      "question": "Was the customer greeted positively and respectfully in every call?",
      "answer_options": ["Yes", "No"]
    }
  ]
}

Additional Resources​

  • Processing Audio with LeMUR: This guide helps you understand how to use LeMUR from the start of transcribing audio to interacting with LLMs.
  • LeMUR Best Practices: Use the best practices to help figure out the best way to get the info you need from your transcriptions.
© AssemblyAI