Skip to main content

LeMUR

AssemblyAI's LeMUR (Leveraging Large Language Models to Understand Recognized Speech) framework to process audio files with an LLM.

LeMUR API Parameters​

Question & Answer​

Question & Answer allows you to ask free form questions about one or many transcripts. This can be any question you find useful, such as judging the outcome or determining facts about the audio. For instance, you can ask for action items from a meeting, did the customer respond positively, or count how many times a word or phrase was said.

transcript_idsstring[]YesN/ANoneA list of completed transcripts with text. Up to 10 files max, or 10 hours max. Whichever is lower.
questionsquestion[]YesN/ANoneA list of questions to ask. Question format listed below. Maximum of 5.
contextanyNoN/ANoneContext to provide the model. This can be a string or a free form json value.
modelstringNodefault, basicdefaultThe type of model to chose. Either the more robust default model, or the simplified basic model.

Question​

These are the fields to be used in the list of questions. We only allow a maximum of 5 questions at this time.

questionstringYesN/ANoneThe question you wish to ask. For more complex questions use default model.
contextanyNoN/ANoneAny context about the transcripts you wish to provide. This can be a string, or free form json.
answer_formatstringNoN/ANoneHow you want the answer to be returned. This can be any text. Cannot be used with answer_options. Examples: "short sentence", "bullet points"
answer_optionsstring[]NoN/ANoneWhat discrete options to return. Useful for precise responses. Cannot be used with answer_format. Example: ["Yes", "No"]

Example JSON:

{
  "transcript_ids": ["abc-def-123", "def-abc-123"],
  "questions": [
    {
      "question": "Were the transcripts about politics?"
    },
    {
      "question": "Were the policies related to taxes?",
      "answer_options": ["yes", "no"]
    }
  ],
  "context": "this is a political meeting"
}

Summary​

Summary allows you to distill a piece of audio into a few impactful sentences. You can give the model context to get more pinpoint results while outputting the results in a variety of formats described in human language.

transcript_idsstring[]YesN/ANoneA list of completed transcripts with text. Up to 10 files max, or 10 hours max. Whichever is lower.
contextanyNoN/ANoneContext to provide the model. This can be a string or a free form json value.
answer_formatstringNoN/ANoneHow you want the answer to be returned. This can be any text. Cannot be used with answer_options. Examples: "short sentence", "bullet points"
modelstringNodefault, basicdefaultThe type of model to chose. Either the more robust default model, or the simplified basic model.

Example JSON:

{
  "transcript_ids": ["abc-def-123", "def-abc-123"],
  "context": "these are sales calls",
  "answer_formt": "bullet points"
  "model": "basic"
}

AI Coach​

AI Coach allows you to gain feedback on the audio. For example, it can help the people in a customer call to improve their way of talking to achieve better results. You can provide context on the type of call/meeting to allow the model to determine the type of feedback to provide.

transcript_idsstring[]YesN/ANoneA list of completed transcripts with text. Up to 10 files max, or 10 hours max. Whichever is lower.
contextanyNoN/ANoneContext to provide the model. This can be a string or a free form json value.
modelstringNodefault, basicdefaultThe type of model to chose. Either the more robust default model, or the simplified basic model.

Example JSON:

{
  "transcript_ids": ["abc-def-123", "def-abc-123"],
  "context": "these are product meetings"
}

LeMUR Basic & Default​

LeMUR features two model modes, Basic & Default, that allow you to configure your request to suit your needs. These options tell LeMUR whether to use the more advanced Default model or the cheaper, faster, but simplified Basic model. The implicit setting is Default when no option is explicitly passed in.

LeMUR Default​

LeMUR Default is the standard model to use. It is capable of handling any task as well, or better than LeMUR Basic. Default is capable of more nuanced and complex questions, where Basic would provide poor results. Additionally, our AI Coach feature provides more insightful responses with Default. The drawback of this expanded functionality and quality is execution speed and cost. Default is up to 20% slower than Basic and has an increased cost (see Pricing).

LeMUR Basic​

LeMUR Basic is a simplified model optimized for speed and cost allowing you to complete simple requests quickly, and cheaply. LeMUR Basic can complete requests up to 20% faster than Default.

The best use cases for Basic include summary and simple questions with factual answers. It is not recommended to use Basic for complex/subjective questions, or AI Coach where the answer requires more nuance to be effective.

Some example Questions that work well with Basic.

{
  "questions": [
    {
      "question": "Was a sale made?",
      "answer_options": [
          "Yes",
          "No"
      ]
    },
    {
      "question": "What were the names of people in the transcript?",
      "answer_format": "comma separated list"
    },
    {
      "question": "Was the customer greeted in every call?",
      "answer_options": [
          "Yes",
          "No"
      ]
    }
  ]
}

And here are some example questions that are too complex to be recommended with Basic. These questions ask for nuanced responses that require subjective analysis of the transcript. Basic excels at summarizing and finding facts.

{
 "questions": [
   {
     "question": "How did the customer respond to the sale?",
     "answer_format": "short sentence"
   },
   {
     "question": "Could you describe each person's behavior in the call?"
   },
   {
     "question": "Was the customer greeted positively and respectfully in every call?",
     "answer_options": [
         "Yes",
         "No"
     ]
   }
 ]
}
© AssemblyAI