Generate Action Items with LLM Gateway

This tutorial will demonstrate how to use AssemblyAI’s LLM Gateway framework to create action items from a transcript.

Quickstart

Python
JavaScript

import requests
import time

API_KEY = "YOUR_API_KEY"
audio_url = "https://storage.googleapis.com/aai-web-samples/meeting.mp4"

# Step 1: Upload or provide audio URL and start transcription
transcript_request = requests.post(
    "https://api.assemblyai.com/v2/transcript",
    headers={"authorization": API_KEY, "content-type": "application/json"},
    json={"audio_url": audio_url},
)

transcript_id = transcript_request.json()["id"]

# Step 2: Poll until transcription completes
while True:
    polling_response = requests.get(
        f"https://api.assemblyai.com/v2/transcript/{transcript_id}",
        headers={"authorization": API_KEY},
    )
    status = polling_response.json()["status"]

    if status == "completed":
        break
    elif status == "error":
        raise RuntimeError(f"Transcription failed: {polling_response.json()['error']}")
    else:
        print(f"Transcription status: {status}")
        time.sleep(3)

# Step 3: Build the prompt
prompt = """
Here are guidelines to follow:
- You are an expert at understanding transcripts of conversations, calls and meetings.
- You are an expert at coming up with ideal action items based on the contents of the transcripts.
- Action items are things that the transcript implies should get done.
- Your action item ideas do not make stuff up that isn't relevant to the transcript.
- You do not needlessly make up action items - you stick to important tasks.
- You are useful, true and concise, and write in perfect English.
- Your action items can be tied back to direct quotes in the transcript.
- You do not cite the quotes the action items relate to.
- The action items are written succinctly.
- Please give useful action items based on the transcript.
"""

answer_format = "Bullet Points"
if answer_format:
    prompt += f"\nYour response should have the following format: {answer_format}"

# Step 4: Send transcript ID to LLM Gateway
headers = {"authorization": API_KEY}

response = requests.post(
    "https://llm-gateway.assemblyai.com/v1/chat/completions",
    headers=headers,
    json={
        "model": "claude-sonnet-4-6",
        "messages": [
            {
                "role": "user",
                "content": f"{prompt}\n\n{{{{ transcript }}}}",
            }
        ],
        "transcript_id": transcript_id,
        "max_tokens": 1000,
    },
)

# Step 5: Print the LLM-generated action items
response_json = response.json()
print(response_json["choices"][0]["message"]["content"])

const API_KEY = "YOUR_API_KEY";
const headers = { authorization: API_KEY };
const audioUrl = "https://storage.googleapis.com/aai-web-samples/meeting.mp4";

// Step 1: Upload or provide audio URL and start transcription
let res = await fetch("https://api.assemblyai.com/v2/transcript", {
  method: "POST",
  headers: { ...headers, "Content-Type": "application/json" },
  body: JSON.stringify({ audio_url: audioUrl }),
});
if (!res.ok) throw new Error(`Error: ${res.status}`);
const transcriptRequest = await res.json();

const transcriptId = transcriptRequest.id;

// Step 2: Poll until transcription completes
while (true) {
  res = await fetch(`https://api.assemblyai.com/v2/transcript/${transcriptId}`, { headers });
  if (!res.ok) throw new Error(`Error: ${res.status}`);
  const pollingResponse = await res.json();
  const status = pollingResponse.status;

  if (status === "completed") {
    break;
  } else if (status === "error") {
    throw new Error(`Transcription failed: ${pollingResponse.error}`);
  } else {
    console.log(`Transcription status: ${status}`);
    await new Promise((resolve) => setTimeout(resolve, 3000));
  }
}

// Step 3: Build the prompt
let prompt = `
Here are guidelines to follow:
- You are an expert at understanding transcripts of conversations, calls and meetings.
- You are an expert at coming up with ideal action items based on the contents of the transcripts.
- Action items are things that the transcript implies should get done.
- Your action item ideas do not make stuff up that isn't relevant to the transcript.
- You do not needlessly make up action items - you stick to important tasks.
- You are useful, true and concise, and write in perfect English.
- Your action items can be tied back to direct quotes in the transcript.
- You do not cite the quotes the action items relate to.
- The action items are written succinctly.
- Please give useful action items based on the transcript.
`;

const answerFormat = "Bullet Points";
if (answerFormat) {
  prompt += `\nYour response should have the following format: ${answerFormat}`;
}

// Step 4: Send transcript ID to LLM Gateway
res = await fetch("https://llm-gateway.assemblyai.com/v1/chat/completions", {
  method: "POST",
  headers: { ...headers, "Content-Type": "application/json" },
  body: JSON.stringify({
    model: "claude-sonnet-4-6",
    messages: [
      {
        role: "user",
        content: `${prompt}\n\n{{ transcript }}`,
      },
    ],
    transcript_id: transcriptId,
    max_tokens: 1000,
  }),
});
if (!res.ok) throw new Error(`Error: ${res.status}`);
const response = await res.json();

// Step 5: Print the LLM-generated action items
console.log(response.choices[0].message.content);

Getting Started

Before we begin, make sure you have an AssemblyAI account and an API key. You can sign up for an AssemblyAI account and get your API key from your dashboard. Find more details on the current LLM Gateway pricing in the AssemblyAI pricing page.

Step-by-Step Instructions

In this guide, we will prompt LLM Gateway to generate action items from our transcript. Install the required packages:

Python

pip install requests

Import the necessary libraries and set AssemblyAI API key.

Python
JavaScript

import requests
import time

API_KEY = "YOUR_API_KEY"

const API_KEY = "YOUR_API_KEY";
const headers = { authorization: API_KEY };

Use AssemblyAI to transcribe a file and save the transcript.

Python
JavaScript

# Step 1: Upload or provide audio URL and start transcription
audio_url = "https://storage.googleapis.com/aai-web-samples/meeting.mp4"

transcript_request = requests.post(
    "https://api.assemblyai.com/v2/transcript",
    headers={"authorization": API_KEY, "content-type": "application/json"},
    json={"audio_url": audio_url},
)

transcript_id = transcript_request.json()["id"]

# Step 2: Poll until transcription completes
while True:
    polling_response = requests.get(
        f"https://api.assemblyai.com/v2/transcript/{transcript_id}",
        headers={"authorization": API_KEY},
    )
    status = polling_response.json()["status"]

    if status == "completed":
        break
    elif status == "error":
        raise RuntimeError(f"Transcription failed: {polling_response.json()['error']}")
    else:
        print(f"Transcription status: {status}")
        time.sleep(3)

// Step 1: Upload or provide audio URL and start transcription
const audioUrl = "https://storage.googleapis.com/aai-web-samples/meeting.mp4";

let res = await fetch("https://api.assemblyai.com/v2/transcript", {
  method: "POST",
  headers: { ...headers, "Content-Type": "application/json" },
  body: JSON.stringify({ audio_url: audioUrl }),
});
if (!res.ok) throw new Error(`Error: ${res.status}`);
const transcriptRequest = await res.json();

const transcriptId = transcriptRequest.id;

// Step 2: Poll until transcription completes
while (true) {
  res = await fetch(`https://api.assemblyai.com/v2/transcript/${transcriptId}`, { headers });
  if (!res.ok) throw new Error(`Error: ${res.status}`);
  const pollingResponse = await res.json();
  const status = pollingResponse.status;

  if (status === "completed") {
    break;
  } else if (status === "error") {
    throw new Error(`Transcription failed: ${pollingResponse.error}`);
  } else {
    console.log(`Transcription status: ${status}`);
    await new Promise((resolve) => setTimeout(resolve, 3000));
  }
}

Provide detailed instructions to prompt LLM Gateway to create action items from the transcript.

Python
JavaScript

prompt = f"""
        Here are guidelines to follow:
        - You are an expert at understanding transcripts of conversations, calls and meetings.
        - You are an expert at coming up with ideal action items based on the contents of the transcripts.
        - Action items are things that the transcript implies should get done.
        - Your action item ideas do not make stuff up that isn't relevant to the transcript.
        - You do not needlessly make up action items - you stick to important tasks.
        - You are useful, true and concise, and write in perfect English.
        - Your action items can be tied back to direct quotes in the transcript.
        - You do not cite the quotes the action items relate to.
        - The action items are written succinctly.
        - Please give useful action items based on the transcript.
        - Your response should be formatted in bullet points.
        """

let prompt = `
        Here are guidelines to follow:
        - You are an expert at understanding transcripts of conversations, calls and meetings.
        - You are an expert at coming up with ideal action items based on the contents of the transcripts.
        - Action items are things that the transcript implies should get done.
        - Your action item ideas do not make stuff up that isn't relevant to the transcript.
        - You do not needlessly make up action items - you stick to important tasks.
        - You are useful, true and concise, and write in perfect English.
        - Your action items can be tied back to direct quotes in the transcript.
        - You do not cite the quotes the action items relate to.
        - The action items are written succinctly.
        - Please give useful action items based on the transcript.
        - Your response should be formatted in bullet points.
        `;

Generate the custom action items using LLM Gateway.

Python
JavaScript

answer_format = "Bullet Points"
if answer_format:
    prompt += f"\nYour response should have the following format: {answer_format}"

const answerFormat = "Bullet Points";
if (answerFormat) {
  prompt += `\nYour response should have the following format: ${answerFormat}`;
}

Prompt LLM Gateway using the transcript results and return the response.

Python
JavaScript

# Step 4: Send transcript ID to LLM Gateway
headers = {"authorization": API_KEY}

response = requests.post(
    "https://llm-gateway.assemblyai.com/v1/chat/completions",
    headers=headers,
    json={
        "model": "claude-sonnet-4-6",
        "messages": [
            {
                "role": "user",
                "content": f"{prompt}\n\n{{{{ transcript }}}}",
            }
        ],
        "transcript_id": transcript_id,
        "max_tokens": 1000,
    },
)

# Step 5: Print the LLM-generated action items
response_json = response.json()
print(response_json["choices"][0]["message"]["content"])

// Step 4: Send transcript ID to LLM Gateway
let res = await fetch("https://llm-gateway.assemblyai.com/v1/chat/completions", {
  method: "POST",
  headers: { ...headers, "Content-Type": "application/json" },
  body: JSON.stringify({
    model: "claude-sonnet-4-6",
    messages: [
      {
        role: "user",
        content: `${prompt}\n\n{{ transcript }}`,
      },
    ],
    transcript_id: transcriptId,
    max_tokens: 1000,
  }),
});
if (!res.ok) throw new Error(`Error: ${res.status}`);
const response = await res.json();

// Step 5: Print the LLM-generated action items
console.log(response.choices[0].message.content);

Getting started

Features

API reference

Core concepts

Voice AI Guides

Generate Action Items with LLM Gateway

Quickstart

Getting Started

Step-by-Step Instructions

​Quickstart

​Getting Started

​Step-by-Step Instructions

Quickstart

Getting Started

Step-by-Step Instructions