Create Chapter Summaries Using LLM Gateway

In this guide, we’ll show you how to use AssemblyAI’s LLM Gateway to process an audio file and summarize it into chapters using paragraph data from the transcription.

This guide shows how to use paragraph data from transcriptions to create chapter summaries. You can customize the paragraph grouping and prompts to fit your specific use case for creating structured summaries.

Quickstart

1import requests
2import time
3
4base_url = "https://api.assemblyai.com"
5headers = {"authorization": "<YOUR_API_KEY>"}
6
7# Step 1: Transcribe and get paragraphs
8with open("./my-audio.mp3", "rb") as f:
9 response = requests.post(base_url + "/v2/upload", headers=headers, data=f)
10
11upload_url = response.json()["upload_url"]
12data = {"audio_url": upload_url}
13
14response = requests.post(base_url + "/v2/transcript", json=data, headers=headers)
15transcript_id = response.json()['id']
16polling_endpoint = base_url + "/v2/transcript/" + transcript_id
17
18while True:
19 transcription_result = requests.get(polling_endpoint, headers=headers).json()
20 if transcription_result['status'] == 'completed':
21 break
22 elif transcription_result['status'] == 'error':
23 raise RuntimeError(f"Transcription failed: {transcription_result['error']}")
24 else:
25 time.sleep(3)
26
27# Get paragraphs
28paragraphs = requests.get(polling_endpoint + '/paragraphs', headers=headers).json()['paragraphs']
29
30# Step 2: Combine paragraphs and create chapter summaries
31combined_paragraphs = []
32step = 2 # Adjust as needed if you want combined paragraphs to be shorter or longer in length.
33
34# Combine paragraphs into groups, finding the appropriate timestamps and combining all their text into one string.
35for i in range(0, len(paragraphs), step):
36 paragraph_group = paragraphs[i : i + step]
37 start = paragraph_group[0]['start']
38 end = paragraph_group[-1]['end']
39 text = ""
40 for paragraph in paragraph_group:
41 text += f"{paragraph['text']} "
42 combined_paragraphs.append(f"Paragraph: {text} Start: {start} End: {end}")
43
44results = []
45
46# Step 3: Generate summaries with LLM Gateway
47for paragraph in combined_paragraphs:
48 prompt = "Summarize this text as a whole and provide start and end timestamps."
49
50 llm_gateway_data = {
51 "model": "claude-sonnet-4-5-20250929",
52 "messages": [
53 {"role": "user", "content": f"{prompt}\n\n{paragraph}"}
54 ],
55 "max_tokens": 1000
56 }
57
58 response = requests.post(
59 "https://llm-gateway.assemblyai.com/v1/chat/completions",
60 headers=headers,
61 json=llm_gateway_data
62 )
63
64 result = response.json()["choices"][0]["message"]["content"]
65 results.append(result)
66
67for result in results:
68 print(f"{result}\n")

Get Started

Before we begin, make sure you have an AssemblyAI account and an API key. You can sign up for an account and get your API key from your dashboard.

Step-by-Step Instructions

Install the required packages:

$pip install requests

Set up your API client and transcribe the audio file:

1import requests
2import time
3
4base_url = "https://api.assemblyai.com"
5headers = {"authorization": "<YOUR_API_KEY>"}
6
7with open("./my-audio.mp3", "rb") as f:
8 response = requests.post(base_url + "/v2/upload", headers=headers, data=f)
9
10upload_url = response.json()["upload_url"]
11data = {"audio_url": upload_url} # You can also use a URL to an audio or video file on the web
12
13response = requests.post(base_url + "/v2/transcript", json=data, headers=headers)
14transcript_id = response.json()['id']
15polling_endpoint = base_url + "/v2/transcript/" + transcript_id
16
17while True:
18 transcription_result = requests.get(polling_endpoint, headers=headers).json()
19 if transcription_result['status'] == 'completed':
20 print(f"Transcript ID:", transcript_id)
21 break
22 elif transcription_result['status'] == 'error':
23 raise RuntimeError(f"Transcription failed: {transcription_result['error']}")
24 else:
25 time.sleep(3)

Fetch all paragraphs from the transcript and combine them into groups. We set a step variable that controls how many paragraphs we combine into one overall paragraph to provide LLM Gateway with more context to create better summaries.

We also extract the appropriate start and end timestamps, and save all of our combined paragraphs in string form to send to LLM Gateway in a later step.

1paragraphs = requests.get(polling_endpoint + '/paragraphs', headers=headers).json()['paragraphs']
2combined_paragraphs = []
3step = 2 # Adjust as needed if you want combined paragraphs to be shorter or longer in length.
4
5# Combine paragraphs into groups, finding the appropriate timestamps and combining all their text into one string.
6for i in range(0, len(paragraphs), step):
7 paragraph_group = paragraphs[i : i + step]
8 start = paragraph_group[0]['start']
9 end = paragraph_group[-1]['end']
10 text = ""
11 for paragraph in paragraph_group:
12 text += f"{paragraph['text']} "
13 combined_paragraphs.append(f"Paragraph: {text} Start: {start} End: {end}")

Now we’ll use LLM Gateway to process all of our combined_paragraphs and create summaries for each one.

The summary for each paragraph then gets saved to a results array so we can output all of them at the same time.

1results = []
2
3for paragraph in combined_paragraphs:
4 prompt = "Summarize this text as a whole and provide start and end timestamps."
5
6 llm_gateway_data = {
7 "model": "claude-sonnet-4-5-20250929",
8 "messages": [
9 {"role": "user", "content": f"{prompt}\n\n{paragraph}"}
10 ],
11 "max_tokens": 1000
12 }
13
14 response = requests.post(
15 "https://llm-gateway.assemblyai.com/v1/chat/completions",
16 headers=headers,
17 json=llm_gateway_data
18 )
19
20 result = response.json()["choices"][0]["message"]["content"]
21 results.append(result)
22
23for result in results:
24 print(f"{result}\n")

The output will look similar to the example below.

Summary:
This transcript discusses the widespread impact of Canadian wildfires on air quality across the United States. Smoke from these fires is causing hazy conditions and air quality alerts in multiple states, prompting warnings to stay indoors in some areas. Peter DeCarlo, an environmental health expert from Johns Hopkins University, explains that dry conditions and specific weather patterns are channeling the smoke southward, affecting the mid-Atlantic and Northeast regions.
Start timestamp: 240
End timestamp: 60890
Summary:
The transcript discusses the unhealthy air quality in Baltimore due to high levels of particulate matter. These microscopic particles can affect respiratory, cardiovascular, and neurological systems. The concentration of particulate matter has reached dangerous levels, measuring 150 micrograms per cubic meter, which is 10 times higher than the annual average and 4 times higher than the recommended 24-hour average.
Start timestamp: 62270
End timestamp: 113214
Summary:
The text discusses the health impacts of high levels of air pollution, likely due to smoke from wildfires. It explains that the concentration of particles in the air is much higher than usual, leading to various health problems. The most vulnerable groups are identified as children, the elderly, and those with pre-existing health conditions, particularly respiratory or heart issues. The situation is severe enough that outdoor activities are being canceled in places like New York City, despite it being early summer.
Start timestamp: 113342
End timestamp: 158870
Summary:
The text discusses air quality issues related to smoke from wildfires. It mentions that some areas, like New York, are experiencing higher concentrations of smoke, but this will change as the air moves. The impact will vary across different areas over the next few days. The speaker doesn't expect concentrations to increase significantly even if more fires start. The duration of the smoke's impact on the US depends on weather system changes, while the fires themselves are expected to continue burning for some time.
Start timestamp: 162090
End timestamp: 203856
Summary:
The transcript discusses the impact of weather systems on smoke from wildfires affecting the Mid-Atlantic and Northeast regions. It predicts that changing weather patterns will soon push the smoke away from these areas. The speaker also addresses the connection between climate change and increased wildfires, suggesting that such air quality issues may become more frequent in the future, particularly in the western United States.
Start timestamp: 203968
End timestamp: 258010
Summary:
Peter DeCarlo, an associate professor at Johns Hopkins University, discusses how climate change is likely to increase the frequency of extreme weather events in the eastern United States. He suggests that while such events are currently unusual for the region, they may become more common in the future due to climate change.
Start timestamp: 258130
End timestamp: 280290