🦜️🔗 LangChain JavaScript Integration with AssemblyAI | AssemblyAI

To apply LLMs to speech, you first need to transcribe the audio to text, which is what the AssemblyAI integration for LangChain helps you with.

Looking for the Python integration?
Go to the LangChain Python integration.

Quickstart

Add the AssemblyAI SDK to your project:

npm

yarn

pnpm

bun

$ npm install langchain @langchain/community

To use the loaders, you need an AssemblyAI account and get your AssemblyAI API key from the dashboard. Configure the API key as the ASSEMBLYAI_API_KEY environment variable or the apiKey options parameter.

1   AudioTranscriptLoader,
2   // AudioTranscriptParagraphsLoader,
3   // AudioTranscriptSentencesLoader
4 } from "@langchain/community/document_loaders/web/assemblyai";
5 
6 // You can also use a local file path and the loader will upload it to AssemblyAI for you.
7 const audioUrl = "https://assembly.ai/espn.m4a";
8 
9 // Use `AudioTranscriptParagraphsLoader` or `AudioTranscriptSentencesLoader` for splitting the transcript into paragraphs or sentences
10 const loader = new AudioTranscriptLoader(
11   {
12     audio: audioUrl,
13     // any other parameters as documented here: https://www.assemblyai.com/docs/api-reference/transcript#create-a-transcript
14   },
15   {
16     apiKey: "<ASSEMBLYAI_API_KEY>", // or set the `ASSEMBLYAI_API_KEY` env variable
17   }
18 );
19 const docs = await loader.load();
20 console.dir(docs, { depth: Infinity });

You can use the AudioTranscriptParagraphsLoader or AudioTranscriptSentencesLoader to split the transcript into paragraphs or sentences. - If the audio_file is a local file path, the loader will upload it to AssemblyAI for you. - The audio_file can also be a video file. See the list of supported file types in the FAQ doc.
If you don’t pass in the apiKey option, the loader will use the ASSEMBLYAI_API_KEY environment variable. - You can add more properties in addition to audio. Find the full list of request parameters in the AssemblyAI API docs.

You can also use the AudioSubtitleLoader to get srt or vtt subtitles as a document.

1 // You can also use a local file path and the loader will upload it to AssemblyAI for you.
2 const audioUrl = "https://assembly.ai/espn.m4a";
3 
4 const loader = new AudioSubtitleLoader(
5   {
6     audio: audioUrl,
7     // any other parameters as documented here: https://www.assemblyai.com/docs/api-reference/transcript#create-a-transcript
8   },
9   "srt", // srt or vtt
10   {
11     apiKey: "<ASSEMBLYAI_API_KEY>", // or set the `ASSEMBLYAI_API_KEY` env variable
12   }
13 );
14 
15 const docs = await loader.load();
16 console.dir(docs, { depth: Infinity });

Additional resources

You can learn more about using LangChain with AssemblyAI in these resources: