Supported languages

English, Spanish, French, German, Italian, and Portuguese

Coming soon

Our multilingual feature is currently in Beta. We’re actively working on improvements to our multilingual offering including reduced latency, returning language codes, and adding more languages. Check our public roadmap to see what’s currently being worked on.

Multilingual streaming allows you to transcribe audio streams in multiple languages.

Configuration

Keyterms prompting is not supported with multilingual streaming.

To utilize multilingual streaming, you need to include "language":"multi" as a query parameter in the WebSocket URL.

Supported languages

Multilingual currently supports English, Spanish, French, German, Italian, and Portuguese.

Understanding formatting

The multilingual model produces transcripts with punctuation and capitalization already built into the model outputs. This means you’ll receive properly formatted text without requiring any additional post-processing.

While the API still returns the turn_is_formatted parameter to maintain interface consistency with other streaming models, the multilingual model doesn’t perform additional formatting operations. All transcripts from the multilingual model are already formatted as they’re generated.

In the future, this built-in formatting capability will be extended to our English-only streaming model as well.

Quickstart

Python

Javascript

Firstly, install the required dependencies.

$ pip install websockets pyaudio

Python

Javascript

1 import websockets
2 import asyncio
3 import json
4 from urllib.parse import urlencode
5 
6 import pyaudio
7 
8 FRAMES_PER_BUFFER = 3200
9 FORMAT = pyaudio.paInt16
10 CHANNELS = 1
11 RATE = 48000
12 p = pyaudio.PyAudio()
13 
14 stream = p.open(
15     format=FORMAT,
16     channels=CHANNELS,
17     rate=RATE,
18     input=True,
19     frames_per_buffer=FRAMES_PER_BUFFER
20 )
21 
22 BASE_URL = "wss://streaming.assemblyai.com/v3/ws"
23 CONNECTION_PARAMS = {
24     "sample_rate": RATE,
25     "format_turns": True,
26     "language": "multi",
27 }
28 URL = f"{BASE_URL}?{urlencode(CONNECTION_PARAMS)}"
29 
30 async def send_receive():
31 
32     print(f'Connecting websocket to url ${URL}')
33 
34     async with websockets.connect(
35         URL,
36         extra_headers={"Authorization": "YOUR-API-KEY"},
37         ping_interval=5,
38         ping_timeout=20
39     ) as _ws:
40         await asyncio.sleep(0.1)
41         print("Receiving SessionBegins ...")
42 
43         session_begins = await _ws.recv()
44         print(session_begins)
45         print("Sending messages ...")
46 
47         async def send():
48             while True:
49                 try:
50                     data = stream.read(FRAMES_PER_BUFFER, exception_on_overflow=False)
51                     await _ws.send(data)
52                 except websockets.exceptions.ConnectionClosedError as e:
53                     print(e)
54                 except Exception as e:
55                     print(e)
56                 await asyncio.sleep(0.01)
57 
58         async def receive():
59             while True:
60                 try:
61                     result_str = await _ws.recv()
62                     data = json.loads(result_str)
63                     transcript = data['transcript']
64 
65                     if data['type'] == 'Turn':
66                         if data.get('turn_is_formatted'):
67                             print(f"\r{transcript}")
68                         else:
69                             print(f"\r{transcript}", end="")
70                             print(data)
71                     else:
72                         pass
73 
74                 except websockets.exceptions.ConnectionClosed:
75                     break
76                 except Exception as e:
77                     print(f"\nError receiving data: {e}")
78                     break
79 
80         try:
81             await asyncio.gather(send(), receive())
82         except KeyboardInterrupt:
83             await _ws.send({"type": "Terminate"})
84             # Wait for the server to close the connection after receiving the message
85             await _ws.wait_closed()
86             print("Session terminated and connection closed.")
87 
88 if __name__ == "__main__":
89     try:
90         asyncio.run(send_receive())
91     finally:
92         stream.stop_stream()
93         stream.close()
94         p.terminate()