We say the funniest things when no one is listening. But what if someone did, all the time? In this article, we will learn how to make an app that will listen to you and tweet the funniest, smartest or most relatable things you say out loud.
The app will work by listening to you and transcribing your sentences. After you say something you would like to tweet, you can say the keyword "tweet" and it will post your latest sentence on X (formerly Twitter).
We will make the app with Python using AssemblyAI's industry-leading Universal-Streaming speech-to-text model, which delivers ultra-fast, ultra-accurate transcription with ~300ms latency and immutable transcripts perfect for real-time applications.
The main libraries will be:
- PyAudio for listening to the input source
- Tweepy for easy use of the X API
- AssemblyAI's Universal-Streaming for Speech-to-Text transcription
Setting up the dependencies
Before coding at all, we need X and AssemblyAI credentials. Getting an AssemblyAI API token is very simple. Just sign up for AssemblyAI and log in to find your token. New users get $50 in free credits to get started.
In order to use the X API, go to the X Developer Portal and create an account. After providing some information to X, you need to create a project and get the necessary credentials. For this project, you need read and write permissions.
There will be two files in this project. The main Python script and a configuration file. Fill in your configuration file with the authentication key from AssemblyAI and other credentials from X like so:
In the main script, we start by importing all the libraries we need.
Audio configuration
Next up is setting up the parameters of PyAudio and the WebSocket.
The main point to pay attention to here is the SAMPLE_RATE parameter. When setting up the connection to AssemblyAI's Universal-Streaming endpoint, the same sample rate needs to be specified. Since sentences will be transcribed in real-time with immutable results, we'll use AssemblyAI's Universal-Streaming endpoint.
Connecting to X
This section sets up the core functionality for detecting tweet commands and posting to X (formerly Twitter). The code defines two main command patterns: a simple "tweet" or “tweet that” command that posts previously recognized speech. The try_command_and_extract_payload() function uses regex patterns to identify tweet commands, and we use the Tweepy library to handle authentication and posting.
This foundation allows your speech-to-tweet application to reliably process voice commands and convert them into social media posts with proper error handling and user feedback.
Establishing the WebSocket connection
Our application uses WebSocket connections to maintain a persistent connection with AssemblyAI's streaming service. We'll need four main handler functions to manage the connection lifecycle and process incoming messages.The on_open function is called when our WebSocket connection is successfully established with AssemblyAI's servers.
Once connected, this function immediately starts streaming audio data. The key improvement here is that we're sending raw PCM audio data as binary frames directly to AssemblyAI, rather than encoding it as JSON. This creates a more efficient streaming experience with lower latency.
Processing transcription results
The heart of our application lies in the on_message function, which processes all incoming messages from AssemblyAI's streaming service.
The new streaming API provides structured message types that give us much more control over the transcription process. When a session begins, we receive important metadata like the session ID and expiration time.
Handling real-time transcripts
The most exciting part is processing the actual transcription results. The API sends us both partial results (as you're speaking) and final results (when you finish a sentence).
When turn_is_formatted is true, we know this is a final, polished transcript. At this point, we check if the user said a tweet command. Our application supports two types of commands: saying "tweet" or "tweet that" to post the previous sentence.
Displaying live transcription
For partial results, we create a dynamic display that updates in real-time as you speak:
This creates a user experience where you can see your words appearing on screen as you speak, before they're finalized and tweeted.
Managing session lifecycle
The API also tells us when sessions end and provides useful statistics:
This information is valuable for understanding how much audio was processed and optimizing performance.
Error handling and connection management
Robust error handling ensures our application gracefully manages connection issues:
When connections close or errors occur, we properly clean up all audio resources and stop background threads.
Running the application
The main function ties everything together, initializing our microphone, WebSocket connection, and Twitter client:
After setting up the audio stream, we create our WebSocket connection with all the handler functions we defined:
The application runs the WebSocket connection in a separate thread, allowing the main thread to handle user interrupts and cleanup.
Graceful Shutdown
When the user presses Ctrl+C, our application performs a clean shutdown:
We send a termination message to AssemblyAI's servers before closing the connection, ensuring proper cleanup on both ends.
Key improvements with Universal-Streaming
This updated implementation leverages several advantages of AssemblyAI's Universal-Streaming:
- Immutable transcripts: Results are final from the start, no revisions
- Ultra-low latency: ~300ms response time for better user experience
- Superior accuracy: Industry-leading accuracy with proper nouns and formatting
- Cost-effective: $0.15/hour pricing with unlimited concurrent streams
- Enhanced reliability: Built for production voice applications
Production considerations
For production deployment, consider these additional features:
- Temporary tokens: For client-side applications, use temporary authentication tokens instead of exposing your API key
- Connection management: Implement reconnection logic for dropped connections
- Rate limiting: Monitor your usage to stay within X API limits
- Error recovery: Add more sophisticated error handling and retry logic
You can find the complete updated code on GitHub.
AssemblyAI's Universal-Streaming makes building real-time voice applications like this auto-tweeter both technically straightforward and cost-effective, enabling developers to focus on creating unique user experiences rather than wrestling with speech recognition infrastructure.
