Over the past few years, the amount of audio and video content available online has been increasing at an exponential rate. This has created both a challenge and opportunity for product teams building with audio. On the one hand, there’s a tremendous amount of information encoded into audio/video files that can be leveraged to build powerful features and products. On the other hand, it’s very hard to work with audio data in its default state! Automatic transcription models, like those we build at AssemblyAI, have been a great first step to help turn audio data into a more pliable format: text. But there still leaves a lot to be desired.
To help developers and product teams build exciting products that automatically summarize audio and video content, we are introducing new AI-powered Summarization models that achieve impressive, state-of-the-art results on conversational data (phone calls, podcasts, video meetings, etc).
These models are powered by state-of-the-art AI research, and can automatically generate accurate summaries for audio or video files sent to our API.
Here’s a quick example of how powerful our latest AI Summarization models can be:
Bullets
- Josh Seiden and Brian Donohue discuss the topic of outcome versus output on Inside Intercom. Josh Seiden is a product consultant and author who has just released a book called Outcomes Over Output. Brian is product management director and he's looking forward to the chat.
- The main premise of the book is that by defining outcomes precisely, it's possible to apply this idea of outcomes in our work. It's in contrast to a really broad and undefined definition of the word ”outcome”.
- Paul, one of the design managers at Intercom, was struggling to differentiate between customer outcomes and business impact. In Lean Startup, teams focus on what they can change in their behavior to make their customers more satisfied. They focus on the business impact instead of on the customer outcomes. They have a hypothesis and they test their hypothesis with an experiment. They don't have to be 100% certain, but they need to have a hunch. There is a difference between problem-focused and outcome-focused approaches to building and prioritizing projects. For example, a company is working on improving the inbox search feature in their product. They hope it will improve user retention and improve the business impact of the change.
- Product teams need to focus on the outcome of their work rather than on the business impact of their product. They need to be more aware of the customer experience and their relationship with their business.
- As a business owner, you have to build a theory of how the business works. The more you know about your business as your business goes on, the more you can build a business model. The business model is reflected in roadmaps and prioritizations.
- Josh's book is available on Amazon, in print, in ebook and in audiobook on Audible.com. Brian's advice for teams looking to change their way of working is to start small and to use retrospectives and improve your process as you try to implement this. Josh and Brian enjoyed their conversation.
Paragraph
Headline
Gist
Our Summarization models in action
Over the past few years, AI has shown rapid advances in a number of fields. You can now generate high quality, realistic images and art with a single text prompt. Transformers have taken the world by storm and opened up the opportunity for Large Language Models (LLMs) that can generate code and write articles.
Our newest Summarization models are built on the same AI technology (Transformers) that are behind most of these advances. We’ve purpose-built our Summarization models to work extremely well on conversational data (phone calls, zoom meetings, screen recordings, videos, etc.). To demonstrate the power of these models, we include a number of examples below:
A deep dive into how our models work
AssemblyAI’s new Summarization models are fast, scalable, and continuously updated by our in-house team of AI experts to keep it state-of-the-art as new research emerges. These models are accessible through a single API call, making it easy for teams of all sizes to embed the models into their products.
For example, here is how you’d request a summary for an audio file using our API:
As you’ll see above, developers have access to customizable summary types, which can be controlled via the summary_type parameter. This offers developers the flexibility and control they need to generate different types of summaries depending on their use case. Here is a description of the available summary types we offer today:
Use cases for Summarization
Our Summarization models will help customers across a wide range of industries and use cases, including:
Conversational Intelligence
Speed up post-call Quality Assurance review by identifying key takeaways from the phone calls and removing the need for manual summarization.
Video/Media Platforms
Distill long educational courses, lectures, media broadcasts, and more into their most essential points for faster consumption.
Podcasts
Give podcast listeners a quick summary of what the podcast is about before they listen, and make podcast episodes more searchable.
Virtual Meeting Platforms
Offer summaries of full meetings, make meeting recordings easier to consume, and readily identify key takeaways and post-call action items.
Test our Summarization models today
You can use our CLI to quickly test our Summarization models on any audio or video file (even YouTube videos!) right from your terminal. For example:
If you want to start building applications with these Summarization models, you can see more complete code examples in our API documentation.
What’s next
Our Summarization models seek to empower developers and product teams to build new features that automatically extract essential information for their customers at scale.
Summarization is an active area of research — even measuring summary quality is difficult given its inherent subjectivity. There are many promising research avenues in the field of Summarization, and our AI research team is excited to explore these avenues to advance the state-of-the-art.
