With PII Redaction, the API can automatically remove Personally Identifiable Information (PII), such as phone numbers and social security numbers, from the transcription text before it is returned to you.

All redacted text will be replaced with # characters. For example, if the phone number 111-2222 was spoken in the audio, it would be transcribed as ###-#### in the text.

Control Which Types of PII to Redact

To best-fit PII Redaction to your use case and data, you can select from a set of redaction policies when using PII Redaction. Include any or all of the policy names below in the redact_pii_policies array when making your POST request as shown on the right.

By default, any PII that is spoken will be transcribed with a hash #. For example, the credit card number 1111-2222-3333-4444 will be transcribed as ####-####-####-####.

By including the redact_pii_sub parameter in your POST request, you can customize how the PII is replaced.

Here are the options for the redact_pii_sub parameter:

When you transcribe a file with PII Redaction, the API can optionally generate a version of your original audio file with the PII "beeped" out when it is being spoken. To do so, include the redact_pii_audio parameter in your POST request, as shown on the right, when submitting files for transcription.

Get the Redacted Audio File

You can make a GET request to the below API endpoint, also shown on the right, to retrieve a URL that points to your redacted audio file:

https://api.assemblyai.com/v2/transcript/<your transcript id>/redacted-audio

In the JSON response shown on the right, you'll see the API responds with a redacted_audio_url key. This key contains a URL that points to your redacted audio file.

Please note that the redacted_audio_url link is only accessible for 24 hours. You'll want to download the redacted audio file from the URL, and save a copy on your end before the link expires.

Alternative Status Codes and Responses

Sometimes, you may not receive a 200 status code from the below API endpoint. In the table below, we outline the non-200 status codes you may receive, and what they mean.

https://api.assemblyai.com/v2/transcript/<your transcript id>/redacted-audio

Receive a Webhook

If a webhook_url was provided in your POST request when submitting your audio file for transcription, we will send a POST to your webhook_url when the redacted audio is ready. The POST request headers and JSON body will look like this:

headers
---
content-length: 79
accept-encoding: gzip, deflate
accept: */*
user-agent: python-requests/2.21.0
content-type: application/json

params
--
status: 'redacted_audio_ready'
redacted_audio_url: 'https://link-to-redacted-audio'

With Automatic Transcript Highlights, the AssemblyAI API can automatically detect important phrases and words in your transcription text.

For example, consider the following text:

We smirk because we believe that synthetic happiness is not of the same quality as what we might call natural happiness. What are these terms? Natural happiness is what we get when we get what we wanted. And synthetic happiness is what we make when we don't get what we wanted. And in our society...

Automatic Transcript Highlights will automatically detect the following key phrases/words in the text:

"synthetic happiness"
"natural happiness"
...

To enable this feature, include the auto_highlights parameter in your POST request when submitting files for transcription, and set this parameter to true.

Once your transcription is complete, and you GET the result, you'll see an auto_highlights_result key in the JSON response.

The auto_highlights_result key in the JSON response will contain the key phrases/words the API found in your transcription text. Here is a close-up of only that key's response, and what each value means:

With Content Safety Detection, AssemblyAI can detect if any of the following sensitive content is spoken in your audio/video files, and pinpoint exactly when and what was spoken:

Include the content_safety parameter in your POST request when submitting audio files for transcription, and set this parameter to true.

Interpreting Content Safety Detection Results

Once the transcription is complete, and you get the result, there will be an additional key content_safety_labels in the JSON response. Below, we'll drill into the data that is returned in the content_safety_labels key.

Understanding Severity Scores and Confidence Scores

Each label will be returned with a confidence score and a severity score. It is important to note that these two keys measure two very different things. The severity key will produce a score that shows how severe the flagged content is on a scale of 0–1. For example, a natural disaster with mass casualties would be a 1, whereas a wind storm that broke a lamppost would be a 0.1.

In comparison, confidence displays how confident the model was in predicting the label it predicted, also on a scale of 0-1.

We can break this down further by reviewing the following label:

"labels": [
    {
        "label": "health_issues",
        "confidence": 0.8225132822990417,
        "severity": 0.15090347826480865
    }
],

In the above example, the Content Safety model is indicating it is 82.25% confident that the spoken content is about Health Issues; however, it is measured at a low severity of 0.1509. This means the model is very confident the content is about health issues, but the content was not severe in nature (ie, was likely about a minor health issue).

Understanding the Severity Score Summary

The severity_score_summary key lists each label that was detected along with low, medium, and high keys.

"severity_score_summary": {
    "health_issues": {
        "low": 0.7210625030587972,
        "medium": 0.2789374969412028,
        "high": 0.0
    }
}

The value of the low, medium, and high keys reflect the API's confidence that the label is "low," "medium," or "high" in severity throughout the entire audio file. This score is based on the intersection of the length of the audio file, the frequency of low/medium/high severity tags through the file, and the confidence score for each of those occurrences.

Controlling the Threshold of Surfaced Results

By default, the content safety model will return any label with a confidence of 50% or greater. If you wish to set a higher or lower threshold, add the content_safety_confidence: {number} parameter to your POST request. This parameter will accept an integer value between 25 and 100, inclusive.

With Topic Detection, AssemblyAI can label the topics that are spoken in your audio/video files. The predicted topic labels follow the standardized IAB Taxonomy, which makes them suitable for Contextual Targeting use cases. The below table shows the 698 potential topics the API can predict.

To use our Topic Detection feature, include the iab_categories parameter in your POST request when submitting audio files for transcription, and set this parameter to true, as shown in the cURL request on the right.

Once the transcription is complete, and you get the transcription result, there will be an additional key iab_categories_result in the JSON response. Below, we drill into that key and what data it includes.

With Sentiment Analysis, AssemblyAI can detect the sentiment of each sentence of speech spoken in your audio files. Sentiment Analysis returns a result of POSITIVE, NEGATIVE, or NEUTRAL for each sentence in the transcript.

To include Sentiment Analysis in your transcript results, add the sentiment_analysis parameter in your POST request when submitting files for transcription and set this parameter to true, as shown in the cURL request on the right.

Once the transcription is complete, and you get the result, there will be an additional key sentiment_analysis_results in the JSON response.

For each sentence in the transcription text, the API will return the sentiment, confidence score, the start and end time for when that sentence was spoken, and, if applicable, the speaker label for that sentence. A detailed explanation of each key in the list of objects returned in the sentiment_analysis_results array can be found in the below table.

With Summarization, AssemblyAI can generate a single abstractive summary of entire audio files submitted for transcription.

When submitting a file for transcription, include the summarization parameter in your POST request, and set this to true. By default, Summarization produces a bullet-point style summary using our Informative model. Optionally, you can customize the summararization to best fit your use case using the summary_modeland summary_type parameters.

The summary_model parameters specifies which Summarization model will be used. The summary_type parameter determines the type of summarization results you will receive.

The table below shows more information on the options available for the summary_model parameter:

The table below shows more information on the options available for the summary_type parameter:

When the transcription finishes, you will see the generated summary key in the JSON response containing the summary of their submitted audio/video file in the format (summary_type) you requested.

Auto Chapters provides a "summary over time" for audio files transcribed with AssemblyAI. It works by first segmenting your audio files into logical "chapters" as the topic of conversation changes, and then provides an automatically generated summary for each "chapter" of content. For more information on Auto Chapters, check out this announcement from our blog.

When submitting a file for transcription, include the auto_chapters parameter in your POST request, and set this to true.

When your transcription is completed, you'll see a chapters key in the JSON response, as shown on the right. For each chapter that was detected, the API will include the start and end timestamps (in milliseconds), a summary - which is a few sentence summary of the content spoken during that timeframe, a short headline, which can be thought of as a "summary of the summary", and a gist, which is an ultra-short, few word summary of the chapter of content.

With Entity Detection, you can identify a wide range of entities that are spoken in your audio files, such as person and company names, email addresses, dates, and locations.

To include Entity Detection in your transcript response, add the entity_detection parameter in your POST request when submitting audio files for transcription, and set this parameter to true.

When your transcription is complete, you will see an entities key in the JSON response, as shown on the right. Below, we drill into the data that is returned within the list of results in the entities key.

Entity Types Detected

When Entity Detection is enabled, the entity types listed below are automatically detected and, if found in the transcription text, will be included in the entities key as shown above. They will be listed individually in the order that they appear in the transcript.