Entity Detection

The Entity Detection model lets you automatically identify and categorize key information in transcribed audio content.

Here are a few examples of what you can detect:

• Names of people
• Organizations
• Addresses
• Phone numbers
• Medical data
• Social security numbers

For the full list of entities that you can detect, check out Supported entities.

Quickstart

Enable Entity Detection by setting entity_detection to true in the transcription config.

Example output

Canada
location
Timestamp: 2548 - 3130

the US
location
Timestamp: 5498 - 6350

...

API reference

Request

curl https://api.assemblyai.com/v2/transcript \
--header "Authorization: <YOUR_API_KEY>" \
--header "Content-Type: application/json" \
--data '{
  "audio_url": "YOUR_AUDIO_URL",
  "entity_detection": true
}'

Key	Type	Description
entity_detection	boolean	Enable Entity Detection.

Response

Key	Type	Description
entities	array	An array of detected entities.
entities[i].entity_type	string	The type of entity for the i-th detected entity.
entities[i].text	string	The text for the i-th detected entity.
entities[i].start	number	The starting time, in milliseconds, at which the i-th detected entity appears in the audio file.
entities[i].end	number	The ending time, in milliseconds, for the i-th detected entity in the audio file.

The response also includes the request parameters used to generate the transcript.

Supported entities

The model is designed to automatically detect and classify various types of entities within the transcription text. The detected entities and their corresponding types is listed individually in the entities key of the response object, ordered by when they first appear in the transcript.

Entity name	Description
banking_information	Banking information, including account and routing numbers.
blood_type	Blood type (e.g., O-, AB positive).
credit_card_cvv	Credit card verification code (e.g., CVV: 080).
credit_card_expiration	Expiration date of a credit card.
credit_card_number	Credit card number.
date	Specific calendar date (e.g., December 18).
date_of_birth	Date of Birth (e.g., Date of Birth: March 7, 1961).
drivers_license	Driver's license number (e.g., DL #356933-540).
drug	Medications, vitamins, or supplements (e.g., Advil, Acetaminophen, Panadol).
email_address	Email address (e.g., support@assemblyai.com).
event	Name of an event or holiday (e.g., Olympics, Yom Kippur).
injury	Bodily injury (e.g., I broke my arm, I have a sprained wrist).
language	Name of a natural language (e.g., Spanish, French).
location	Any location reference including mailing address, postal code, city, state, province, or country.
medical_condition	Name of a medical condition, disease, syndrome, deficit, or disorder (e.g., chronic fatigue syndrome, arrhythmia, depression).
medical_process	Medical process, including treatments, procedures, and tests (e.g., heart surgery, CT scan).
money_amount	Name and/or amount of currency (e.g., 15 pesos, $94.50).
nationality	Terms indicating nationality, ethnicity, or race (e.g., American, Asian, Caucasian).
occupation	Job title or profession (e.g., professor, actors, engineer, CPA).
organization	Name of an organization (e.g., CNN, McDonalds, University of Alaska).
password	Account passwords, PINs, access keys, or verification answers (e.g., 27%alfalfa, temp1234, My mother's maiden name is Smith).
person_age	Number associated with an age (e.g., 27, 75).
person_name	Name of a person, such as "Bob" and "Doug Jones".
phone_number	Telephone or fax number.
political_affiliation	Terms referring to a political party, movement, or ideology. For example, "Republican" and "Liberal".
religion	Terms indicating religious affiliation, such as "Hindu" and "Catholic".
time	Expressions indicating clock times, such as "19:37:28" and "10pm EST".
url	Internet addresses, such as "www.assemblyai.com".
us_social_security_number	Social Security Number or equivalent.

Frequently asked questions

How does the Entity Detection model handle misspellings or variations of entities?

The model is capable of identifying entities with variations in spelling or formatting. However, the accuracy of the detection may depend on the severity of the variation or misspelling.

Can the Entity Detection model identify custom entity types?

No, the Entity Detection model currently doesn't support the detection of custom entity types. However, the model is capable of detecting a wide range of predefined entity types, including people, organizations, locations, dates, times, addresses, phone numbers, medical data, and banking information, among others.

How can I improve the accuracy of the Entity Detection model?

To improve the accuracy of the Entity Detection model, it's recommended to provide high-quality audio files with clear and distinct speech. In addition, it's important to ensure that the audio content is relevant to the use case and that the entities being detected are relevant to the intended analysis. Finally, it may be helpful to review and adjust the model's configuration parameters, such as the confidence threshold for entity detection, to optimize the results.