> ## Documentation Index
> Fetch the complete documentation index at: https://assemblyai.com/docs/llms.txt
> Use this file to discover all available pages before exploring further.

# Entity Detection

export const LanguageTable = ({languages, columns = 3}) => {
  return <div className="grid gap-2" style={{
    gridTemplateColumns: `repeat(${columns}, 1fr)`
  }}>
      {languages.map(language => <div key={language.code} className="flex justify-between items-center">
          <span>{language.name}</span>
          <code className="text-sm bg-gray-100 px-2 py-1 rounded">
            {language.code}
          </code>
        </div>)}
    </div>;
};

<AccordionGroup>
  <Accordion title="Supported languages">
    <LanguageTable
      languages={[
  { name: "Global English", code: "en" },
  { name: "Australian English", code: "en_au" },
  { name: "British English", code: "en_uk" },
  { name: "US English", code: "en_us" },
  { name: "Spanish", code: "es" },
  { name: "French", code: "fr" },
  { name: "German", code: "de" },
  { name: "Italian", code: "it" },
  { name: "Portuguese", code: "pt" },
  { name: "Dutch", code: "nl" },
  { name: "Hindi", code: "hi" },
  { name: "Japanese", code: "ja" },
  { name: "Chinese", code: "zh" },
  { name: "Finnish", code: "fi" },
  { name: "Korean", code: "ko" },
  { name: "Polish", code: "pl" },
  { name: "Russian", code: "ru" },
  { name: "Turkish", code: "tr" },
  { name: "Ukrainian", code: "uk" },
  { name: "Vietnamese", code: "vi" },
  { name: "Afrikaans", code: "af" },
  { name: "Arabic", code: "ar" },
  { name: "Belarusian", code: "be" },
  { name: "Bulgarian", code: "bg" },
  { name: "Burmese", code: "my" },
  { name: "Catalan", code: "ca" },
  { name: "Croatian", code: "hr" },
  { name: "Czech", code: "cs" },
  { name: "Danish", code: "da" },
  { name: "Estonian", code: "et" },
  { name: "Georgian", code: "ka" },
  { name: "Greek", code: "el" },
  { name: "Hebrew", code: "he" },
  { name: "Hungarian", code: "hu" },
  { name: "Icelandic", code: "is" },
  { name: "Indonesian", code: "id" },
  { name: "Khmer", code: "km" },
  { name: "Latvian", code: "lv" },
  { name: "Lithuanian", code: "lt" },
  { name: "Luxembourgish", code: "lb" },
  { name: "Malay", code: "ms" },
  { name: "Norwegian", code: "no" },
  { name: "Persian", code: "fa" },
  { name: "Romanian", code: "ro" },
  { name: "Slovak", code: "sk" },
  { name: "Slovenian", code: "sl" },
  { name: "Swahili", code: "sw" },
  { name: "Swedish", code: "sv" },
  { name: "Tagalog", code: "tl" },
  { name: "Tamil", code: "ta" },
]}
      columns={4}
    />

    <br />
  </Accordion>

  <Accordion title="Supported models">
    <LanguageTable
      languages={[
  { name: "Universal-3.5 Pro", code: "universal-3-5-pro" },
  { name: "Universal-2", code: "universal-2" },
]}
      columns={2}
    />

    <br />
  </Accordion>

  <Accordion title="Supported regions">
    US & EU <br />
  </Accordion>
</AccordionGroup>

The Entity Detection model lets you automatically identify and categorize key information in transcribed audio content.

Here are a few examples of what you can detect:

* Names of people
* Organizations
* Addresses
* Phone numbers
* Medical data
* Social security numbers

For the full list of entities that you can detect, see [Supported entities](#supported-entities).

<Tip>
  **Supported Languages**

  Entity Detection is available in multiple languages. See [Supported
  languages](/pre-recorded-audio/supported-languages).
</Tip>

## Quickstart

<Tabs groupId="language">
  <Tab language="python" title="Python" default>
    Enable Entity Detection by setting `entity_detection` to `True` in the JSON payload.

    ```python {19} expandable theme={null}
    import requests
    import time

    base_url = "https://api.assemblyai.com"

    headers = {
        "authorization": "<YOUR_API_KEY>"
    }

    with open("./local_file.mp3", "rb") as f:
        response = requests.post(base_url + "/v2/upload",
                                headers=headers,
                                data=f)

    upload_url = response.json()["upload_url"]

    data = {
        "audio_url": upload_url, # You can also use a URL to an audio or video file on the web
        "language_detection": True,
        "entity_detection": True
    }

    url = base_url + "/v2/transcript"
    response = requests.post(url, json=data, headers=headers)

    transcript_id = response.json()['id']
    polling_endpoint = base_url + "/v2/transcript/" + transcript_id

    print(f"Transcript ID: {transcript_id}")

    while True:
        transcription_result = requests.get(polling_endpoint, headers=headers).json()

        if transcription_result['status'] == 'completed':
          for entity in transcription_result['entities']:
            print(entity['text'])
            print(entity['entity_type'])
            print(f"Timestamp: {entity['start']} - {entity['end']}\n")
          break
        elif transcription_result['status'] == 'error':
            raise RuntimeError(f"Transcription failed: {transcription_result['error']}")
        else:
            time.sleep(3)
    ```
  </Tab>

  <Tab language="python-sdk" title="Python SDK">
    Enable Entity Detection by setting `entity_detection` to `True` in the transcription config.

    ```python {8} theme={null}
    import assemblyai as aai

    aai.settings.api_key = "<YOUR_API_KEY>"

    # audio_file = "./local_file.mp3"
    audio_file = "https://assembly.ai/wildfires.mp3"

    config = aai.TranscriptionConfig(
        language_detection=True,
        entity_detection=True
    )

    transcript = aai.Transcriber().transcribe(audio_file, config)
    print(f"Transcript ID: {transcript.id}")

    for entity in transcript.entities:
        print(entity.text)
        print(entity.entity_type)
        print(f"Timestamp: {entity.start} - {entity.end}\n")
    ```
  </Tab>

  <Tab language="javascript" title="JavaScript">
    Enable Entity Detection by setting `entity_detection` to `true` in the JSON payload.

    ```javascript {19} expandable theme={null}
    import fs from "fs-extra";

    const baseUrl = "https://api.assemblyai.com";

    const headers = {
      authorization: "<YOUR_API_KEY>",
    };

    const path = "./my-audio.mp3";
    const audioData = await fs.readFile(path);
    let res = await fetch(`${baseUrl}/v2/upload`, {
      method: "POST",
      headers,
      body: audioData,
    });
    if (!res.ok) throw new Error(`Error: ${res.status}`);
    const uploadResponse = await res.json();
    const uploadUrl = uploadResponse.upload_url;

    const data = {
      audio_url: uploadUrl, // You can also use a URL to an audio or video file on the web
      language_detection: true,
      entity_detection: true,
    };

    const url = `${baseUrl}/v2/transcript`;
    res = await fetch(url, {
      method: "POST",
      headers: { ...headers, "Content-Type": "application/json" },
      body: JSON.stringify(data),
    });
    if (!res.ok) throw new Error(`Error: ${res.status}`);
    const response = await res.json();

    const transcriptId = response.id;
    console.log("Transcript ID: ", transcriptId);

    const pollingEndpoint = `${baseUrl}/v2/transcript/${transcriptId}`;

    while (true) {
      res = await fetch(pollingEndpoint, { headers });
      if (!res.ok) throw new Error(`Error: ${res.status}`);
      const transcriptionResult = await res.json();

      if (transcriptionResult.status === "completed") {
        for (const entity of transcriptionResult.entities) {
          console.log(entity.text);
          console.log(entity.entity_type);
          console.log(`Timestamp: ${entity.start} - ${entity.end}\n`);
        }
        break;
      } else if (transcriptionResult.status === "error") {
        throw new Error(`Transcription failed: ${transcriptionResult.error}`);
      } else {
        await new Promise((resolve) => setTimeout(resolve, 3000));
      }
    }
    ```
  </Tab>

  <Tab language="javascript-sdk" title="JavaScript SDK">
    Enable Entity Detection by setting `entity_detection` to `true` in the transcription config.

    ```javascript {12} expandable theme={null}
    import { AssemblyAI } from "assemblyai";

    const client = new AssemblyAI({
      apiKey: "<YOUR_API_KEY>",
    });

    // const audioFile = './local_file.mp3'
    const audioFile = "https://assembly.ai/wildfires.mp3";

    const params = {
      audio: audioFile,
      language_detection: true,
      entity_detection: true,
    };

    const run = async () => {
      const transcript = await client.transcripts.transcribe(params);

      for (const entity of transcript.entities) {
        console.log(entity.text);
        console.log(entity.entity_type);
        console.log(`Timestamp: ${entity.start} - ${entity.end}\n`);
      }
    };

    run();
    ```
  </Tab>
</Tabs>

### Example output

```plain theme={null}
Canada
location
Timestamp: 2548 - 3130

the US
location
Timestamp: 5498 - 6350

...
```

## API reference

### Request

```bash {6} theme={null}
curl https://api.assemblyai.com/v2/transcript \
--header "Authorization: <YOUR_API_KEY>" \
--header "Content-Type: application/json" \
--data '{
  "audio_url": "YOUR_AUDIO_URL",
  "entity_detection": true
}'
```

| Key                | Type    | Description              |
| ------------------ | ------- | ------------------------ |
| `entity_detection` | boolean | Enable Entity Detection. |

### Response

```js expandable theme={null}
{
  entities: [
    {
      entity_type: "location",
      text: "Canada",
      start: 2548,
      end: 3130,
    },
    {
      entity_type: "location",
      text: "the US",
      start: 5498,
      end: 6382,
    },
    {
      entity_type: "location",
      text: "Maine",
      start: 7492,
      end: 7914,
    },
    {
      entity_type: "location",
      text: "Maryland",
      start: 8212,
      end: 8634,
    },
    {
      entity_type: "location",
      text: "Minnesota",
      start: 8932,
      end: 9578,
    },
    {
      entity_type: "person_name",
      text: "Peter de Carlo",
      start: 18948,
      end: 19930,
    },
    {
      entity_type: "occupation",
      text: "associate professor",
      start: 20292,
      end: 21194,
    },
    {
      entity_type: "organization",
      text: "Department of Environmental Health and Engineering",
      start: 21508,
      end: 23706,
    },
    {
      entity_type: "organization",
      text: "Johns Hopkins University Varsity",
      start: 23972,
      end: 25490,
    },
    {
      entity_type: "occupation",
      text: "professor",
      start: 26076,
      end: 26950,
    },
    {
      entity_type: "location",
      text: "the US",
      start: 45184,
      end: 45898,
    },
    {
      entity_type: "nationality",
      text: "Canadian",
      start: 49728,
      end: 50086,
    },
    {
      entity_type: "location",
      text: "Pennsylvania",
      start: 51680,
      end: 52326,
    },
    {
      entity_type: "location",
      text: "Mid Atlantic",
      start: 52624,
      end: 53178,
    },
    {
      entity_type: "location",
      text: "Northeast",
      start: 53428,
      end: 53866,
    },
    {
      entity_type: "location",
      text: "Baltimore",
      start: 65064,
      end: 65534,
    },
    {
      entity_type: "occupation",
      text: "science",
      start: 101168,
      end: 101446,
    },
    {
      entity_type: "location",
      text: "New York City",
      start: 125768,
      end: 126274,
    },
    {
      entity_type: "medical_condition",
      text: "respiratory conditions",
      start: 152964,
      end: 153786,
    },
    {
      entity_type: "medical_condition",
      text: "heart conditions",
      start: 153988,
      end: 154506,
    },
    {
      entity_type: "location",
      text: "New York",
      start: 171448,
      end: 171938,
    },
    {
      entity_type: "location",
      text: "New York",
      start: 176008,
      end: 176322,
    },
    {
      entity_type: "location",
      text: "the US",
      start: 201824,
      end: 202202,
    },
    {
      entity_type: "location",
      text: "mid Atlantic",
      start: 209010,
      end: 209866,
    },
    {
      entity_type: "location",
      text: "Northeast region",
      start: 210196,
      end: 211082,
    },
    {
      entity_type: "location",
      text: "Western US",
      start: 257364,
      end: 258046,
    },
    {
      entity_type: "location",
      text: "eastern US",
      start: 258484,
      end: 259054,
    },
    {
      entity_type: "person_name",
      text: "Peter De Carlo",
      start: 268298,
      end: 269194,
    },
    {
      entity_type: "occupation",
      text: "associate professor",
      start: 269242,
      end: 270186,
    },
    {
      entity_type: "organization",
      text: "Department of Environmental Health and Engineering",
      start: 270404,
      end: 272762,
    },
    {
      entity_type: "organization",
      text: "Johns Hopkins University",
      start: 273156,
      end: 274850,
    },
    {
      entity_type: "occupation",
      text: "Sergeant",
      start: 274970,
      end: 275298,
    },
    {
      entity_type: "person_name",
      text: "Carlo",
      start: 275314,
      end: 275634,
    },
  ],
}
```

| Key                       | Type   | Description                                                                                      |
| ------------------------- | ------ | ------------------------------------------------------------------------------------------------ |
| `entities`                | array  | An array of detected entities.                                                                   |
| `entities[i].entity_type` | string | The type of entity for the i-th detected entity.                                                 |
| `entities[i].text`        | string | The text for the i-th detected entity.                                                           |
| `entities[i].start`       | number | The starting time, in milliseconds, at which the i-th detected entity appears in the audio file. |
| `entities[i].end`         | number | The ending time, in milliseconds, for the i-th detected entity in the audio file.                |

The response also includes the request parameters used to generate the transcript.

## Supported entities

The model is designed to automatically detect and classify various types of entities within the transcription text. The detected entities and their corresponding types is listed individually in the entities key of the response object, ordered by when they first appear in the transcript.

| Entity name                 | Description                                                                                                    | Example                                                          |
| --------------------------- | -------------------------------------------------------------------------------------------------------------- | ---------------------------------------------------------------- |
| `account_number`            | Customer account or membership identification number                                                           | Policy No. 10042992; Member ID: HZ-5235-001                      |
| `banking_information`       | Banking information, including account and routing numbers                                                     |                                                                  |
| `blood_type`                | Blood type                                                                                                     | O-, AB positive                                                  |
| `credit_card_cvv`           | Credit card verification code                                                                                  | CVV: 080                                                         |
| `credit_card_expiration`    | Expiration date of a credit card                                                                               |                                                                  |
| `credit_card_number`        | Credit card number                                                                                             |                                                                  |
| `date`                      | Specific calendar date                                                                                         | December 18                                                      |
| `date_interval`             | Broader time periods, including date ranges, months, seasons, years, and decades                               | 2020-2021; 5-9 May; January 1984                                 |
| `date_of_birth`             | Date of birth                                                                                                  | Date of Birth: March 7, 1961                                     |
| `drivers_license`           | Driver's license number                                                                                        | DL# 356933-540                                                   |
| `drug`                      | Medications, vitamins, or supplements                                                                          | Advil, Acetaminophen, Panadol                                    |
| `duration`                  | Periods of time, specified as a number and a unit of time                                                      | 8 months; 2 years                                                |
| `email_address`             | Email address                                                                                                  | [support@assemblyai.com](mailto:support@assemblyai.com)          |
| `event`                     | Name of an event or holiday                                                                                    | Olympics, Yom Kippur                                             |
| `filename`                  | Names of computer files, including the extension or filepath                                                   | Taxes/2012/brad-tax-returns.pdf                                  |
| `gender_sexuality`          | Terms indicating gender identity or sexual orientation, including slang terms                                  | female; bisexual; trans                                          |
| `healthcare_number`         | Healthcare numbers and health plan beneficiary numbers                                                         | Policy No.: 5584-486-674-YM                                      |
| `injury`                    | Injuries or health issues                                                                                      | I broke my arm, I have a sprained wrist                          |
| `ip_address`                | Internet IP address, including IPv4 and IPv6 formats                                                           | 192.168.0.1                                                      |
| `language`                  | Name of a natural language                                                                                     | Spanish, French                                                  |
| `location`                  | Any Location reference including mailing address, postal code, city, state, province, country, or coordinates. | Lake Victoria, 145 Windsor St., 90210                            |
| `marital_status`            | Terms indicating marital status                                                                                | Single; common-law; ex-wife; married                             |
| `medical_condition`         | Name of a medical condition, disease, syndrome, deficit, or disorder                                           | chronic fatigue syndrome, arrhythmia, depression                 |
| `medical_process`           | Medical process, including treatments, procedures, and tests                                                   | heart surgery, CT scan                                           |
| `money_amount`              | Name and/or amount of currency                                                                                 | 15 pesos, \$94.50                                                |
| `nationality`               | Terms indicating nationality, ethnicity, or race                                                               | American, Asian, Caucasian                                       |
| `number_sequence`           | Numerical PII (including alphanumeric strings) that doesn't fall under other categories                        |                                                                  |
| `occupation`                | Job title or profession                                                                                        | professor, actors, engineer, CPA                                 |
| `organization`              | Name of an organization                                                                                        | CNN, McDonalds, University of Alaska, Northwest General Hospital |
| `passport_number`           | Passport numbers, issued by any country                                                                        | PA4568332; NU3C6L86S12                                           |
| `password`                  | Account passwords, PINs, access keys, or verification answers                                                  | 27%alfalfa, temp1234, My mother's maiden name is Smith           |
| `person_age`                | Number associated with an age                                                                                  | 27, 75                                                           |
| `person_name`               | Name of a person                                                                                               | Bob, Doug Jones, Dr. Kay Martinez, MD                            |
| `phone_number`              | Telephone or fax number                                                                                        |                                                                  |
| `physical_attribute`        | Distinctive bodily attributes, including terms indicating race                                                 | I'm 190cm tall; He belongs to the Black students' association    |
| `political_affiliation`     | Terms referring to a political party, movement, or ideology                                                    | Republican, Liberal                                              |
| `religion`                  | Terms indicating religious affiliation                                                                         | Hindu, Catholic                                                  |
| `statistics`                | Medical statistics                                                                                             | 18%, 18 percent                                                  |
| `time`                      | Expressions indicating clock times                                                                             | 19:37:28; 10pm EST                                               |
| `url`                       | Internet addresses                                                                                             | [https://www.assemblyai.com/](https://www.assemblyai.com/)       |
| `us_social_security_number` | Social Security Number or equivalent                                                                           |                                                                  |
| `username`                  | Usernames, login names, or handles                                                                             | @AssemblyAI                                                      |
| `vehicle_id`                | Vehicle identification numbers (VINs), vehicle serial numbers, and license plate numbers                       | 5FNRL38918B111818; BIF7547                                       |
| `zodiac_sign`               | Names of Zodiac signs                                                                                          | Aries; Taurus                                                    |

## Frequently asked questions

<AccordionGroup>
  <Accordion title="How does the Entity Detection model handle misspellings or variations of entities?" theme="dark" iconColor="white">
    The model is capable of identifying entities with variations in spelling or formatting. However, the accuracy of the detection may depend on the severity of the variation or misspelling.
  </Accordion>

  <Accordion title="Can the Entity Detection model identify custom entity types?" theme="dark" iconColor="white">
    No, the Entity Detection model doesn't support the detection of custom entity types. However, the model is capable of detecting a wide range of predefined entity types, including people, organizations, locations, dates, times, addresses, phone numbers, medical data, and banking information, among others.
  </Accordion>

  <Accordion title="How can I improve the accuracy of the Entity Detection model?" theme="dark" iconColor="white">
    To improve the accuracy of the Entity Detection model, it's recommended to provide high-quality audio files with clear and distinct speech. In addition, it's important to ensure that the audio content is relevant to the use case and that the entities being detected are relevant to the intended analysis. Finally, it may be helpful to review and adjust the model's configuration parameters, such as the confidence threshold for entity detection, to optimize the results.
  </Accordion>
</AccordionGroup>