For AI agents: a documentation index is available at the root level at /llms.txt and /llms-full.txt. Append /llms.txt to any URL for a page-level index, or .md for the markdown version of any page.
PlaygroundChangelogSign In
OverviewAPI ReferencePre-recorded STTStreaming STTVoice AgentsSpeech UnderstandingGuardrailsLLM GatewayFAQ
OverviewAPI ReferencePre-recorded STTStreaming STTVoice AgentsSpeech UnderstandingGuardrailsLLM GatewayFAQ
  • Overview
      • Understanding Input and Output Tokens for LLM Gateway
LogoLogo
PlaygroundChangelogSign In
On this page
  • What are Input Tokens?
  • What are Output Tokens?
  • Key Differences
  • Why This Matters
OverviewLLM Gateway

Understanding Input and Output Tokens for LLM Gateway

Was this page helpful?
Previous

Can you use the Playground with files in languages other than English?

Next
Built with

What are Input Tokens?

Input tokens represent the number of tokens provided to the AI model in your initial request. These include:

  • The prompt or question
  • Transcript text
  • Context information
  • Any other data included in your request

What are Output Tokens?

Output tokens represent the number of tokens generated by the AI model in response to your input. This typically includes:

  • The text response from the model

Key Differences

There are two main differences between input and output tokens:

  1. Quantity: Usually, the number of input tokens is higher than the number of output tokens. This is because your initial request often contains more information than the model’s response.

  2. Cost: Output tokens generally cost more than input tokens. This is due to the computational resources required for the model to generate a response.

Why This Matters

Understanding the difference between input and output tokens can help you:

  • Optimize your requests to the AI model
  • Manage costs associated with using the AI service
  • Improve the efficiency of your interactions with the AI model

If you have any further questions about input and output tokens or how they affect your usage of our AI service, please don’t hesitate to contact our support team.