OpenAI Token Usage

This article provides a rough estimation, as this aligns with the nature of AI and how token usage is calculated. The result varies significantly depending on individual usage factors such as how frequently the AI integration is used, the number of users involved, and the length and complexity of the prompts submitted.

General Information on BOC AI Features can be found in our ADONIS AI Assistant Documentation.
 

Token Usage "Process Design"

Based on extended logs material, we can provide the following approximative numbers based on a small input prompt (1-2 sentences) for creating a process.
The prompt tokens are sent while the completion tokens are those received as an answer from the AI.
 
Tested on gpt-4o-mini with max complexity (i.e. max level of detail).
 

Token Counts per Prompt and Completion
 

GPT-4o-mini Token Costs
 

 
Costs per model and Models per $ Spent
 

 
Token Usage "Process Analysis"

API usage tested on gpt-4o. The amount of prompt tokens varies on model size and amount of information model contains (number of elements, element properties, relations etc.)
 

General Information on Token usage

Token usage is not based on directly based characters; there is only a rule of thumb:

• 1 token ≈ 4 characters (English average)
• 1 token ≈ 0.75 words (English average)
• Tokenization splits text into chunks that can be whole words, subwords, or characters, depending on frequency and model vocabulary. Increasing the complexity of an estimation.
• Non-English languages often have a higher token-to-character ratio.
   

Regarding Tokenization

Tokenization is active whenever text is processed by an OpenAI language model, such as when you provide input prompts or the model generates output. The model always breaks down text into tokens to understand and generate language. 

This happens behind the backend during both input encoding (when text is sent to the model) and output decoding (when the model produces text). It ensures text is converted into manageable chunks (tokens) that the model can read and predict.
 
Each word or symbol might be one or split into multiple tokens depending on the model's tokenizer rules:
 

This demonstrates how tokenization varies by word length, punctuation, and symbol use. Each word or symbol might be one or split into multiple tokens, depending on the individual model's tokenizer rules.
 
OpenAI usually offers a dashboard to monitor your token consumption accurately: Rate limits - OpenAI API