Skip to content

improvement(usage): update usage limit in realtime, standardize token output object across providers #2553

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
waleedlatif1 merged 5 commits into from
Dec 23, 2025
Merged

improvement(usage): update usage limit in realtime, standardize token output object across providers #2553

waleedlatif1 merged 5 commits into from
Dec 23, 2025

Conversation

@waleedlatif1
Copy link
Collaborator

@waleedlatif1 waleedlatif1 commented Dec 23, 2025 •
edited
Loading

Summary

  • update usage limit in realtime
  • standardize token output object across providers

Type of Change

  • Bug fix
  • Code cleanup

Testing

Tested manually

Checklist

  • Code follows project style guidelines
  • Self-reviewed my changes
  • Tests added/updated and passing
  • No new warnings introduced
  • I confirm that I have read and agree to the terms outlined in the Contributor License Agreement (CLA)

@vercel
Copy link

vercel bot commented Dec 23, 2025 •
edited
Loading

The latest updates on your projects. Learn more about Vercel for GitHub.

1 Skipped Deployment
Project Deployment Review Updated (UTC)
docs Skipped Skipped Dec 23, 2025 8:56pm

@greptile-apps
Copy link
Contributor

greptile-apps bot commented Dec 23, 2025 •
edited
Loading

Greptile Summary

This PR standardizes token field naming across all AI providers from prompt/completion to input/output, and implements real-time usage limit updates through query invalidation.

Key Changes:

  • Renamed token fields in core types (TokenInfo, TokenUsage, ProviderResponse) from prompt/completion to input/output
  • Updated all 13 AI providers (OpenAI, Anthropic, Gemini, Azure, Cerebras, DeepSeek, Groq, Mistral, Ollama, OpenRouter, vLLM, XAI) to use new naming
  • Refactored QueryProvider to singleton pattern with getQueryClient() export for cache operations outside React
  • Implemented real-time usage updates by invalidating subscription queries with 1s setTimeout after workflow/copilot executions
  • Added backward compatibility fallbacks in ExecutionLogger.mergeCostModels() and evaluator-handler.ts to handle old prompt/completion fields
  • Updated executionData schema to include tokens and models at top level
  • Comprehensively updated test mocks across all handler tests

Implementation Quality:
The refactor is thorough and well-executed with proper backward compatibility handling. The real-time usage update mechanism uses a 1s delay to allow backend processing before cache invalidation.

Confidence Score: 5/5

  • This PR is safe to merge with minimal risk
  • The changes are systematic, well-tested, and include backward compatibility. All providers consistently updated, tests comprehensively modified, and the refactor follows a clear pattern across 51 files.
  • No files require special attention

Important Files Changed

Filename Overview
apps/sim/providers/types.ts Updated TokenInfo and ProviderResponse.tokens from prompt/completion to input/output, establishing standard naming convention
apps/sim/lib/tokenization/types.ts Standardized TokenUsage interface to use input/output field names instead of prompt/completion
apps/sim/lib/logs/execution/logger.ts Added backward compatibility fallbacks for prompt/completion when merging costs, updated token field names to new standard
apps/sim/app/_shell/providers/query-provider.tsx Refactored to singleton pattern with getQueryClient() export for cache invalidation outside React components
apps/sim/app/workspace/[workspaceId]/w/[workflowId]/hooks/use-workflow-execution.ts Updated to use getQueryClient() with 1s setTimeout for real-time usage limit updates after workflow execution
apps/sim/lib/logs/types.ts Added tokens and models to executionData, updated cost token fields, kept backward-compatible TokenInfo

Sequence Diagram

sequenceDiagram
    participant User
    participant WorkflowExec as Workflow Execution
    participant Provider as AI Provider
    participant Logger as Execution Logger
    participant UsageLog as Usage Log DB
    participant QueryClient as Query Client
    participant UI as User Interface

    User->>WorkflowExec: Execute workflow
    WorkflowExec->>Provider: Request with model params
    Provider-->>WorkflowExec: Response with tokens {input, output, total}
    WorkflowExec->>Logger: Log execution with standardized tokens
    Logger->>Logger: Merge costs (backward compat: prompt→input, completion→output)
    Logger->>UsageLog: Batch insert usage records
    WorkflowExec->>QueryClient: setTimeout(invalidateQueries, 1000ms)
    QueryClient->>UI: Refetch subscription/usage data
    UI-->>User: Display updated usage in real-time
Loading

greptile-apps[bot]
greptile-apps bot reviewed Dec 23, 2025
View reviewed changes
Copy link
Contributor

@greptile-apps greptile-apps bot left a comment •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Additional Comments (1)

  1. apps/sim/lib/logs/execution/logger.ts, line 62-63 (link)

    logic: token fields still use old prompt/completion names instead of input/output

    the calculateCostSummary function in logging-factory.ts now returns models with input/output token fields, but this merge function still expects prompt/completion, causing token data to be lost during model cost merging

28 files reviewed, 1 comment

Edit Code Review Agent Settings | Greptile

@waleedlatif1
Copy link
Collaborator Author

waleedlatif1 commented Dec 23, 2025

merge has the new format (input, output) taking precedence but falling back to the old format (prompt, completions)

@waleedlatif1
Copy link
Collaborator Author

waleedlatif1 commented Dec 23, 2025

@greptile

icecrasher321
icecrasher321 reviewed Dec 23, 2025
View reviewed changes
greptile-apps[bot]
greptile-apps bot reviewed Dec 23, 2025
View reviewed changes
Copy link
Contributor

@greptile-apps greptile-apps bot left a comment •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Additional Comments (1)

  1. apps/sim/lib/logs/execution/logger.ts, line 62-63 (link)

    logic: still using old prompt/completion keys instead of input/output

28 files reviewed, 2 comments

Edit Code Review Agent Settings | Greptile

@waleedlatif1
Copy link
Collaborator Author

waleedlatif1 commented Dec 23, 2025

@greptile

@waleedlatif1 waleedlatif1 merged commit 4e09c38 into staging Dec 23, 2025
10 checks passed
@waleedlatif1 waleedlatif1 deleted the improvement/usage-limit branch December 23, 2025 21:04
waleedlatif1 added a commit that referenced this pull request Dec 23, 2025
@waleedlatif1
improvement(usage): update usage limit in realtime, standardize token…
a702293 
… output object across providers (#2553)

* improvement(usage-limit): update usage in real time, fix token output object

* updated tokenBreakdown to tokens, standardized input/output/total token object type across providers

* update remaining references

* ack PR comment

* remove singleton query client instance from hooks, leave only in zustand
@waleedlatif1 waleedlatif1 mentioned this pull request Dec 24, 2025
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@icecrasher321 icecrasher321 icecrasher321 left review comments

+1 more reviewer

@greptile-apps greptile-apps[bot] greptile-apps[bot] left review comments

Reviewers whose approvals may not affect merge requirements

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

3 participants

@waleedlatif1 @icecrasher321