-
Notifications
You must be signed in to change notification settings - Fork 3.1k
feat(anthropic): add native structured outputs support #2528
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
feat(anthropic): add native structured outputs support #2528
Conversation
Implements Anthropic's native structured outputs (November 2025 feature) for supported Claude models using constrained decoding. Changes: - Add STRUCTURED_OUTPUT_MODELS list for Claude 4.x models that support native structured outputs (haiku-4-5, sonnet-4-5, opus-4-1, opus-4-5) - Add supportsNativeStructuredOutput() to detect supported models - Add ensureAdditionalPropertiesFalse() to prepare schemas - Add createMessage() and createStreamingMessage() helpers that use anthropic.beta.messages.create() with the structured-outputs-2025-11-13 beta header when native structured outputs are enabled - Automatically detect model capability and use native structured outputs when available, falling back to prompt-based approach for older models Native structured outputs use grammar-based constrained decoding, guaranteeing valid JSON output matching the specified schema.
|
@aadamsx is attempting to deploy a commit to the Sim Team on Vercel. A member of the Team first needs to authorize it. |
Greptile SummaryThis PR adds Anthropic's native structured outputs (November 2025 feature) to the Anthropic provider, replacing unreliable prompt-based approaches with constrained decoding for Claude 4.x models. The implementation automatically detects model capabilities and falls back to prompt-based instructions for older Claude 3.x models. Key Changes:
Issues Found:
Confidence Score: 3/5
Important Files Changed
Sequence DiagramsequenceDiagram
actor Client
participant Provider as Anthropic Provider
participant DetectSupport as supportsNativeStructuredOutput()
participant CreateMsg as createMessage()
participant APIStandard as Standard API
participant APIBeta as Beta API (Structured Output)
participant Fallback as Fallback Prompt-Based
Client->>Provider: executeRequest(responseFormat, model)
Provider->>DetectSupport: supportsNativeStructuredOutput(model)
alt Model supports native structured output
DetectSupport-->>Provider: true
Provider->>Provider: ensureAdditionalPropertiesFalse(schema)
Provider->>CreateMsg: useNativeStructuredOutput=true
CreateMsg->>APIBeta: create({output_format: json_schema, betas: [structured-outputs-2025-11-13]})
APIBeta-->>CreateMsg: Anthropic.Message (constrained JSON)
CreateMsg-->>Provider: response
else Model does not support native structured output
DetectSupport-->>Provider: false
Provider->>Fallback: Build JSON template instructions
Provider->>CreateMsg: useNativeStructuredOutput=false
CreateMsg->>APIStandard: create({system: "...JSON schema instructions..."})
APIStandard-->>CreateMsg: Anthropic.Message (prompt-based)
CreateMsg-->>Provider: response
end
Provider-->>Client: ProviderResponse | StreamingExecution
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Additional Comments (5)
-
apps/sim/providers/anthropic/index.ts, line 35-39 (link)logic: The model matching logic here is overly permissive. Using both
.includes()and.startsWith()with the same lowercase comparison means a model namedclaude-3-5-sonnetwould matchclaude-sonnet-4-5(contains "sonnet"), creating false positives. This could cause older models to incorrectly attempt native structured outputs. Need exact version matching instead. -
apps/sim/providers/anthropic/index.ts, line 45-68 (link)logic: Schema recursion doesn't handle
oneOf,anyOf,allOf, or discriminated unions - common in JSON Schema. Nested objects in these properties won't be processed, potentially failing validation at the API level if they also needadditionalProperties: false. -
apps/sim/providers/anthropic/index.ts, line 83-96 (link)style: The
loggerparameter is typed asanydespite being available from module scope. Consider using the module-levelloggerinstead of passing it as a parameter - reduces parameter noise and ensures consistency. -
apps/sim/providers/anthropic/index.ts, line 101-127 (link)style: Same as above -
loggeris typedanyand passed as parameter. Use module-level logger to reduce noise. -
apps/sim/providers/anthropic/index.ts, line 84-91 (link)logic: Using
(anthropic.beta.messages as any).create()bypasses TypeScript type checking. If the SDK updates the beta API type signature, this code won't catch breaking changes. Consider updating the SDK types or documenting why the cast is necessary.
1 file reviewed, 5 comments
|
resolved in #2515 |
Summary
Implements Anthropic's native structured outputs (November 2025 feature) for supported Claude models using constrained decoding.
Problem
The current Anthropic provider uses prompt-based structured output, which is unreliable:
Solution
Use Anthropic's native structured outputs with constrained decoding for Claude 4.x models:
structured-outputs-2025-11-13output_formatwithtype: "json_schema"Changes
STRUCTURED_OUTPUT_MODELSlist for Claude 4.x modelssupportsNativeStructuredOutput()to detect supported modelsensureAdditionalPropertiesFalse()to prepare schemas (required by API)createMessage()andcreateStreamingMessage()helpers that use the beta APISupported Models
Native structured outputs work with:
claude-haiku-4-5claude-sonnet-4-5claude-opus-4-1claude-opus-4-5References