feat(anthropic): add native structured outputs support #2531
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Implements Anthropic's native structured outputs (November 2025 feature) for supported Claude models using constrained decoding. Changes: - Add STRUCTURED_OUTPUT_MODELS list for Claude 4.x models that support native structured outputs (haiku-4-5, sonnet-4-5, opus-4-1, opus-4-5) - Add supportsNativeStructuredOutput() to detect supported models - Add ensureAdditionalPropertiesFalse() to prepare schemas - Add createMessage() and createStreamingMessage() helpers that use anthropic.beta.messages.create() with the structured-outputs-2025-11-13 beta header when native structured outputs are enabled - Automatically detect model capability and use native structured outputs when available, falling back to prompt-based approach for older models Native structured outputs use grammar-based constrained decoding, guaranteeing valid JSON output matching the specified schema.
Addresses all issues from PR review: 1. Model matching logic (false positives): - Changed from includes/startsWith to prefix-only matching - Renamed to STRUCTURED_OUTPUT_MODEL_PREFIXES for clarity - Added claude-3-5-haiku which supports structured outputs - Now correctly matches "claude-sonnet-4-*" but not "claude-3-5-sonnet" 2. Incomplete schema recursion: - Added support for oneOf, anyOf, allOf keywords - Added support for definitions/$defs (reusable schema components) - Now properly processes all nested object types 3. Logger parameter anti-pattern: - Removed logger parameter from createMessage and createStreamingMessage - Now uses module-level logger directly - Cleaner function signatures 4. TypeScript type safety: - Added documented type assertions with explanatory comments - Created explicit interface for beta.messages.create - Documented why cast is needed (SDK types lag behind beta features)
|
@aadamsx is attempting to deploy a commit to the Sim Team on Vercel. A member of the Team first needs to authorize it. |
Contributor
Greptile Summary
Important Files Changed
Confidence score: 2/5
Sequence DiagramsequenceDiagram
participant User
participant anthropicProvider
participant Anthropic as "Anthropic API"
participant Tool as "Tool System"
User->>anthropicProvider: "executeRequest(request)"
anthropicProvider->>anthropicProvider: "Transform messages to Anthropic format"
anthropicProvider->>anthropicProvider: "Check if model supports native structured outputs"
alt Native structured outputs supported
anthropicProvider->>anthropicProvider: "ensureAdditionalPropertiesFalse(schema)"
anthropicProvider->>anthropicProvider: "Set useNativeStructuredOutput = true"
else Fallback to prompt-based
anthropicProvider->>anthropicProvider: "Add JSON schema instructions to system prompt"
end
anthropicProvider->>anthropicProvider: "Prepare tools with usage control"
anthropicProvider->>anthropicProvider: "Build request payload"
alt Streaming with no tools
anthropicProvider->>Anthropic: "createStreamingMessage()"
Anthropic-->>anthropicProvider: "Stream response"
anthropicProvider-->>User: "StreamingExecution with readable stream"
else Non-streaming or tools present
anthropicProvider->>Anthropic: "createMessage()"
Anthropic-->>anthropicProvider: "Response with content/tool calls"
loop Tool execution (max iterations)
alt Tool calls present
anthropicProvider->>Tool: "executeTool(toolName, params)"
Tool-->>anthropicProvider: "Tool result"
anthropicProvider->>anthropicProvider: "Add tool result to messages"
anthropicProvider->>Anthropic: "createMessage() with updated messages"
Anthropic-->>anthropicProvider: "Next response"
end
end
alt Final streaming requested
anthropicProvider->>Anthropic: "createStreamingMessage() for final response"
Anthropic-->>anthropicProvider: "Final stream"
anthropicProvider-->>User: "StreamingExecution with all tool data"
else Standard response
anthropicProvider-->>User: "ProviderResponse with content and tool results"
end
end
|
Contributor
Greptile's behavior is changing!From now on, if a review finishes with no comments, we will not post an additional "statistics" comment to confirm that our review found nothing to comment on. However, you can confirm that we reviewed your changes in the status check section. This feature can be toggled off in your Code Review Settings by deselecting "Create a status check for each PR". |
Collaborator
|
not true, we use the new structured output from anthropic for the models that support it. |
Author
|
I've checked the current Current main branch (line ~157): // If response format is specified, add strict formatting instructions
// Build a system prompt for structured output based on the JSON schemaThis is the prompt-based approach - adding JSON instructions to the system prompt. Claude is "asked nicely" but not constrained. Native structured outputs (November 2025) uses: anthropic.beta.messages.create({
betas: ['structured-outputs-2025-11-13'],
output_format: { type: 'json_schema', schema: ... }
})A search for Docs: https://docs.anthropic.com/en/docs/build-with-claude/structured-outputs |
Author
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Summary
Implements Anthropic's native structured outputs (November 2025 feature) for supported Claude models using constrained decoding.
Problem
The current Anthropic provider uses prompt-based structured output, which is unreliable:
Solution
Use Anthropic's native structured outputs with constrained decoding for Claude 3.5+ and 4.x models:
structured-outputs-2025-11-13output_formatwithtype: "json_schema"Changes
STRUCTURED_OUTPUT_MODEL_PREFIXESlist with prefix matching to avoid false positivessupportsNativeStructuredOutput()with proper prefix-based detectionensureAdditionalPropertiesFalse()with full schema recursion including:oneOf,anyOf,allOfkeywordsdefinitionsand$defsfor reusable schema componentscreateMessage()andcreateStreamingMessage()helpers using beta APISupported Models
Native structured outputs work with:
claude-3-5-haiku-*claude-sonnet-4-*claude-opus-4-*Code Review Feedback Addressed
All issues from the previous review have been fixed:
oneOf,anyOf,allOf,definitions,$defsReferences