feat: render raw curl for aws-bedrock #2319

ethanlijin · 2025-08-14T20:55:55Z

Split out only the raw-curl implementation from previous branch

Important

Adds render_raw_curl to AwsClient for generating AWS Bedrock API curl commands, handling text and media content, and formatting with environment variables.

Behavior:
- Adds render_raw_curl method to AwsClient in aws_client.rs to generate raw curl commands for AWS Bedrock API requests.
- Handles text and media content, constructing JSON payloads for system and messages.
- Formats curl command with environment variables for AWS_REGION and AWS_PROFILE.
Functions:
- Introduces strip_mime_prefix and to_cli_content_block helper functions for media handling.
- Uses shell_escape::escape for safe command-line argument formatting.
Error Handling:
- Logs errors for unsupported media types and incomplete configurations.

^{This description was created by}^{for 4a10e4e. You can customize this summary. It will automatically update as commits are pushed.}

vercel · 2025-08-14T20:56:02Z

@ethanlijin is attempting to deploy a commit to the Boundary Team on Vercel.

A member of the Team first needs to authorize it.

sxlijin · 2025-08-16T18:53:59Z

engine/baml-runtime/src/internal/llm_client/primitive/aws/aws_client.rs

+        // region/profile hints
+        if let Some(region) = &self.properties.region {
+            cmd.push_str(&format!(" --region '{}'", escape(Cow::Borrowed(region))));
+        }
+        if let Some(profile) = &self.properties.profile {
+            cmd.push_str(&format!(" --profile '{}'", escape(Cow::Borrowed(profile))));
+        }


these should be formatted to prefix the command e.g. AWS_REGION=us-east2 AWS_PROFILE=boundaryml-dev aws bedrock-runtime converse

sxlijin · 2025-08-16T18:56:32Z

engine/baml-runtime/src/internal/llm_client/primitive/aws/aws_client.rs

+        // --model-id
+        cmd.push_str(&format!(
+            " --model-id '{}'",
+            escape(Cow::Borrowed(&self.properties.model))
+        ));
+
+        // --messages
+        let messages_str = serde_json::to_string(&messages_json)?;
+        let messages_escaped = escape(Cow::Borrowed(&messages_str));
+        cmd.push_str(&format!(" --messages {}", messages_escaped));
+
+        // --system (optional)
+        if let Some(blocks) = system_blocks {
+            let system_str = serde_json::to_string(&blocks)?;
+            let system_escaped = escape(Cow::Borrowed(&system_str));
+            cmd.push_str(&format!(" --system {}", system_escaped));
+        }
+
+        // --inference-config (optional)
+        if let Some(cfg) = &self.properties.inference_config {
+            let mut map = serde_json::Map::new();
+            if let Some(v) = cfg.max_tokens {
+                map.insert("maxTokens".into(), json!(v));
+            }
+            if let Some(v) = cfg.temperature {
+                map.insert("temperature".into(), json!(v));
+            }
+            if let Some(v) = cfg.top_p {
+                map.insert("topP".into(), json!(v));
+            }
+            if let Some(v) = cfg.stop_sequences.as_ref() {
+                map.insert("stopSequences".into(), json!(v));
+            }
+            if !map.is_empty() {
+                let cfg_str = serde_json::to_string(&map)?;
+                let cfg_escaped = escape(Cow::Borrowed(&cfg_str));
+                cmd.push_str(&format!(" --inference-config {}", cfg_escaped));
+            }
+        }
+
+        // --additional-model-request-fields (optional)
+        if !self.properties.additional_model_request_fields.is_empty() {
+            let addl = serde_json::to_value(&self.properties.additional_model_request_fields)?;
+            let addl_str = serde_json::to_string(&addl)?;
+            let addl_escaped = escape(Cow::Borrowed(&addl_str));
+            cmd.push_str(&format!(
+                " --additional-model-request-fields {}",
+                addl_escaped
+            ));
+        }


instead of building these fields one at a time, use --cli-input-json and format the request the way the other raw curl requests are formatted, with a json-escaped string that's spread across multiple lines

(you might still need to specify --model-id explicitly)

sxlijin · 2025-08-16T18:57:27Z

engine/baml-runtime/src/internal/llm_client/primitive/aws/aws_client.rs

+        fn image_format_from_mime(mime: &str) -> String {
+            match mime.strip_prefix("image/") {
+                Some(s) => s.to_string(),
+                None => mime.to_string(),
+            }
+        }
+
+        fn video_format_from_mime(mime: &str) -> Result<&'static str> {
+            Ok(match mime {
+                "video/mp4" => "mp4",
+                "video/mpeg" => "mpeg",
+                "video/mov" => "mov",
+                "video/x-flv" => "flv",
+                "video/mkv" => "mkv",
+                "video/webm" => "webm",
+                _ => anyhow::bail!(
+                    "AWS Bedrock video format not supported: {}. Supported formats: mp4, mpeg, mov, flv, mkv, webm",
+                    mime
+                ),
+            })
+        }


don't do this, just always strip the mime-type prefix (aka before the /) and then let the aws api return errors

…ws-rawcurl-clean

ellipsis-dev

Caution

Changes requested ❌

Reviewed everything up to 4a10e4e in 1 minute and 49 seconds. Click for details.

Reviewed 229 lines of code in 1 files
Skipped 0 files when reviewing.
Skipped posting 3 draft comments. View those below.
Modify your settings and rules to customize what types of comments Ellipsis leaves. And don't forget to react with 👍 or 👎 to teach Ellipsis.

1. engine/baml-runtime/src/internal/llm_client/primitive/aws/aws_client.rs:586

Draft comment:
The helper function 'strip_mime_prefix' is quite naive. Consider a more robust MIME parsing approach if inputs might be nonstandard.
Reason this comment was not posted:
Confidence changes required: 33% <= threshold 50% None

2. engine/baml-runtime/src/internal/llm_client/primitive/aws/aws_client.rs:723

Draft comment:
Pretty-printing JSON for the CLI argument may introduce newlines that can affect shell command behavior. Consider using a compact JSON string if unexpected formatting issues arise.
Reason this comment was not posted:
Decided after close inspection that this draft comment was likely wrong and/or not actionable: usefulness confidence = 10% vs. threshold = 50% The code is already using shell_escape::escape() which properly escapes special characters including newlines for shell usage. The shell_escape crate is specifically designed for this use case. Pretty-printing the JSON won't cause issues since any newlines will be properly escaped. The comment seems overly cautious and unnecessary given the existing safety measures. Could there be edge cases where shell_escape doesn't handle newlines correctly? Should we err on the side of caution with shell commands? The shell_escape crate is well-tested and specifically designed for this purpose. Pretty-printed JSON with proper escaping is actually more readable and debuggable for users seeing the curl command. The comment should be deleted. The code is already safely handling shell escaping via the shell_escape crate, and pretty-printed JSON provides better readability.

3. engine/baml-runtime/src/internal/llm_client/primitive/aws/aws_client.rs:647

Draft comment:
There is duplicated logic converting ChatMessagePart variants to CLI JSON blocks. Consider refactoring this into a shared helper to reduce code duplication.
Reason this comment was not posted:
Marked as duplicate.

Workflow ID: wflow_F7zU5zjPnq6R1vkf

^{You can customize}^{by changing your verbosity settings, reacting with 👍 or 👎, replying to comments, or adding code review rules.}

ellipsis-dev · 2025-08-16T23:38:21Z

engine/baml-runtime/src/internal/llm_client/primitive/aws/aws_client.rs


-        // TODO(sam): this is fucked up. The SDK actually hides all the serializers inside the crate and doesn't let the user access them.
+        if let Some((first, remainder_slice)) = chat_slice.split_first() {


Consider extracting the system block conversion logic into a helper. The same pattern for processing ChatMessagePart (for system messages) appears later, and merging them could improve maintainability.

…ws-rawcurl-clean

…to --cli-input-json; always strip MIME-type prefix for media in raw curl

…ON and prefixes env (AWS_REGION/AWS_PROFILE); stop validating mime in raw CLI (strip subtype only)

…ws-rawcurl-clean

sxlijin

LGTM, with two last things that need fixing

sxlijin · 2025-08-21T05:52:25Z

engine/baml-runtime/src/internal/llm_client/primitive/aws/aws_client.rs

+                        // Always strip MIME prefix and let AWS validate/err.
+                        let format = bedrock::types::VideoFormat::from(strip_mime_prefix(
+                            &media.mime_type_as_ok()?,
+                        ));


aaggghhh. i was wrong, we should put back the manual mime type matching - this will not delegate to the bedrock api server but to the bedrock sdk and will actually result in worse error messages.

sxlijin · 2025-08-21T05:52:55Z

...common/src/shared/baml-project-panel/playground-panel/prompt-preview/prompt-preview-curl.tsx

let's undo the changes to this file - i don't really understand why we're changing this

…ws-rawcurl-clean

codecov · 2025-08-27T17:24:45Z

Codecov Report

❌ Patch coverage is 0% with 128 lines in your changes missing coverage. Please review.

Files with missing lines	Patch %	Lines
...rc/internal/llm_client/primitive/aws/aws_client.rs	0.00%	128 Missing ⚠️

📢 Thoughts on this report? Let us know!

@sxlijin

Split out only the raw-curl implementation from previous branch @sxlijin  ---- > [!IMPORTANT] > Adds `render_raw_curl` to `AwsClient` for generating AWS Bedrock API curl commands, handling text and media content, and formatting with environment variables. > > - **Behavior**: > - Adds `render_raw_curl` method to `AwsClient` in `aws_client.rs` to generate raw curl commands for AWS Bedrock API requests. > - Handles text and media content, constructing JSON payloads for `system` and `messages`. > - Formats curl command with environment variables for `AWS_REGION` and `AWS_PROFILE`. > - **Functions**: > - Introduces `strip_mime_prefix` and `to_cli_content_block` helper functions for media handling. > - Uses `shell_escape::escape` for safe command-line argument formatting. > - **Error Handling**: > - Logs errors for unsupported media types and incomplete configurations. > > <sup>This description was created by </sup>[<img alt="Ellipsis" src="https://img.shields.io/badge/Ellipsis-blue?color=175173">](https://www.ellipsis.dev?ref=BoundaryML%2Fbaml&utm_source=github&utm_medium=referral)<sup> for 4a10e4e. You can [customize](https://app.ellipsis.dev/BoundaryML/settings/summaries) this summary. It will automatically update as commits are pushed.</sup>  --------- Co-authored-by: Sam Lijin <sam@boundaryml.com>

wrote render_raw_curl but still gotta do some more testing

7a5280f

sxlijin reviewed Aug 16, 2025

View reviewed changes

ethanlijin added 2 commits August 16, 2025 19:36

fixed some issues with the render_raw_curl method

38f418b

Merge branch 'canary' of github.com:BoundaryML/baml into ethanlijin/a…

4a10e4e

…ws-rawcurl-clean

ethanlijin marked this pull request as ready for review August 16, 2025 23:36

ellipsis-dev bot reviewed Aug 16, 2025

View reviewed changes

ethanlijin added 6 commits August 17, 2025 22:23

playground: add Reload button to force cURL preview recompute

d871248

Merge branch 'canary' of github.com:BoundaryML/baml into ethanlijin/a…

a305114

…ws-rawcurl-clean

aws(raw-curl): prefix command with AWS_REGION/AWS_PROFILE and switch …

272844f

…to --cli-input-json; always strip MIME-type prefix for media in raw curl

runtime(aws): render_raw_curl uses --cli-input-json with multiline JS…

1215345

…ON and prefixes env (AWS_REGION/AWS_PROFILE); stop validating mime in raw CLI (strip subtype only)

Merge branch 'canary' of github.com:BoundaryML/baml into ethanlijin/a…

1068d7a

…ws-rawcurl-clean

just fixed indenting in the code

393280b

sxlijin reviewed Aug 21, 2025

View reviewed changes

ethanlijin added 3 commits August 21, 2025 10:56

undid changes to prompt-preview-curl.tsx

18a065c

Reverted back to manual mime checking

5b3ca71

Merge branch 'canary' of github.com:BoundaryML/baml into ethanlijin/a…

bb8706f

…ws-rawcurl-clean

sxlijin changed the title ~~AWS: raw curl support (clean split)~~ feat: implement raw curl for aws-berock Aug 27, 2025

sxlijin changed the title ~~feat: implement raw curl for aws-berock~~ feat: implement raw curl for aws-bedrock Aug 27, 2025

sxlijin added 6 commits August 26, 2025 20:47

delete unused code

e9d0008

fix more

29af423

more

c37e1a5

more

4127e51

Merge branch 'canary' into ethanlijin/aws-rawcurl-clean

a1ac80a

undo cffi

a73e7b9

sxlijin changed the title ~~feat: implement raw curl for aws-bedrock~~ feat: render raw curl for aws-bedrock Aug 27, 2025

sxlijin enabled auto-merge August 27, 2025 04:38

sxlijin had a problem deploying to boundary-tools-dev August 27, 2025 17:17 — with GitHub Actions Failure

sxlijin added this pull request to the merge queue Aug 27, 2025

Merged via the queue into BoundaryML:canary with commit 2bbb267 Aug 27, 2025
14 of 19 checks passed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

feat: render raw curl for aws-bedrock #2319

feat: render raw curl for aws-bedrock #2319

Uh oh!

ethanlijin commented Aug 14, 2025 •

edited by ellipsis-dev bot

Loading

Uh oh!

vercel bot commented Aug 14, 2025

Uh oh!

sxlijin Aug 16, 2025

Uh oh!

sxlijin Aug 16, 2025

Uh oh!

sxlijin Aug 16, 2025

Uh oh!

ellipsis-dev bot left a comment

Uh oh!

ellipsis-dev bot Aug 16, 2025

Uh oh!

sxlijin left a comment

Uh oh!

sxlijin Aug 21, 2025

Uh oh!

sxlijin Aug 21, 2025 •

edited

Loading

Uh oh!

codecov bot commented Aug 27, 2025

Uh oh!

Uh oh!

Uh oh!


		// TODO(sam): this is fucked up. The SDK actually hides all the serializers inside the crate and doesn't let the user access them.
		if let Some((first, remainder_slice)) = chat_slice.split_first() {

feat: render raw curl for aws-bedrock #2319

feat: render raw curl for aws-bedrock #2319

Uh oh!

Conversation

ethanlijin commented Aug 14, 2025 • edited by ellipsis-dev bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

vercel bot commented Aug 14, 2025

Uh oh!

sxlijin Aug 16, 2025

Choose a reason for hiding this comment

Uh oh!

sxlijin Aug 16, 2025

Choose a reason for hiding this comment

Uh oh!

sxlijin Aug 16, 2025

Choose a reason for hiding this comment

Uh oh!

ellipsis-dev bot left a comment

Choose a reason for hiding this comment

Uh oh!

ellipsis-dev bot Aug 16, 2025

Choose a reason for hiding this comment

Uh oh!

sxlijin left a comment

Choose a reason for hiding this comment

Uh oh!

sxlijin Aug 21, 2025

Choose a reason for hiding this comment

Uh oh!

sxlijin Aug 21, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

codecov bot commented Aug 27, 2025

Codecov Report

Uh oh!

Uh oh!

Uh oh!

ethanlijin commented Aug 14, 2025 •

edited by ellipsis-dev bot

Loading

sxlijin Aug 21, 2025 •

edited

Loading