Fix token usage tracking for streaming providers

[jajimajp]

Problem

When using streaming providers with the goose CLI, the context token count remains at 0 and doesn't update properly:

Context: ○○○○○○○○○○ 0% (0/200000 tokens)
( O)> Hello

(...snip...)

Context: ○○○○○○○○○○ 0% (0/200000 tokens)
( O)>

Root Cause

The current implementation overwrites usage metrics with each streaming event instead of properly accumulating them. When streaming APIs send a final event with None values, it resets the usage count to 0.

Solution

Modified the update_session_metrics method to:

• Properly track cumulative token usage across streaming events
• Handle None values gracefully without resetting existing counts

[michaelneale]

thanks @jajimajp - what is odd, I use anthropic/databricks and didn't see it show zero, can you confirm what provider you were using and model etc?
(good catch if there are some cases, but I wonder how it worked at all)

[jajimajp]

thanks @jajimajp - what is odd, I use anthropic/databricks and didn't see it show zero, can you confirm what provider you were using and model etc? (good catch if there are some cases, but I wonder how it worked at all)

I am using an OPENAI-compatible proxy with the model sonnet-4. Therefore, the provider is OPENAI. This change worked well in my environment, but let me verify again whether the same issue occurs with the regular OPENAI API.

[michaelneale]

also if you can run cargo fmt and clippy etc.

[michaelneale]

I hand tested this with databricks, openai direct, and anthropic, seems ok, but @katzdave sorry to hassle again - but you ok with this? I am just not sure how it works. I am wary that it works without with openai direct, but with a proxy, was different (sounds like 3rd party proxy may be returning incorrect values?)

[katzdave]

Can we just add a test that verifies it does the the same thing for the non-streaming case. Generally makes sense to me though on the streaming side.

I can't remember my exact setup but I was seeing some kind of 0 issue a couple weeks ago; I think with openai on databricks.

[michaelneale]

@jajimajp able to add any tests?

[jajimajp]

@michaelneale @katzdave

Thanks for the feedback. I've added a test case to verify the non-streaming behavior as requested.

Additionally, I manually verified that non-streaming works correctly with my OpenAI provider by temporarily making the following change:

--- a/crates/goose/src/providers/openai.rs
+++ b/crates/goose/src/providers/openai.rs
@@ -206,7 +206,7 @@ impl Provider for OpenAiProvider {
  }
 
  fn supports_streaming(&self) -> bool {
-     true
+     false
  }
 
  async fn stream(

It seems to be working correctly. Let me know if there's anything else.

[jajimajp]

force pushed to sign previous commits

[zanesq]

Still in progress?

[jajimajp]

Still in progress?

No, I am waiting for comments on this: #3909 (comment)