provider lead/worker cleanup and config #3980
michaelneale
started this conversation in
Ideas
Replies: 0 comments
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
-
I started looking at how we can consolidate things around lead/worker and provider config (vs env vars), lead worker is currently special cased (and wont' support streaming for example), and uses environment variables, a whole variety of confusing things.
What we could do is have a more general config something like:
where there are multiple providers configured for different purposes. Lead/worker is just one case (it could have just one by default, similar to before).
oracle is for one of consultation, second-opinion for when assistance is needed and so on (can add extra metadata to help the routing/gating decisions, but don't want to get bogged in that detail).
This would also go along with simplifying the lead/worker code to be less of a special case, and more about tracking what provider is in use and switching, and loading config from one place ideally (and not only rely on environment variables).
While making that change, this can also use a simpler config format (of course for compat/command line etc - can still have env vars set when that isn't used)
Beta Was this translation helpful? Give feedback.
All reactions