Skip to content

Add GibsonAI toolkit #493

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 3 commits into
base: main
Choose a base branch
from
Open

Add GibsonAI toolkit #493

wants to merge 3 commits into from

Conversation

Boburmirzo
Copy link

Summary

Add support for GibsonAI database integration to the Arcade toolkit, enabling agents to evolve database schemas and run SQL queries against multiple relational databases.

What is GibsonAI?

GibsonAI is your AI-powered “database engineer” that lets you design, build, deploy, and scale production-ready serverless databases using natural-language prompts. You can create a database schema for your new project idea using the GibsonAI App by chatting with AI. Under the hood, GibsonAI spun up a new real MySQL/Postgres database. You can create multiple database environments and update the existing schema in AI chat.

For AI Agents, GibsonAI can be used as Test labs for AI Agent where you can run SQL queries in a sandbox environment and Durable Memory Layer – “Short, mid & long term memory for Agents”.

Description

Adding a new tool package arcade_gibsonai that provides database interaction capabilities with GibsonAI's Data API. The integration follows the similar patterns as existing database tools (like arcade_postgres) but communicates via HTTP API instead of direct database connections.

Motivation

  • API-First Architecture: GibsonAI provides database access through a REST API, making it ideal for cloud-native applications
  • Full SQL Support: Enable agents to perform all SQL operations including SELECT, INSERT, UPDATE, DELETE, CREATE, ALTER
  • Secure Access: Use API key authentication for secure database access

Demo

I recorded the demo of using the new tool: https://drive.google.com/file/d/1tZYx7NnnBLu5z0ku1o7Kr7UYAq4Eo1M0/view?usp=sharing

@torresmateo torresmateo self-requested a review July 16, 2025 20:41
@Boburmirzo
Copy link
Author

@torresmateo Now I fixed them accordingly. Let me know what you think.

@Boburmirzo Boburmirzo requested a review from torresmateo July 20, 2025 22:04
raise ValueError(f"Validation error: {error_msg}") from e


@tool(requires_secrets=["GIBSONAI_API_KEY"])
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does GibsonAI gives full read and write access with this API key?
I would love to see this scoped to a scope like sql:delete. Otherwise, any user with access to this tool may delete the database entirely, and the secret is shared for the entire project.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@torresmateo I can understand your concerns. Thanks for sharing them!

If you are thinking it is too risky to allow users to make edit, create or delete operations. Lets keep only Query tool and we are okay to showcase this capability. We can wait until you support remote MCP server connections.

We just onboarded our docs and there might be missing docs. But Nathan can reply soon for this question.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not immediately against tools that insert or delete data. I worry about tools that do so very openly and without scopes. This could be ok if the database has proper RBAC setup, but I can't see how that might work with the current setup, as the user executing the query will be identified solely by GIBSONAI_API_KEY, right?

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Keys are scoped with distinct permissions (e.g. read/write/delete)
These API keys are managed/named on the GibsonAI side (either via the CLI or web app)
We identify the project, specific database, and permissions all from the API key

The /query endpoint that this tool uses parses the query to determine what types of statements are allowed:

  • Schema statements, e.g. CREATE / ALTER, are always forbidden
  • SELECT statements require read on the API key
  • UPDATE/INSERT statements require write on the API key
  • DELETE statements require delete on the API key

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @nathanchapman, this is indeed a better pattern than what I originally thought.

I have a couple of concerns about DML verbs, however, a couple of questions here:

  1. Arcade is a platform that is intended to offer tools in multi-tenant apps, and the API key will be set at the project level. As such, it seems that the only option for agent builders that need to store anything is to use a single overly-permissive API key. I'd love to hear your ideas on how an agent builder may implement a pattern where one user authenticates to GibsonAI and allows write operations, while allowing a different user to connect to the same database with read only access.
  2. @Boburmirzo, I think the validation attempt here is a good starting point, but I would prefer a specific allow-list type of validation. If you want to make this very versatile, perhaps this may come from the configured schema on the GibsonAI platform (e.g., internally check against the schema endpoint to see if what comes in the table name matches allowed tables, etc). For example, while I agree that escaping those strings in your DeleteRequest is useful, but it's one of the strongly discouraged defense options by OWASP, in favor of prepared-statements, stored procedures, and allow-list input validation. Of the top of my head, your approach is open to having a number of sub-queries pass the table name validation, for example.



@tool(requires_secrets=["GIBSONAI_API_KEY"])
async def insert_records(
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Here again I worry about allowing inserts too broadly. A lot of the AI-based SQL attacks are injecting instructions into tables through endpoints that allow write operations openly. @evantahler I would love your input on this.

I think:

  • We should have purpose-built insert tools that validate each argument using prepared queries under the hood
  • With the right permissions, we could ask the LLM to prepare a query and then someone with the right access may decide if it's worth running it or not.

@Boburmirzo could you point me to the GibsonAI permissions docs? I can't find it easily on the docs.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Here's generally how we are thinking about SQL tools - https://blog.arcade.dev/sql-tools-ai-agents-security. If tools are writing, it's best to scope down the permissions and then make very specific tools that only modify known params, rather than crafting the whole SQL. You could have a few secrets - GIBSONAI_API_KEY_WRITE and GIBSONAI_API_KEY_READ for different tools.

Yes, we care a lot about roles and access :D

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@torresmateo @evantahler I definitely understand where y'all are coming from.

on Arcade, is it possible for this particular tool to be a sub-call/agent-delegation from a user-implemented tool? e.g. the only path to gibson (as the storage layer) would be via an explicitly supported tool in a wrapping agent?
e.g. combination of this pattern: https://ai.pydantic.dev/multi-agent-applications/#agent-delegation-and-dependencies and what y'all mention in the Examples of Operational Tools section of the post you linked above.

This would be nice in that it allows the LLM to still craft the exact query, but can scope it's capabilities to supported tool definitions

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Totally agree with what Nathan is highlighting here — I support the direction he's pointing to like proposing agent-side control to protect downstream tools.

From an Arcade product positioning perspective, one of the key opportunities I see is simplifying authentication and authorization between agents and providers like MCP. Right now, it feels like all the responsibility for securing these interactions is being pushed down to the provider side. But I believe this is actually a great moment for Arcade to shine — by taking on more of the security model (like how API Management platforms protect APIs), Arcade could become the trusted middle layer that standardizes and enforces access control between agents and tools.

This could also open doors for stronger partnerships and broader adoption, especially in enterprise settings where fine-grained control and auditability are non-negotiable.

Would love to hear your thoughts on that. @torresmateo @evantahler

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@nathanchapman Arcade can't control how agent builders will connect the tools to their agent. So while it is possible to implement this pattern, it's not a guarantee that it will be a sub-agent. If I understand your suggestion correctly, this will put the enforcement of defenses such as prepared statements on the agent developer, rather than closer to the data.

I'm not against having a tool that stores memories, records, etc into a database, as long as they're tightly scoped and the roles for write operations are clearly delineated and only users with such access can do those operations.

@Boburmirzo While I can see your point, I do not agree that it's a good idea to move the protection boundary away from the data layer. For traditional APIs this may be more sensible, as we have to assume the developer is a trusted actor that will not intentionally allow SQL injections to happen. LLMs are not trusted actors in the system however, and should be outside of the trust boundary for the data layer.



@tool(requires_secrets=["GIBSONAI_API_KEY"])
async def update_records(
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is another problematic tool for me with respect to data security. Without properly scoped permissions it could be too easy for somebody to replace all the data with something else

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this is an artifact of @Boburmirzo trying to address your first comment on the PR.
I'd vote for a single tool (e.g. execute_query) to keep the implementation as simple as possible and because we handle all permissions checks on our end on the API endpoint.
If we're able to protect the singular execute_query behind explicit tool definitions in other agents, that'd make the most sense to me

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants