Documented DRAFT proposal for a POTENTIAL Validation Working Group

[hesreallyhim]

As stated in this document:

DISCLAIMER: This is not (yet) an actual Working Group, and has not gone through all of the preparatory steps. It was recommended to me by a core contributor that I open this up for discussion by opening a draft. The intention of this document is to clarify what a Validation Working Group would be like, what purpose it would serve, what goals it would have, and so on. All of this is subject to change if and when a group is formed. Obviously I have used the Hosting Group README as a template.

The "addendum" is an attempt to classify and enumerate some of the main linguistic issues I have identified in the documentation. It is opinionated and even slightly polemical, but I tried to be direct and concise, instead of long-winded and more nuanced. The connection between this document and the validation WG proposal is basically: you can't validate a Spec that is not clearly defined. So maybe there is even the need for two WG's, if there are enough hands - one focusing on the language, one focusing on validation tools. I see these more as dovetailing because, e.g. building a validation tool might help clarify which linguistic issues are really problematic, and which aren't, and vice versa.

[olaservo]

Hey thanks for opening this PR! I don't think the group gets notifications on these since I just stumbled across it by accident. Will leave comments and get it more 👀

[hesreallyhim]

@olaservo thanks I was probably going to share it amongst the wg, but i just posted the "addendum" part yesterday

[atrawog]

I think what's really critical for the whole MCP ecosystem to grow are generally accepted reference implementations and automated testing tools.

Because even if you want to follow the MCP specs down to the letter. There is so much ambiguity in the MCP specs that we are going to end up with a myriad of slightly incompatible implementations sooner or later with the current approach.

[hesreallyhim]

@atrawog if by "automated testing tools" you mean, or include, tools that test compliance with the Spec (validation tools), then I totally agree, which is why I am pitching this is a validation working-group and not a semantics working-group. On the other hand, you can't write tests for a Spec that is ill-defined. So a nice path forward might be to incrementally build out validation-tools for at least the parts of the Spec that are unambiguous, and in doing so it would help the group to pin-point the critical ambiguities and resolve them as needed.

[atrawog]

I think at the end of the day you need both. You need a validation tool that is testing against the core MCP protocol specifications and a reference implementation that's implementing the complete MCP ecosystem.

Because the issues I'm currently bumping into aren't directly related to MCP. They are all related to the OAuth2 RFCs that are a MUST requirement in the latest MCP protocol specifications.

Using and referring to the OAuth2 RFCs makes tons of sense. But once you start referring to half a dozen of RFCs with each having its own list of things you might or might not implement. You really need a reference implementation to establish a list of baseline features that should be implemented by everyone.

[hesreallyhim]

@atrawog yeah that makes sense... I'll just say to me, these are all different tactics towards the same goal I think. If you're saying the spec is unclear/ambiguous about the requirements on integrating OAuth2, then a reference implementation is doing more than just being a good reference - it would be actually adding to the Spec, whereas I think of a reference implementation as a pristine implementation of what the Spec already states. Anyway, not to get side-tracked I think it's also important and a mature standard generally involves reference implementations as well.

[hevangel]

Thanks for creating this pull request. I would like to contributing to the validation work group. My background is hardware verification with domain knowledge in various communication protocol PCIe/Ethernet/OTN where it has a similar interoperability problems like in MCP.

[hesreallyhim]

@olaservo hey what do you think we should do with this PR? I think it's mostly achieved its purpose since we have the discord channel now, but it also contains some (hopefully) useful context.