[Feature] Zero-Copy JSON-RPC Processing for High-Performance MCP Servers

[bonpiedlaroute]

[Feature] Zero-Copy JSON-RPC Processing for High-Performance MCP Servers

Problem Statement

Current JSON-RPC processing in the MCP Rust SDK:

• Creates unnecessary string allocations during parsing
• Uses standard serde_json which isn't optimized for high-throughput scenarios
• Lacks optimization options for memory-constrained or high-concurrency environments

Proposal

Zero-copy JSON-RPC parser with:

• SIMD-optimized parsing using simd-json
• Borrowed string references instead of owned strings where possible
• Hybrid architecture with automatic fallback for compatibility
• Feature-gated implementation (zero-copy feature) - no breaking changes

Proof of Concept

I've implemented a working POC to validate this approach:

• Branch: feature/zero-copy-json-rpc
• Status: Functional implementation with comprehensive benchmarks
• Coverage: Core parsing, hybrid processing, performance measurement

What's Implemented

✅ Zero-copy parser with simd-json
✅ Hybrid processor with fallback logic
✅ Performance benchmarks showing +35% gains
✅ Feature-gated architecture

Benchmark Results

Significant Gains on Larger Payloads

Parsing + field extraction performance:
- Large JSON (1.5KB):   1,371ns vs 1,774ns → +23% faster
- XLarge JSON (3KB+):   2,145ns vs 3,284ns → +35% faster

Pure parsing performance:
- Large JSON:   1,285ns vs 1,802ns → +29% faster  
- XLarge JSON:  2,220ns vs 3,236ns → +31% faster

Real-world MCP Scenarios

• Small payloads (~500B): Performance neutral (expected with SIMD libraries)
• Large tool responses: Significant improvements (23-35%)
• Batch processing: Major memory efficiency gains

What's Missing (pending discussion)

⏳ Comprehensive unit test suite
⏳ Edge case handling
⏳ Documentation

Code available here for review

❓ Questions for Discussion

Please, before investing in a complete implementation, I'd appreciate feedbacks

1. Does this approach align with MCP's performance goals?
2. Any concerns about the simd-json dependency, the architectural approach?
3. Suggestions for the API design?

Thank you

[4t145]

Feature-gated implementation (zero-copy feature) - no breaking changes

How can we remain current interface with zero copy deserialization? Since we are using String for all data type now, will we have two set of data type?

Borrowed string references instead of owned strings where possible

Will this bring the deserialization lifetime all the way down to tool call? I haven't verified this, but it seems like it could be more complicated.

And this could be impossible under currently using tokio_utils::codec::Decoder,

fn decode(&mut self, buf: &mut BytesMut) -> Result<Option<Self::Item>, JsonRpcMessageCodecError>

We cannot let Self::Item bring the lifetime of parameter buf. So this will be a very huge refactor.

[bonpiedlaroute]

How can we remain current interface with zero copy deserialization? Since we are using String for all data type now, will we have two set of data type?

Current interfaces remain 100% unchanged. In fact zero-copy happens internally with immediate conversion at module boundaries. No new types exposed in public APIs. The POC maintains compatibility through conversion at boundaries

// Public API remains completely unchanged - no new types exposed
pub fn process_request(&mut self, input: &[u8]) -> Result<ProcessingResult<String>, Error> {
    let mut input_copy = input.to_vec();
    
    // Zero-copy happens internally
    match self.processor.process_bytes(&mut input_copy) {
        Ok(borrowed_result) => Ok(ProcessingResult {
            result: borrowed_result.result.method().to_string(), // Convert at boundary
            used_zero_copy: borrowed_result.used_zero_copy,
            processing_time_ns: borrowed_result.processing_time_ns,
        }),
        Err(_) => // fallback
    }
}

Will this bring the deserialization lifetime all the way down to tool call?

No, lifetimes are contained within the parsing module, conversion to owned types happens before data leaves the zero-copy module. Tool calls continue to receive the same String - based types. The current poc implementation deliberately contains lifetimes within the parsing phase.

Lifetime scope: input buffer -> parsing -> immediate conversion -> owned types

And this could be impossible under currently using tokio_utils::codec::Decoder,

Correct - and the poc intentionally avoids this contraint. The idea is to provide an alternative path for non-streaming scenarios. Existing streaming/codec infrastructure remains untouched

Based on your feedback i see this fitting into MCP Rust sdk as complementary performance path

// Existing streaming path (unchanged)
#[cfg(feature = "streaming")]
pub fn serve_streaming<T: Decoder>(...) -> impl Stream { ... }

// New high-performance path (opt-in)
#[cfg(feature = "zero-copy")]  
pub fn serve_batch<T>(...) -> Result<BatchProcessor<T>, Error> { ... }

or internal optimization only within existing streaming

❓ Questions

1- does MCP have scenarios where batch/high-throughput processing vs streaming would be valuable ?

2- Would you prefer this as completely separate batch processing path ? or internal optimization within existing streaming ? or it is not suitable for current MCP architecture at all ?

3- are the 35% improvements on large payload worth the added complexity ?

Thanks

[4t145]

No, lifetimes are contained within the parsing module, conversion to owned types happens before data leaves the zero-copy module. Tool calls continue to receive the same String - based types. The current poc implementation deliberately contains lifetimes within the parsing phase.

According to my understanding, the copy still happened if you store the deserialized value into owned type like String. So I doubt that will simd-json still have preformance advantage in this case? (In your test case, the deserilized type don't own the data.)

does MCP have scenarios where batch/high-throughput processing vs streaming would be valuable ?

If it's easy to implement, of course we should do it. But if not, it's okay to remain non-zero-copy interface. Just look at axum, I think we should keep similar performance level as axum.

[bonpiedlaroute]

You're right about the String conversion negating zero-copy benefits.
The performance gains in my benchmarks come primarily from simd-json's faster
parsing rather than true zero-copy. Given the codec constraints and the need
for String conversion, the complexity may not justify the modest gains.

Would you be interested in simpler simd-json integration that doesn't claim
zero-copy but still provides the SIMD parsing benefits?

Thanks

[4t145]

If we can just replace serde_json::from_slice to simd_json::serde::from_slice to gain perfermaince improvements, I think we should support it.

[bonpiedlaroute]

You're absolutely right. This is much cleaner than my zero-copy approach.

The implementation would be straightforward:

#[cfg(feature = "simd-json")]
use simd_json::serde as json_parser;

#[cfg(not(feature = "simd-json"))]
use serde_json as json_parser;

// Then replace existing calls:
let value: serde_json::Value = json_parser::from_slice(&mut data)?;

This gives us:

• SIMD parsing performance benefits
• Same serde_json::Value types everywhere
• No breaking changes to existing APIs
• Full compatibility with the current codec infrastructure

Should I implement this approach? It would involve:

• Adding simd-json as optional dependency with serde_impl feature
• Feature-gated imports throughout the JSON processing code
• Benchmarks comparing the two parsing approaches

This addresses your valid critique about my zero-copy implementation - we get performance improvements without the architectural complexity or lifetime management issues.

Thank you

[4t145]

That would be good!