Skip to content

draft blog post #5

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Draft
wants to merge 1 commit into
base: main
Choose a base branch
from
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 3 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -129,3 +129,6 @@ dist
.yarn/build-state.yml
.yarn/install-state.gz
.pnp.*

# Claude Code
.claude/
313 changes: 313 additions & 0 deletions blog-post.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,313 @@
---
title: "Building Scalable AI Integration with the MCP Everything Server"
date: 2025-08-12
author: "MCP Team"
tags: ["MCP", "Model Context Protocol", "AI", "Architecture", "Tutorial"]
---

# Building Scalable AI Integration with the MCP Everything Server

The Model Context Protocol (MCP) is revolutionizing how AI applications interact with external systems. Today, we're diving deep into the **MCP Everything Server** – a production-ready reference implementation that showcases every feature of the MCP specification while demonstrating enterprise-grade scalability patterns.

Whether you're building your first MCP integration or architecting a distributed AI system, this comprehensive implementation serves as both a learning tool and a production blueprint.

## What is the MCP Everything Server?

The MCP Everything Server is an open-source, horizontally-scalable implementation that demonstrates the complete Model Context Protocol specification. Unlike minimal examples that show isolated features, this server integrates everything – tools, resources, prompts, sampling, authentication, and multiple transport methods – into a cohesive, production-ready system.

### Why "Everything"?

The name isn't hyperbole. This server implements:
- **All 7 core MCP tool patterns** including long-running operations and LLM sampling
- **100 example resources** with real-time subscriptions
- **Complete OAuth 2.0 authentication** with PKCE support
- **Both modern and legacy transports** (Streamable HTTP and SSE)
- **Horizontal scaling** via Redis-backed session management
- **Enterprise security features** including CSP headers and session isolation

## Real-World Use Cases

### 1. Learning and Understanding MCP

**Who it's for:** Developers new to MCP or those wanting to understand advanced patterns

The Everything Server is the ultimate learning resource. Unlike documentation that explains concepts in isolation, here you can see how all MCP features work together in a real system.

For example, you can trace how a tool invocation flows through the authentication layer, gets routed via Redis pub/sub, executes in the MCP server, and returns results through either Streamable HTTP or SSE transport. Every pattern recommended in the MCP specification is implemented here.

### 2. Reference Architecture for Production Systems

**Who it's for:** Teams building production MCP servers

Starting a production MCP implementation from scratch? The Everything Server provides battle-tested patterns for:
- **Session management** that survives server restarts
- **Horizontal scaling** across multiple instances
- **Security implementation** with proper OAuth flows
- **Error handling** and structured logging
- **Message routing** through distributed systems

Teams can fork this repository and replace the example tools and resources with their domain-specific implementations while keeping the robust infrastructure intact.

### 3. Testing and Development Environment

**Who it's for:** Client application developers

Developing an MCP client? You need a full-featured server for testing. The Everything Server provides:
- **Predictable test data** with 100 numbered resources
- **Various tool behaviors** from simple echo to long-running operations
- **Error scenarios** for testing client resilience
- **Multiple transport options** to test compatibility
- **Authentication flows** for security testing

The fake OAuth provider means you can test authentication flows without external dependencies.

### 4. Integration Testing Platform

**Who it's for:** QA teams and CI/CD pipelines

The server's comprehensive test suite (unit, integration, and multi-user tests) demonstrates how to properly test MCP implementations. The included test utilities can be adapted for your own testing needs, and the server itself can be containerized for use in CI/CD pipelines.

### 5. Distributed System Blueprint

**Who it's for:** Architects designing scalable AI systems

The Redis-backed architecture demonstrates how to build stateless MCP servers that can scale horizontally. This pattern is essential for:
- **High-availability deployments** where any server can handle any request
- **Load-balanced environments** with multiple server instances
- **Microservices architectures** where MCP is one component
- **Cloud-native deployments** with auto-scaling

## Step-by-Step Getting Started Guide

Let's get the MCP Everything Server running on your machine. We'll go from zero to a fully functional MCP server with authentication in about 10 minutes.

### Prerequisites

Before starting, ensure you have:
- **Node.js 16 or higher** (check with `node --version`)
- **npm or yarn** (comes with Node.js)
- **Redis server** (we'll show you how to install this)
- **Git** for cloning the repository

### Step 1: Install Redis

Redis is essential for the server's session management and message routing.

**On macOS:**
```bash
brew install redis
brew services start redis
```

**On Ubuntu/Debian:**
```bash
sudo apt update
sudo apt install redis-server
sudo systemctl start redis
```

**On Windows:**
Use WSL2 or Docker:
```bash
docker run -d -p 6379:6379 redis:latest
```

Verify Redis is running:
```bash
redis-cli ping
# Should return: PONG
```

### Step 2: Clone and Setup the Repository

```bash
# Clone the repository
git clone https://github.com/modelcontextprotocol/example-remote-server.git
cd example-remote-server

# Install dependencies
npm install
```

### Step 3: Configure Environment

Create your environment configuration:

```bash
# Copy the example environment file
cp .env.example .env
```

Edit `.env` with your preferred editor:
```bash
PORT=3232 # Server port
BASE_URI=http://localhost:3232 # Base URI for OAuth redirects
REDIS_HOST=localhost # Redis server host
REDIS_PORT=6379 # Redis server port
REDIS_PASSWORD= # Leave empty for local Redis
```

### Step 4: Start the Development Server

```bash
# Start with hot-reload for development
npm run dev
```

You should see:
```
MCP Everything Server listening on port 3232
Redis client connected
```

### Step 5: Verify the Server is Running

Open a new terminal and test the server:

```bash
# Check the server is responding
curl http://localhost:3232/health

# You should see: {"status":"ok"}
```

### Step 6: Explore the OAuth Flow

The server includes a fake OAuth provider for testing. Open your browser and navigate to:

```
http://localhost:3232/fakeupstreamauth/authorize?client_id=test&redirect_uri=http://localhost:3232/oauth/callback
```

This simulates the OAuth flow without external dependencies.

### Step 7: Run the Test Suite

Understand the server's capabilities by running tests:

```bash
# Run all tests
npm test

# Run specific test suites
npm test -- --testNamePattern="session ownership"
```

### Step 8: Monitor Real-Time Activity

Open a new terminal to watch Redis activity:

```bash
# Monitor all Redis commands in real-time
redis-cli MONITOR | grep "mcp:"
```

Keep this running while you interact with the server to see message flow.

### Step 9: Explore with Development Tools

The `scratch/` directory contains helpful development scripts:

```bash
# Test OAuth flows
./scratch/oauth.sh

# Run a simple test client
node scratch/simple-test-client.js

# Debug MCP message flows
./scratch/debug-mcp-flow.sh
```

### Step 10: Connect Your MCP Client

Now you're ready to connect your MCP client application. Use these connection details:

- **Endpoint:** `http://localhost:3232/mcp`
- **Transport:** Streamable HTTP (recommended) or SSE at `/sse`
- **Authentication:** Bearer token from OAuth flow

Example client connection (using MCP SDK):
```javascript
import { Client } from "@modelcontextprotocol/sdk/client/index.js";
import { StreamableHTTPClientTransport } from "@modelcontextprotocol/sdk/client/streamableHttp.js";

const transport = new StreamableHTTPClientTransport({
url: "http://localhost:3232/mcp",
headers: {
Authorization: `Bearer ${yourAccessToken}`
}
});

const client = new Client({
name: "my-client",
version: "1.0.0"
});

await client.connect(transport);
```

### Troubleshooting Common Issues

**Redis connection errors:**
- Ensure Redis is running: `redis-cli ping`
- Check Redis port isn't blocked by firewall
- Verify REDIS_HOST and REDIS_PORT in `.env`

**Port already in use:**
- Change PORT in `.env` to another value (e.g., 3233)
- Or find and stop the process using port 3232

**Authentication failures:**
- The fake auth provider uses localStorage in the browser
- Clear browser cache if seeing unexpected user IDs
- Check bearer token is properly formatted

## Architecture Deep Dive

The Everything Server's architecture is designed for production scalability:

### Stateless Server Design
Every server instance is stateless – session state lives in Redis, not memory. This means:
- Servers can be added or removed without losing sessions
- Load balancers can route requests to any instance
- Zero-downtime deployments are possible

### Redis as the Backbone
Redis serves three critical functions:
1. **Session state storage** with automatic TTL expiration
2. **Pub/sub message routing** between client connections and MCP servers
3. **Connection tracking** via subscription counts

### Security-First Approach
- OAuth 2.0 with PKCE prevents token interception
- Session ownership ensures user isolation
- Security headers protect against common web vulnerabilities
- Structured logging sanitizes sensitive data

## Next Steps

Now that you have the Everything Server running:

1. **Explore the codebase:** Start with `src/services/mcp.ts` to understand tool implementations
2. **Modify tools:** Replace example tools with your domain-specific functionality
3. **Test scaling:** Spin up multiple instances and verify session sharing
4. **Build your client:** Use the server to develop and test your MCP client application
5. **Deploy to production:** Containerize and deploy to your cloud platform

## Resources

- [MCP Specification](https://modelcontextprotocol.io/specification)
- [MCP Documentation](https://modelcontextprotocol.io)
- [TypeScript SDK](https://github.com/modelcontextprotocol/typescript-sdk)
- [Everything Server Repository](https://github.com/modelcontextprotocol/example-remote-server)

## Conclusion

The MCP Everything Server is more than just example code – it's a production-ready foundation for building scalable AI integrations. Whether you're learning MCP, building a production server, or testing client applications, this comprehensive implementation provides the patterns and infrastructure you need.

Start with the Everything Server today and build the next generation of AI-powered applications with confidence.

---

*The MCP Everything Server is open source and welcomes contributions. Join us in building the future of AI integration at [github.com/modelcontextprotocol/example-remote-server](https://github.com/modelcontextprotocol/example-remote-server).*