Compare commits
12 Commits
d359a78a4d
...
main
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
66cf3e851e | ||
|
|
15127c3cbd | ||
|
|
3e0ed2d6c6 | ||
|
|
50c2c43fd9 | ||
|
|
8512f031f7 | ||
|
|
3d9d07b210 | ||
|
|
436ac8d07f | ||
|
|
1fd46839ef | ||
|
|
baf1270e89 | ||
|
|
3f9840d884 | ||
|
|
74d2997547 | ||
|
|
b8c6e0c8be |
17
.gitignore
vendored
17
.gitignore
vendored
@@ -48,3 +48,20 @@ ehthumbs_vista.db
|
||||
# Testing
|
||||
coverage/
|
||||
.nyc_output/
|
||||
|
||||
# Environment variables
|
||||
.env
|
||||
.env.local
|
||||
.env.development.local
|
||||
.env.test.local
|
||||
.env.production.local
|
||||
|
||||
# Runtime data
|
||||
pids
|
||||
*.pid
|
||||
*.seed
|
||||
*.pid.lock
|
||||
|
||||
# OS files
|
||||
.DS_Store
|
||||
Thumbs.db
|
||||
|
||||
@@ -39,9 +39,15 @@ RUN npx tsc && \
|
||||
# Switch to production for runtime
|
||||
ENV NODE_ENV=production
|
||||
|
||||
# Default port for HTTP transport
|
||||
ENV PORT=3001
|
||||
|
||||
# The API key should be passed at runtime
|
||||
# ENV OPENROUTER_API_KEY=your-api-key-here
|
||||
# ENV OPENROUTER_DEFAULT_MODEL=your-default-model
|
||||
|
||||
# Run the server
|
||||
# Expose HTTP port
|
||||
EXPOSE 3001
|
||||
|
||||
# Run the server in HTTP mode (default)
|
||||
CMD ["node", "dist/index.js"]
|
||||
|
||||
196
README.md
196
README.md
@@ -30,6 +30,28 @@ An MCP (Model Context Protocol) server that provides chat and image analysis cap
|
||||
- Exponential backoff for retries
|
||||
- Automatic rate limit handling
|
||||
|
||||
## What's New in 1.5.0
|
||||
|
||||
- **Improved OS Compatibility:**
|
||||
- Enhanced path handling for Windows, macOS, and Linux
|
||||
- Better support for Windows-style paths with drive letters
|
||||
- Normalized path processing for consistent behavior across platforms
|
||||
|
||||
- **MCP Configuration Support:**
|
||||
- Cursor MCP integration without requiring environment variables
|
||||
- Direct configuration via MCP parameters
|
||||
- Flexible API key and model specification options
|
||||
|
||||
- **Robust Error Handling:**
|
||||
- Improved fallback mechanisms for image processing
|
||||
- Better error reporting with specific diagnostics
|
||||
- Multiple backup strategies for file reading
|
||||
|
||||
- **Image Processing Enhancements:**
|
||||
- More reliable base64 encoding for all image types
|
||||
- Fallback options when Sharp module is unavailable
|
||||
- Better handling of large images with automatic optimization
|
||||
|
||||
## Installation
|
||||
|
||||
### Option 1: Install via npm
|
||||
@@ -68,7 +90,7 @@ Add one of the following configurations to your MCP settings file (e.g., `cline_
|
||||
],
|
||||
"env": {
|
||||
"OPENROUTER_API_KEY": "your-api-key-here",
|
||||
"OPENROUTER_DEFAULT_MODEL": "anthropic/claude-3.5-sonnet"
|
||||
"DEFAULT_MODEL": "qwen/qwen2.5-vl-32b-instruct:free"
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -89,7 +111,7 @@ Add one of the following configurations to your MCP settings file (e.g., `cline_
|
||||
],
|
||||
"env": {
|
||||
"OPENROUTER_API_KEY": "your-api-key-here",
|
||||
"OPENROUTER_DEFAULT_MODEL": "anthropic/claude-3.5-sonnet"
|
||||
"DEFAULT_MODEL": "qwen/qwen2.5-vl-32b-instruct:free"
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -108,7 +130,7 @@ Add one of the following configurations to your MCP settings file (e.g., `cline_
|
||||
"--rm",
|
||||
"-i",
|
||||
"-e", "OPENROUTER_API_KEY=your-api-key-here",
|
||||
"-e", "OPENROUTER_DEFAULT_MODEL=anthropic/claude-3.5-sonnet",
|
||||
"-e", "DEFAULT_MODEL=qwen/qwen2.5-vl-32b-instruct:free",
|
||||
"stabgandocker/openrouter-mcp-multimodal:latest"
|
||||
]
|
||||
}
|
||||
@@ -129,23 +151,45 @@ Add one of the following configurations to your MCP settings file (e.g., `cline_
|
||||
],
|
||||
"env": {
|
||||
"OPENROUTER_API_KEY": "your-api-key-here",
|
||||
"OPENROUTER_DEFAULT_MODEL": "anthropic/claude-3.5-sonnet"
|
||||
"DEFAULT_MODEL": "qwen/qwen2.5-vl-32b-instruct:free"
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
## Examples
|
||||
|
||||
For comprehensive examples of how to use this MCP server, check out the [examples directory](./examples/). We provide:
|
||||
|
||||
- JavaScript examples for Node.js applications
|
||||
- Python examples with interactive chat capabilities
|
||||
- Code snippets for integrating with various applications
|
||||
|
||||
Each example comes with clear documentation and step-by-step instructions.
|
||||
|
||||
## Dependencies
|
||||
|
||||
This project uses the following key dependencies:
|
||||
|
||||
- `@modelcontextprotocol/sdk`: ^1.8.0 - Latest MCP SDK for tool implementation
|
||||
- `openai`: ^4.89.1 - OpenAI-compatible API client for OpenRouter
|
||||
- `sharp`: ^0.33.5 - Fast image processing library
|
||||
- `axios`: ^1.8.4 - HTTP client for API requests
|
||||
- `node-fetch`: ^3.3.2 - Modern fetch implementation
|
||||
|
||||
Node.js 18 or later is required. All dependencies are regularly updated to ensure compatibility and security.
|
||||
|
||||
## Available Tools
|
||||
|
||||
### chat_completion
|
||||
### mcp_openrouter_chat_completion
|
||||
|
||||
Send text or multimodal messages to OpenRouter models:
|
||||
|
||||
```javascript
|
||||
use_mcp_tool({
|
||||
server_name: "openrouter",
|
||||
tool_name: "chat_completion",
|
||||
tool_name: "mcp_openrouter_chat_completion",
|
||||
arguments: {
|
||||
model: "google/gemini-2.5-pro-exp-03-25:free", // Optional if default is set
|
||||
messages: [
|
||||
@@ -168,7 +212,7 @@ For multimodal messages with images:
|
||||
```javascript
|
||||
use_mcp_tool({
|
||||
server_name: "openrouter",
|
||||
tool_name: "chat_completion",
|
||||
tool_name: "mcp_openrouter_chat_completion",
|
||||
arguments: {
|
||||
model: "anthropic/claude-3.5-sonnet",
|
||||
messages: [
|
||||
@@ -191,141 +235,3 @@ use_mcp_tool({
|
||||
}
|
||||
});
|
||||
```
|
||||
|
||||
### analyze_image
|
||||
|
||||
Analyze a single image with an optional question:
|
||||
|
||||
```javascript
|
||||
use_mcp_tool({
|
||||
server_name: "openrouter",
|
||||
tool_name: "analyze_image",
|
||||
arguments: {
|
||||
image_path: "/absolute/path/to/image.jpg",
|
||||
question: "What objects are in this image?", // Optional
|
||||
model: "anthropic/claude-3.5-sonnet" // Optional if default is set
|
||||
}
|
||||
});
|
||||
```
|
||||
|
||||
### multi_image_analysis
|
||||
|
||||
Analyze multiple images with a single prompt:
|
||||
|
||||
```javascript
|
||||
use_mcp_tool({
|
||||
server_name: "openrouter",
|
||||
tool_name: "multi_image_analysis",
|
||||
arguments: {
|
||||
images: [
|
||||
{ url: "https://example.com/image1.jpg" },
|
||||
{ url: "file:///absolute/path/to/image2.jpg" },
|
||||
{
|
||||
url: "https://example.com/image3.jpg",
|
||||
alt: "Optional description of image 3"
|
||||
}
|
||||
],
|
||||
prompt: "Compare these images and tell me their similarities and differences",
|
||||
markdown_response: true, // Optional, defaults to true
|
||||
model: "anthropic/claude-3-opus" // Optional if default is set
|
||||
}
|
||||
});
|
||||
```
|
||||
|
||||
### search_models
|
||||
|
||||
Search and filter available models:
|
||||
|
||||
```javascript
|
||||
use_mcp_tool({
|
||||
server_name: "openrouter",
|
||||
tool_name: "search_models",
|
||||
arguments: {
|
||||
query: "claude", // Optional text search
|
||||
provider: "anthropic", // Optional provider filter
|
||||
capabilities: {
|
||||
vision: true // Filter for models with vision capabilities
|
||||
},
|
||||
limit: 5 // Optional, defaults to 10
|
||||
}
|
||||
});
|
||||
```
|
||||
|
||||
### get_model_info
|
||||
|
||||
Get detailed information about a specific model:
|
||||
|
||||
```javascript
|
||||
use_mcp_tool({
|
||||
server_name: "openrouter",
|
||||
tool_name: "get_model_info",
|
||||
arguments: {
|
||||
model: "anthropic/claude-3.5-sonnet"
|
||||
}
|
||||
});
|
||||
```
|
||||
|
||||
### validate_model
|
||||
|
||||
Check if a model ID is valid:
|
||||
|
||||
```javascript
|
||||
use_mcp_tool({
|
||||
server_name: "openrouter",
|
||||
tool_name: "validate_model",
|
||||
arguments: {
|
||||
model: "google/gemini-2.5-pro-exp-03-25:free"
|
||||
}
|
||||
});
|
||||
```
|
||||
|
||||
## Error Handling
|
||||
|
||||
The server provides detailed error messages for various failure cases:
|
||||
|
||||
- Invalid input parameters
|
||||
- Network errors
|
||||
- Rate limiting issues
|
||||
- Invalid image formats
|
||||
- Authentication problems
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
### Common Issues
|
||||
|
||||
- **"fetch is not defined" error**: This often occurs when the Node.js environment doesn't have global fetch. Use Node.js v18+ or add the PATH environment variable to your configuration as shown below:
|
||||
|
||||
```json
|
||||
{
|
||||
"mcpServers": {
|
||||
"openrouter": {
|
||||
"command": "npx",
|
||||
"args": [
|
||||
"-y",
|
||||
"@stabgan/openrouter-mcp-multimodal"
|
||||
],
|
||||
"env": {
|
||||
"OPENROUTER_API_KEY": "your-api-key-here",
|
||||
"PATH": "/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin"
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
- **Image analysis failures**: Make sure your image path is absolute and the file format is supported.
|
||||
|
||||
## Development
|
||||
|
||||
To build from source:
|
||||
|
||||
```bash
|
||||
git clone https://github.com/stabgan/openrouter-mcp-multimodal.git
|
||||
cd openrouter-mcp-multimodal
|
||||
npm install
|
||||
npm run build
|
||||
```
|
||||
|
||||
## License
|
||||
|
||||
MIT License
|
||||
|
||||
131
convert_to_base64.html
Normal file
131
convert_to_base64.html
Normal file
@@ -0,0 +1,131 @@
|
||||
<!DOCTYPE html>
|
||||
<html lang="en">
|
||||
<head>
|
||||
<meta charset="UTF-8">
|
||||
<meta name="viewport" content="width=device-width, initial-scale=1.0">
|
||||
<title>Image to Base64 Converter</title>
|
||||
<style>
|
||||
body {
|
||||
font-family: Arial, sans-serif;
|
||||
max-width: 800px;
|
||||
margin: 0 auto;
|
||||
padding: 20px;
|
||||
}
|
||||
.container {
|
||||
display: flex;
|
||||
flex-direction: column;
|
||||
gap: 20px;
|
||||
}
|
||||
.preview {
|
||||
margin-top: 20px;
|
||||
max-width: 100%;
|
||||
}
|
||||
.preview img {
|
||||
max-width: 100%;
|
||||
max-height: 300px;
|
||||
border: 1px solid #ddd;
|
||||
}
|
||||
.result {
|
||||
margin-top: 20px;
|
||||
}
|
||||
textarea {
|
||||
width: 100%;
|
||||
height: 100px;
|
||||
margin-top: 10px;
|
||||
}
|
||||
button {
|
||||
padding: 10px 15px;
|
||||
background-color: #4CAF50;
|
||||
color: white;
|
||||
border: none;
|
||||
border-radius: 4px;
|
||||
cursor: pointer;
|
||||
font-size: 16px;
|
||||
}
|
||||
button:hover {
|
||||
background-color: #45a049;
|
||||
}
|
||||
.copy-button {
|
||||
margin-top: 10px;
|
||||
}
|
||||
.code-block {
|
||||
background-color: #f5f5f5;
|
||||
padding: 15px;
|
||||
border-radius: 4px;
|
||||
overflow-x: auto;
|
||||
margin-top: 20px;
|
||||
}
|
||||
</style>
|
||||
</head>
|
||||
<body>
|
||||
<h1>Image to Base64 Converter for MCP Testing</h1>
|
||||
<p>Use this tool to convert a local image to a base64 string that can be used with the MCP server's multi_image_analysis tool.</p>
|
||||
|
||||
<div class="container">
|
||||
<div>
|
||||
<label for="imageInput">Select an image:</label><br>
|
||||
<input type="file" id="imageInput" accept="image/*">
|
||||
</div>
|
||||
|
||||
<div class="preview" id="preview">
|
||||
<h3>Image Preview:</h3>
|
||||
<div id="imagePreview"></div>
|
||||
</div>
|
||||
|
||||
<div class="result" id="result">
|
||||
<h3>Base64 String:</h3>
|
||||
<textarea id="base64Output" readonly></textarea>
|
||||
<button class="copy-button" id="copyButton">Copy to Clipboard</button>
|
||||
</div>
|
||||
|
||||
<div>
|
||||
<h3>How to use with MCP:</h3>
|
||||
<div class="code-block">
|
||||
<pre>
|
||||
{
|
||||
"images": [
|
||||
{
|
||||
"url": "PASTE_BASE64_STRING_HERE"
|
||||
}
|
||||
],
|
||||
"prompt": "Please describe this image in detail. What does it show?",
|
||||
"model": "qwen/qwen2.5-vl-32b-instruct:free"
|
||||
}
|
||||
</pre>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<script>
|
||||
document.getElementById('imageInput').addEventListener('change', function(event) {
|
||||
const file = event.target.files[0];
|
||||
if (!file) return;
|
||||
|
||||
// Display image preview
|
||||
const preview = document.getElementById('imagePreview');
|
||||
preview.innerHTML = '';
|
||||
const img = document.createElement('img');
|
||||
img.src = URL.createObjectURL(file);
|
||||
preview.appendChild(img);
|
||||
|
||||
// Convert to base64
|
||||
const reader = new FileReader();
|
||||
reader.onload = function(e) {
|
||||
const base64String = e.target.result; // This already includes "data:image/jpeg;base64,"
|
||||
document.getElementById('base64Output').value = base64String;
|
||||
};
|
||||
reader.readAsDataURL(file);
|
||||
});
|
||||
|
||||
document.getElementById('copyButton').addEventListener('click', function() {
|
||||
const textarea = document.getElementById('base64Output');
|
||||
textarea.select();
|
||||
document.execCommand('copy');
|
||||
this.textContent = 'Copied!';
|
||||
setTimeout(() => {
|
||||
this.textContent = 'Copy to Clipboard';
|
||||
}, 2000);
|
||||
});
|
||||
</script>
|
||||
</body>
|
||||
</html>
|
||||
89
convert_to_base64.py
Normal file
89
convert_to_base64.py
Normal file
@@ -0,0 +1,89 @@
|
||||
#!/usr/bin/env python3
|
||||
import base64
|
||||
import argparse
|
||||
import os
|
||||
import sys
|
||||
from pathlib import Path
|
||||
|
||||
|
||||
def convert_image_to_base64(image_path):
|
||||
"""Convert an image file to base64 encoding with data URI prefix"""
|
||||
# Get file extension and determine mime type
|
||||
file_ext = os.path.splitext(image_path)[1].lower()
|
||||
mime_type = {
|
||||
'.png': 'image/png',
|
||||
'.jpg': 'image/jpeg',
|
||||
'.jpeg': 'image/jpeg',
|
||||
'.gif': 'image/gif',
|
||||
'.webp': 'image/webp',
|
||||
'.bmp': 'image/bmp'
|
||||
}.get(file_ext, 'application/octet-stream')
|
||||
|
||||
# Read binary data and encode to base64
|
||||
try:
|
||||
with open(image_path, 'rb') as img_file:
|
||||
img_data = img_file.read()
|
||||
base64_data = base64.b64encode(img_data).decode('utf-8')
|
||||
return f"data:{mime_type};base64,{base64_data}"
|
||||
except Exception as e:
|
||||
print(f"Error: {e}", file=sys.stderr)
|
||||
return None
|
||||
|
||||
|
||||
def save_base64_to_file(base64_data, output_path):
|
||||
"""Save base64 data to a file"""
|
||||
try:
|
||||
with open(output_path, 'w') as out_file:
|
||||
out_file.write(base64_data)
|
||||
print(f"Base64 data saved to {output_path}")
|
||||
return True
|
||||
except Exception as e:
|
||||
print(f"Error saving file: {e}", file=sys.stderr)
|
||||
return False
|
||||
|
||||
|
||||
def main():
|
||||
parser = argparse.ArgumentParser(description='Convert image to base64 for MCP server testing')
|
||||
parser.add_argument('image_path', help='Path to the image file')
|
||||
parser.add_argument('-o', '--output', help='Output file path (if not provided, output to console)')
|
||||
|
||||
args = parser.parse_args()
|
||||
|
||||
# Check if file exists
|
||||
image_path = Path(args.image_path)
|
||||
if not image_path.exists():
|
||||
print(f"Error: File not found: {args.image_path}", file=sys.stderr)
|
||||
return 1
|
||||
|
||||
# Convert image to base64
|
||||
base64_data = convert_image_to_base64(args.image_path)
|
||||
if not base64_data:
|
||||
return 1
|
||||
|
||||
# Output base64 data
|
||||
if args.output:
|
||||
success = save_base64_to_file(base64_data, args.output)
|
||||
if not success:
|
||||
return 1
|
||||
else:
|
||||
print("\nBase64 Image Data:")
|
||||
print(base64_data[:100] + "..." if len(base64_data) > 100 else base64_data)
|
||||
print("\nTotal length:", len(base64_data))
|
||||
print("\nTo use with MCP server in multi_image_analysis:")
|
||||
print('''
|
||||
{
|
||||
"images": [
|
||||
{
|
||||
"url": "''' + base64_data[:20] + '... (full base64 string)" ' + '''
|
||||
}
|
||||
],
|
||||
"prompt": "Please describe this image in detail. What does it show?",
|
||||
"model": "qwen/qwen2.5-vl-32b-instruct:free"
|
||||
}
|
||||
''')
|
||||
|
||||
return 0
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
sys.exit(main())
|
||||
78
encode_image.sh
Normal file
78
encode_image.sh
Normal file
@@ -0,0 +1,78 @@
|
||||
#!/bin/bash
|
||||
|
||||
# Check if an image file is provided
|
||||
if [ $# -lt 1 ]; then
|
||||
echo "Usage: $0 <image_file> [output_file]"
|
||||
echo "Example: $0 test.png base64_output.txt"
|
||||
exit 1
|
||||
fi
|
||||
|
||||
IMAGE_FILE="$1"
|
||||
OUTPUT_FILE="${2:-}" # Use the second argument as output file, if provided
|
||||
|
||||
# Check if the image file exists
|
||||
if [ ! -f "$IMAGE_FILE" ]; then
|
||||
echo "Error: Image file '$IMAGE_FILE' does not exist."
|
||||
exit 1
|
||||
fi
|
||||
|
||||
# Get the file extension and determine MIME type
|
||||
FILE_EXT="${IMAGE_FILE##*.}"
|
||||
MIME_TYPE="application/octet-stream" # Default MIME type
|
||||
|
||||
case "$FILE_EXT" in
|
||||
png|PNG)
|
||||
MIME_TYPE="image/png"
|
||||
;;
|
||||
jpg|jpeg|JPG|JPEG)
|
||||
MIME_TYPE="image/jpeg"
|
||||
;;
|
||||
gif|GIF)
|
||||
MIME_TYPE="image/gif"
|
||||
;;
|
||||
webp|WEBP)
|
||||
MIME_TYPE="image/webp"
|
||||
;;
|
||||
*)
|
||||
echo "Warning: Unknown file extension. Using generic MIME type."
|
||||
;;
|
||||
esac
|
||||
|
||||
# Convert image to base64
|
||||
echo "Converting '$IMAGE_FILE' to base64..."
|
||||
|
||||
# Different commands based on OS
|
||||
if [ "$(uname)" == "Darwin" ]; then
|
||||
# macOS
|
||||
BASE64_DATA="data:$MIME_TYPE;base64,$(base64 -i "$IMAGE_FILE")"
|
||||
else
|
||||
# Linux and others
|
||||
BASE64_DATA="data:$MIME_TYPE;base64,$(base64 -w 0 "$IMAGE_FILE")"
|
||||
fi
|
||||
|
||||
# Output the base64 data
|
||||
if [ -n "$OUTPUT_FILE" ]; then
|
||||
# Save to file if output file is specified
|
||||
echo "$BASE64_DATA" > "$OUTPUT_FILE"
|
||||
echo "Base64 data saved to '$OUTPUT_FILE'"
|
||||
echo "Total length: ${#BASE64_DATA} characters"
|
||||
else
|
||||
# Display a preview and length if no output file
|
||||
echo "Base64 Image Data (first 100 chars):"
|
||||
echo "${BASE64_DATA:0:100}..."
|
||||
echo "Total length: ${#BASE64_DATA} characters"
|
||||
|
||||
echo ""
|
||||
echo "To use with MCP server in multi_image_analysis:"
|
||||
echo '{
|
||||
"images": [
|
||||
{
|
||||
"url": "'"${BASE64_DATA:0:20}"'... (full base64 string)"
|
||||
}
|
||||
],
|
||||
"prompt": "Please describe this image in detail. What does it show?",
|
||||
"model": "qwen/qwen2.5-vl-32b-instruct:free"
|
||||
}'
|
||||
fi
|
||||
|
||||
exit 0
|
||||
119
examples/README.md
Normal file
119
examples/README.md
Normal file
@@ -0,0 +1,119 @@
|
||||
# OpenRouter MCP Server Examples
|
||||
|
||||
This directory contains example scripts demonstrating how to use the OpenRouter MCP Server for various tasks such as text chat, image analysis, and model searching.
|
||||
|
||||
## Prerequisites
|
||||
|
||||
Before running these examples, ensure you have:
|
||||
|
||||
1. Node.js 18 or later installed
|
||||
2. OpenRouter API key (get one from [OpenRouter](https://openrouter.ai))
|
||||
3. Set up the environment variable:
|
||||
```
|
||||
OPENROUTER_API_KEY=your_api_key_here
|
||||
```
|
||||
You can create a `.env` file in the root directory with this variable.
|
||||
|
||||
## JavaScript Example
|
||||
|
||||
The `index.js` file demonstrates how to use the MCP server from Node.js:
|
||||
|
||||
1. Starting the MCP server
|
||||
2. Connecting to the server
|
||||
3. Simple text chat
|
||||
4. Single image analysis
|
||||
5. Multiple image analysis
|
||||
6. Model search
|
||||
|
||||
### Running the JavaScript Example
|
||||
|
||||
```bash
|
||||
# Install dependencies if you haven't already
|
||||
npm install
|
||||
|
||||
# Run the example
|
||||
npm run examples
|
||||
```
|
||||
|
||||
## Python Example
|
||||
|
||||
The `python_example.py` script demonstrates how to use the MCP server from Python:
|
||||
|
||||
1. Connecting to the MCP server
|
||||
2. Converting MCP tool definitions to OpenAI format
|
||||
3. Interactive chat loop with tool calling
|
||||
|
||||
### Running the Python Example
|
||||
|
||||
```bash
|
||||
# Install required Python packages
|
||||
pip install python-mcp openai python-dotenv
|
||||
|
||||
# Run the example
|
||||
python examples/python_example.py
|
||||
```
|
||||
|
||||
## Using the MCP Server in Your Projects
|
||||
|
||||
To use the OpenRouter MCP Server in your own projects:
|
||||
|
||||
1. Install the package:
|
||||
```bash
|
||||
npm install @stabgan/openrouter-mcp-multimodal
|
||||
```
|
||||
|
||||
2. Create a client connection using the MCP client libraries:
|
||||
```javascript
|
||||
import { ClientSession, StdioServerParameters } from '@modelcontextprotocol/sdk/client/index.js';
|
||||
import { stdio_client } from '@modelcontextprotocol/sdk/client/stdio.js';
|
||||
|
||||
// Configure server
|
||||
const serverConfig = {
|
||||
command: 'npx',
|
||||
args: ['-y', '@stabgan/openrouter-mcp-multimodal'],
|
||||
env: { OPENROUTER_API_KEY: 'your_api_key_here' }
|
||||
};
|
||||
|
||||
// Create connection
|
||||
const serverParams = new StdioServerParameters(
|
||||
serverConfig.command,
|
||||
serverConfig.args,
|
||||
serverConfig.env
|
||||
);
|
||||
const client = await stdio_client(serverParams);
|
||||
const [stdio, write] = client;
|
||||
|
||||
// Initialize session
|
||||
const session = new ClientSession(stdio, write);
|
||||
await session.initialize();
|
||||
```
|
||||
|
||||
3. Call tools:
|
||||
```javascript
|
||||
// Get available tools
|
||||
const response = await session.list_tools();
|
||||
console.log('Available tools:', response.tools.map(tool => tool.name).join(', '));
|
||||
|
||||
// Call a tool
|
||||
const result = await session.call_tool('mcp_openrouter_chat_completion', {
|
||||
messages: [
|
||||
{ role: 'user', content: 'Hello, what can you do?' }
|
||||
],
|
||||
model: 'deepseek/deepseek-chat-v3-0324:free'
|
||||
});
|
||||
|
||||
console.log('Response:', result.content[0].text);
|
||||
```
|
||||
|
||||
## Available Tools
|
||||
|
||||
The OpenRouter MCP Server provides the following tools:
|
||||
|
||||
1. `mcp_openrouter_chat_completion` - Text chat with LLMs
|
||||
2. `mcp_openrouter_analyze_image` - Analyze a single image
|
||||
3. `mcp_openrouter_multi_image_analysis` - Analyze multiple images
|
||||
4. `search_models` - Search for available models
|
||||
5. `get_model_info` - Get details about a specific model
|
||||
6. `validate_model` - Check if a model ID is valid
|
||||
|
||||
For detailed information about each tool's parameters, see the [main README](../README.md) file.
|
||||
262
examples/index.js
Normal file
262
examples/index.js
Normal file
@@ -0,0 +1,262 @@
|
||||
#!/usr/bin/env node
|
||||
/**
|
||||
* OpenRouter MCP Server Examples
|
||||
*
|
||||
* This script demonstrates how to use the OpenRouter MCP Server for various tasks:
|
||||
* 1. Text chat with LLMs
|
||||
* 2. Single image analysis
|
||||
* 3. Multiple image analysis
|
||||
* 4. Model search and selection
|
||||
*/
|
||||
import { ClientSession, StdioServerParameters } from '@modelcontextprotocol/sdk/client/index.js';
|
||||
import { stdio_client } from '@modelcontextprotocol/sdk/client/stdio.js';
|
||||
import OpenAI from 'openai';
|
||||
import dotenv from 'dotenv';
|
||||
import path from 'path';
|
||||
import { fileURLToPath } from 'url';
|
||||
import fs from 'fs';
|
||||
import { exec } from 'child_process';
|
||||
import { promisify } from 'util';
|
||||
|
||||
// Get the directory name of the current module
|
||||
const __dirname = path.dirname(fileURLToPath(import.meta.url));
|
||||
const execPromise = promisify(exec);
|
||||
|
||||
// Load environment variables
|
||||
dotenv.config();
|
||||
const API_KEY = process.env.OPENROUTER_API_KEY;
|
||||
|
||||
if (!API_KEY) {
|
||||
console.error('Error: OPENROUTER_API_KEY environment variable is missing');
|
||||
console.error('Please set it in a .env file or in your environment');
|
||||
process.exit(1);
|
||||
}
|
||||
|
||||
// OpenAI client for direct API calls if needed
|
||||
const openai = new OpenAI({
|
||||
apiKey: API_KEY,
|
||||
baseURL: 'https://openrouter.ai/api/v1',
|
||||
defaultHeaders: {
|
||||
'HTTP-Referer': 'https://github.com/stabgan/openrouter-mcp-multimodal',
|
||||
'X-Title': 'OpenRouter MCP Multimodal Examples',
|
||||
},
|
||||
});
|
||||
|
||||
// Image file paths for examples
|
||||
const testImage = path.join(__dirname, '..', 'test.png');
|
||||
|
||||
/**
|
||||
* Convert an image to base64
|
||||
*/
|
||||
async function imageToBase64(imagePath) {
|
||||
try {
|
||||
const imageBuffer = fs.readFileSync(imagePath);
|
||||
return imageBuffer.toString('base64');
|
||||
} catch (error) {
|
||||
console.error(`Error reading image ${imagePath}: ${error.message}`);
|
||||
throw error;
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Example 1: Start the MCP server
|
||||
*/
|
||||
async function startMcpServer() {
|
||||
try {
|
||||
// Path to the project's main script
|
||||
const serverScriptPath = path.join(__dirname, '..', 'dist', 'index.js');
|
||||
|
||||
// Start the MCP server as a child process
|
||||
console.log('Starting MCP server...');
|
||||
|
||||
// Command to start the server with environment variables
|
||||
const command = `OPENROUTER_API_KEY=${API_KEY} node ${serverScriptPath}`;
|
||||
|
||||
const { stdout, stderr } = await execPromise(command);
|
||||
|
||||
if (stderr) {
|
||||
console.error('Server start error:', stderr);
|
||||
}
|
||||
|
||||
console.log('MCP server output:', stdout);
|
||||
console.log('MCP server started successfully!');
|
||||
|
||||
return serverScriptPath;
|
||||
} catch (error) {
|
||||
console.error('Failed to start MCP server:', error.message);
|
||||
throw error;
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Example 2: Connect to the MCP server
|
||||
*/
|
||||
async function connectToMcpServer(serverPath) {
|
||||
try {
|
||||
// Configuration for the MCP server
|
||||
const serverConfig = {
|
||||
command: 'node',
|
||||
args: [serverPath],
|
||||
env: {
|
||||
OPENROUTER_API_KEY: API_KEY,
|
||||
}
|
||||
};
|
||||
|
||||
// Connect to the server
|
||||
const session = await establishMcpSession(serverConfig);
|
||||
console.log('Connected to MCP server');
|
||||
|
||||
return session;
|
||||
} catch (error) {
|
||||
console.error('Failed to connect to MCP server:', error.message);
|
||||
throw error;
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Establish an MCP session
|
||||
*/
|
||||
async function establishMcpSession(serverConfig) {
|
||||
// Set up server parameters
|
||||
const serverParams = new StdioServerParameters(
|
||||
serverConfig.command,
|
||||
serverConfig.args,
|
||||
serverConfig.env
|
||||
);
|
||||
|
||||
// Create client connection
|
||||
const client = await stdio_client(serverParams);
|
||||
const [stdio, write] = client;
|
||||
|
||||
// Create and initialize session
|
||||
const session = new ClientSession(stdio, write);
|
||||
await session.initialize();
|
||||
|
||||
// List available tools
|
||||
const response = await session.list_tools();
|
||||
console.log('Available tools:', response.tools.map(tool => tool.name).join(', '));
|
||||
|
||||
return session;
|
||||
}
|
||||
|
||||
/**
|
||||
* Example 3: Simple text chat using the MCP server
|
||||
*/
|
||||
async function textChatExample(session) {
|
||||
console.log('\n--- Text Chat Example ---');
|
||||
|
||||
try {
|
||||
// Call the text chat tool
|
||||
const result = await session.call_tool('mcp_openrouter_chat_completion', {
|
||||
messages: [
|
||||
{ role: 'user', content: 'What is the Model Context Protocol (MCP) and how is it useful?' }
|
||||
],
|
||||
model: 'deepseek/deepseek-chat-v3-0324:free'
|
||||
});
|
||||
|
||||
console.log('Response:', result.content[0].text);
|
||||
} catch (error) {
|
||||
console.error('Text chat error:', error.message);
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Example 4: Image analysis using the MCP server
|
||||
*/
|
||||
async function imageAnalysisExample(session) {
|
||||
console.log('\n--- Image Analysis Example ---');
|
||||
|
||||
try {
|
||||
// Convert image to base64
|
||||
const base64Image = await imageToBase64(testImage);
|
||||
|
||||
// Call the image analysis tool
|
||||
const result = await session.call_tool('mcp_openrouter_analyze_image', {
|
||||
image_path: testImage,
|
||||
question: 'What can you see in this image? Please describe it in detail.'
|
||||
});
|
||||
|
||||
console.log('Response:', result.content[0].text);
|
||||
} catch (error) {
|
||||
console.error('Image analysis error:', error.message);
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Example 5: Multiple image analysis using the MCP server
|
||||
*/
|
||||
async function multiImageAnalysisExample(session) {
|
||||
console.log('\n--- Multiple Image Analysis Example ---');
|
||||
|
||||
try {
|
||||
// Call the multi-image analysis tool
|
||||
const result = await session.call_tool('mcp_openrouter_multi_image_analysis', {
|
||||
images: [
|
||||
{ url: testImage }
|
||||
],
|
||||
prompt: 'What can you see in this image? Please describe it in detail.',
|
||||
markdown_response: true
|
||||
});
|
||||
|
||||
console.log('Response:', result.content[0].text);
|
||||
} catch (error) {
|
||||
console.error('Multi-image analysis error:', error.message);
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Example 6: Search available models
|
||||
*/
|
||||
async function searchModelsExample(session) {
|
||||
console.log('\n--- Search Models Example ---');
|
||||
|
||||
try {
|
||||
// Call the search models tool
|
||||
const result = await session.call_tool('search_models', {
|
||||
query: 'free',
|
||||
capabilities: {
|
||||
vision: true
|
||||
},
|
||||
limit: 5
|
||||
});
|
||||
|
||||
console.log('Available free vision models:');
|
||||
result.content[0].models.forEach((model, index) => {
|
||||
console.log(`${index + 1}. ${model.id} - Context length: ${model.context_length}`);
|
||||
});
|
||||
} catch (error) {
|
||||
console.error('Search models error:', error.message);
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Run all examples
|
||||
*/
|
||||
async function runExamples() {
|
||||
try {
|
||||
// Start the MCP server
|
||||
const serverPath = await startMcpServer();
|
||||
|
||||
// Connect to the MCP server
|
||||
const session = await connectToMcpServer(serverPath);
|
||||
|
||||
// Run the text chat example
|
||||
await textChatExample(session);
|
||||
|
||||
// Run the image analysis example
|
||||
await imageAnalysisExample(session);
|
||||
|
||||
// Run the multi-image analysis example
|
||||
await multiImageAnalysisExample(session);
|
||||
|
||||
// Run the search models example
|
||||
await searchModelsExample(session);
|
||||
|
||||
console.log('\nAll examples completed successfully!');
|
||||
} catch (error) {
|
||||
console.error('Error running examples:', error.message);
|
||||
}
|
||||
}
|
||||
|
||||
// Run the examples
|
||||
runExamples().catch(console.error);
|
||||
187
examples/python_example.py
Normal file
187
examples/python_example.py
Normal file
@@ -0,0 +1,187 @@
|
||||
#!/usr/bin/env python3
|
||||
"""
|
||||
OpenRouter MCP Server - Python Example
|
||||
|
||||
This script demonstrates how to use the OpenRouter MCP Server from Python,
|
||||
for various tasks such as text chat and image analysis.
|
||||
"""
|
||||
|
||||
import os
|
||||
import sys
|
||||
import json
|
||||
import asyncio
|
||||
import subprocess
|
||||
from typing import Optional, Dict, Any, List
|
||||
from contextlib import AsyncExitStack
|
||||
from dotenv import load_dotenv
|
||||
|
||||
# Try to import MCP client libraries, show a helpful error if not available
|
||||
try:
|
||||
from mcp import ClientSession, StdioServerParameters
|
||||
from mcp.client.stdio import stdio_client
|
||||
except ImportError:
|
||||
print("Error: MCP client libraries not found. Please install them with:")
|
||||
print("pip install python-mcp")
|
||||
sys.exit(1)
|
||||
|
||||
# Try to import OpenAI, show a helpful error if not available
|
||||
try:
|
||||
from openai import OpenAI
|
||||
except ImportError:
|
||||
print("Error: OpenAI client not found. Please install it with:")
|
||||
print("pip install openai")
|
||||
sys.exit(1)
|
||||
|
||||
# Load environment variables from .env file
|
||||
load_dotenv()
|
||||
|
||||
# Get API key from environment, or show error
|
||||
API_KEY = os.getenv("OPENROUTER_API_KEY")
|
||||
if not API_KEY:
|
||||
print("Error: OPENROUTER_API_KEY environment variable is missing")
|
||||
print("Please create a .env file with OPENROUTER_API_KEY=your_key")
|
||||
sys.exit(1)
|
||||
|
||||
# Default model to use
|
||||
MODEL = "anthropic/claude-3-5-sonnet"
|
||||
|
||||
# Configuration for the MCP server
|
||||
SERVER_CONFIG = {
|
||||
"command": "npx",
|
||||
"args": ["-y", "@stabgan/openrouter-mcp-multimodal"],
|
||||
"env": {"OPENROUTER_API_KEY": API_KEY}
|
||||
}
|
||||
|
||||
def convert_tool_format(tool):
|
||||
"""Convert MCP tool definition to OpenAI tool format"""
|
||||
converted_tool = {
|
||||
"type": "function",
|
||||
"function": {
|
||||
"name": tool.name,
|
||||
"description": tool.description,
|
||||
"parameters": {
|
||||
"type": "object",
|
||||
"properties": tool.inputSchema["properties"],
|
||||
"required": tool.inputSchema["required"]
|
||||
}
|
||||
}
|
||||
}
|
||||
return converted_tool
|
||||
|
||||
class MCPClient:
|
||||
"""MCP Client for interacting with the OpenRouter MCP server"""
|
||||
|
||||
def __init__(self):
|
||||
self.session: Optional[ClientSession] = None
|
||||
self.exit_stack = AsyncExitStack()
|
||||
self.openai = OpenAI(
|
||||
base_url="https://openrouter.ai/api/v1",
|
||||
api_key=API_KEY
|
||||
)
|
||||
self.messages = []
|
||||
|
||||
async def connect_to_server(self, server_config):
|
||||
"""Connect to the MCP server"""
|
||||
server_params = StdioServerParameters(**server_config)
|
||||
stdio_transport = await self.exit_stack.enter_async_context(stdio_client(server_params))
|
||||
self.stdio, self.write = stdio_transport
|
||||
self.session = await self.exit_stack.enter_async_context(ClientSession(self.stdio, self.write))
|
||||
|
||||
await self.session.initialize()
|
||||
|
||||
# List available tools from the MCP server
|
||||
response = await self.session.list_tools()
|
||||
print("\nConnected to server with tools:", [tool.name for tool in response.tools])
|
||||
|
||||
return response.tools
|
||||
|
||||
async def process_query(self, query: str) -> str:
|
||||
"""Process a text query using the MCP server"""
|
||||
self.messages.append({
|
||||
"role": "user",
|
||||
"content": query
|
||||
})
|
||||
|
||||
# Get available tools from the MCP server
|
||||
response = await self.session.list_tools()
|
||||
available_tools = [convert_tool_format(tool) for tool in response.tools]
|
||||
|
||||
# Make the initial OpenRouter API call with tool definitions
|
||||
response = self.openai.chat.completions.create(
|
||||
model=MODEL,
|
||||
tools=available_tools,
|
||||
messages=self.messages
|
||||
)
|
||||
self.messages.append(response.choices[0].message.model_dump())
|
||||
|
||||
final_text = []
|
||||
content = response.choices[0].message
|
||||
|
||||
# Process tool calls if any
|
||||
if content.tool_calls is not None:
|
||||
tool_name = content.tool_calls[0].function.name
|
||||
tool_args = content.tool_calls[0].function.arguments
|
||||
tool_args = json.loads(tool_args) if tool_args else {}
|
||||
|
||||
# Execute tool call
|
||||
try:
|
||||
result = await self.session.call_tool(tool_name, tool_args)
|
||||
final_text.append(f"[Calling tool {tool_name} with args {tool_args}]")
|
||||
except Exception as e:
|
||||
print(f"Error calling tool {tool_name}: {e}")
|
||||
result = None
|
||||
|
||||
# Add tool result to messages
|
||||
self.messages.append({
|
||||
"role": "tool",
|
||||
"tool_call_id": content.tool_calls[0].id,
|
||||
"name": tool_name,
|
||||
"content": result.content if result else "Error executing tool call"
|
||||
})
|
||||
|
||||
# Make a follow-up API call with the tool results
|
||||
response = self.openai.chat.completions.create(
|
||||
model=MODEL,
|
||||
max_tokens=1000,
|
||||
messages=self.messages,
|
||||
)
|
||||
|
||||
final_text.append(response.choices[0].message.content)
|
||||
else:
|
||||
final_text.append(content.content)
|
||||
|
||||
return "\n".join(final_text)
|
||||
|
||||
async def chat_loop(self):
|
||||
"""Run an interactive chat loop"""
|
||||
print("\nMCP Client Started!")
|
||||
print("Type your queries or 'quit' to exit.")
|
||||
|
||||
while True:
|
||||
try:
|
||||
query = input("\nQuery: ").strip()
|
||||
if query.lower() in ['quit', 'exit']:
|
||||
break
|
||||
|
||||
result = await self.process_query(query)
|
||||
print("Result:")
|
||||
print(result)
|
||||
|
||||
except Exception as e:
|
||||
print(f"Error: {str(e)}")
|
||||
|
||||
async def cleanup(self):
|
||||
"""Clean up resources"""
|
||||
await self.exit_stack.aclose()
|
||||
|
||||
async def main():
|
||||
"""Main entry point for the example script"""
|
||||
client = MCPClient()
|
||||
try:
|
||||
await client.connect_to_server(SERVER_CONFIG)
|
||||
await client.chat_loop()
|
||||
finally:
|
||||
await client.cleanup()
|
||||
|
||||
if __name__ == "__main__":
|
||||
asyncio.run(main())
|
||||
1594
full_base64.txt
Normal file
1594
full_base64.txt
Normal file
File diff suppressed because it is too large
Load Diff
1611
lena_base64.txt
Normal file
1611
lena_base64.txt
Normal file
File diff suppressed because it is too large
Load Diff
183
openrouter-image-python.py
Normal file
183
openrouter-image-python.py
Normal file
@@ -0,0 +1,183 @@
|
||||
#!/usr/bin/env python3
|
||||
import base64
|
||||
import os
|
||||
import mimetypes
|
||||
import requests
|
||||
from openai import OpenAI
|
||||
|
||||
# Constants
|
||||
OPENROUTER_API_KEY = "your_openrouter_api_key" # Replace with your actual key
|
||||
IMAGE_PATH = "path/to/your/image.jpg" # Replace with your image path
|
||||
|
||||
def image_to_base64(image_path):
|
||||
"""Convert an image file to base64 with data URI prefix"""
|
||||
try:
|
||||
# Determine MIME type
|
||||
mime_type, _ = mimetypes.guess_type(image_path)
|
||||
if not mime_type:
|
||||
# Default to generic binary if type cannot be determined
|
||||
mime_type = "application/octet-stream"
|
||||
|
||||
# Read and encode the image
|
||||
with open(image_path, "rb") as image_file:
|
||||
encoded_string = base64.b64encode(image_file.read()).decode("utf-8")
|
||||
|
||||
# Return data URI
|
||||
return f"data:{mime_type};base64,{encoded_string}"
|
||||
except Exception as e:
|
||||
print(f"Error converting image to base64: {e}")
|
||||
raise
|
||||
|
||||
def send_image_direct_api(base64_image, question="What's in this image?"):
|
||||
"""Send an image to OpenRouter using direct API call"""
|
||||
try:
|
||||
print("Sending image via direct API call...")
|
||||
|
||||
headers = {
|
||||
"Authorization": f"Bearer {OPENROUTER_API_KEY}",
|
||||
"Content-Type": "application/json",
|
||||
"HTTP-Referer": "https://your-site-url.com", # Optional
|
||||
"X-Title": "Your Site Name" # Optional
|
||||
}
|
||||
|
||||
payload = {
|
||||
"model": "anthropic/claude-3-opus", # Choose an appropriate model with vision capabilities
|
||||
"messages": [
|
||||
{
|
||||
"role": "user",
|
||||
"content": [
|
||||
{
|
||||
"type": "text",
|
||||
"text": question
|
||||
},
|
||||
{
|
||||
"type": "image_url",
|
||||
"image_url": {
|
||||
"url": base64_image
|
||||
}
|
||||
}
|
||||
]
|
||||
}
|
||||
]
|
||||
}
|
||||
|
||||
response = requests.post(
|
||||
"https://openrouter.ai/api/v1/chat/completions",
|
||||
headers=headers,
|
||||
json=payload
|
||||
)
|
||||
|
||||
response.raise_for_status() # Raise exception for non-200 responses
|
||||
data = response.json()
|
||||
|
||||
print("Response from direct API:")
|
||||
print(data["choices"][0]["message"]["content"])
|
||||
except Exception as e:
|
||||
print(f"Error sending image via direct API: {e}")
|
||||
if hasattr(e, "response") and e.response:
|
||||
print(f"API error details: {e.response.text}")
|
||||
|
||||
def send_image_openai_sdk(base64_image, question="What's in this image?"):
|
||||
"""Send an image to OpenRouter using OpenAI SDK"""
|
||||
try:
|
||||
print("Sending image via OpenAI SDK...")
|
||||
|
||||
# Initialize the OpenAI client with OpenRouter base URL
|
||||
client = OpenAI(
|
||||
api_key=OPENROUTER_API_KEY,
|
||||
base_url="https://openrouter.ai/api/v1",
|
||||
default_headers={
|
||||
"HTTP-Referer": "https://your-site-url.com", # Optional
|
||||
"X-Title": "Your Site Name" # Optional
|
||||
}
|
||||
)
|
||||
|
||||
# Create the message with text and image
|
||||
completion = client.chat.completions.create(
|
||||
model="anthropic/claude-3-opus", # Choose an appropriate model with vision capabilities
|
||||
messages=[
|
||||
{
|
||||
"role": "user",
|
||||
"content": [
|
||||
{
|
||||
"type": "text",
|
||||
"text": question
|
||||
},
|
||||
{
|
||||
"type": "image_url",
|
||||
"image_url": {
|
||||
"url": base64_image
|
||||
}
|
||||
}
|
||||
]
|
||||
}
|
||||
]
|
||||
)
|
||||
|
||||
print("Response from OpenAI SDK:")
|
||||
print(completion.choices[0].message.content)
|
||||
except Exception as e:
|
||||
print(f"Error sending image via OpenAI SDK: {e}")
|
||||
|
||||
def send_image_from_base64_file(base64_file_path, question="What's in this image?"):
|
||||
"""Use a pre-encoded base64 file (e.g., from bash script)"""
|
||||
try:
|
||||
print("Sending image from base64 file...")
|
||||
|
||||
# Read the base64 data from file
|
||||
with open(base64_file_path, "r") as file:
|
||||
base64_data = file.read().strip()
|
||||
|
||||
# Initialize the OpenAI client
|
||||
client = OpenAI(
|
||||
api_key=OPENROUTER_API_KEY,
|
||||
base_url="https://openrouter.ai/api/v1"
|
||||
)
|
||||
|
||||
# Create the message with text and image
|
||||
completion = client.chat.completions.create(
|
||||
model="anthropic/claude-3-opus",
|
||||
messages=[
|
||||
{
|
||||
"role": "user",
|
||||
"content": [
|
||||
{
|
||||
"type": "text",
|
||||
"text": question
|
||||
},
|
||||
{
|
||||
"type": "image_url",
|
||||
"image_url": {
|
||||
"url": base64_data
|
||||
}
|
||||
}
|
||||
]
|
||||
}
|
||||
]
|
||||
)
|
||||
|
||||
print("Response when using base64 file:")
|
||||
print(completion.choices[0].message.content)
|
||||
except Exception as e:
|
||||
print(f"Error sending image from base64 file: {e}")
|
||||
|
||||
def main():
|
||||
try:
|
||||
# Convert the image to base64
|
||||
base64_image = image_to_base64(IMAGE_PATH)
|
||||
print("Image converted to base64 successfully")
|
||||
|
||||
# Example 1: Using direct API call
|
||||
send_image_direct_api(base64_image)
|
||||
|
||||
# Example 2: Using OpenAI SDK
|
||||
send_image_openai_sdk(base64_image)
|
||||
|
||||
# Example 3: Using a base64 file (if you have one)
|
||||
# send_image_from_base64_file("path/to/base64.txt")
|
||||
|
||||
except Exception as e:
|
||||
print(f"Error in main function: {e}")
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
247
openrouter-image-sdk.js
Normal file
247
openrouter-image-sdk.js
Normal file
@@ -0,0 +1,247 @@
|
||||
/**
|
||||
* OpenRouter Image Analysis using OpenAI SDK
|
||||
*
|
||||
* This script demonstrates how to analyze local images using OpenRouter's API
|
||||
* through the OpenAI SDK. It supports both command-line usage and can be imported
|
||||
* as a module for use in other applications.
|
||||
*
|
||||
* Usage:
|
||||
* - Direct: node openrouter-image-sdk.js <image_path> [prompt]
|
||||
* - As module: import { analyzeImage } from './openrouter-image-sdk.js'
|
||||
*
|
||||
* Environment variables:
|
||||
* - OPENROUTER_API_KEY: Your OpenRouter API key (required)
|
||||
*/
|
||||
|
||||
import 'dotenv/config';
|
||||
import { promises as fs } from 'fs';
|
||||
import path from 'path';
|
||||
import { fileURLToPath } from 'url';
|
||||
import { dirname } from 'path';
|
||||
import { OpenAI } from 'openai';
|
||||
|
||||
// ES Module compatibility
|
||||
const __filename = fileURLToPath(import.meta.url);
|
||||
const __dirname = dirname(__filename);
|
||||
|
||||
// Constants
|
||||
const DEFAULT_MODEL = 'qwen/qwen2.5-vl-32b-instruct:free';
|
||||
const MAX_RETRIES = 2;
|
||||
const RETRY_DELAY = 1000; // milliseconds
|
||||
|
||||
/**
|
||||
* Convert a local image file to base64 format
|
||||
*
|
||||
* @param {string} filePath - Path to the image file
|
||||
* @returns {Promise<string>} - Base64 encoded image with data URI prefix
|
||||
*/
|
||||
export async function imageToBase64(filePath) {
|
||||
try {
|
||||
// Ensure the file exists
|
||||
try {
|
||||
await fs.access(filePath);
|
||||
} catch (error) {
|
||||
throw new Error(`Image file not found: ${filePath}`);
|
||||
}
|
||||
|
||||
// Read the file
|
||||
const imageBuffer = await fs.readFile(filePath);
|
||||
|
||||
// Determine MIME type based on file extension
|
||||
const fileExt = path.extname(filePath).toLowerCase();
|
||||
let mimeType = 'application/octet-stream';
|
||||
|
||||
switch (fileExt) {
|
||||
case '.png':
|
||||
mimeType = 'image/png';
|
||||
break;
|
||||
case '.jpg':
|
||||
case '.jpeg':
|
||||
mimeType = 'image/jpeg';
|
||||
break;
|
||||
case '.webp':
|
||||
mimeType = 'image/webp';
|
||||
break;
|
||||
case '.gif':
|
||||
mimeType = 'image/gif';
|
||||
break;
|
||||
default:
|
||||
console.warn(`Unknown file extension: ${fileExt}, using default MIME type`);
|
||||
}
|
||||
|
||||
// Convert to base64 and add the data URI prefix
|
||||
const base64 = imageBuffer.toString('base64');
|
||||
return `data:${mimeType};base64,${base64}`;
|
||||
} catch (error) {
|
||||
console.error('Error converting image to base64:', error);
|
||||
throw new Error(`Failed to convert image to base64: ${error.message}`);
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Sleep for a specified amount of time
|
||||
*
|
||||
* @param {number} ms - Milliseconds to sleep
|
||||
* @returns {Promise<void>}
|
||||
*/
|
||||
const sleep = (ms) => new Promise(resolve => setTimeout(resolve, ms));
|
||||
|
||||
/**
|
||||
* Analyze an image using OpenRouter's API via OpenAI SDK
|
||||
*
|
||||
* @param {Object} options - Options for image analysis
|
||||
* @param {string} options.imagePath - Path to the local image file
|
||||
* @param {string} [options.imageBase64] - Base64 encoded image (alternative to imagePath)
|
||||
* @param {string} [options.prompt="Please describe this image in detail."] - The prompt to send with the image
|
||||
* @param {string} [options.model=DEFAULT_MODEL] - The model to use for analysis
|
||||
* @param {string} [options.apiKey] - OpenRouter API key (defaults to OPENROUTER_API_KEY env var)
|
||||
* @returns {Promise<Object>} - The analysis results
|
||||
*/
|
||||
export async function analyzeImage({
|
||||
imagePath,
|
||||
imageBase64,
|
||||
prompt = "Please describe this image in detail.",
|
||||
model = DEFAULT_MODEL,
|
||||
apiKey
|
||||
}) {
|
||||
// Check for API key
|
||||
const openrouterApiKey = apiKey || process.env.OPENROUTER_API_KEY;
|
||||
if (!openrouterApiKey) {
|
||||
throw new Error('OpenRouter API key is required. Set OPENROUTER_API_KEY in your environment or pass it as an option.');
|
||||
}
|
||||
|
||||
// Check that we have either imagePath or imageBase64
|
||||
if (!imagePath && !imageBase64) {
|
||||
throw new Error('Either imagePath or imageBase64 must be provided.');
|
||||
}
|
||||
|
||||
// Get base64 data if not provided
|
||||
let base64Data = imageBase64;
|
||||
if (!base64Data && imagePath) {
|
||||
console.log(`Converting image at ${imagePath} to base64...`);
|
||||
base64Data = await imageToBase64(imagePath);
|
||||
console.log('Image converted successfully!');
|
||||
}
|
||||
|
||||
// Initialize the OpenAI client with OpenRouter base URL
|
||||
const openai = new OpenAI({
|
||||
apiKey: openrouterApiKey,
|
||||
baseURL: 'https://openrouter.ai/api/v1',
|
||||
defaultHeaders: {
|
||||
'HTTP-Referer': 'https://github.com/stabgan/openrouter-mcp-multimodal',
|
||||
'X-Title': 'OpenRouter Local Image Analysis'
|
||||
}
|
||||
});
|
||||
|
||||
// Implement retry logic
|
||||
let lastError = null;
|
||||
for (let attempt = 0; attempt <= MAX_RETRIES; attempt++) {
|
||||
try {
|
||||
if (attempt > 0) {
|
||||
console.log(`Retry attempt ${attempt}/${MAX_RETRIES}...`);
|
||||
await sleep(RETRY_DELAY * attempt); // Exponential backoff
|
||||
}
|
||||
|
||||
console.log(`Sending image analysis request to model: ${model}`);
|
||||
|
||||
// Create the message with text and image
|
||||
const completion = await openai.chat.completions.create({
|
||||
model,
|
||||
messages: [
|
||||
{
|
||||
role: 'user',
|
||||
content: [
|
||||
{
|
||||
type: 'text',
|
||||
text: prompt
|
||||
},
|
||||
{
|
||||
type: 'image_url',
|
||||
image_url: {
|
||||
url: base64Data
|
||||
}
|
||||
}
|
||||
]
|
||||
}
|
||||
]
|
||||
});
|
||||
|
||||
// Extract the relevant information from the response
|
||||
if (completion && completion.choices && completion.choices.length > 0) {
|
||||
const result = {
|
||||
analysis: completion.choices[0].message.content,
|
||||
model: completion.model,
|
||||
usage: completion.usage,
|
||||
requestId: completion.id,
|
||||
finishReason: completion.choices[0].finish_reason
|
||||
};
|
||||
|
||||
return result;
|
||||
} else {
|
||||
throw new Error('Unexpected response structure from OpenRouter API.');
|
||||
}
|
||||
} catch (error) {
|
||||
lastError = error;
|
||||
|
||||
// If this is a 402 Payment Required error, we won't retry
|
||||
if (error.status === 402 || (error.response && error.response.status === 402)) {
|
||||
console.error('Payment required error. Not retrying.');
|
||||
break;
|
||||
}
|
||||
|
||||
if (attempt === MAX_RETRIES) {
|
||||
console.error('Maximum retry attempts reached.');
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// If we've exhausted all retries, throw the last error
|
||||
throw lastError || new Error('Failed to analyze image after multiple attempts.');
|
||||
}
|
||||
|
||||
/**
|
||||
* Command line interface for image analysis
|
||||
*/
|
||||
async function main() {
|
||||
try {
|
||||
const args = process.argv.slice(2);
|
||||
|
||||
if (args.length === 0) {
|
||||
console.log('Usage: node openrouter-image-sdk.js <image_path> [prompt]');
|
||||
console.log('Example: node openrouter-image-sdk.js test.png "What objects do you see in this image?"');
|
||||
process.exit(0);
|
||||
}
|
||||
|
||||
const imagePath = args[0];
|
||||
const prompt = args[1] || "Please describe this image in detail. What do you see?";
|
||||
|
||||
console.log(`Analyzing image: ${imagePath}`);
|
||||
console.log(`Prompt: ${prompt}`);
|
||||
|
||||
const result = await analyzeImage({ imagePath, prompt });
|
||||
|
||||
console.log('\n----- Analysis Results -----\n');
|
||||
console.log(result.analysis);
|
||||
console.log('\n----------------------------\n');
|
||||
|
||||
console.log('Model used:', result.model);
|
||||
if (result.usage) {
|
||||
console.log('Token usage:');
|
||||
console.log('- Prompt tokens:', result.usage.prompt_tokens);
|
||||
console.log('- Completion tokens:', result.usage.completion_tokens);
|
||||
console.log('- Total tokens:', result.usage.total_tokens);
|
||||
}
|
||||
} catch (error) {
|
||||
console.error('Error:', error.message);
|
||||
if (error.response) {
|
||||
console.error('API error details:', JSON.stringify(error.response, null, 2));
|
||||
}
|
||||
process.exit(1);
|
||||
}
|
||||
}
|
||||
|
||||
// Run the main function directly
|
||||
main().catch(error => {
|
||||
console.error('Fatal error:', error);
|
||||
process.exit(1);
|
||||
});
|
||||
1102
package-lock.json
generated
1102
package-lock.json
generated
File diff suppressed because it is too large
Load Diff
21
package.json
21
package.json
@@ -1,6 +1,6 @@
|
||||
{
|
||||
"name": "@stabgan/openrouter-mcp-multimodal",
|
||||
"version": "1.1.0",
|
||||
"version": "1.5.0",
|
||||
"description": "MCP server for OpenRouter providing text chat and image analysis tools",
|
||||
"type": "module",
|
||||
"main": "dist/index.js",
|
||||
@@ -16,7 +16,9 @@
|
||||
"build": "tsc && shx chmod +x dist/*.js",
|
||||
"prepare": "npm run build",
|
||||
"start": "node dist/index.js",
|
||||
"watch": "tsc --watch"
|
||||
"watch": "tsc --watch",
|
||||
"examples": "node examples/index.js",
|
||||
"audit": "npm audit fix"
|
||||
},
|
||||
"keywords": [
|
||||
"mcp",
|
||||
@@ -41,17 +43,20 @@
|
||||
"node": ">=18.0.0"
|
||||
},
|
||||
"dependencies": {
|
||||
"@modelcontextprotocol/sdk": "^1.4.1",
|
||||
"axios": "^1.7.9",
|
||||
"@modelcontextprotocol/sdk": "^1.12.0",
|
||||
"axios": "^1.8.4",
|
||||
"dotenv": "^16.4.7",
|
||||
"express": "^4.21.2",
|
||||
"node-fetch": "^3.3.2",
|
||||
"openai": "^4.83.0",
|
||||
"sharp": "^0.33.3"
|
||||
"openai": "^4.89.1",
|
||||
"sharp": "^0.33.5"
|
||||
},
|
||||
"devDependencies": {
|
||||
"@types/node": "^22.13.1",
|
||||
"@types/express": "^5.0.0",
|
||||
"@types/node": "^22.13.14",
|
||||
"@types/sharp": "^0.32.0",
|
||||
"shx": "^0.3.4",
|
||||
"typescript": "^5.7.3"
|
||||
"typescript": "^5.8.2"
|
||||
},
|
||||
"overrides": {
|
||||
"uri-js": "npm:uri-js-replace",
|
||||
|
||||
9
railway.toml
Normal file
9
railway.toml
Normal file
@@ -0,0 +1,9 @@
|
||||
[build]
|
||||
builder = "dockerfile"
|
||||
dockerfilePath = "Dockerfile"
|
||||
|
||||
[deploy]
|
||||
healthcheckPath = "/health"
|
||||
healthcheckTimeout = 30
|
||||
restartPolicyType = "on_failure"
|
||||
restartPolicyMaxRetries = 3
|
||||
259
send_image_to_openrouter.js
Normal file
259
send_image_to_openrouter.js
Normal file
@@ -0,0 +1,259 @@
|
||||
// Send an image to OpenRouter using JavaScript
|
||||
import { promises as fs } from 'fs';
|
||||
import path from 'path';
|
||||
import axios from 'axios';
|
||||
import { OpenAI } from 'openai';
|
||||
import { fileURLToPath } from 'url';
|
||||
import { dirname } from 'path';
|
||||
|
||||
console.log("Starting script...");
|
||||
|
||||
// Constants
|
||||
const OPENROUTER_API_KEY = process.env.OPENROUTER_API_KEY || 'your_openrouter_api_key'; // Get from env or replace
|
||||
const IMAGE_PATH = process.argv[2] || 'test.png'; // Get from command line or use default
|
||||
const DEFAULT_MODEL = 'qwen/qwen2.5-vl-32b-instruct:free';
|
||||
|
||||
console.log(`Arguments: ${process.argv.join(', ')}`);
|
||||
console.log(`Using image path: ${IMAGE_PATH}`);
|
||||
|
||||
// Load environment variables from .env file
|
||||
async function loadEnv() {
|
||||
try {
|
||||
const __filename = fileURLToPath(import.meta.url);
|
||||
const __dirname = dirname(__filename);
|
||||
const envPath = path.join(__dirname, '.env');
|
||||
const envFile = await fs.readFile(envPath, 'utf-8');
|
||||
|
||||
envFile.split('\n').forEach(line => {
|
||||
const match = line.match(/^\s*([\w.-]+)\s*=\s*(.*)?\s*$/);
|
||||
if (match) {
|
||||
const key = match[1];
|
||||
let value = match[2] || '';
|
||||
|
||||
// Remove quotes if they exist
|
||||
if (value.length > 0 && value.charAt(0) === '"' && value.charAt(value.length - 1) === '"') {
|
||||
value = value.replace(/^"|"$/g, '');
|
||||
}
|
||||
|
||||
process.env[key] = value;
|
||||
}
|
||||
});
|
||||
|
||||
console.log('Environment variables loaded from .env file');
|
||||
} catch (error) {
|
||||
console.error('Error loading .env file:', error.message);
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Convert an image file to base64
|
||||
*/
|
||||
async function imageToBase64(filePath) {
|
||||
try {
|
||||
// Read the file
|
||||
const imageBuffer = await fs.readFile(filePath);
|
||||
|
||||
// Determine MIME type based on file extension
|
||||
const fileExt = path.extname(filePath).toLowerCase();
|
||||
let mimeType = 'application/octet-stream';
|
||||
|
||||
switch (fileExt) {
|
||||
case '.png':
|
||||
mimeType = 'image/png';
|
||||
break;
|
||||
case '.jpg':
|
||||
case '.jpeg':
|
||||
mimeType = 'image/jpeg';
|
||||
break;
|
||||
case '.webp':
|
||||
mimeType = 'image/webp';
|
||||
break;
|
||||
// Add other supported types as needed
|
||||
}
|
||||
|
||||
// Convert to base64 and add the data URI prefix
|
||||
const base64 = imageBuffer.toString('base64');
|
||||
return `data:${mimeType};base64,${base64}`;
|
||||
} catch (error) {
|
||||
console.error('Error converting image to base64:', error);
|
||||
throw error;
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Example 1: Send a base64 image using the MCP server analyze_image tool
|
||||
*/
|
||||
async function testMcpAnalyzeImage(base64Image, question = "What's in this image?") {
|
||||
try {
|
||||
console.log('Testing MCP analyze_image tool with base64 image...');
|
||||
|
||||
// This would normally be handled by the MCP server client
|
||||
// This is a simulation of how to structure the data for the MCP server
|
||||
console.log(`
|
||||
To analyze the image using MCP, send this request to the MCP server:
|
||||
|
||||
{
|
||||
"tool": "mcp_openrouter_analyze_image",
|
||||
"arguments": {
|
||||
"image_path": "${base64Image.substring(0, 50)}...", // Truncated for display
|
||||
"question": "${question}",
|
||||
"model": "${DEFAULT_MODEL}"
|
||||
}
|
||||
}
|
||||
|
||||
The MCP server will convert the image path (which is already a base64 data URL)
|
||||
and send it to OpenRouter in the correct format.
|
||||
`);
|
||||
} catch (error) {
|
||||
console.error('Error testing MCP analyze_image:', error);
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Example 2: Send multiple base64 images using the MCP server multi_image_analysis tool
|
||||
*/
|
||||
async function testMcpMultiImageAnalysis(base64Images, prompt = "Describe these images in detail.") {
|
||||
try {
|
||||
console.log('Testing MCP multi_image_analysis tool with base64 images...');
|
||||
|
||||
// Create the images array for the MCP request
|
||||
const images = base64Images.map(base64 => ({ url: base64 }));
|
||||
|
||||
// This would normally be handled by the MCP server client
|
||||
// This is a simulation of how to structure the data for the MCP server
|
||||
console.log(`
|
||||
To analyze multiple images using MCP, send this request to the MCP server:
|
||||
|
||||
{
|
||||
"tool": "mcp_openrouter_multi_image_analysis",
|
||||
"arguments": {
|
||||
"images": [
|
||||
{ "url": "${base64Images[0].substring(0, 50)}..." } // Truncated for display
|
||||
${base64Images.length > 1 ? `, { "url": "${base64Images[1].substring(0, 50)}..." }` : ''}
|
||||
${base64Images.length > 2 ? ', ...' : ''}
|
||||
],
|
||||
"prompt": "${prompt}",
|
||||
"model": "${DEFAULT_MODEL}"
|
||||
}
|
||||
}
|
||||
|
||||
The MCP server will process these base64 images and send them to OpenRouter
|
||||
in the correct format.
|
||||
`);
|
||||
} catch (error) {
|
||||
console.error('Error testing MCP multi_image_analysis:', error);
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Example 3: Direct OpenRouter API call with base64 image (for comparison)
|
||||
*/
|
||||
async function sendImageDirectAPI(base64Image, question = "What's in this image?", apiKey) {
|
||||
try {
|
||||
console.log('Sending image directly to OpenRouter API (for comparison)...');
|
||||
|
||||
const response = await axios.post(
|
||||
'https://openrouter.ai/api/v1/chat/completions',
|
||||
{
|
||||
model: DEFAULT_MODEL,
|
||||
messages: [
|
||||
{
|
||||
role: 'user',
|
||||
content: [
|
||||
{
|
||||
type: 'text',
|
||||
text: question
|
||||
},
|
||||
{
|
||||
type: 'image_url',
|
||||
image_url: {
|
||||
url: base64Image
|
||||
}
|
||||
}
|
||||
]
|
||||
}
|
||||
]
|
||||
},
|
||||
{
|
||||
headers: {
|
||||
'Authorization': `Bearer ${apiKey}`,
|
||||
'Content-Type': 'application/json',
|
||||
'HTTP-Referer': 'https://github.com/yourusername/your-repo',
|
||||
'X-Title': 'MCP Server Demo'
|
||||
}
|
||||
}
|
||||
);
|
||||
|
||||
console.log('\nDirect API response:');
|
||||
console.log(response.data.choices[0].message.content);
|
||||
} catch (error) {
|
||||
console.error('Error sending image via direct API:', error);
|
||||
if (error.response) {
|
||||
console.error('API error details:', error.response.data);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Main function to run the examples
|
||||
*/
|
||||
async function main() {
|
||||
try {
|
||||
// Load environment variables from .env file
|
||||
await loadEnv();
|
||||
|
||||
// Get API key from environment after loading
|
||||
const apiKey = process.env.OPENROUTER_API_KEY || OPENROUTER_API_KEY;
|
||||
|
||||
// Debug: Show if API key is set in environment
|
||||
console.log(`API key from environment: ${process.env.OPENROUTER_API_KEY ? 'Yes (set)' : 'No (not set)'}`);
|
||||
console.log(`Using API key: ${apiKey === 'your_openrouter_api_key' ? 'Default placeholder (update needed)' : 'From environment'}`);
|
||||
|
||||
// Check if API key is provided
|
||||
if (apiKey === 'your_openrouter_api_key') {
|
||||
console.error('Please set the OPENROUTER_API_KEY environment variable or update the script.');
|
||||
return;
|
||||
}
|
||||
|
||||
console.log(`Converting image: ${IMAGE_PATH}`);
|
||||
|
||||
// Check if the image file exists
|
||||
try {
|
||||
await fs.access(IMAGE_PATH);
|
||||
console.log(`Image file exists: ${IMAGE_PATH}`);
|
||||
} catch (err) {
|
||||
console.error(`Error: Image file does not exist: ${IMAGE_PATH}`);
|
||||
return;
|
||||
}
|
||||
|
||||
// Convert the image to base64
|
||||
const base64Image = await imageToBase64(IMAGE_PATH);
|
||||
console.log('Image converted to base64 successfully.');
|
||||
console.log(`Base64 length: ${base64Image.length} characters`);
|
||||
console.log(`Base64 starts with: ${base64Image.substring(0, 50)}...`);
|
||||
|
||||
// For multiple images demo, we'll use the same image twice
|
||||
const base64Images = [base64Image, base64Image];
|
||||
|
||||
// Example 1: MCP server with analyze_image
|
||||
await testMcpAnalyzeImage(base64Image);
|
||||
|
||||
// Example 2: MCP server with multi_image_analysis
|
||||
await testMcpMultiImageAnalysis(base64Images);
|
||||
|
||||
// Example 3: Direct API call (if API key is available)
|
||||
if (apiKey !== 'your_openrouter_api_key') {
|
||||
await sendImageDirectAPI(base64Image, "What's in this image?", apiKey);
|
||||
}
|
||||
|
||||
console.log('\nDone! You can now use the MCP server with base64 encoded images.');
|
||||
} catch (error) {
|
||||
console.error('Error in main function:', error);
|
||||
}
|
||||
}
|
||||
|
||||
// Run the main function directly
|
||||
console.log("Running main function...");
|
||||
main().catch(error => {
|
||||
console.error("Unhandled error in main:", error);
|
||||
});
|
||||
160
send_image_to_openrouter.ts
Normal file
160
send_image_to_openrouter.ts
Normal file
@@ -0,0 +1,160 @@
|
||||
import fs from 'fs/promises';
|
||||
import path from 'path';
|
||||
import OpenAI from 'openai';
|
||||
import axios from 'axios';
|
||||
|
||||
// Constants
|
||||
const OPENROUTER_API_KEY = 'your_openrouter_api_key'; // Replace with your actual key
|
||||
const IMAGE_PATH = 'path/to/your/image.jpg'; // Replace with your image path
|
||||
|
||||
/**
|
||||
* Convert an image file to base64
|
||||
*/
|
||||
async function imageToBase64(filePath: string): Promise<string> {
|
||||
try {
|
||||
// Read the file
|
||||
const imageBuffer = await fs.readFile(filePath);
|
||||
|
||||
// Determine MIME type based on file extension
|
||||
const fileExt = path.extname(filePath).toLowerCase();
|
||||
let mimeType = 'application/octet-stream';
|
||||
|
||||
switch (fileExt) {
|
||||
case '.png':
|
||||
mimeType = 'image/png';
|
||||
break;
|
||||
case '.jpg':
|
||||
case '.jpeg':
|
||||
mimeType = 'image/jpeg';
|
||||
break;
|
||||
case '.webp':
|
||||
mimeType = 'image/webp';
|
||||
break;
|
||||
// Add other supported types as needed
|
||||
}
|
||||
|
||||
// Convert to base64 and add the data URI prefix
|
||||
const base64 = imageBuffer.toString('base64');
|
||||
return `data:${mimeType};base64,${base64}`;
|
||||
} catch (error) {
|
||||
console.error('Error converting image to base64:', error);
|
||||
throw error;
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Method 1: Send an image to OpenRouter using direct API call
|
||||
*/
|
||||
async function sendImageDirectAPI(base64Image: string, question: string = "What's in this image?"): Promise<void> {
|
||||
try {
|
||||
console.log('Sending image via direct API call...');
|
||||
|
||||
const response = await axios.post(
|
||||
'https://openrouter.ai/api/v1/chat/completions',
|
||||
{
|
||||
model: 'anthropic/claude-3-opus', // Choose an appropriate model with vision capabilities
|
||||
messages: [
|
||||
{
|
||||
role: 'user',
|
||||
content: [
|
||||
{
|
||||
type: 'text',
|
||||
text: question
|
||||
},
|
||||
{
|
||||
type: 'image_url',
|
||||
image_url: {
|
||||
url: base64Image
|
||||
}
|
||||
}
|
||||
]
|
||||
}
|
||||
]
|
||||
},
|
||||
{
|
||||
headers: {
|
||||
'Authorization': `Bearer ${OPENROUTER_API_KEY}`,
|
||||
'Content-Type': 'application/json',
|
||||
'HTTP-Referer': 'https://your-site-url.com', // Optional
|
||||
'X-Title': 'Your Site Name' // Optional
|
||||
}
|
||||
}
|
||||
);
|
||||
|
||||
console.log('Response from direct API:');
|
||||
console.log(response.data.choices[0].message.content);
|
||||
} catch (error) {
|
||||
console.error('Error sending image via direct API:', error);
|
||||
if (axios.isAxiosError(error) && error.response) {
|
||||
console.error('API error details:', error.response.data);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Method 2: Send an image to OpenRouter using OpenAI SDK
|
||||
*/
|
||||
async function sendImageOpenAISDK(base64Image: string, question: string = "What's in this image?"): Promise<void> {
|
||||
try {
|
||||
console.log('Sending image via OpenAI SDK...');
|
||||
|
||||
// Initialize the OpenAI client with OpenRouter base URL
|
||||
const openai = new OpenAI({
|
||||
apiKey: OPENROUTER_API_KEY,
|
||||
baseURL: 'https://openrouter.ai/api/v1',
|
||||
defaultHeaders: {
|
||||
'HTTP-Referer': 'https://your-site-url.com', // Optional
|
||||
'X-Title': 'Your Site Name' // Optional
|
||||
}
|
||||
});
|
||||
|
||||
// Create the message with text and image
|
||||
const completion = await openai.chat.completions.create({
|
||||
model: 'anthropic/claude-3-opus', // Choose an appropriate model with vision capabilities
|
||||
messages: [
|
||||
{
|
||||
role: 'user',
|
||||
content: [
|
||||
{
|
||||
type: 'text',
|
||||
text: question
|
||||
},
|
||||
{
|
||||
type: 'image_url',
|
||||
image_url: {
|
||||
url: base64Image
|
||||
}
|
||||
}
|
||||
]
|
||||
}
|
||||
]
|
||||
});
|
||||
|
||||
console.log('Response from OpenAI SDK:');
|
||||
console.log(completion.choices[0].message.content);
|
||||
} catch (error) {
|
||||
console.error('Error sending image via OpenAI SDK:', error);
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Main function to run the examples
|
||||
*/
|
||||
async function main() {
|
||||
try {
|
||||
// Convert the image to base64
|
||||
const base64Image = await imageToBase64(IMAGE_PATH);
|
||||
console.log('Image converted to base64 successfully');
|
||||
|
||||
// Example 1: Using direct API call
|
||||
await sendImageDirectAPI(base64Image);
|
||||
|
||||
// Example 2: Using OpenAI SDK
|
||||
await sendImageOpenAISDK(base64Image);
|
||||
} catch (error) {
|
||||
console.error('Error in main function:', error);
|
||||
}
|
||||
}
|
||||
|
||||
// Run the examples
|
||||
main();
|
||||
@@ -8,8 +8,33 @@ image:
|
||||
|
||||
entrypoint: ["node", "dist/index.js"]
|
||||
|
||||
startCommand:
|
||||
type: stdio
|
||||
configSchema:
|
||||
type: object
|
||||
properties:
|
||||
OPENROUTER_API_KEY:
|
||||
type: string
|
||||
description: OpenRouter API key for authentication
|
||||
OPENROUTER_DEFAULT_MODEL:
|
||||
type: string
|
||||
description: Default model to use if none specified in requests
|
||||
required: ["OPENROUTER_API_KEY"]
|
||||
commandFunction: |
|
||||
function getCommand(config) {
|
||||
return {
|
||||
command: "node",
|
||||
args: ["dist/index.js"],
|
||||
env: {
|
||||
OPENROUTER_API_KEY: config.OPENROUTER_API_KEY,
|
||||
OPENROUTER_DEFAULT_MODEL: config.OPENROUTER_DEFAULT_MODEL || "anthropic/claude-3.5-sonnet"
|
||||
}
|
||||
};
|
||||
}
|
||||
|
||||
build:
|
||||
dockerfile: Dockerfile
|
||||
dockerBuildPath: "."
|
||||
|
||||
publish:
|
||||
smithery: true
|
||||
|
||||
222
src/index.ts
222
src/index.ts
@@ -1,29 +1,44 @@
|
||||
#!/usr/bin/env node
|
||||
// OpenRouter Multimodal MCP Server
|
||||
// Supports both stdio (local) and Streamable HTTP (remote) transports
|
||||
|
||||
import { Server } from '@modelcontextprotocol/sdk/server/index.js';
|
||||
import { StdioServerTransport } from '@modelcontextprotocol/sdk/server/stdio.js';
|
||||
import { StreamableHTTPServerTransport } from '@modelcontextprotocol/sdk/server/streamableHttp.js';
|
||||
import { isInitializeRequest } from '@modelcontextprotocol/sdk/types.js';
|
||||
import express, { Request, Response } from 'express';
|
||||
|
||||
import { ToolHandlers } from './tool-handlers.js';
|
||||
|
||||
// Define the default model to use when none is specified
|
||||
const DEFAULT_MODEL = 'openai/gpt-4o-mini';
|
||||
|
||||
interface ServerOptions {
|
||||
apiKey?: string;
|
||||
defaultModel?: string;
|
||||
}
|
||||
|
||||
class OpenRouterMultimodalServer {
|
||||
private server: Server;
|
||||
private toolHandlers!: ToolHandlers; // Using definite assignment assertion
|
||||
private toolHandlers!: ToolHandlers;
|
||||
private apiKey: string;
|
||||
private defaultModel: string;
|
||||
|
||||
constructor() {
|
||||
// Get API key and default model from environment variables
|
||||
const apiKey = process.env.OPENROUTER_API_KEY;
|
||||
const defaultModel = process.env.OPENROUTER_DEFAULT_MODEL;
|
||||
constructor(options?: ServerOptions) {
|
||||
// Retrieve API key from options or environment variables
|
||||
this.apiKey = options?.apiKey || process.env.OPENROUTER_API_KEY || '';
|
||||
this.defaultModel = options?.defaultModel || process.env.OPENROUTER_DEFAULT_MODEL || DEFAULT_MODEL;
|
||||
|
||||
// Check if API key is provided
|
||||
if (!apiKey) {
|
||||
throw new Error('OPENROUTER_API_KEY environment variable is required');
|
||||
if (!this.apiKey) {
|
||||
throw new Error('OpenRouter API key is required. Provide it via options or OPENROUTER_API_KEY environment variable');
|
||||
}
|
||||
|
||||
// Initialize the server
|
||||
this.server = new Server(
|
||||
{
|
||||
name: 'openrouter-multimodal-server',
|
||||
version: '1.0.0',
|
||||
version: '1.5.0',
|
||||
},
|
||||
{
|
||||
capabilities: {
|
||||
@@ -38,8 +53,8 @@ class OpenRouterMultimodalServer {
|
||||
// Initialize tool handlers
|
||||
this.toolHandlers = new ToolHandlers(
|
||||
this.server,
|
||||
apiKey,
|
||||
defaultModel
|
||||
this.apiKey,
|
||||
this.defaultModel
|
||||
);
|
||||
|
||||
process.on('SIGINT', async () => {
|
||||
@@ -48,20 +63,187 @@ class OpenRouterMultimodalServer {
|
||||
});
|
||||
}
|
||||
|
||||
async run() {
|
||||
getServer(): Server {
|
||||
return this.server;
|
||||
}
|
||||
|
||||
getDefaultModel(): string {
|
||||
return this.toolHandlers.getDefaultModel() || this.defaultModel;
|
||||
}
|
||||
|
||||
async runStdio() {
|
||||
const transport = new StdioServerTransport();
|
||||
await this.server.connect(transport);
|
||||
console.error('OpenRouter Multimodal MCP server running on stdio');
|
||||
console.error('Using API key from environment variable');
|
||||
console.error('Note: To use OpenRouter Multimodal, add the API key to your environment variables:');
|
||||
console.error(' OPENROUTER_API_KEY=your-api-key');
|
||||
if (process.env.OPENROUTER_DEFAULT_MODEL) {
|
||||
console.error(` Using default model: ${process.env.OPENROUTER_DEFAULT_MODEL}`);
|
||||
} else {
|
||||
console.error(' No default model set. You will need to specify a model in each request.');
|
||||
console.error(`Using default model: ${this.getDefaultModel()}`);
|
||||
console.error('Server is ready to process tool calls. Waiting for input...');
|
||||
}
|
||||
|
||||
async runHttp(port: number) {
|
||||
const app = express();
|
||||
app.use(express.json());
|
||||
|
||||
// Store transports by session ID for stateful mode
|
||||
const transports: Record<string, StreamableHTTPServerTransport> = {};
|
||||
|
||||
// Health check endpoint
|
||||
app.get('/health', (_req: Request, res: Response) => {
|
||||
res.json({
|
||||
status: 'ok',
|
||||
server: 'openrouter-multimodal-mcp',
|
||||
version: '1.5.0',
|
||||
defaultModel: this.getDefaultModel()
|
||||
});
|
||||
});
|
||||
|
||||
// Main MCP endpoint - handles POST and GET
|
||||
app.all('/mcp', async (req: Request, res: Response) => {
|
||||
console.log(`[MCP] ${req.method} request received`);
|
||||
|
||||
// Handle GET for SSE stream (part of Streamable HTTP spec)
|
||||
if (req.method === 'GET') {
|
||||
console.log('[MCP] SSE stream requested');
|
||||
const sessionId = req.headers['mcp-session-id'] as string | undefined;
|
||||
|
||||
if (!sessionId || !transports[sessionId]) {
|
||||
res.status(400).json({ error: 'No active session. Send POST first.' });
|
||||
return;
|
||||
}
|
||||
|
||||
// Let the transport handle the SSE stream (no body for GET)
|
||||
const transport = transports[sessionId];
|
||||
await transport.handleRequest(req, res, undefined);
|
||||
return;
|
||||
}
|
||||
|
||||
// Handle DELETE for session termination
|
||||
if (req.method === 'DELETE') {
|
||||
const sessionId = req.headers['mcp-session-id'] as string | undefined;
|
||||
if (sessionId && transports[sessionId]) {
|
||||
await transports[sessionId].close();
|
||||
delete transports[sessionId];
|
||||
res.status(200).json({ message: 'Session terminated' });
|
||||
} else {
|
||||
res.status(404).json({ error: 'Session not found' });
|
||||
}
|
||||
return;
|
||||
}
|
||||
|
||||
// Handle POST requests
|
||||
if (req.method !== 'POST') {
|
||||
res.status(405).json({ error: 'Method not allowed' });
|
||||
return;
|
||||
}
|
||||
|
||||
const sessionId = req.headers['mcp-session-id'] as string | undefined;
|
||||
let transport: StreamableHTTPServerTransport;
|
||||
|
||||
// Check if this is an initialization request
|
||||
if (isInitializeRequest(req.body)) {
|
||||
console.log('[MCP] Initialize request - creating new session');
|
||||
|
||||
// Create new transport for stateless mode (no session persistence)
|
||||
transport = new StreamableHTTPServerTransport({
|
||||
sessionIdGenerator: () => crypto.randomUUID(),
|
||||
onsessioninitialized: (newSessionId) => {
|
||||
console.log(`[MCP] Session initialized: ${newSessionId}`);
|
||||
transports[newSessionId] = transport;
|
||||
}
|
||||
});
|
||||
|
||||
// Connect a new server instance for this transport
|
||||
const sessionServer = new Server(
|
||||
{ name: 'openrouter-multimodal-server', version: '1.5.0' },
|
||||
{ capabilities: { tools: {} } }
|
||||
);
|
||||
sessionServer.onerror = (error) => console.error('[MCP Session Error]', error);
|
||||
|
||||
// Initialize tool handlers for this session
|
||||
new ToolHandlers(sessionServer, this.apiKey, this.defaultModel);
|
||||
|
||||
await sessionServer.connect(transport);
|
||||
} else {
|
||||
// Existing session - find transport
|
||||
if (!sessionId || !transports[sessionId]) {
|
||||
res.status(400).json({
|
||||
error: 'Invalid or missing session ID',
|
||||
message: 'Send initialize request first to create a session'
|
||||
});
|
||||
return;
|
||||
}
|
||||
transport = transports[sessionId];
|
||||
}
|
||||
|
||||
// Handle the request - pass body for POST, omit for GET/DELETE
|
||||
try {
|
||||
await transport.handleRequest(req, res, req.body);
|
||||
} catch (error) {
|
||||
console.error('[MCP] Request handling error:', error);
|
||||
if (!res.headersSent) {
|
||||
res.status(500).json({ error: 'Internal server error' });
|
||||
}
|
||||
}
|
||||
});
|
||||
|
||||
// Legacy SSE endpoint for backwards compatibility
|
||||
app.get('/sse', (_req: Request, res: Response) => {
|
||||
res.status(410).json({
|
||||
error: 'SSE transport is deprecated',
|
||||
message: 'Use Streamable HTTP at /mcp endpoint instead'
|
||||
});
|
||||
});
|
||||
|
||||
// Start server
|
||||
app.listen(port, () => {
|
||||
console.log(`OpenRouter Multimodal MCP server running on HTTP port ${port}`);
|
||||
console.log(`MCP endpoint: http://localhost:${port}/mcp`);
|
||||
console.log(`Health check: http://localhost:${port}/health`);
|
||||
console.log(`Using default model: ${this.getDefaultModel()}`);
|
||||
});
|
||||
}
|
||||
}
|
||||
|
||||
const server = new OpenRouterMultimodalServer();
|
||||
server.run().catch(console.error);
|
||||
// Determine transport mode from environment or arguments
|
||||
const transportMode = process.env.MCP_TRANSPORT || 'http';
|
||||
const httpPort = parseInt(process.env.PORT || '3001', 10);
|
||||
|
||||
// Get MCP configuration if provided
|
||||
let mcpOptions: ServerOptions | undefined;
|
||||
|
||||
// Check for config argument
|
||||
if (process.argv.length > 2) {
|
||||
try {
|
||||
const configArg = process.argv.find(arg => arg.startsWith('--config='));
|
||||
if (configArg) {
|
||||
const configPath = configArg.split('=')[1];
|
||||
const configData = require(configPath);
|
||||
mcpOptions = {
|
||||
apiKey: configData.OPENROUTER_API_KEY || configData.apiKey,
|
||||
defaultModel: configData.OPENROUTER_DEFAULT_MODEL || configData.defaultModel
|
||||
};
|
||||
if (mcpOptions.apiKey) {
|
||||
console.error('Using API key from MCP configuration');
|
||||
}
|
||||
}
|
||||
|
||||
// Check for --stdio flag
|
||||
if (process.argv.includes('--stdio')) {
|
||||
const server = new OpenRouterMultimodalServer(mcpOptions);
|
||||
server.runStdio().catch(console.error);
|
||||
} else {
|
||||
const server = new OpenRouterMultimodalServer(mcpOptions);
|
||||
server.runHttp(httpPort);
|
||||
}
|
||||
} catch (error) {
|
||||
console.error('Error parsing configuration:', error);
|
||||
process.exit(1);
|
||||
}
|
||||
} else {
|
||||
// Default: HTTP mode unless MCP_TRANSPORT=stdio
|
||||
const server = new OpenRouterMultimodalServer(mcpOptions);
|
||||
if (transportMode === 'stdio') {
|
||||
server.runStdio().catch(console.error);
|
||||
} else {
|
||||
server.runHttp(httpPort);
|
||||
}
|
||||
}
|
||||
|
||||
@@ -15,8 +15,8 @@ import { handleChatCompletion, ChatCompletionToolRequest } from './tool-handlers
|
||||
import { handleSearchModels, SearchModelsToolRequest } from './tool-handlers/search-models.js';
|
||||
import { handleGetModelInfo, GetModelInfoToolRequest } from './tool-handlers/get-model-info.js';
|
||||
import { handleValidateModel, ValidateModelToolRequest } from './tool-handlers/validate-model.js';
|
||||
import { handleAnalyzeImage, AnalyzeImageToolRequest } from './tool-handlers/analyze-image.js';
|
||||
import { handleMultiImageAnalysis, MultiImageAnalysisToolRequest } from './tool-handlers/multi-image-analysis.js';
|
||||
import { handleAnalyzeImage, AnalyzeImageToolRequest } from './tool-handlers/analyze-image.js';
|
||||
|
||||
export class ToolHandlers {
|
||||
private server: Server;
|
||||
@@ -52,7 +52,7 @@ export class ToolHandlers {
|
||||
tools: [
|
||||
// Chat Completion Tool
|
||||
{
|
||||
name: 'chat_completion',
|
||||
name: 'mcp_openrouter_chat_completion',
|
||||
description: 'Send a message to OpenRouter.ai and get a response',
|
||||
inputSchema: {
|
||||
type: 'object',
|
||||
@@ -128,16 +128,16 @@ export class ToolHandlers {
|
||||
maxContextTokens: 200000
|
||||
},
|
||||
|
||||
// Image Analysis Tool
|
||||
// Single Image Analysis Tool
|
||||
{
|
||||
name: 'analyze_image',
|
||||
name: 'mcp_openrouter_analyze_image',
|
||||
description: 'Analyze an image using OpenRouter vision models',
|
||||
inputSchema: {
|
||||
type: 'object',
|
||||
properties: {
|
||||
image_path: {
|
||||
type: 'string',
|
||||
description: 'Path to the image file to analyze (must be an absolute path)',
|
||||
description: 'Path to the image file to analyze (can be an absolute file path, URL, or base64 data URL starting with "data:")',
|
||||
},
|
||||
question: {
|
||||
type: 'string',
|
||||
@@ -154,7 +154,7 @@ export class ToolHandlers {
|
||||
|
||||
// Multi-Image Analysis Tool
|
||||
{
|
||||
name: 'multi_image_analysis',
|
||||
name: 'mcp_openrouter_multi_image_analysis',
|
||||
description: 'Analyze multiple images at once with a single prompt and receive detailed responses',
|
||||
inputSchema: {
|
||||
type: 'object',
|
||||
@@ -167,7 +167,7 @@ export class ToolHandlers {
|
||||
properties: {
|
||||
url: {
|
||||
type: 'string',
|
||||
description: 'URL or data URL of the image (can be a file:// URL to read from local filesystem)',
|
||||
description: 'URL or data URL of the image (use http(s):// for web images, absolute file paths for local files, or data:image/xxx;base64,... for base64 encoded images)',
|
||||
},
|
||||
alt: {
|
||||
type: 'string',
|
||||
@@ -188,7 +188,7 @@ export class ToolHandlers {
|
||||
},
|
||||
model: {
|
||||
type: 'string',
|
||||
description: 'OpenRouter model to use (defaults to claude-3.5-sonnet if not specified)',
|
||||
description: 'OpenRouter model to use. If not specified, the system will use a free model with vision capabilities or the default model.',
|
||||
},
|
||||
},
|
||||
required: ['images', 'prompt'],
|
||||
@@ -294,21 +294,21 @@ export class ToolHandlers {
|
||||
|
||||
this.server.setRequestHandler(CallToolRequestSchema, async (request) => {
|
||||
switch (request.params.name) {
|
||||
case 'chat_completion':
|
||||
case 'mcp_openrouter_chat_completion':
|
||||
return handleChatCompletion({
|
||||
params: {
|
||||
arguments: request.params.arguments as unknown as ChatCompletionToolRequest
|
||||
}
|
||||
}, this.openai, this.defaultModel);
|
||||
|
||||
case 'analyze_image':
|
||||
case 'mcp_openrouter_analyze_image':
|
||||
return handleAnalyzeImage({
|
||||
params: {
|
||||
arguments: request.params.arguments as unknown as AnalyzeImageToolRequest
|
||||
}
|
||||
}, this.openai, this.defaultModel);
|
||||
|
||||
case 'multi_image_analysis':
|
||||
case 'mcp_openrouter_multi_image_analysis':
|
||||
return handleMultiImageAnalysis({
|
||||
params: {
|
||||
arguments: request.params.arguments as unknown as MultiImageAnalysisToolRequest
|
||||
@@ -344,4 +344,11 @@ export class ToolHandlers {
|
||||
}
|
||||
});
|
||||
}
|
||||
|
||||
/**
|
||||
* Get the default model configured for this server
|
||||
*/
|
||||
getDefaultModel(): string | undefined {
|
||||
return this.defaultModel;
|
||||
}
|
||||
}
|
||||
|
||||
@@ -1,8 +1,31 @@
|
||||
import path from 'path';
|
||||
import { promises as fs } from 'fs';
|
||||
import sharp from 'sharp';
|
||||
import fetch from 'node-fetch';
|
||||
import { McpError, ErrorCode } from '@modelcontextprotocol/sdk/types.js';
|
||||
import OpenAI from 'openai';
|
||||
import { findSuitableFreeModel } from './multi-image-analysis.js';
|
||||
|
||||
// Default model for image analysis
|
||||
const DEFAULT_FREE_MODEL = 'qwen/qwen2.5-vl-32b-instruct:free';
|
||||
|
||||
let sharp: any;
|
||||
try {
|
||||
sharp = require('sharp');
|
||||
} catch (e) {
|
||||
console.error('Warning: sharp module not available, using fallback image processing');
|
||||
// Mock implementation that just passes through the base64 data
|
||||
sharp = (buffer: Buffer) => ({
|
||||
metadata: async () => ({ width: 800, height: 600 }),
|
||||
resize: () => ({
|
||||
jpeg: () => ({
|
||||
toBuffer: async () => buffer
|
||||
})
|
||||
}),
|
||||
jpeg: () => ({
|
||||
toBuffer: async () => buffer
|
||||
})
|
||||
});
|
||||
}
|
||||
|
||||
export interface AnalyzeImageToolRequest {
|
||||
image_path: string;
|
||||
@@ -10,32 +33,110 @@ export interface AnalyzeImageToolRequest {
|
||||
model?: string;
|
||||
}
|
||||
|
||||
export async function handleAnalyzeImage(
|
||||
request: { params: { arguments: AnalyzeImageToolRequest } },
|
||||
openai: OpenAI,
|
||||
defaultModel?: string
|
||||
) {
|
||||
const args = request.params.arguments;
|
||||
|
||||
try {
|
||||
// Validate image path
|
||||
const imagePath = args.image_path;
|
||||
if (!path.isAbsolute(imagePath)) {
|
||||
throw new McpError(ErrorCode.InvalidParams, 'Image path must be absolute');
|
||||
/**
|
||||
* Normalizes a file path to be OS-neutral
|
||||
* Handles Windows backslashes, drive letters, etc.
|
||||
*/
|
||||
function normalizePath(filePath: string): string {
|
||||
// Skip normalization for URLs and data URLs
|
||||
if (filePath.startsWith('http://') ||
|
||||
filePath.startsWith('https://') ||
|
||||
filePath.startsWith('data:')) {
|
||||
return filePath;
|
||||
}
|
||||
|
||||
// Read image file
|
||||
const imageBuffer = await fs.readFile(imagePath);
|
||||
console.error(`Successfully read image buffer of size: ${imageBuffer.length}`);
|
||||
// Handle Windows paths and convert them to a format that's usable
|
||||
// First normalize the path according to the OS
|
||||
let normalized = path.normalize(filePath);
|
||||
|
||||
// Make sure any Windows backslashes are handled
|
||||
normalized = normalized.replace(/\\/g, '/');
|
||||
|
||||
return normalized;
|
||||
}
|
||||
|
||||
async function fetchImageAsBuffer(url: string): Promise<Buffer> {
|
||||
try {
|
||||
// Handle data URLs
|
||||
if (url.startsWith('data:')) {
|
||||
const matches = url.match(/^data:([A-Za-z-+\/]+);base64,(.+)$/);
|
||||
if (!matches || matches.length !== 3) {
|
||||
throw new Error('Invalid data URL');
|
||||
}
|
||||
return Buffer.from(matches[2], 'base64');
|
||||
}
|
||||
|
||||
// Normalize the path before proceeding
|
||||
const normalizedUrl = normalizePath(url);
|
||||
|
||||
// Handle file URLs
|
||||
if (normalizedUrl.startsWith('file://')) {
|
||||
const filePath = normalizedUrl.replace('file://', '');
|
||||
return await fs.readFile(filePath);
|
||||
}
|
||||
|
||||
// Handle http/https URLs
|
||||
if (normalizedUrl.startsWith('http://') || normalizedUrl.startsWith('https://')) {
|
||||
const response = await fetch(normalizedUrl);
|
||||
if (!response.ok) {
|
||||
throw new Error(`HTTP error! status: ${response.status}`);
|
||||
}
|
||||
return Buffer.from(await response.arrayBuffer());
|
||||
}
|
||||
|
||||
// Handle regular file paths
|
||||
try {
|
||||
return await fs.readFile(normalizedUrl);
|
||||
} catch (error: any) {
|
||||
// Try the original path as a fallback
|
||||
if (normalizedUrl !== url) {
|
||||
try {
|
||||
return await fs.readFile(url);
|
||||
} catch (secondError) {
|
||||
console.error(`Failed to read file with normalized path (${normalizedUrl}) and original path (${url})`);
|
||||
throw error; // Throw the original error
|
||||
}
|
||||
}
|
||||
throw error;
|
||||
}
|
||||
} catch (error) {
|
||||
console.error(`Error fetching image from ${url}:`, error);
|
||||
throw error;
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Processes an image with minimal processing when sharp isn't available
|
||||
*/
|
||||
async function processImageFallback(buffer: Buffer): Promise<string> {
|
||||
try {
|
||||
// Just return the buffer as base64 without processing
|
||||
return buffer.toString('base64');
|
||||
} catch (error) {
|
||||
console.error('Error in fallback image processing:', error);
|
||||
throw error;
|
||||
}
|
||||
}
|
||||
|
||||
async function processImage(buffer: Buffer): Promise<string> {
|
||||
try {
|
||||
if (typeof sharp !== 'function') {
|
||||
console.warn('Using fallback image processing (sharp not available)');
|
||||
return processImageFallback(buffer);
|
||||
}
|
||||
|
||||
// Get image metadata
|
||||
const metadata = await sharp(imageBuffer).metadata();
|
||||
console.error('Image metadata:', metadata);
|
||||
let metadata;
|
||||
try {
|
||||
metadata = await sharp(buffer).metadata();
|
||||
} catch (error) {
|
||||
console.warn('Error getting image metadata, using fallback:', error);
|
||||
return processImageFallback(buffer);
|
||||
}
|
||||
|
||||
// Calculate dimensions to keep base64 size reasonable
|
||||
const MAX_DIMENSION = 800; // Larger than original example for better quality
|
||||
const JPEG_QUALITY = 80; // Higher quality
|
||||
let resizedBuffer = imageBuffer;
|
||||
const MAX_DIMENSION = 800;
|
||||
const JPEG_QUALITY = 80;
|
||||
|
||||
if (metadata.width && metadata.height) {
|
||||
const largerDimension = Math.max(metadata.width, metadata.height);
|
||||
@@ -44,50 +145,200 @@ export async function handleAnalyzeImage(
|
||||
? { width: MAX_DIMENSION }
|
||||
: { height: MAX_DIMENSION };
|
||||
|
||||
resizedBuffer = await sharp(imageBuffer)
|
||||
const resizedBuffer = await sharp(buffer)
|
||||
.resize(resizeOptions)
|
||||
.jpeg({ quality: JPEG_QUALITY })
|
||||
.toBuffer();
|
||||
} else {
|
||||
resizedBuffer = await sharp(imageBuffer)
|
||||
|
||||
return resizedBuffer.toString('base64');
|
||||
}
|
||||
}
|
||||
|
||||
// If no resizing needed, just convert to JPEG
|
||||
const jpegBuffer = await sharp(buffer)
|
||||
.jpeg({ quality: JPEG_QUALITY })
|
||||
.toBuffer();
|
||||
|
||||
return jpegBuffer.toString('base64');
|
||||
} catch (error) {
|
||||
console.error('Error processing image, using fallback:', error);
|
||||
return processImageFallback(buffer);
|
||||
}
|
||||
}
|
||||
|
||||
// Convert to base64
|
||||
const base64Image = resizedBuffer.toString('base64');
|
||||
/**
|
||||
* Processes an image from a path or base64 string to a proper base64 format for APIs
|
||||
*/
|
||||
async function prepareImage(imagePath: string): Promise<{ base64: string; mimeType: string }> {
|
||||
try {
|
||||
// Check if already a base64 data URL
|
||||
if (imagePath.startsWith('data:')) {
|
||||
const matches = imagePath.match(/^data:([A-Za-z-+\/]+);base64,(.+)$/);
|
||||
if (!matches || matches.length !== 3) {
|
||||
throw new McpError(ErrorCode.InvalidParams, 'Invalid base64 data URL format');
|
||||
}
|
||||
return { base64: matches[2], mimeType: matches[1] };
|
||||
}
|
||||
|
||||
// Select model
|
||||
const model = args.model || defaultModel || 'anthropic/claude-3.5-sonnet';
|
||||
// Normalize the path first
|
||||
const normalizedPath = normalizePath(imagePath);
|
||||
|
||||
// Prepare message with image
|
||||
const messages = [
|
||||
{
|
||||
role: 'user',
|
||||
content: [
|
||||
// Check if image is a URL
|
||||
if (normalizedPath.startsWith('http://') || normalizedPath.startsWith('https://')) {
|
||||
try {
|
||||
const buffer = await fetchImageAsBuffer(normalizedPath);
|
||||
const processed = await processImage(buffer);
|
||||
return { base64: processed, mimeType: 'image/jpeg' }; // We convert everything to JPEG
|
||||
} catch (error: any) {
|
||||
throw new McpError(ErrorCode.InvalidParams, `Failed to fetch image from URL: ${error.message}`);
|
||||
}
|
||||
}
|
||||
|
||||
// Handle file paths
|
||||
let absolutePath = normalizedPath;
|
||||
|
||||
// For local file paths, ensure they are absolute
|
||||
// Don't check URLs or data URIs
|
||||
if (!normalizedPath.startsWith('data:') &&
|
||||
!normalizedPath.startsWith('http://') &&
|
||||
!normalizedPath.startsWith('https://')) {
|
||||
|
||||
if (!path.isAbsolute(normalizedPath)) {
|
||||
throw new McpError(ErrorCode.InvalidParams, 'Image path must be absolute');
|
||||
}
|
||||
|
||||
// For Windows paths that include a drive letter but aren't recognized as absolute
|
||||
// by path.isAbsolute in some environments
|
||||
if (/^[A-Za-z]:/.test(normalizedPath) && !path.isAbsolute(normalizedPath)) {
|
||||
absolutePath = path.resolve(normalizedPath);
|
||||
}
|
||||
}
|
||||
|
||||
try {
|
||||
// Check if the file exists
|
||||
await fs.access(absolutePath);
|
||||
} catch (error) {
|
||||
// Try the original path as a fallback
|
||||
try {
|
||||
await fs.access(imagePath);
|
||||
absolutePath = imagePath; // Use the original path if that works
|
||||
} catch (secondError) {
|
||||
throw new McpError(ErrorCode.InvalidParams, `File not found: ${absolutePath}`);
|
||||
}
|
||||
}
|
||||
|
||||
// Read the file as a buffer
|
||||
let buffer;
|
||||
try {
|
||||
buffer = await fs.readFile(absolutePath);
|
||||
} catch (error) {
|
||||
// Try the original path as a fallback
|
||||
try {
|
||||
buffer = await fs.readFile(imagePath);
|
||||
} catch (secondError) {
|
||||
throw new McpError(ErrorCode.InvalidParams, `Failed to read file: ${absolutePath}`);
|
||||
}
|
||||
}
|
||||
|
||||
// Determine MIME type from file extension
|
||||
const extension = path.extname(absolutePath).toLowerCase();
|
||||
let mimeType: string;
|
||||
|
||||
switch (extension) {
|
||||
case '.png':
|
||||
mimeType = 'image/png';
|
||||
break;
|
||||
case '.jpg':
|
||||
case '.jpeg':
|
||||
mimeType = 'image/jpeg';
|
||||
break;
|
||||
case '.webp':
|
||||
mimeType = 'image/webp';
|
||||
break;
|
||||
case '.gif':
|
||||
mimeType = 'image/gif';
|
||||
break;
|
||||
case '.bmp':
|
||||
mimeType = 'image/bmp';
|
||||
break;
|
||||
default:
|
||||
mimeType = 'application/octet-stream';
|
||||
}
|
||||
|
||||
// Process and optimize the image
|
||||
const processed = await processImage(buffer);
|
||||
return { base64: processed, mimeType };
|
||||
} catch (error) {
|
||||
console.error('Error preparing image:', error);
|
||||
throw error;
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Handler for analyzing a single image
|
||||
*/
|
||||
export async function handleAnalyzeImage(
|
||||
request: { params: { arguments: AnalyzeImageToolRequest } },
|
||||
openai: OpenAI,
|
||||
defaultModel?: string
|
||||
) {
|
||||
const args = request.params.arguments;
|
||||
|
||||
try {
|
||||
// Validate inputs
|
||||
if (!args.image_path) {
|
||||
throw new McpError(ErrorCode.InvalidParams, 'An image path, URL, or base64 data is required');
|
||||
}
|
||||
|
||||
const question = args.question || "What's in this image?";
|
||||
|
||||
console.error(`Processing image: ${args.image_path.substring(0, 100)}${args.image_path.length > 100 ? '...' : ''}`);
|
||||
|
||||
// Convert the image to base64
|
||||
const { base64, mimeType } = await prepareImage(args.image_path);
|
||||
|
||||
// Create the content array for the OpenAI API
|
||||
const content = [
|
||||
{
|
||||
type: 'text',
|
||||
text: args.question || "What's in this image?"
|
||||
text: question
|
||||
},
|
||||
{
|
||||
type: 'image_url',
|
||||
image_url: {
|
||||
url: `data:image/jpeg;base64,${base64Image}`
|
||||
url: `data:${mimeType};base64,${base64}`
|
||||
}
|
||||
}
|
||||
]
|
||||
}
|
||||
];
|
||||
|
||||
console.error('Sending request to OpenRouter...');
|
||||
// Select model with priority:
|
||||
// 1. User-specified model
|
||||
// 2. Default model from environment
|
||||
// 3. Default free vision model (qwen/qwen2.5-vl-32b-instruct:free)
|
||||
let model = args.model || defaultModel || DEFAULT_FREE_MODEL;
|
||||
|
||||
// Call OpenRouter API
|
||||
// If a model is specified but not our default free model, verify it exists
|
||||
if (model !== DEFAULT_FREE_MODEL) {
|
||||
try {
|
||||
await openai.models.retrieve(model);
|
||||
} catch (error) {
|
||||
console.error(`Specified model ${model} not found, falling back to auto-selection`);
|
||||
model = await findSuitableFreeModel(openai);
|
||||
}
|
||||
}
|
||||
|
||||
console.error(`Making API call with model: ${model}`);
|
||||
|
||||
// Make the API call
|
||||
const completion = await openai.chat.completions.create({
|
||||
model,
|
||||
messages: messages as any,
|
||||
messages: [{
|
||||
role: 'user',
|
||||
content
|
||||
}] as any
|
||||
});
|
||||
|
||||
// Return the analysis result
|
||||
return {
|
||||
content: [
|
||||
{
|
||||
@@ -95,9 +346,13 @@ export async function handleAnalyzeImage(
|
||||
text: completion.choices[0].message.content || '',
|
||||
},
|
||||
],
|
||||
metadata: {
|
||||
model: completion.model,
|
||||
usage: completion.usage
|
||||
}
|
||||
};
|
||||
} catch (error) {
|
||||
console.error('Error analyzing image:', error);
|
||||
console.error('Error in image analysis:', error);
|
||||
|
||||
if (error instanceof McpError) {
|
||||
throw error;
|
||||
@@ -111,6 +366,10 @@ export async function handleAnalyzeImage(
|
||||
},
|
||||
],
|
||||
isError: true,
|
||||
metadata: {
|
||||
error_type: error instanceof Error ? error.constructor.name : 'Unknown',
|
||||
error_message: error instanceof Error ? error.message : String(error)
|
||||
}
|
||||
};
|
||||
}
|
||||
}
|
||||
|
||||
@@ -66,6 +66,53 @@ function truncateMessagesToFit(
|
||||
return truncated;
|
||||
}
|
||||
|
||||
// Find a suitable free model with the largest context window
|
||||
async function findSuitableFreeModel(openai: OpenAI): Promise<string> {
|
||||
try {
|
||||
// Query available models with 'free' in their name
|
||||
const modelsResponse = await openai.models.list();
|
||||
if (!modelsResponse || !modelsResponse.data || modelsResponse.data.length === 0) {
|
||||
return 'deepseek/deepseek-chat-v3-0324:free'; // Fallback to a known model
|
||||
}
|
||||
|
||||
// Filter models with 'free' in ID
|
||||
const freeModels = modelsResponse.data
|
||||
.filter(model => model.id.includes('free'))
|
||||
.map(model => {
|
||||
// Try to extract context length from the model object
|
||||
let contextLength = 0;
|
||||
try {
|
||||
const modelAny = model as any; // Cast to any to access non-standard properties
|
||||
if (typeof modelAny.context_length === 'number') {
|
||||
contextLength = modelAny.context_length;
|
||||
} else if (modelAny.context_window) {
|
||||
contextLength = parseInt(modelAny.context_window, 10);
|
||||
}
|
||||
} catch (e) {
|
||||
console.error(`Error parsing context length for model ${model.id}:`, e);
|
||||
}
|
||||
|
||||
return {
|
||||
id: model.id,
|
||||
contextLength: contextLength || 0
|
||||
};
|
||||
});
|
||||
|
||||
if (freeModels.length === 0) {
|
||||
return 'deepseek/deepseek-chat-v3-0324:free'; // Fallback if no free models found
|
||||
}
|
||||
|
||||
// Sort by context length and pick the one with the largest context window
|
||||
freeModels.sort((a, b) => b.contextLength - a.contextLength);
|
||||
console.error(`Selected free model: ${freeModels[0].id} with context length: ${freeModels[0].contextLength}`);
|
||||
|
||||
return freeModels[0].id;
|
||||
} catch (error) {
|
||||
console.error('Error finding suitable free model:', error);
|
||||
return 'deepseek/deepseek-chat-v3-0324:free'; // Fallback to a known model
|
||||
}
|
||||
}
|
||||
|
||||
export async function handleChatCompletion(
|
||||
request: { params: { arguments: ChatCompletionToolRequest } },
|
||||
openai: OpenAI,
|
||||
@@ -73,20 +120,6 @@ export async function handleChatCompletion(
|
||||
) {
|
||||
const args = request.params.arguments;
|
||||
|
||||
// Validate model selection
|
||||
const model = args.model || defaultModel;
|
||||
if (!model) {
|
||||
return {
|
||||
content: [
|
||||
{
|
||||
type: 'text',
|
||||
text: 'No model specified and no default model configured in MCP settings. Please specify a model or set OPENROUTER_DEFAULT_MODEL in the MCP configuration.',
|
||||
},
|
||||
],
|
||||
isError: true,
|
||||
};
|
||||
}
|
||||
|
||||
// Validate message array
|
||||
if (args.messages.length === 0) {
|
||||
return {
|
||||
@@ -101,9 +134,22 @@ export async function handleChatCompletion(
|
||||
}
|
||||
|
||||
try {
|
||||
// Select model with priority:
|
||||
// 1. User-specified model
|
||||
// 2. Default model from environment
|
||||
// 3. Free model with the largest context window (selected automatically)
|
||||
let model = args.model || defaultModel;
|
||||
|
||||
if (!model) {
|
||||
model = await findSuitableFreeModel(openai);
|
||||
console.error(`Using auto-selected model: ${model}`);
|
||||
}
|
||||
|
||||
// Truncate messages to fit within context window
|
||||
const truncatedMessages = truncateMessagesToFit(args.messages, MAX_CONTEXT_TOKENS);
|
||||
|
||||
console.error(`Making API call with model: ${model}`);
|
||||
|
||||
const completion = await openai.chat.completions.create({
|
||||
model,
|
||||
messages: truncatedMessages,
|
||||
|
||||
@@ -1,7 +1,48 @@
|
||||
import fetch from 'node-fetch';
|
||||
import sharp from 'sharp';
|
||||
// Remove the sharp import to avoid conflicts with our dynamic import
|
||||
// import sharp from 'sharp';
|
||||
import { McpError, ErrorCode } from '@modelcontextprotocol/sdk/types.js';
|
||||
import OpenAI from 'openai';
|
||||
import path from 'path';
|
||||
import { promises as fs } from 'fs';
|
||||
import { tmpdir } from 'os';
|
||||
// Remove uuid import as we'll use a simple random string generator instead
|
||||
// import { v4 as uuidv4 } from 'uuid';
|
||||
|
||||
// Setup sharp with fallback
|
||||
let sharp: any;
|
||||
try {
|
||||
sharp = require('sharp');
|
||||
} catch (e) {
|
||||
console.error('Warning: sharp module not available, using fallback image processing');
|
||||
// Mock implementation that just passes through the base64 data
|
||||
sharp = (buffer: Buffer) => ({
|
||||
metadata: async () => ({ width: 800, height: 600 }),
|
||||
resize: () => ({
|
||||
jpeg: () => ({
|
||||
toBuffer: async () => buffer
|
||||
})
|
||||
}),
|
||||
jpeg: () => ({
|
||||
toBuffer: async () => buffer
|
||||
})
|
||||
});
|
||||
}
|
||||
|
||||
// Default model for image analysis
|
||||
const DEFAULT_FREE_MODEL = 'qwen/qwen2.5-vl-32b-instruct:free';
|
||||
|
||||
// Image processing constants
|
||||
const MAX_DIMENSION = 800;
|
||||
const JPEG_QUALITY = 80;
|
||||
const MAX_RETRY_ATTEMPTS = 3;
|
||||
const RETRY_DELAY = 1000; // ms
|
||||
|
||||
// Simple random ID generator to replace uuid
|
||||
function generateRandomId(): string {
|
||||
return Math.random().toString(36).substring(2, 15) +
|
||||
Math.random().toString(36).substring(2, 15);
|
||||
}
|
||||
|
||||
export interface MultiImageAnalysisToolRequest {
|
||||
images: Array<{
|
||||
@@ -13,45 +54,193 @@ export interface MultiImageAnalysisToolRequest {
|
||||
model?: string;
|
||||
}
|
||||
|
||||
/**
|
||||
* Sleep function for retry mechanisms
|
||||
*/
|
||||
const sleep = (ms: number) => new Promise(resolve => setTimeout(resolve, ms));
|
||||
|
||||
/**
|
||||
* Normalizes a file path to be OS-neutral
|
||||
* Handles Windows backslashes, drive letters, etc.
|
||||
*/
|
||||
function normalizePath(filePath: string): string {
|
||||
// Skip normalization for URLs and data URLs
|
||||
if (filePath.startsWith('http://') ||
|
||||
filePath.startsWith('https://') ||
|
||||
filePath.startsWith('data:')) {
|
||||
return filePath;
|
||||
}
|
||||
|
||||
// Handle Windows paths and convert them to a format that's usable
|
||||
// First normalize the path according to the OS
|
||||
let normalized = path.normalize(filePath);
|
||||
|
||||
// Make sure any Windows backslashes are handled
|
||||
normalized = normalized.replace(/\\/g, '/');
|
||||
|
||||
return normalized;
|
||||
}
|
||||
|
||||
/**
|
||||
* Get MIME type from file extension or data URL
|
||||
*/
|
||||
function getMimeType(url: string): string {
|
||||
if (url.startsWith('data:')) {
|
||||
const match = url.match(/^data:([^;]+);/);
|
||||
return match ? match[1] : 'application/octet-stream';
|
||||
}
|
||||
|
||||
const extension = path.extname(url.split('?')[0]).toLowerCase();
|
||||
|
||||
switch (extension) {
|
||||
case '.png': return 'image/png';
|
||||
case '.jpg':
|
||||
case '.jpeg': return 'image/jpeg';
|
||||
case '.webp': return 'image/webp';
|
||||
case '.gif': return 'image/gif';
|
||||
case '.bmp': return 'image/bmp';
|
||||
case '.svg': return 'image/svg+xml';
|
||||
default: return 'application/octet-stream';
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Fetch image from various sources: data URLs, file paths, or remote URLs
|
||||
*/
|
||||
async function fetchImageAsBuffer(url: string): Promise<Buffer> {
|
||||
try {
|
||||
// Handle data URLs
|
||||
if (url.startsWith('data:')) {
|
||||
const matches = url.match(/^data:([A-Za-z-+\/]+);base64,(.+)$/);
|
||||
if (!matches || matches.length !== 3) {
|
||||
throw new Error('Invalid data URL');
|
||||
throw new Error('Invalid data URL format');
|
||||
}
|
||||
return Buffer.from(matches[2], 'base64');
|
||||
}
|
||||
|
||||
// Handle file URLs
|
||||
if (url.startsWith('file://')) {
|
||||
const filePath = url.replace('file://', '');
|
||||
const fs = await import('fs/promises');
|
||||
// Normalize the path before proceeding
|
||||
const normalizedUrl = normalizePath(url);
|
||||
|
||||
// Handle file URLs with file:// protocol
|
||||
if (normalizedUrl.startsWith('file://')) {
|
||||
const filePath = normalizedUrl.replace('file://', '');
|
||||
try {
|
||||
return await fs.readFile(filePath);
|
||||
} catch (error) {
|
||||
console.error(`Error reading file at ${filePath}:`, error);
|
||||
throw new Error(`Failed to read file: ${filePath}`);
|
||||
}
|
||||
}
|
||||
|
||||
// Handle absolute and relative file paths
|
||||
if (normalizedUrl.startsWith('/') || normalizedUrl.startsWith('./') || normalizedUrl.startsWith('../') || /^[A-Za-z]:\\/.test(normalizedUrl) || /^[A-Za-z]:\//.test(normalizedUrl)) {
|
||||
try {
|
||||
// Try with normalized path
|
||||
return await fs.readFile(normalizedUrl);
|
||||
} catch (error) {
|
||||
// Fallback to original path if normalized path doesn't work
|
||||
if (normalizedUrl !== url) {
|
||||
try {
|
||||
return await fs.readFile(url);
|
||||
} catch (secondError) {
|
||||
console.error(`Failed to read file with both normalized path (${normalizedUrl}) and original path (${url})`);
|
||||
throw new Error(`Failed to read file: ${url}`);
|
||||
}
|
||||
}
|
||||
console.error(`Error reading file at ${normalizedUrl}:`, error);
|
||||
throw new Error(`Failed to read file: ${normalizedUrl}`);
|
||||
}
|
||||
}
|
||||
|
||||
// Handle http/https URLs
|
||||
const response = await fetch(url);
|
||||
if (normalizedUrl.startsWith('http://') || normalizedUrl.startsWith('https://')) {
|
||||
for (let attempt = 0; attempt < MAX_RETRY_ATTEMPTS; attempt++) {
|
||||
try {
|
||||
// Use AbortController for timeout instead of timeout option
|
||||
const controller = new AbortController();
|
||||
const timeoutId = setTimeout(() => controller.abort(), 15000);
|
||||
|
||||
const response = await fetch(normalizedUrl, {
|
||||
signal: controller.signal,
|
||||
headers: {
|
||||
'User-Agent': 'OpenRouter-MCP-Server/1.0'
|
||||
}
|
||||
});
|
||||
|
||||
// Clear the timeout to prevent memory leaks
|
||||
clearTimeout(timeoutId);
|
||||
|
||||
if (!response.ok) {
|
||||
throw new Error(`HTTP error! status: ${response.status}`);
|
||||
}
|
||||
|
||||
return Buffer.from(await response.arrayBuffer());
|
||||
} catch (error) {
|
||||
console.error(`Error fetching URL (attempt ${attempt + 1}/${MAX_RETRY_ATTEMPTS}): ${normalizedUrl}`, error);
|
||||
|
||||
if (attempt < MAX_RETRY_ATTEMPTS - 1) {
|
||||
// Exponential backoff with jitter
|
||||
const delay = RETRY_DELAY * Math.pow(2, attempt) * (0.5 + Math.random() * 0.5);
|
||||
await sleep(delay);
|
||||
} else {
|
||||
throw error;
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// If we get here, the URL format is unsupported
|
||||
throw new Error(`Unsupported URL format: ${url}`);
|
||||
} catch (error) {
|
||||
console.error(`Error fetching image from ${url}:`, error);
|
||||
throw error;
|
||||
}
|
||||
|
||||
// TypeScript requires a return statement here, but this is unreachable
|
||||
return Buffer.from([]);
|
||||
}
|
||||
|
||||
async function processImage(buffer: Buffer): Promise<string> {
|
||||
/**
|
||||
* Fallback image processing when sharp isn't available
|
||||
*/
|
||||
function processImageFallback(buffer: Buffer, mimeType: string): Promise<string> {
|
||||
return Promise.resolve(buffer.toString('base64'));
|
||||
}
|
||||
|
||||
/**
|
||||
* Process and optimize image for API consumption
|
||||
*/
|
||||
async function processImage(buffer: Buffer, mimeType: string): Promise<string> {
|
||||
try {
|
||||
// Get image metadata
|
||||
const metadata = await sharp(buffer).metadata();
|
||||
if (typeof sharp !== 'function') {
|
||||
console.warn('Using fallback image processing (sharp not available)');
|
||||
return processImageFallback(buffer, mimeType);
|
||||
}
|
||||
|
||||
// Calculate dimensions to keep base64 size reasonable
|
||||
const MAX_DIMENSION = 800;
|
||||
const JPEG_QUALITY = 80;
|
||||
// Create a temporary directory for processing if needed
|
||||
const tempDir = path.join(tmpdir(), `openrouter-mcp-${generateRandomId()}`);
|
||||
await fs.mkdir(tempDir, { recursive: true });
|
||||
|
||||
// Get image info
|
||||
let sharpInstance = sharp(buffer);
|
||||
let metadata;
|
||||
|
||||
try {
|
||||
metadata = await sharpInstance.metadata();
|
||||
} catch (error) {
|
||||
console.warn('Error getting image metadata, using fallback:', error);
|
||||
return processImageFallback(buffer, mimeType);
|
||||
}
|
||||
|
||||
// Skip processing for small images
|
||||
if (metadata.width && metadata.height &&
|
||||
metadata.width <= MAX_DIMENSION &&
|
||||
metadata.height <= MAX_DIMENSION &&
|
||||
(mimeType === 'image/jpeg' || mimeType === 'image/webp')) {
|
||||
return buffer.toString('base64');
|
||||
}
|
||||
|
||||
// Resize larger images
|
||||
if (metadata.width && metadata.height) {
|
||||
const largerDimension = Math.max(metadata.width, metadata.height);
|
||||
if (largerDimension > MAX_DIMENSION) {
|
||||
@@ -59,27 +248,101 @@ async function processImage(buffer: Buffer): Promise<string> {
|
||||
? { width: MAX_DIMENSION }
|
||||
: { height: MAX_DIMENSION };
|
||||
|
||||
const resizedBuffer = await sharp(buffer)
|
||||
.resize(resizeOptions)
|
||||
.jpeg({ quality: JPEG_QUALITY })
|
||||
.toBuffer();
|
||||
|
||||
return resizedBuffer.toString('base64');
|
||||
sharpInstance = sharpInstance.resize(resizeOptions);
|
||||
}
|
||||
}
|
||||
|
||||
// If no resizing needed, just convert to JPEG
|
||||
const jpegBuffer = await sharp(buffer)
|
||||
try {
|
||||
// Convert to JPEG for consistency and small size
|
||||
const processedBuffer = await sharpInstance
|
||||
.jpeg({ quality: JPEG_QUALITY })
|
||||
.toBuffer();
|
||||
|
||||
return jpegBuffer.toString('base64');
|
||||
return processedBuffer.toString('base64');
|
||||
} catch (error) {
|
||||
console.error('Error processing image:', error);
|
||||
throw error;
|
||||
console.warn('Error in final image processing, using fallback:', error);
|
||||
return processImageFallback(buffer, mimeType);
|
||||
}
|
||||
} catch (error) {
|
||||
console.error('Error processing image, using fallback:', error);
|
||||
return processImageFallback(buffer, mimeType);
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Find a suitable free model with vision capabilities, defaulting to Qwen
|
||||
*/
|
||||
export async function findSuitableFreeModel(openai: OpenAI): Promise<string> {
|
||||
try {
|
||||
// First try with an exact match for our preferred model
|
||||
const preferredModel = DEFAULT_FREE_MODEL;
|
||||
|
||||
try {
|
||||
// Check if our preferred model is available
|
||||
const modelInfo = await openai.models.retrieve(preferredModel);
|
||||
if (modelInfo && modelInfo.id) {
|
||||
console.error(`Using preferred model: ${preferredModel}`);
|
||||
return preferredModel;
|
||||
}
|
||||
} catch (error) {
|
||||
console.error(`Preferred model ${preferredModel} not available, searching for alternatives...`);
|
||||
}
|
||||
|
||||
// Query available models
|
||||
const modelsResponse = await openai.models.list();
|
||||
if (!modelsResponse?.data || modelsResponse.data.length === 0) {
|
||||
console.error('No models found, using default fallback model');
|
||||
return DEFAULT_FREE_MODEL;
|
||||
}
|
||||
|
||||
// First, try to find free vision models
|
||||
const freeVisionModels = modelsResponse.data
|
||||
.filter(model => {
|
||||
const modelId = model.id.toLowerCase();
|
||||
return modelId.includes('free') &&
|
||||
(modelId.includes('vl') || modelId.includes('vision') || modelId.includes('claude') ||
|
||||
modelId.includes('gemini') || modelId.includes('gpt-4') || modelId.includes('qwen'));
|
||||
})
|
||||
.map(model => {
|
||||
// Extract context length if available
|
||||
let contextLength = 0;
|
||||
try {
|
||||
const modelAny = model as any;
|
||||
if (typeof modelAny.context_length === 'number') {
|
||||
contextLength = modelAny.context_length;
|
||||
} else if (modelAny.context_window) {
|
||||
contextLength = parseInt(modelAny.context_window, 10);
|
||||
}
|
||||
} catch (e) {
|
||||
console.error(`Error parsing context length for model ${model.id}:`, e);
|
||||
}
|
||||
|
||||
return {
|
||||
id: model.id,
|
||||
contextLength: contextLength || 0
|
||||
};
|
||||
});
|
||||
|
||||
if (freeVisionModels.length > 0) {
|
||||
// Sort by context length and pick the one with the largest context window
|
||||
freeVisionModels.sort((a, b) => b.contextLength - a.contextLength);
|
||||
const selectedModel = freeVisionModels[0].id;
|
||||
console.error(`Selected free vision model: ${selectedModel} with context length: ${freeVisionModels[0].contextLength}`);
|
||||
return selectedModel;
|
||||
}
|
||||
|
||||
// If no free vision models found, fallback to our default
|
||||
console.error('No free vision models found, using default fallback model');
|
||||
return DEFAULT_FREE_MODEL;
|
||||
} catch (error) {
|
||||
console.error('Error finding suitable model:', error);
|
||||
return DEFAULT_FREE_MODEL;
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Main handler for multi-image analysis
|
||||
*/
|
||||
export async function handleMultiImageAnalysis(
|
||||
request: { params: { arguments: MultiImageAnalysisToolRequest } },
|
||||
openai: OpenAI,
|
||||
@@ -89,47 +352,87 @@ export async function handleMultiImageAnalysis(
|
||||
|
||||
try {
|
||||
// Validate inputs
|
||||
if (!args.images || args.images.length === 0) {
|
||||
if (!args.images || !Array.isArray(args.images) || args.images.length === 0) {
|
||||
throw new McpError(ErrorCode.InvalidParams, 'At least one image is required');
|
||||
}
|
||||
|
||||
if (!args.prompt) {
|
||||
throw new McpError(ErrorCode.InvalidParams, 'A prompt is required');
|
||||
throw new McpError(ErrorCode.InvalidParams, 'A prompt for analyzing the images is required');
|
||||
}
|
||||
|
||||
// Prepare content array for the message
|
||||
const content: Array<any> = [{
|
||||
console.error(`Processing ${args.images.length} images`);
|
||||
|
||||
// Process each image and convert to base64 if needed
|
||||
const processedImages = await Promise.all(
|
||||
args.images.map(async (image, index) => {
|
||||
try {
|
||||
// Skip processing if already a data URL
|
||||
if (image.url.startsWith('data:')) {
|
||||
console.error(`Image ${index + 1} is already in base64 format`);
|
||||
return image;
|
||||
}
|
||||
|
||||
console.error(`Processing image ${index + 1}: ${image.url.substring(0, 100)}${image.url.length > 100 ? '...' : ''}`);
|
||||
|
||||
// Get MIME type
|
||||
const mimeType = getMimeType(image.url);
|
||||
|
||||
// Fetch and process the image
|
||||
const buffer = await fetchImageAsBuffer(image.url);
|
||||
const base64 = await processImage(buffer, mimeType);
|
||||
|
||||
return {
|
||||
url: `data:${mimeType === 'application/octet-stream' ? 'image/jpeg' : mimeType};base64,${base64}`,
|
||||
alt: image.alt
|
||||
};
|
||||
} catch (error: any) {
|
||||
console.error(`Error processing image ${index + 1}:`, error);
|
||||
throw new Error(`Failed to process image ${index + 1}: ${image.url}. Error: ${error.message}`);
|
||||
}
|
||||
})
|
||||
);
|
||||
|
||||
// Select model with priority:
|
||||
// 1. User-specified model
|
||||
// 2. Default model from environment
|
||||
// 3. Default free vision model
|
||||
let model = args.model || defaultModel || DEFAULT_FREE_MODEL;
|
||||
|
||||
// If a model is specified but not our default free model, verify it exists
|
||||
if (model !== DEFAULT_FREE_MODEL) {
|
||||
try {
|
||||
await openai.models.retrieve(model);
|
||||
} catch (error) {
|
||||
console.error(`Specified model ${model} not found, falling back to auto-selection`);
|
||||
model = await findSuitableFreeModel(openai);
|
||||
}
|
||||
}
|
||||
|
||||
console.error(`Making API call with model: ${model}`);
|
||||
|
||||
// Build content array for the API call
|
||||
const content: Array<{
|
||||
type: string;
|
||||
text?: string;
|
||||
image_url?: {
|
||||
url: string
|
||||
}
|
||||
}> = [
|
||||
{
|
||||
type: 'text',
|
||||
text: args.prompt
|
||||
}];
|
||||
}
|
||||
];
|
||||
|
||||
// Process each image
|
||||
for (const image of args.images) {
|
||||
try {
|
||||
// Fetch and process the image
|
||||
const imageBuffer = await fetchImageAsBuffer(image.url);
|
||||
const base64Image = await processImage(imageBuffer);
|
||||
|
||||
// Add to content
|
||||
// Add each processed image to the content array
|
||||
processedImages.forEach(image => {
|
||||
content.push({
|
||||
type: 'image_url',
|
||||
image_url: {
|
||||
url: `data:image/jpeg;base64,${base64Image}`
|
||||
url: image.url
|
||||
}
|
||||
});
|
||||
} catch (error) {
|
||||
console.error(`Error processing image ${image.url}:`, error);
|
||||
// Continue with other images if one fails
|
||||
}
|
||||
}
|
||||
|
||||
// If no images were successfully processed
|
||||
if (content.length === 1) {
|
||||
throw new Error('Failed to process any of the provided images');
|
||||
}
|
||||
|
||||
// Select model
|
||||
const model = args.model || defaultModel || 'anthropic/claude-3.5-sonnet';
|
||||
});
|
||||
|
||||
// Make the API call
|
||||
const completion = await openai.chat.completions.create({
|
||||
@@ -140,15 +443,35 @@ export async function handleMultiImageAnalysis(
|
||||
}] as any
|
||||
});
|
||||
|
||||
// Get response text and format if requested
|
||||
let responseText = completion.choices[0].message.content || '';
|
||||
|
||||
// Format as markdown if requested
|
||||
if (args.markdown_response) {
|
||||
// Simple formatting enhancements
|
||||
responseText = responseText
|
||||
// Add horizontal rule after sections
|
||||
.replace(/^(#{1,3}.*)/gm, '$1\n\n---')
|
||||
// Ensure proper spacing for lists
|
||||
.replace(/^(\s*[-*•]\s.+)$/gm, '\n$1')
|
||||
// Convert plain URLs to markdown links
|
||||
.replace(/(https?:\/\/[^\s]+)/g, '[$1]($1)');
|
||||
}
|
||||
|
||||
// Return the analysis result
|
||||
return {
|
||||
content: [
|
||||
{
|
||||
type: 'text',
|
||||
text: completion.choices[0].message.content || '',
|
||||
text: responseText,
|
||||
},
|
||||
],
|
||||
metadata: {
|
||||
model: completion.model,
|
||||
usage: completion.usage
|
||||
}
|
||||
};
|
||||
} catch (error) {
|
||||
} catch (error: any) {
|
||||
console.error('Error in multi-image analysis:', error);
|
||||
|
||||
if (error instanceof McpError) {
|
||||
@@ -159,10 +482,14 @@ export async function handleMultiImageAnalysis(
|
||||
content: [
|
||||
{
|
||||
type: 'text',
|
||||
text: `Error analyzing images: ${error instanceof Error ? error.message : String(error)}`,
|
||||
text: `Error analyzing images: ${error.message}`,
|
||||
},
|
||||
],
|
||||
isError: true,
|
||||
metadata: {
|
||||
error_type: error.constructor.name,
|
||||
error_message: error.message
|
||||
}
|
||||
};
|
||||
}
|
||||
}
|
||||
|
||||
65
start-server.js
Normal file
65
start-server.js
Normal file
@@ -0,0 +1,65 @@
|
||||
// Load environment variables and start the MCP server
|
||||
import { promises as fs } from 'fs';
|
||||
import path from 'path';
|
||||
import { fileURLToPath } from 'url';
|
||||
import { dirname } from 'path';
|
||||
import { spawn } from 'child_process';
|
||||
|
||||
// Get current directory
|
||||
const __filename = fileURLToPath(import.meta.url);
|
||||
const __dirname = dirname(__filename);
|
||||
|
||||
// Path to .env file
|
||||
const envPath = path.join(__dirname, '.env');
|
||||
|
||||
async function loadEnvAndStartServer() {
|
||||
try {
|
||||
console.log('Loading environment variables from .env file...');
|
||||
|
||||
// Read .env file
|
||||
const envContent = await fs.readFile(envPath, 'utf8');
|
||||
|
||||
// Parse .env file and set environment variables
|
||||
const envVars = {};
|
||||
envContent.split('\n').forEach(line => {
|
||||
const match = line.match(/^\s*([\w.-]+)\s*=\s*(.*)?\s*$/);
|
||||
if (match) {
|
||||
const key = match[1];
|
||||
let value = match[2] || '';
|
||||
|
||||
// Remove quotes if they exist
|
||||
if (value.length > 0 && value.charAt(0) === '"' && value.charAt(value.length - 1) === '"') {
|
||||
value = value.replace(/^"|"$/g, '');
|
||||
}
|
||||
|
||||
envVars[key] = value;
|
||||
process.env[key] = value;
|
||||
}
|
||||
});
|
||||
|
||||
console.log('Environment variables loaded successfully');
|
||||
console.log(`API Key found: ${process.env.OPENROUTER_API_KEY ? 'Yes' : 'No'}`);
|
||||
|
||||
// Start the server process with environment variables
|
||||
console.log('Starting MCP server...');
|
||||
const serverProcess = spawn('node', ['dist/index.js'], {
|
||||
env: { ...process.env, ...envVars },
|
||||
stdio: 'inherit'
|
||||
});
|
||||
|
||||
// Handle server process events
|
||||
serverProcess.on('close', (code) => {
|
||||
console.log(`MCP server exited with code ${code}`);
|
||||
});
|
||||
|
||||
serverProcess.on('error', (err) => {
|
||||
console.error('Failed to start MCP server:', err);
|
||||
});
|
||||
|
||||
} catch (error) {
|
||||
console.error('Error:', error);
|
||||
}
|
||||
}
|
||||
|
||||
// Run the function
|
||||
loadEnvAndStartServer();
|
||||
155
test-openai-sdk.js
Normal file
155
test-openai-sdk.js
Normal file
@@ -0,0 +1,155 @@
|
||||
import 'dotenv/config';
|
||||
import { promises as fs } from 'fs';
|
||||
import path from 'path';
|
||||
import { fileURLToPath } from 'url';
|
||||
import { dirname } from 'path';
|
||||
import { OpenAI } from 'openai';
|
||||
|
||||
// Get the directory name for ES modules
|
||||
const __filename = fileURLToPath(import.meta.url);
|
||||
const __dirname = dirname(__filename);
|
||||
|
||||
// Constants
|
||||
const TEST_IMAGE_PATH = 'test.png'; // Adjust to your image path
|
||||
|
||||
/**
|
||||
* Convert an image file to base64
|
||||
*/
|
||||
async function imageToBase64(filePath) {
|
||||
try {
|
||||
// Read the file
|
||||
const imageBuffer = await fs.readFile(filePath);
|
||||
|
||||
// Determine MIME type based on file extension
|
||||
const fileExt = path.extname(filePath).toLowerCase();
|
||||
let mimeType = 'application/octet-stream';
|
||||
|
||||
switch (fileExt) {
|
||||
case '.png':
|
||||
mimeType = 'image/png';
|
||||
break;
|
||||
case '.jpg':
|
||||
case '.jpeg':
|
||||
mimeType = 'image/jpeg';
|
||||
break;
|
||||
case '.webp':
|
||||
mimeType = 'image/webp';
|
||||
break;
|
||||
default:
|
||||
console.log(`Using default MIME type for extension: ${fileExt}`);
|
||||
}
|
||||
|
||||
// Convert to base64 and add the data URI prefix
|
||||
const base64 = imageBuffer.toString('base64');
|
||||
return `data:${mimeType};base64,${base64}`;
|
||||
} catch (error) {
|
||||
console.error('Error converting image to base64:', error);
|
||||
throw error;
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Send an image to OpenRouter using OpenAI SDK
|
||||
*/
|
||||
async function analyzeImageWithOpenRouter(base64Image, question = "What's in this image?") {
|
||||
try {
|
||||
console.log('Initializing OpenAI client with OpenRouter...');
|
||||
|
||||
// Initialize the OpenAI client with OpenRouter base URL
|
||||
const openai = new OpenAI({
|
||||
apiKey: process.env.OPENROUTER_API_KEY,
|
||||
baseURL: 'https://openrouter.ai/api/v1',
|
||||
defaultHeaders: {
|
||||
'HTTP-Referer': 'https://github.com/stabgan/openrouter-mcp-multimodal',
|
||||
'X-Title': 'OpenRouter MCP Test'
|
||||
}
|
||||
});
|
||||
|
||||
console.log('Sending image for analysis to Qwen free model...');
|
||||
// Create the message with text and image
|
||||
const completion = await openai.chat.completions.create({
|
||||
model: 'qwen/qwen2.5-vl-32b-instruct:free', // Using Qwen free model with vision capabilities
|
||||
messages: [
|
||||
{
|
||||
role: 'user',
|
||||
content: [
|
||||
{
|
||||
type: 'text',
|
||||
text: question
|
||||
},
|
||||
{
|
||||
type: 'image_url',
|
||||
image_url: {
|
||||
url: base64Image
|
||||
}
|
||||
}
|
||||
]
|
||||
}
|
||||
]
|
||||
});
|
||||
|
||||
// Debug the completion response structure
|
||||
console.log('\n----- Debug: API Response -----');
|
||||
console.log(JSON.stringify(completion, null, 2));
|
||||
console.log('----- End Debug -----\n');
|
||||
|
||||
// Check if completion has expected structure before accessing properties
|
||||
if (completion && completion.choices && completion.choices.length > 0 && completion.choices[0].message) {
|
||||
console.log('\n----- Analysis Results -----\n');
|
||||
console.log(completion.choices[0].message.content);
|
||||
console.log('\n----------------------------\n');
|
||||
|
||||
// Print additional information about the model used and token usage
|
||||
console.log('Model used:', completion.model);
|
||||
if (completion.usage) {
|
||||
console.log('Token usage:');
|
||||
console.log('- Prompt tokens:', completion.usage.prompt_tokens);
|
||||
console.log('- Completion tokens:', completion.usage.completion_tokens);
|
||||
console.log('- Total tokens:', completion.usage.total_tokens);
|
||||
}
|
||||
} else {
|
||||
console.log('Unexpected response structure from OpenRouter API.');
|
||||
}
|
||||
|
||||
return completion;
|
||||
} catch (error) {
|
||||
console.error('Error analyzing image with OpenRouter:');
|
||||
if (error.response) {
|
||||
console.error('API error status:', error.status);
|
||||
console.error('API error details:', JSON.stringify(error.response, null, 2));
|
||||
} else if (error.cause) {
|
||||
console.error('Error cause:', error.cause);
|
||||
} else {
|
||||
console.error(error);
|
||||
}
|
||||
throw error;
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Main function
|
||||
*/
|
||||
async function main() {
|
||||
try {
|
||||
if (!process.env.OPENROUTER_API_KEY) {
|
||||
throw new Error('OPENROUTER_API_KEY not found in environment variables. Create a .env file with your API key.');
|
||||
}
|
||||
|
||||
console.log(`Converting image at ${TEST_IMAGE_PATH} to base64...`);
|
||||
const base64Image = await imageToBase64(TEST_IMAGE_PATH);
|
||||
console.log('Image converted successfully!');
|
||||
|
||||
// Log the first 100 chars of the base64 string to verify format
|
||||
console.log('Base64 string preview:', base64Image.substring(0, 100) + '...');
|
||||
|
||||
// Analyze the image
|
||||
await analyzeImageWithOpenRouter(base64Image, "Please describe this image in detail. What do you see?");
|
||||
|
||||
} catch (error) {
|
||||
console.error('Error in main function:', error);
|
||||
process.exit(1);
|
||||
}
|
||||
}
|
||||
|
||||
// Run the script
|
||||
main();
|
||||
1
test_base64.txt
Normal file
1
test_base64.txt
Normal file
File diff suppressed because one or more lines are too long
94
test_mcp_server.js
Normal file
94
test_mcp_server.js
Normal file
@@ -0,0 +1,94 @@
|
||||
// Test MCP server with image analysis
|
||||
import { promises as fs } from 'fs';
|
||||
import path from 'path';
|
||||
|
||||
// Path to test image
|
||||
const IMAGE_PATH = process.argv[2] || 'test.png';
|
||||
|
||||
// Function to convert image to base64
|
||||
async function imageToBase64(imagePath) {
|
||||
try {
|
||||
// Read the file
|
||||
const imageBuffer = await fs.readFile(imagePath);
|
||||
|
||||
// Determine MIME type based on file extension
|
||||
const fileExt = path.extname(imagePath).toLowerCase();
|
||||
let mimeType = 'application/octet-stream';
|
||||
|
||||
switch (fileExt) {
|
||||
case '.png':
|
||||
mimeType = 'image/png';
|
||||
break;
|
||||
case '.jpg':
|
||||
case '.jpeg':
|
||||
mimeType = 'image/jpeg';
|
||||
break;
|
||||
case '.webp':
|
||||
mimeType = 'image/webp';
|
||||
break;
|
||||
default:
|
||||
mimeType = 'image/png'; // Default to PNG
|
||||
}
|
||||
|
||||
// Convert to base64 and add the data URI prefix
|
||||
const base64 = imageBuffer.toString('base64');
|
||||
return `data:${mimeType};base64,${base64}`;
|
||||
} catch (error) {
|
||||
console.error('Error converting image to base64:', error);
|
||||
throw error;
|
||||
}
|
||||
}
|
||||
|
||||
// Main function to test the MCP server
|
||||
async function main() {
|
||||
try {
|
||||
console.log(`Converting image: ${IMAGE_PATH}`);
|
||||
|
||||
// Check if the image file exists
|
||||
try {
|
||||
await fs.access(IMAGE_PATH);
|
||||
console.log(`Image file exists: ${IMAGE_PATH}`);
|
||||
} catch (err) {
|
||||
console.error(`Error: Image file does not exist: ${IMAGE_PATH}`);
|
||||
return;
|
||||
}
|
||||
|
||||
// Convert the image to base64
|
||||
const base64Image = await imageToBase64(IMAGE_PATH);
|
||||
console.log('Image converted to base64 successfully.');
|
||||
console.log(`Base64 length: ${base64Image.length} characters`);
|
||||
|
||||
// Create the request for analyze_image
|
||||
const analyzeImageRequest = {
|
||||
jsonrpc: '2.0',
|
||||
id: '1',
|
||||
method: 'mcp/call_tool',
|
||||
params: {
|
||||
tool: 'mcp_openrouter_analyze_image',
|
||||
arguments: {
|
||||
image_path: base64Image,
|
||||
question: "What's in this image?",
|
||||
model: 'qwen/qwen2.5-vl-32b-instruct:free'
|
||||
}
|
||||
}
|
||||
};
|
||||
|
||||
// Send the request to the MCP server's stdin
|
||||
console.log('Sending request to MCP server...');
|
||||
process.stdout.write(JSON.stringify(analyzeImageRequest) + '\n');
|
||||
|
||||
// The MCP server will write the response to stdout, which we can read
|
||||
console.log('Waiting for response...');
|
||||
|
||||
// In a real application, you would read from the server's stdout stream
|
||||
// Here we just wait for input to be processed by the MCP server
|
||||
console.log('Request sent to MCP server. Check the server logs for the response.');
|
||||
} catch (error) {
|
||||
console.error('Error in main function:', error);
|
||||
}
|
||||
}
|
||||
|
||||
// Run the main function
|
||||
main().catch(error => {
|
||||
console.error("Unhandled error in main:", error);
|
||||
});
|
||||
Reference in New Issue
Block a user