MCP Service User Guide

·2026-08-08 00:30:49

MCP Service User Guide

Expose AI Agent phone automation capabilities as standard tools via MCP (Model Context Protocol), for use by external AI Agents such as OpenClaw, Qoder, Claude Desktop, etc.

Provides two access channels:

Channel Transport Supported Clients Status
A. Local MCP Server (Recommended) stdio JSON-RPC + WebSocket relay OpenClaw / Qoder / Claude Desktop and any standard MCP client ✅ Stable
B. WebMCP (Browser Native) document.modelContext Chrome built-in AI (Gemini Nano) and other in-page Agents 🧪 Experimental (Chrome 149+ Origin Trial)

Table of Contents

Architecture Overview

┌──────────────────────────────────────────────────────────────────────┐
│                        External AI Agent                             │
│              (OpenClaw / Qoder / Claude Desktop / Chrome AI)         │
└───────────┬──────────────────────────────────────────┬───────────────┘
            │ Channel A: stdio JSON-RPC 2.0            │ Channel B:
            ▼                                          │ document.modelContext
┌───────────────────────────────┐                      │   .callTool()
│  mcp-server/dist/server.js    │                      ▼
│  (Node.js McpServer)          │      ┌───────────────────────────────┐
│  6 Tools registered on stdio  │      │   Chrome WebMCP Runtime       │
└───────────┬───────────────────┘      │   (#enable-webmcp-testing)    │
            │ WebSocket                └───────────────┬───────────────┘
            │ ws://localhost:7788                      │ execute()
            ▼                                          ▼
┌──────────────────────────────────────────────────────────────────────┐
│                    Browser Page (Mobile AI Use)                       │
│   McpRelayClient (Channel A)    WebMcpService (Channel B)            │
│                └────────────┬────────────┘                           │
│                             ▼  McpServiceContext (shared)             │
│            Plan-and-Resolve Agent / ReAct Agent                      │
└─────────────────────────────┬────────────────────────────────────────┘
                              │  Web ADB
                              ▼
┌──────────────────────────────────────────────────────────────────────┐
│                       Android / iOS Device                           │
└──────────────────────────────────────────────────────────────────────┘

Both channels ultimately share the same McpServiceContext, with identical tool semantics; the only differences are in the transport layer and some return formats (see Tool Definitions).

Flow Diagrams

Channel A: Local MCP Server Call Flow

sequenceDiagram
    participant Agent as External Agent
(OpenClaw/Qoder)
    participant MCP as mcp-server
(Node.js, stdio)
    participant WS as WebSocket Relay
ws://localhost:7788
    participant Relay as McpRelayClient
(Browser)
    participant Panel as AiChatPanel
    participant Device as Phone Device

    Note over MCP,Relay: Relay auto-connects after page opens
(reconnects every 30s on disconnect)
    Agent->>MCP: tools/call run_agent_task {goal}
    MCP->>WS: {id, tool, args} (JSON)
    WS->>Relay: Forward request
    Relay->>Relay: Check device connection & Agent idle
    Relay->>Panel: runAgentForMcp(goal, signal)
    Panel->>Device: Plan-and-Resolve: screenshot → vision analysis → touch
    Device-->>Panel: Operation result
    Panel-->>Relay: AgentTask (finished)
    Relay->>WS: {id, result: Markdown report}
    WS->>MCP: Match pending request
    MCP-->>Agent: CallToolResult

Channel B: WebMCP Service Registration Flow

flowchart TD
    A[Start] --> B[Mobile AI Use]
    B --> C{document.modelContext exists?}
    C -->|No| D[WebMCP not registered
Relay client only]
    C -->|Yes| E[Create WebMcpService instance]
    E --> F[register: register 6 Tools]
    F --> G[External in-page Agent can discover and call]
    D --> H[McpRelayClient.connect]
    G --> H
    H --> I{Connected to ws://localhost:7788?}
    I -->|Yes| J[Relay badge green]
    I -->|No| K[Relay badge gray
Auto-retry in 30s]

Task Execution State Machine

stateDiagram-v2
    [*] --> idle: Initialize
    idle --> planning: run_agent_task triggered
    planning --> executing: Plan complete
    executing --> finished: All steps succeeded / Scroll extraction complete
    executing --> failed: Step execution failed
    executing --> aborted: abort_task called / Timeout
    finished --> [*]
    failed --> [*]
    aborted --> [*]

💡 Special termination condition: When the Agent executes a scroll extraction successfully, the extraction result is returned directly as the task conclusion.

Channel A: Local MCP Server (Recommended)

Project Download

mcp-server download

Directory Structure

mcp-server/
├── src/server.ts       ← McpServer + WebSocket relay implementation
├── dist/server.js      ← Build output (MCP client config points to this file)
├── package.json        ← name: phone-agent-mcp
└── tsconfig.json

How It Works

  1. MCP Protocol Layer: Uses @modelcontextprotocol/sdk's McpServer + StdioServerTransport, communicating with MCP clients via stdio JSON-RPC 2.0; Server name is phone-agent
  2. WebSocket Relay Layer: Embedded WebSocketServer (default port 7788), the browser page's McpRelayClient connects actively
  3. Call Forwarding: Upon receiving tools/call, sends {id, tool, args} to the browser via callBrowser(), waits for the browser to return {id, result}, then returns to the MCP client
  4. Connection Self-Check: 3 seconds after the browser connects, the Server automatically calls get_device_status once to verify the link
  5. Single Connection Model: Only one browser connection is supported at a time; new connections replace old ones

Startup Steps

cd mcp-server
npm install       # Install dependencies on first run
npm run build     # Compile TypeScript → dist/server.js
npm start         # Start (stdio + WebSocket :7788)
# Development mode (no-build hot execution):
npm run dev       # tsx src/server.ts

Then:

  1. Open the Mobile AI Use page (e.g. https://mobile-ai-use.com/mobileAi), McpRelayClient auto-connects to the relay after page load
  2. Confirm the MCP Relay badge is green at the top of the AI chat panel
  3. Connect an Android phone via USB and confirm screen mirroring is working

Environment Variables

Variable Default Description
PHONE_AGENT_WS_PORT 7788 WebSocket relay port. After modification, the browser-side McpRelayClient connection URL must be updated accordingly

MCP Client Configuration

OpenClaw (openclaw.json):

{
  mcp: {
    servers: {
      "phone-agent": {
        command: "node",
        // Replace <PROJECT_ROOT> with the actual absolute path
        args: ["<PROJECT_ROOT>/mcp-server/dist/server.js"],
        env: { PHONE_AGENT_WS_PORT: "7788" }
      }
    }
  }
}

Or via CLI:

openclaw mcp set phone-agent --command node --args "<PROJECT_ROOT>/mcp-server/dist/server.js"

Qoder (.qoder/mcp.json or global MCP config):

{
  "mcpServers": {
    "phone-agent": {
      "command": "node",
      "args": ["<PROJECT_ROOT>/mcp-server/dist/server.js"],
      "env": { "PHONE_AGENT_WS_PORT": "7788" }
    }
  }
}

Claude Desktop (claude_desktop_config.json):

{
  "mcpServers": {
    "phone-agent": {
      "command": "node",
      "args": ["<PROJECT_ROOT>/mcp-server/dist/server.js"]
    }
  }
}

WebSocket Relay Protocol

// Server → Browser (Request)
interface WsRequest {
    id: string;                      // Unique request ID (timestamp + random string)
    tool: string;                    // Tool name, e.g. 'run_agent_task'
    args: Record<string, unknown>;   // Tool parameters
}

// Browser → Server (Response)
interface WsResponse {
    id: string;                      // Corresponds to request ID
    result: McpToolResult;           // { content: [...], isError? }
}

// Browser → Server (Intermediate progress, only pushed for run_agent_task)
interface WsProgress {
    id: string;                      // Corresponds to request ID
    type: 'progress';                // Server uses this field to distinguish from final response
    seq: number;                     // Progress sequence number, incrementing from 1
    message: string;                 // Human-readable process description
}

Multiple WsProgress messages may appear on the same id before a final WsResponse closes it. When the Server receives a progress frame, it only forwards — it does not end the request or reset the timeout.

Real-Time Agent Execution Progress Reporting

During run_agent_task execution, the browser pushes the Agent's thinking and execution process to the Server in real time, which then converts it to MCP notifications for external clients (OpenClaw / Qoder / Claude Desktop) to display, preventing the "frozen" feeling during long tasks.

Reporting Points (corresponding to Agent lifecycle callbacks in runAgent):

Point Reported Content
onPlanned 📋 Execution plan generated (N steps total) + step title list
onStepStart ▶️ Step N: Title + step description
onStepText Buffered, not directly reported (triggered per token; direct passthrough would flood the channel)
onToolCall First flush buffered thinking text (💭 ...), then report 🔧 Calling tool name(args)
onToolResult → Result (truncated to 300 characters)
onStepEnd Flush remaining thinking + ✅/❌/⏭️ Step N completed (status, duration Xs)
onError ❌ Agent execution error: ...

Server-Side Forwarding Strategy (mcp-server/src/server.ts):

  • Client declares willingness to receive progress via _meta.progressToken in the request → send notifications/progress (progress takes seq, message takes the process text)
  • No progressToken declared → falls back to notifications/message (logging, level: info, logger: phone-agent), ensuring process visibility
  • Notification send failures are logged to stderr only — Agent task execution is never interrupted

Channel B (WebMCP) has no corresponding notification mechanism; the onProgress parameter is optional and not passed on the WebMCP side.

Browser-Side McpRelayClient

File: relay-client.ts

  • Auto connect() on page load, auto-reconnects every 30 seconds on disconnect
  • Shares the exact same McpServiceContext as WebMCP; tool dispatch logic centralized in _dispatch()
  • onStatusChange(connected) callback drives UI badge state
  • run_agent_task return value is formatted by _formatTaskMarkdown() into a readable Markdown report (task summary + per-step title/status/duration/tool call list) for easy reference by external Agents
  • _sendProgress() maintains incrementing sequence per reqId; silently ignored on disconnect or send error (progress display is supplementary — must not affect the main flow)

Channel B: WebMCP (Browser Native)

🧪 Experimental capability, depends on Chrome Origin Trial, suitable for in-page Agents (e.g. Chrome built-in AI). External process clients should use Channel A.

Browser Requirements

Item Requirement
Browser Chrome 149+ (Canary / Dev Channel)
Flag chrome://flags/#enable-webmcp-testing → Enabled
Protocol Page must be accessed via HTTPS or localhost
API Entry document.modelContext (legacy navigator.modelContext is deprecated; code uses it only as fallback)

Enable Steps

  1. Enable WebMCP Flag: Enter chrome://flags/#enable-webmcp-testing in the address bar, set to Enabled, then restart the browser
  2. Verify API Availability:
    // DevTools Console
console.log(typeof document.modelContext);              // "object"
console.log(typeof document.modelContext.registerTool); // "function"
  3. Open the WebADB Page and Connect Phone: On page load, WebMcpService.register() automatically registers 6 Tools (console shows [WebMCP] 6 tools registered log)

Tool Capability Overview

Both channels register tools with identical names and parameters:

# Tool Name Type Description Server-Side Timeout (Channel A)
1 run_agent_task Write Run AI Agent automation task on the phone timeoutMs + 5s (default 20 min)
2 abort_task Write Abort the currently running Agent task 10s
3 get_task_result Read Query detailed task results by taskId 10s
4 get_latest_task Read Get the latest task result in the current conversation 10s
5 take_screenshot Read Phone screen screenshot (returns PNG base64) 30s
6 get_device_status Read Get connected device information 15s

In Channel B (WebMCP), run_agent_task has a requiresUserApproval: true annotation — Chrome will show a user confirmation dialog. In Channel A, the corresponding destructiveHint: true annotation means confirmation depends on the MCP client's policy.

Tool Definitions

1. run_agent_task

Run an AI Agent automation task on the connected phone. The Agent uses a Plan-and-Resolve architecture, achieving goals through screen visual understanding and touch operations.

Parameters:

Parameter Type Required Description
goal string ✅ Natural language task goal, e.g. "Open WeChat and send a message to Zhang San: see you tomorrow"
timeoutMs number ❌ Maximum execution time (ms). Channel A default 1200000 (20 min), Channel B default 120000 (2 min)

Pre-Checks: Returns No phone device connected. if device is not connected; returns Agent is already running. Please wait or call abort_task first. if Agent is running.

Return Format (Channel A — Markdown Report):

## ✅ Agent Task Completed

**Goal**: Open WeChat and send a message to Zhang San
**Status**: finished  **Steps**: 4/4 succeeded  **Duration**: 32.5s

**Conclusion**: WeChat message sent successfully

---

### ✅ Step 1: Open WeChat (5.2s)
- `tap({"x":540,"y":1200})` → tapped at (540,1200)
...

Return Format (Channel B — JSON):

{
  "content": [{
    "type": "text",
    "text": "{\"taskId\":\"abc123\",\"status\":\"finished\",\"conclusion\":\"WeChat message sent successfully\",\"stepsCompleted\":4,\"totalSteps\":4,\"durationMs\":32500,\"error\":null}"
  }]
}

💡 Scroll Content Extraction Task: If the goal involves "extract list content / scroll capture", the Agent will call the ExtractScrollingContent tool (auto-scroll screenshots → stitch long image → AI analysis). After extraction completes, the task auto-terminates, with the extraction summary returned as conclusion.

2. abort_task

Abort the currently running Agent task. No parameters.

Return Example:

{ "content": [{ "type": "text", "text": "Task aborted." }] }

Returns No task is currently running. when no task is running.

3. get_task_result

Query detailed results of a completed task by taskId (including per-step execution records).

Parameters:

Parameter Type Required Description
taskId string ✅ Task ID returned by run_agent_task
includeSteps boolean ❌ Whether to include per-step details (default true)

Return Example:

{
  "content": [{
    "type": "text",
    "text": "{\"taskId\":\"abc123\",\"goal\":\"Open WeChat and send message\",\"status\":\"finished\",\"conclusion\":\"Task completed\",\"stepsCompleted\":3,\"totalSteps\":3,\"durationMs\":28000,\"steps\":[{\"index\":1,\"title\":\"Open WeChat\",\"status\":\"success\",\"durationMs\":5200,\"error\":null}]}"
  }]
}

4. get_latest_task

Get the full result (including steps) of the most recent Agent task in the current conversation. No parameters. Return format is the same as get_task_result.

5. take_screenshot

Take a screenshot of the connected phone screen, returns a PNG image (base64 encoded). No parameters.

Return Example:

{
  "content": [{
    "type": "image",
    "data": "iVBORw0KGgoAAAANSUhEUg...",
    "mimeType": "image/png"
  }]
}

6. get_device_status

Get basic information about the currently connected phone device. No parameters.

Return Example:

{
  "content": [{
    "type": "text",
    "text": "{\"connected\":true,\"model\":\"Pixel 7\",\"brand\":\"google\",\"release\":\"14\",\"sdk\":\"34\"}"
  }]
}

Returns {"connected": false} when no device is connected.

Skill Configuration

The project includes a built-in Skill definition file: /skills/phone-agent/SKILL.md download, which can be directly recognized and loaded by OpenClaw / Qoder.

---
name: phone-agent
description: Invoke WebADB AI Agent via local MCP Server to execute automation tasks on the connected phone
tools:
  - mcp__phone-agent__run_agent_task
  - mcp__phone-agent__abort_task
  - mcp__phone-agent__take_screenshot
  - mcp__phone-agent__get_device_status
  - mcp__phone-agent__get_task_result
  - mcp__phone-agent__get_latest_task
---

After configuration, simply describe your task in natural language:

  • "Open WeChat, find the conversation with Zhang San, and send a 'Hello'"
  • "Take a screenshot to see what's currently on the phone screen"
  • "Open Xiaohongshu and search for deepseek, extract the titles and authors of the top 10 notes"
  • "Abort the currently running task"
  • "What was the result of the most recent Agent task"

Typical tool call sequence:

get_device_status          # Confirm device is online
run_agent_task {goal}      # Dispatch task (blocks until complete, returns Markdown report)
get_latest_task            # Supplementary query when structured data is needed

Integration Code Details

Core Files

mcp-server/src/server.ts                ← Local MCP Server (Channel A server)
skills/phone-agent/SKILL.md             ← Skill definition

McpServiceContext Interface

Method Return Type Description
runAgent(goal, signal) Promise<AgentTask | void> Execute Agent task
getTask(taskId) AgentTask | undefined Query task by ID
getAllTasks() AgentTask[] All tasks in current conversation
screenshot() Promise<string> Screenshot (data URL)
deviceInfo() Promise<DeviceInfo> Device information
isRunning() boolean Whether Agent is running
abort() void Abort current task
isConnected() boolean Whether device is connected

Key Implementation Details

  1. execute Property Name (Channel B): WebMCP spec requires tool handler functions to be named execute (not handler)
  2. document.modelContext: The API is mounted on document. Earlier versions mounted on navigator; Chrome still retains it but outputs a navigator.modelContext is deprecated warning on access; WebMcpService.getModelContext() prioritizes document, only falling back to navigator after checking with in (directly reading the property would trigger the warning)
  3. Concurrency Protection: run_agent_task checks isRunning() before execution — only one Agent task allowed at a time
  4. Timeout Control: Browser-side via AbortController + setTimeout; Channel A Server-side has an additional timeoutMs + 5s fallback timeout, returning isError: true on timeout instead of hanging
  5. Reconnection: McpRelayClient retries every 30 seconds after disconnect; on the Server side, duplicate browser connections cause the old connection to be replaced
  6. Request Pairing: The relay layer uses unique id to pair requests/responses in the pending Map; browser disconnect does not affect timeout handling for other pending requests

UI Status Indicators

The AI Agent chat panel provides a MCP Relay badge at the top, indicating the connection status between the browser and the local MCP Server (Channel A):

Status Style Meaning
🟢 Green breathing animation mcp-badge--ready Relay connected — OpenClaw / Qoder can call the phone Agent
⚪ Gray mcp-badge--off Relay not connected (local MCP Server not running)

Hover tooltip:

  • Connected: MCP Relay connected: OpenClaw / Qoder can call the phone Agent
  • Not connected: MCP Relay disconnected: please start the local MCP server (cd mcp-server && npm start)

Channel B (WebMCP) registration results are confirmed via console log: on success, outputs [WebMCP] 6 tools registered: ...; when the environment is unsupported, outputs a warn message.

FAQ

Channel A (Local MCP Server)

Q: Tool call returns "Browser not connected. Open the WebADB page first."

Cause: The browser page is not open, or McpRelayClient hasn't connected to the relay.

Troubleshooting:

  1. Confirm mcp-server is running (terminal should show WebSocket relay on ws://localhost:7788)
  2. Open the WebADB page and confirm the Relay badge is green
  3. If the Server was just started, wait up to 30 seconds (browser auto-reconnect interval) or refresh the page

Q: Relay badge stays gray

Troubleshooting:

  1. Is the local MCP Server running: cd mcp-server && npm start
  2. Has the port been changed: PHONE_AGENT_WS_PORT and the browser-side connection URL (default ws://localhost:7788) must match
  3. If the port is occupied, switch to another port and update the env in the MCP client configuration accordingly

Q: Tool call times out (Tool "run_agent_task" timed out after ...ms)

Cause: Agent task execution time exceeds timeoutMs (Server-side fallback is timeoutMs + 5s).

Solution: For complex tasks, explicitly pass a larger timeoutMs; or call abort_task to interrupt, then split the task and retry.

Q: Multiple Mobile-AI-Use pages open, causing abnormal behavior

Cause: The relay only supports one browser connection at a time; the last connected page replaces previous connections.

Solution: Keep only one Mobile-AI-Use page tab open.

Channel B (WebMCP)

Q: Error Failed to read the 'execute' property from 'ModelContextTool'

Cause: The tool definition uses handler instead of execute as the handler function property name.

mc.registerTool({
  name: 'my_tool',
  inputSchema: { ... },
  execute: async (params) => { ... },  // ✅ Correct
  // handler: async (params) => { ... },  // ❌ Wrong
});

Q: document.modelContext is undefined

Troubleshooting Steps:

  1. Confirm Chrome version ≥ 149 (chrome://version)
  2. Confirm chrome://flags/#enable-webmcp-testing is enabled
  3. Confirm the page is accessed via HTTPS or localhost
  4. Restart the browser and retry

Q: External Agent cannot see registered Tools

Possible Causes:

  • The page is not the active tab (Chrome may suspend MCP services for background tabs)
  • isAvailable returned false during registration (check console warn logs)
  • Origin mismatch (the external Agent's configured origin must match the page URL)

General

Q: run_agent_task returns "Agent is already running"

Cause: Only one Agent task is allowed to run at a time.

Solution: Call abort_task first to interrupt the current task, then re-dispatch.

Q: Screenshot returns blank or errors

Troubleshooting:

  1. Confirm the device is connected via USB and screen mirroring is working
  2. Confirm get_device_status returns connected: true
  3. Check if the Scrcpy service is running normally

Appendix: AgentTask Data Structure

interface AgentTask {
  id: string;              // Unique task ID
  goal: string;            // User goal (natural language)
  steps: AgentStep[];      // Step list from the Plan phase
  status: 'idle' | 'planning' | 'executing' | 'finished' | 'failed' | 'aborted';
  conclusion?: string;     // Final conclusion (extraction summary for scroll extraction tasks)
  error?: string;          // Error message
  createdAt: number;       // Creation timestamp
  startedAt?: number;      // Execution start timestamp
  finishedAt?: number;     // Completion timestamp
}

interface AgentStep {
  id: string;
  index: number;           // Step number (starting from 1)
  title: string;           // Step title
  description: string;     // Detailed description
  summary?: string;        // Execution summary (referenced in Markdown reports)
  status: 'pending' | 'running' | 'success' | 'failed' | 'skipped';
  toolCalls: ToolCallRecord[];  // Tool call records
  beforeScreenshot?: string;    // Pre-execution screenshot
  afterScreenshot?: string;     // Post-execution screenshot
  error?: string;
  startedAt?: number;
  finishedAt?: number;
}

目录