### Fetch as Markdown Example Source: https://github.com/zcaceres/fetch-mcp/blob/main/_autodocs/api-reference/CLI.md Example command to fetch content and return it as Markdown. ```bash mcp-fetch markdown https://example.com/article ``` -------------------------------- ### CLI Examples Source: https://github.com/zcaceres/fetch-mcp/blob/main/README.md Examples of using the fetch-mcp CLI for various tasks. ```bash # Fetch a page as markdown mcp-fetch markdown https://example.com # Extract article content without boilerplate mcp-fetch readable https://example.com/blog/post # Get a YouTube transcript in Spanish mcp-fetch youtube https://www.youtube.com/watch?v=dQw4w9WgXcQ --lang es # Fetch with a length limit mcp-fetch html https://example.com --max-length 10000 # Fetch through a proxy mcp-fetch json https://api.example.com/data --proxy http://proxy:8080 ``` -------------------------------- ### CLI Installation (Global) Source: https://github.com/zcaceres/fetch-mcp/blob/main/README.md Install the fetch-mcp CLI globally. ```bash npm install -g mcp-fetch-server mcp-fetch [flags] ``` -------------------------------- ### Using with Proxy Example Source: https://github.com/zcaceres/fetch-mcp/blob/main/_autodocs/api-reference/CLI.md Example command showing how to use the --proxy flag. ```bash mcp-fetch markdown https://example.com --proxy http://proxy.internal:8080 ``` -------------------------------- ### Example Output Source: https://github.com/zcaceres/fetch-mcp/blob/main/_autodocs/api-reference/CLI.md Example of successful output from the CLI. ```bash $ mcp-fetch html https://example.com 2>/dev/null ... ``` -------------------------------- ### CLI Usage Examples Source: https://github.com/zcaceres/fetch-mcp/blob/main/_autodocs/configuration.md Examples demonstrating how to use the mcp-fetch CLI with various flags for different fetching scenarios. ```bash # Fetch with custom limit and proxy mcp-fetch markdown https://example.com --max-length 10000 --proxy http://proxy:8080 # Fetch YouTube transcript in Spanish with 20000 character limit mcp-fetch youtube https://www.youtube.com/watch?v=dQw4w9WgXcQ --lang es --max-length 20000 # Get help or version mcp-fetch --help mcp-fetch --version ``` -------------------------------- ### Fetch HTML Example Source: https://github.com/zcaceres/fetch-mcp/blob/main/_autodocs/api-reference/CLI.md Example command to fetch raw HTML from a URL. ```bash mcp-fetch html https://example.com ``` -------------------------------- ### Start MCP Server Source: https://github.com/zcaceres/fetch-mcp/blob/main/_autodocs/api-reference/Overview.md Starts the stdio server for MCP clients. ```bash npx mcp-fetch-server # Starts stdio server for MCP clients ``` -------------------------------- ### Complex Example: Pagination Source: https://github.com/zcaceres/fetch-mcp/blob/main/_autodocs/api-reference/CLI.md Example demonstrating how to fetch content in chunks using --max-length and --start-index flags. ```bash # Fetch first 5000 characters of HTML mcp-fetch html https://example.com --max-length 5000 --start-index 0 # Fetch next 5000 characters mcp-fetch html https://example.com --max-length 5000 --start-index 5000 ``` -------------------------------- ### Custom Configuration Example Source: https://github.com/zcaceres/fetch-mcp/blob/main/_autodocs/configuration.md Example of how to configure environment variables DEFAULT_LIMIT and MAX_RESPONSE_BYTES in an mcpServers configuration. ```json { "mcpServers": { "fetch": { "command": "npx", "args": ["mcp-fetch-server"], "env": { "DEFAULT_LIMIT": "50000", "MAX_RESPONSE_BYTES": "52428800" } } } } ``` -------------------------------- ### Install fetch-mcp Source: https://github.com/zcaceres/fetch-mcp/blob/main/_autodocs/modules.md Command to install fetch-mcp as an NPM package. ```bash npm install mcp-fetch-server ``` -------------------------------- ### Global Installation Source: https://github.com/zcaceres/fetch-mcp/blob/main/_autodocs/api-reference/CLI.md Installs mcp-fetch-server globally using npm and demonstrates its usage. ```bash npm install -g mcp-fetch-server # Now can use from anywhere mcp-fetch html https://example.com ``` -------------------------------- ### Show Help Example Source: https://github.com/zcaceres/fetch-mcp/blob/main/_autodocs/api-reference/CLI.md Command to display the help message. ```bash mcp-fetch --help ``` -------------------------------- ### Fetch YouTube Transcript Examples Source: https://github.com/zcaceres/fetch-mcp/blob/main/_autodocs/api-reference/CLI.md Examples for fetching YouTube transcripts, including default language, specific language, and with a character limit. ```bash # English (default) mcp-fetch youtube https://www.youtube.com/watch?v=dQw4w9WgXcQ # Spanish mcp-fetch youtube https://www.youtube.com/watch?v=dQw4w9WgXcQ --lang es # With character limit mcp-fetch youtube https://www.youtube.com/watch?v=dQw4w9WgXcQ --max-length 20000 ``` -------------------------------- ### Fetch Plain Text Example Source: https://github.com/zcaceres/fetch-mcp/blob/main/_autodocs/api-reference/CLI.md Example command to fetch raw plain text from a URL. ```bash mcp-fetch txt https://example.com ``` -------------------------------- ### fetch_html Example Response (Success) Source: https://github.com/zcaceres/fetch-mcp/blob/main/_autodocs/api-reference/MCP-Server.md Example success response for the fetch_html tool. ```json { "content": [{ "type": "text", "text": "..." }], "isError": false } ``` -------------------------------- ### Fetch Article Content (Readability) Example Source: https://github.com/zcaceres/fetch-mcp/blob/main/_autodocs/api-reference/CLI.md Example command to fetch article content and return it as Markdown using Readability. ```bash mcp-fetch readable https://example.com/blog/post ``` -------------------------------- ### Error Handling Example Source: https://github.com/zcaceres/fetch-mcp/blob/main/_autodocs/QUICK-REFERENCE.md Example of how to check for and handle errors from Fetcher methods. ```typescript const result = await Fetcher.html({ url: "..." }); if (result.isError) { console.error(result.content[0].text); } ``` -------------------------------- ### Running MCP Server Source: https://github.com/zcaceres/fetch-mcp/blob/main/_autodocs/modules.md Instructions for installing and configuring fetch-mcp to run as an MCP server. ```bash npm install mcp-fetch-server # Then in your MCP client config: { "mcpServers": { "fetch": { "command": "npx", "args": ["mcp-fetch-server"] } } } ``` -------------------------------- ### fetch_json Example Response (Success) Source: https://github.com/zcaceres/fetch-mcp/blob/main/_autodocs/api-reference/MCP-Server.md Example success response for the fetch_json tool. ```json { "content": [{ "type": "text", "text": "{\"key\": \"value\", ...}" }], "isError": false } ``` -------------------------------- ### Pagination with start_index Source: https://github.com/zcaceres/fetch-mcp/blob/main/_autodocs/api-reference/Overview.md Example demonstrating how to paginate content using the start_index and max_length options. ```typescript // First page (0-5000 chars) const page1 = await Fetcher.html({ url: "https://example.com", max_length: 5000, start_index: 0 }); // Second page (5000-10000 chars) const page2 = await Fetcher.html({ url: "https://example.com", max_length: 5000, start_index: 5000 }); ``` -------------------------------- ### Show Version Example Source: https://github.com/zcaceres/fetch-mcp/blob/main/_autodocs/api-reference/CLI.md Command to display the CLI version. ```bash mcp-fetch --version ``` -------------------------------- ### fetch_html Example Response (Error) Source: https://github.com/zcaceres/fetch-mcp/blob/main/_autodocs/api-reference/MCP-Server.md Example error response for the fetch_html tool. ```json { "content": [{ "type": "text", "text": "Failed to fetch https://example.com: HTTP error: 404" }], "isError": true } ``` -------------------------------- ### Fetch JSON API Example Source: https://github.com/zcaceres/fetch-mcp/blob/main/_autodocs/api-reference/CLI.md Example commands to fetch JSON data from an API, including one with a max-length flag. ```bash mcp-fetch json https://api.example.com/data mcp-fetch json https://api.example.com/data --max-length 50000 ``` -------------------------------- ### Get YouTube Transcripts Source: https://github.com/zcaceres/fetch-mcp/blob/main/_autodocs/api-reference/Overview.md Example of using Fetcher.youtubeTranscript to retrieve transcripts for a YouTube video. ```typescript const result = await Fetcher.youtubeTranscript({ url: "https://www.youtube.com/watch?v=VIDEO_ID", lang: "es", max_length: 20000 }); if (!result.isError) { const transcript = result.content[0].text; } ``` -------------------------------- ### Running CLI Source: https://github.com/zcaceres/fetch-mcp/blob/main/_autodocs/modules.md Instructions for installing and using the fetch-mcp command-line interface (CLI) for various fetching tasks. ```bash npm install -g mcp-fetch-server # Then from shell: mcp-fetch html https://example.com mcp-fetch markdown https://example.com/blog > article.md mcp-fetch youtube https://www.youtube.com/watch?v=VIDEO_ID ``` -------------------------------- ### checkYtDlp() Example Source: https://github.com/zcaceres/fetch-mcp/blob/main/_autodocs/api-reference/Fetcher.md Example of how to use the checkYtDlp method to see if yt-dlp is available. ```typescript const hasYtDlp = await Fetcher.checkYtDlp(); if (hasYtDlp) { console.log("yt-dlp is available for faster YouTube transcript extraction"); } ``` -------------------------------- ### Get Caption Tracks Example Source: https://github.com/zcaceres/fetch-mcp/blob/main/_autodocs/api-reference/YouTubeTranscript.md Example of how to extract available caption tracks from the player response object and log them. ```typescript const playerResponse = YouTubeTranscript.extractPlayerResponse(html); const tracks = YouTubeTranscript.getCaptionTracks(playerResponse); console.log("Available languages:"); tracks.forEach(track => { console.log(` ${track.languageCode}: ${track.name.simpleText}`); }); // Find English track const englishTrack = tracks.find(t => t.languageCode === "en"); if (englishTrack) { console.log("English subtitles available"); } ``` -------------------------------- ### MCP Server Startup Command Source: https://github.com/zcaceres/fetch-mcp/blob/main/_autodocs/modules.md Command to start the stdio-based MCP server. ```bash mcp-fetch-server # Starts the stdio-based MCP server ``` -------------------------------- ### fetch_youtube_transcript Example Response (Success) Source: https://github.com/zcaceres/fetch-mcp/blob/main/_autodocs/api-reference/MCP-Server.md An example of a successful response from the fetch_youtube_transcript tool, containing the transcript content. ```json { "content": [{ "type": "text", "text": "[Transcript language: en — English]\n\n[0:00] First line\n[0:05] Second line\n..." }], "isError": false } ``` -------------------------------- ### CLI Usage Source: https://github.com/zcaceres/fetch-mcp/blob/main/_autodocs/api-reference/Overview.md Example of using mcp-fetch as a CLI tool for fetching HTML. ```bash npx mcp-fetch html https://example.com # Or after global install: mcp-fetch html https://example.com ``` -------------------------------- ### parseArgs() Example Usage Source: https://github.com/zcaceres/fetch-mcp/blob/main/_autodocs/api-reference/CLI.md Example of how to use the parseArgs function and the resulting ParsedArgs object. ```typescript const args = parseArgs(["markdown", "https://example.com", "--max-length", "10000"]); // Returns: // { // subcommand: "markdown", // url: "https://example.com", // maxLength: 10000 // } ``` -------------------------------- ### Server Capabilities Source: https://github.com/zcaceres/fetch-mcp/blob/main/_autodocs/configuration.md Example of server capabilities, indicating support for tools. ```typescript { capabilities: { resources: {}, tools: {} } } ``` -------------------------------- ### Library Usage Source: https://github.com/zcaceres/fetch-mcp/blob/main/_autodocs/api-reference/Overview.md Example of using mcp-fetch as a library in TypeScript. ```typescript import { Fetcher } from "mcp-fetch-server"; const result = await Fetcher.html({ url: "https://example.com" }); ``` -------------------------------- ### Usage as Types Source: https://github.com/zcaceres/fetch-mcp/blob/main/_autodocs/modules.md Example of importing and using types from the mcp-fetch-server package. ```typescript import type { RequestPayload, YouTubeTranscriptPayload } from "mcp-fetch-server"; function processFetch(payload: RequestPayload) { // ... } ``` -------------------------------- ### Development Commands Source: https://github.com/zcaceres/fetch-mcp/blob/main/README.md Commands for development, including installation, running in watch mode, testing, and building. ```bash bun install bun run dev # start with watch mode bun test # run tests bun run build # build for production ``` -------------------------------- ### Fetcher.youtubeTranscript() Example Source: https://github.com/zcaceres/fetch-mcp/blob/main/_autodocs/api-reference/Fetcher.md Example of using the Fetcher.youtubeTranscript() method to fetch a YouTube video's captions/transcript. ```typescript const result = await Fetcher.youtubeTranscript({ url: "https://www.youtube.com/watch?v=dQw4w9WgXcQ", lang: "es", // Spanish max_length: 10000, }); if (!result.isError) { console.log("Transcript:\n", result.content[0].text); } ``` -------------------------------- ### Fetch and Process Blog Articles Source: https://github.com/zcaceres/fetch-mcp/blob/main/_autodocs/api-reference/Overview.md Example of using Fetcher.readable to fetch and process the main content of a blog article. ```typescript const result = await Fetcher.readable({ url: "https://example.com/blog/post", max_length: 50000 }); if (!result.isError) { const content = result.content[0].text; // Process markdown content } ``` -------------------------------- ### Fetcher.markdown() Example Source: https://github.com/zcaceres/fetch-mcp/blob/main/_autodocs/api-reference/Fetcher.md Example of using the Fetcher.markdown() method to fetch a URL and convert its HTML content to Markdown. ```typescript const result = await Fetcher.markdown({ url: "https://example.com/blog/post", max_length: 20000, }); if (!result.isError) { console.log("Content as Markdown:\n", result.content[0].text); } ``` -------------------------------- ### Error Output Source: https://github.com/zcaceres/fetch-mcp/blob/main/_autodocs/api-reference/CLI.md Example of error output from the CLI. ```bash $ mcp-fetch html https://invalid-url-that-does-not-exist.internal 2>&1 Error: Failed to fetch https://invalid-url-that-does-not-exist.internal: getaddrinfo ENOTFOUND invalid-url-that-does-not-exist.internal ``` -------------------------------- ### Parse Transcript XML Example Source: https://github.com/zcaceres/fetch-mcp/blob/main/_autodocs/api-reference/YouTubeTranscript.md Example of parsing YouTube caption XML and logging the formatted transcript lines. ```typescript const xmlContent = await response.text(); const lines = YouTubeTranscript.parseTranscriptXml(xmlContent); lines.forEach(line => { console.log(line); }); // Output: // [0:00] Welcome to the video // [0:05] Today we'll discuss... // [0:15] First point to note... ``` -------------------------------- ### Fetcher.readable() Example Source: https://github.com/zcaceres/fetch-mcp/blob/main/_autodocs/api-reference/Fetcher.md Example of using the Fetcher.readable() method to fetch a URL and extract the main article content as Markdown. ```typescript const result = await Fetcher.readable({ url: "https://example.com/blog/deep-article", max_length: 50000, }); if (!result.isError) { console.log("Article content:\n", result.content[0].text); } ``` -------------------------------- ### formatTimestamp Example Source: https://github.com/zcaceres/fetch-mcp/blob/main/_autodocs/api-reference/YouTubeTranscript.md Illustrates the output of the formatTimestamp method for various floating-point seconds inputs. ```plaintext 0.5 → "0:00" 3.75 → "0:03" 65.2 → "1:05" 125.9 → "2:05" 3661.5 → "61:01" ``` -------------------------------- ### Fetcher.txt() Example Source: https://github.com/zcaceres/fetch-mcp/blob/main/_autodocs/api-reference/Fetcher.md Fetches a URL, strips HTML tags and scripts, and returns plain text content. Demonstrates setting max_length and start_index. ```typescript const result = await Fetcher.txt({ url: "https://example.com/article", max_length: 5000, start_index: 0, }); if (!result.isError) { console.log("Article text:", result.content[0].text); } ``` -------------------------------- ### Environment Variables Configuration Source: https://github.com/zcaceres/fetch-mcp/blob/main/README.md Example of configuring environment variables for the fetch server, such as a custom default limit. ```json { "mcpServers": { "fetch": { "command": "npx", "args": ["mcp-fetch-server"], "env": { "DEFAULT_LIMIT": "50000" } } } } ``` -------------------------------- ### Fetcher.json() Example Source: https://github.com/zcaceres/fetch-mcp/blob/main/_autodocs/api-reference/Fetcher.md Fetches a URL, parses the response as JSON, and returns the stringified JSON content. Includes an example of setting request headers. ```typescript const result = await Fetcher.json({ url: "https://api.example.com/data", headers: { "Authorization": "Bearer token" }, }); if (!result.isError) { const jsonData = result.content[0].text; console.log("API response:", jsonData); } ``` -------------------------------- ### Usage as Schemas Source: https://github.com/zcaceres/fetch-mcp/blob/main/_autodocs/modules.md Example of importing and using Zod schemas from the mcp-fetch-server package for data validation. ```typescript import { RequestPayloadSchema } from "mcp-fetch-server"; const parsed = RequestPayloadSchema.parse({ url: "https://example.com", max_length: 5000 }); ``` -------------------------------- ### Usage as Fetcher Class Source: https://github.com/zcaceres/fetch-mcp/blob/main/_autodocs/modules.md Example of how to use the Fetcher class to retrieve and process HTML content from a URL. ```typescript import { Fetcher } from "mcp-fetch-server"; import type { RequestPayload } from "mcp-fetch-server"; const result = await Fetcher.html({ url: "https://example.com" }); if (!result.isError) { console.log(result.content[0].text); } ``` -------------------------------- ### Server Startup Function Source: https://github.com/zcaceres/fetch-mcp/blob/main/_autodocs/api-reference/MCP-Server.md TypeScript code for the main function that starts the server, including creating a stdio transport, connecting the server, and basic error handling. ```typescript async function main() { const transport = new StdioServerTransport(); await server.connect(transport); } main().catch((error) => { console.error("Fatal error in main():", error); process.exit(1); }); ``` -------------------------------- ### Error Handling in MCP Tools Source: https://github.com/zcaceres/fetch-mcp/blob/main/_autodocs/api-reference/Overview.md Example of checking for errors returned by Fetcher methods. ```typescript if (result.isError) { console.error("Error:", result.content[0].text); } ``` -------------------------------- ### Error Handling in CLI Source: https://github.com/zcaceres/fetch-mcp/blob/main/_autodocs/api-reference/Overview.md Example of handling errors in the CLI, showing output to stderr and process exit code. ```bash mcp-fetch html https://invalid-url 2>&1 # Error: Failed to fetch ... # (process exits with code 1) ``` -------------------------------- ### Pattern 5: YouTube Transcripts Source: https://github.com/zcaceres/fetch-mcp/blob/main/_autodocs/README.md Example of fetching YouTube transcripts, with an option to specify the language. ```typescript const result = await Fetcher.youtubeTranscript({ url: "https://www.youtube.com/watch?v=VIDEO_ID", lang: "es", // Spanish max_length: 20000 }); // Returns formatted transcript with timestamps ``` -------------------------------- ### decodeHtmlEntities Example Source: https://github.com/zcaceres/fetch-mcp/blob/main/_autodocs/api-reference/YouTubeTranscript.md Demonstrates how to use the decodeHtmlEntities method to convert HTML entities in a string to their corresponding characters. ```typescript const text = "Q&A: What’s <the> answer?"; const decoded = YouTubeTranscript.decodeHtmlEntities(text); console.log(decoded); // Output: Q&A: What's answer? ``` -------------------------------- ### Extract Data from APIs Source: https://github.com/zcaceres/fetch-mcp/blob/main/_autodocs/api-reference/Overview.md Example of using Fetcher.json to extract JSON data from an API endpoint. ```typescript const result = await Fetcher.json({ url: "https://api.example.com/data", headers: { "Authorization": "Bearer token" }, max_length: 1000000 }); if (!result.isError) { const jsonString = result.content[0].text; const data = JSON.parse(jsonString); } ``` -------------------------------- ### Pattern 3: Custom Headers Source: https://github.com/zcaceres/fetch-mcp/blob/main/_autodocs/README.md Example of passing custom headers, such as for authentication or setting the User-Agent. ```typescript const result = await Fetcher.json({ url: "https://api.example.com/data", headers: { "Authorization": "Bearer token", "User-Agent": "MyApp/1.0" } }); ``` -------------------------------- ### Fetcher.html() Example Source: https://github.com/zcaceres/fetch-mcp/blob/main/_autodocs/api-reference/Fetcher.md Fetches a URL and returns the unmodified HTML content. Handles potential errors by checking the 'isError' flag. ```typescript const result = await Fetcher.html({ url: "https://example.com", max_length: 10000, }); if (result.isError) { console.error("Error:", result.content[0].text); } else { console.log("HTML content:", result.content[0].text); } ``` -------------------------------- ### Fetch and convert to markdown, then grep for keywords Source: https://github.com/zcaceres/fetch-mcp/blob/main/_autodocs/api-reference/CLI.md Fetches content from a URL, converts it to markdown, and then searches for a keyword using grep. ```bash mcp-fetch markdown https://example.com | grep -i "important" ``` -------------------------------- ### Usage with Limits Source: https://github.com/zcaceres/fetch-mcp/blob/main/_autodocs/api-reference/CLI.md Using --max-length and --start-index to control the amount of content fetched. ```bash # First 5000 characters mcp-fetch html https://example.com --max-length 5000 # Characters 5000-10000 mcp-fetch html https://example.com --max-length 5000 --start-index 5000 ``` -------------------------------- ### Extract Player Response Example Source: https://github.com/zcaceres/fetch-mcp/blob/main/_autodocs/api-reference/YouTubeTranscript.md Example of how to extract the ytInitialPlayerResponse JSON object from YouTube video page HTML. ```typescript const result = await Fetcher.html({ url: "https://www.youtube.com/watch?v=dQw4w9WgXcQ" }); if (!result.isError) { const playerResponse = YouTubeTranscript.extractPlayerResponse(result.content[0].text); console.log("Video ID:", playerResponse.videoDetails?.videoId); } ``` -------------------------------- ### Basic Usage Source: https://github.com/zcaceres/fetch-mcp/blob/main/_autodocs/api-reference/CLI.md The most basic way to use the mcp-fetch command. ```bash mcp-fetch markdown https://example.com ``` -------------------------------- ### CLI Commands Source: https://github.com/zcaceres/fetch-mcp/blob/main/_autodocs/QUICK-REFERENCE.md Command-line interface commands for fetching content. ```bash mcp-fetch html [--max-length N] [--start-index N] [--proxy URL] mcp-fetch json [--max-length N] [--start-index N] [--proxy URL] mcp-fetch txt [--max-length N] [--start-index N] [--proxy URL] mcp-fetch markdown [--max-length N] [--start-index N] [--proxy URL] mcp-fetch readable [--max-length N] [--start-index N] [--proxy URL] mcp-fetch youtube [--lang CODE] [--max-length N] [--start-index N] [--proxy URL] mcp-fetch --help # Show help mcp-fetch --version # Show version ``` -------------------------------- ### CLI Usage Source: https://github.com/zcaceres/fetch-mcp/blob/main/README.md General CLI usage pattern. ```bash mcp-fetch [flags] ``` -------------------------------- ### CLI Commands Source: https://github.com/zcaceres/fetch-mcp/blob/main/_autodocs/api-reference/Overview.md Commands for running fetch-mcp from the shell. ```bash mcp-fetch html # Raw HTML mcp-fetch json # JSON data mcp-fetch txt # Plain text mcp-fetch markdown # HTML as Markdown mcp-fetch readable # Article content mcp-fetch youtube # YouTube transcript ``` -------------------------------- ### Common Patterns - Pagination Source: https://github.com/zcaceres/fetch-mcp/blob/main/_autodocs/QUICK-REFERENCE.md Implementing pagination for fetching content. ```typescript const page1 = await Fetcher.txt({ url, max_length: 5000, start_index: 0 }); const page2 = await Fetcher.txt({ url, max_length: 5000, start_index: 5000 }); ``` -------------------------------- ### Imports Source: https://github.com/zcaceres/fetch-mcp/blob/main/_autodocs/QUICK-REFERENCE.md Common imports for using the fetch-mcp library. ```typescript import { Fetcher } from "mcp-fetch-server"; import { YouTubeTranscript } from "mcp-fetch-server"; import type { RequestPayload, YouTubeTranscriptPayload } from "mcp-fetch-server"; import { RequestPayloadSchema, YouTubeTranscriptPayloadSchema } from "mcp-fetch-server"; import { downloadLimit, maxResponseBytes } from "mcp-fetch-server"; ``` -------------------------------- ### Common Patterns - YouTube with Language Source: https://github.com/zcaceres/fetch-mcp/blob/main/_autodocs/QUICK-REFERENCE.md Fetching YouTube transcripts in a specific language. ```typescript const result = await Fetcher.youtubeTranscript({ url: "https://www.youtube.com/watch?v=VIDEO_ID", lang: "es" }); ``` -------------------------------- ### Fetch JSON and pretty-print Source: https://github.com/zcaceres/fetch-mcp/blob/main/_autodocs/api-reference/CLI.md Fetches JSON data from a URL and pretty-prints it using Python's json.tool. ```bash mcp-fetch json https://api.example.com/data | python -m json.tool ``` -------------------------------- ### Dependencies Table Source: https://github.com/zcaceres/fetch-mcp/blob/main/_autodocs/README.md Lists the project's dependencies and their purposes. ```markdown ## Dependencies | Package | Version | Purpose | |---------|---------|---------| | @modelcontextprotocol/sdk | ^1.27.1 | MCP protocol | | @mozilla/readability | ^0.6.0 | Article extraction | | jsdom | ^28.1.0 | HTML parsing | | turndown | ^7.2.2 | HTML → Markdown | | zod | ^4.3.6 | Validation | | private-ip | ^3.0.2 | IP checking | ``` -------------------------------- ### Server Initialization Source: https://github.com/zcaceres/fetch-mcp/blob/main/_autodocs/api-reference/MCP-Server.md TypeScript code snippet demonstrating the initialization of the Server instance with its name, version, and capabilities. ```typescript const server = new Server( { name: "zcaceres/fetch", version: pkg.version, }, { capabilities: { resources: {}, tools: {}, }, }, ); ``` -------------------------------- ### run() Function Signature Source: https://github.com/zcaceres/fetch-mcp/blob/main/_autodocs/api-reference/CLI.md Executes the fetch operation based on parsed arguments. ```typescript async function run(args: ParsedArgs): Promise ``` -------------------------------- ### Core Fetcher Methods Source: https://github.com/zcaceres/fetch-mcp/blob/main/_autodocs/QUICK-REFERENCE.md Methods for fetching content in various formats. ```typescript // HTML await Fetcher.html({ url, headers?, max_length?, start_index?, proxy? }) // JSON await Fetcher.json({ url, headers?, max_length?, start_index?, proxy? }) // Plain Text await Fetcher.txt({ url, headers?, max_length?, start_index?, proxy? }) // Markdown await Fetcher.markdown({ url, headers?, max_length?, start_index?, proxy? }) // Article Content await Fetcher.readable({ url, headers?, max_length?, start_index?, proxy? }) // YouTube Transcript await Fetcher.youtubeTranscript({ url, headers?, max_length?, start_index?, proxy?, lang? }) // Check yt-dlp availability await Fetcher.checkYtDlp() → boolean ``` -------------------------------- ### Piping to File Source: https://github.com/zcaceres/fetch-mcp/blob/main/_autodocs/api-reference/CLI.md Redirecting the output of the mcp-fetch command to a file. ```bash mcp-fetch readable https://example.com/blog > article.md ``` -------------------------------- ### YouTube Helpers Source: https://github.com/zcaceres/fetch-mcp/blob/main/_autodocs/QUICK-REFERENCE.md Utility functions for processing YouTube transcripts. ```typescript // Extract ytInitialPlayerResponse JSON YouTubeTranscript.extractPlayerResponse(html: string) → unknown // Get caption tracks from player response YouTubeTranscript.getCaptionTracks(playerResponse: any) → any[] // Parse caption XML to formatted lines YouTubeTranscript.parseTranscriptXml(xml: string) → string[] // Decode HTML entities (& → &, etc.) YouTubeTranscript.decodeHtmlEntities(text: string) → string ``` -------------------------------- ### CLI Tool Command Source: https://github.com/zcaceres/fetch-mcp/blob/main/_autodocs/modules.md Command to invoke the CLI tool for fetching. ```bash mcp-fetch [flags] # CLI tool ``` -------------------------------- ### Common Patterns - Authentication Source: https://github.com/zcaceres/fetch-mcp/blob/main/_autodocs/QUICK-REFERENCE.md Making authenticated requests using custom headers. ```typescript const result = await Fetcher.json({ url: "https://api.example.com/data", headers: { "Authorization": "Bearer token" } }); ``` -------------------------------- ### MCP Server Configuration Source: https://github.com/zcaceres/fetch-mcp/blob/main/README.md Add this server to your MCP client configuration. ```json { "mcpServers": { "fetch": { "command": "npx", "args": ["mcp-fetch-server"] } } } ``` -------------------------------- ### Project Information Source: https://github.com/zcaceres/fetch-mcp/blob/main/_autodocs/README.md Provides general information about the project. ```markdown ## Project Information - **Name:** mcp-fetch-server - **Version:** 1.1.2 - **License:** MIT - **Repository:** https://github.com/zcaceres/fetch-mcp - **Node Version:** ≥18.0.0 - **Package Manager:** bun/npm ``` -------------------------------- ### Configuration Constants Imports Source: https://github.com/zcaceres/fetch-mcp/blob/main/_autodocs/modules.md Import statements for configuration constants like downloadLimit and maxResponseBytes. ```typescript import { downloadLimit, maxResponseBytes } from "mcp-fetch-server/dist/types.js"; ``` -------------------------------- ### Server Metadata Source: https://github.com/zcaceres/fetch-mcp/blob/main/_autodocs/configuration.md Metadata for the zcaceres/fetch MCP server. ```typescript { name: "zcaceres/fetch", version: "1.1.2" // from package.json } ``` -------------------------------- ### Pattern 2: Pagination Source: https://github.com/zcaceres/fetch-mcp/blob/main/_autodocs/README.md Shows how to fetch content in chunks using the `max_length` and `start_index` parameters. ```typescript // Fetch characters 0-5000 const page1 = await Fetcher.txt({ url: "https://example.com", max_length: 5000, start_index: 0 }); // Fetch characters 5000-10000 const page2 = await Fetcher.txt({ url: "https://example.com", max_length: 5000, start_index: 5000 }); ``` -------------------------------- ### YouTube Transcript Tool Dispatch Logic Source: https://github.com/zcaceres/fetch-mcp/blob/main/_autodocs/api-reference/MCP-Server.md TypeScript code snippet showing how the server dispatches the 'fetch_youtube_transcript' tool, including input validation and calling the Fetcher method. ```typescript if (name === "fetch_youtube_transcript") { const validatedArgs = YouTubeTranscriptPayloadSchema.parse(args); return Fetcher.youtubeTranscript(validatedArgs); } ``` -------------------------------- ### Default User Agent Source: https://github.com/zcaceres/fetch-mcp/blob/main/_autodocs/configuration.md The default User-Agent header used for all fetch operations. ```string Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/120.0.0.0 Safari/537.36 ``` -------------------------------- ### Fetcher Mapping Source: https://github.com/zcaceres/fetch-mcp/blob/main/_autodocs/api-reference/CLI.md Maps subcommand names to their corresponding Fetcher methods. ```typescript const fetchers: Record Promise> = { html: Fetcher.html.bind(Fetcher), markdown: Fetcher.markdown.bind(Fetcher), readable: Fetcher.readable.bind(Fetcher), txt: Fetcher.txt.bind(Fetcher), json: Fetcher.json.bind(Fetcher), youtube: Fetcher.youtubeTranscript.bind(Fetcher), }; ``` -------------------------------- ### Entry Point Logic Source: https://github.com/zcaceres/fetch-mcp/blob/main/_autodocs/api-reference/CLI.md The main execution block of the CLI script. ```typescript if (isMainModule()) { const args = parseArgs(process.argv.slice(2)); run(args).catch((err) => { process.stderr.write(String(err?.message ?? err) + "\n"); process.exit(1); }); } ``` -------------------------------- ### Build Command Source: https://github.com/zcaceres/fetch-mcp/blob/main/_autodocs/modules.md The command used to build the project's TypeScript files into JavaScript, specifying output directory, target, and external dependencies. ```bash bun build src/index.ts src/cli.ts --outdir dist --target node --external jsdom --external @mozilla/readability --external turndown ``` -------------------------------- ### Standard Fetch Tools Dispatch Logic Source: https://github.com/zcaceres/fetch-mcp/blob/main/_autodocs/api-reference/MCP-Server.md TypeScript code snippet illustrating the dispatch logic for standard fetch tools, including checking for tool existence, input validation, and routing to Fetcher methods. ```typescript if (!FETCH_TOOLS.has(name)) { throw new Error(`Tool not found: ${name}`); } const validatedArgs = RequestPayloadSchema.parse(args); if (name === "fetch_html") return Fetcher.html(validatedArgs); if (name === "fetch_json") return Fetcher.json(validatedArgs); if (name === "fetch_txt") return Fetcher.txt(validatedArgs); if (name === "fetch_markdown") return Fetcher.markdown(validatedArgs); return Fetcher.readable(validatedArgs); ``` -------------------------------- ### Module Dependencies (Import Graph) Source: https://github.com/zcaceres/fetch-mcp/blob/main/_autodocs/modules.md An import graph illustrating the dependencies between different modules of the MCP Fetch Server. ```text index.ts (MCP Server) ├── @modelcontextprotocol/sdk ├── Fetcher ├── types └── process cli.ts (CLI Tool) ├── Fetcher ├── types └── fs, path utilities Fetcher.ts (Core) ├── jsdom (JSDOM) ├── turndown (TurndownService) ├── @mozilla/readability (Readability) ├── private-ip ├── dns ├── types ├── YouTubeTranscript └── child_process (for yt-dlp) YouTubeTranscript.ts (Helper) └── (no dependencies - pure utilities) types.ts (Types & Config) └── zod (z) ``` -------------------------------- ### Fetcher Class Return Format Source: https://github.com/zcaceres/fetch-mcp/blob/main/_autodocs/api-reference/Overview.md Common return format for Fetcher methods. ```typescript { content: [{ type: "text"; text: string }]; isError: boolean; } ``` -------------------------------- ### Common Patterns - Article Extraction Source: https://github.com/zcaceres/fetch-mcp/blob/main/_autodocs/QUICK-REFERENCE.md Using Fetcher.readable to extract article content. ```typescript const result = await Fetcher.readable({ url: "https://example.com/article", max_length: 50000 }); ``` -------------------------------- ### Fetcher Class Main Exports Source: https://github.com/zcaceres/fetch-mcp/blob/main/_autodocs/api-reference/Overview.md Static methods available on the Fetcher class for fetching and processing different content types. ```typescript export class Fetcher { static async html(payload: RequestPayload): Promise static async json(payload: RequestPayload): Promise static async txt(payload: RequestPayload): Promise static async markdown(payload: RequestPayload): Promise static async readable(payload: RequestPayload): Promise static async youtubeTranscript(payload: YouTubeTranscriptPayload): Promise static async checkYtDlp(): Promise } ``` -------------------------------- ### Processing in Shell Scripts Source: https://github.com/zcaceres/fetch-mcp/blob/main/_autodocs/api-reference/CLI.md Using mcp-fetch within a bash script to process fetched content. ```bash #!/bin/bash CONTENT=$(mcp-fetch txt https://example.com) if [ $? -eq 0 ]; then echo "Content length: ${#CONTENT}" else echo "Failed to fetch content" exit 1 fi ``` -------------------------------- ### YouTube Exports Source: https://github.com/zcaceres/fetch-mcp/blob/main/_autodocs/api-reference/Overview.md Helper methods for extracting and parsing YouTube transcripts. ```typescript export class YouTubeTranscript { static extractPlayerResponse(html: string): unknown static getCaptionTracks(playerResponse: any): any[] static parseTranscriptXml(xml: string): string[] static decodeHtmlEntities(text: string): string } ```