fetch-mcp is a versatile MCP server designed to seamlessly integrate web content fetching into your AI models. Acting as a crucial tool within the MCP ecosystem, it empowers models to dynamically retrieve and process information from the web.

This server offers functionalities to fetch web content in various formats, including HTML, JSON, plain text, and Markdown, using tools like fetch_html, fetch_json, fetch_txt, and fetch_markdown. Developers can specify URLs and custom headers, enabling flexible data acquisition. fetch-mcp leverages modern fetch API, JSDOM for HTML parsing, and TurndownService for Markdown conversion, ensuring efficient and accurate content extraction.

By providing AI models with direct access to real-time web data, fetch-mcp enhances their ability to provide up-to-date and contextually relevant responses. Integration is straightforward, requiring only a simple configuration within your application's server settings. This eliminates the complexities of manual web scraping and data formatting, allowing developers to focus on core AI model development.

Multi-Format Web Content Fetching

fetch-mcp empowers AI models with the ability to retrieve and process web content in various formats, including HTML, JSON, plain text, and Markdown. This is achieved through specialized tools like fetch_html, fetch_json, fetch_txt, and fetch_markdown, each tailored to handle specific content types. The server utilizes the modern Fetch API for making HTTP requests, ensuring efficient and reliable data retrieval. This capability is crucial for AI models that require up-to-date information from the web, such as news aggregators, research tools, or applications that need to monitor website changes. For example, an AI-powered market analysis tool could use fetch_json to retrieve real-time stock prices from a financial API or fetch_html to scrape product information from e-commerce websites. The server uses JSDOM for HTML parsing and text extraction.

Customizable Request Headers

The ability to include custom headers in HTTP requests provides AI models with greater control and flexibility when interacting with web servers. This feature allows models to authenticate with APIs, specify desired content types, and bypass certain restrictions. The headers parameter, available in each of the fetching tools, enables developers to add custom headers to the request. For instance, an AI model accessing a protected API might need to include an Authorization header with a valid API key. Similarly, a model designed to retrieve content in a specific language could use the Accept-Language header to request content in the desired language. This level of customization ensures that AI models can seamlessly integrate with a wide range of web services and APIs, enhancing their ability to access and process relevant information.

Streamlined HTML to Markdown Conversion

fetch-mcp simplifies the process of converting HTML content into Markdown format, making it easier for AI models to process and understand web data. The fetch_markdown tool leverages TurndownService, a robust HTML to Markdown converter, to transform complex HTML structures into clean, readable Markdown. This is particularly useful for AI models that need to extract text from web pages while preserving formatting elements like headings, lists, and links. For example, an AI-powered content summarization tool could use fetch_markdown to convert news articles into Markdown before extracting the key points. This feature eliminates the need for AI models to handle complex HTML parsing and formatting, streamlining the data processing pipeline and improving overall efficiency.

Technical Implementation

fetch-mcp is built using TypeScript, a strongly typed superset of JavaScript, which enhances code maintainability and reduces the risk of runtime errors. The server leverages the modern Fetch API for making HTTP requests, ensuring efficient and reliable data retrieval. JSDOM is used for HTML parsing and text extraction, providing a consistent and standards-compliant way to interact with HTML documents. TurndownService is employed for HTML to Markdown conversion, offering a flexible and customizable solution for transforming HTML content into Markdown format. The project uses npm as its package manager and provides convenient commands for building, running, and testing the server.