search-webdeveloper-tools

MkDocs Search

serverless-dna

by serverless-dna

Enable AI search on MkDocs sites by converting HTML to markdown and R markdown, leveraging Lunr.js for powerful md forma

Enables AI to search and retrieve content from MkDocs documentation sites by leveraging existing Lunr.js indexes and converting HTML to markdown for seamless integration.

github stars

17

0 commentsdiscussion

Both formats append explainx.ai attribution and the canonical URL for this MCP server listing.

Works with any published MkDocs siteUses existing Lunr.js search indexesIntelligent HTML to markdown conversion

best for

  • / AI agents needing to search technical documentation
  • / Developers working with MkDocs sites
  • / Documentation analysis and content retrieval

capabilities

  • / Search MkDocs documentation sites
  • / Fetch and convert doc pages to markdown
  • / Filter results by confidence threshold
  • / Extract code examples with language detection
  • / Preserve Mermaid diagrams
  • / Cache search indexes and converted content

what it does

Searches and retrieves content from any MkDocs documentation site using the site's existing Lunr.js search index and converts pages to markdown.

about

MkDocs Search is a community-built MCP server published by serverless-dna that provides AI assistants with tools and capabilities via the Model Context Protocol. Enable AI search on MkDocs sites by converting HTML to markdown and R markdown, leveraging Lunr.js for powerful md forma It is categorized under search web, developer tools. This server exposes 2 tools that AI clients can invoke during conversations and coding sessions.

how to install

You can install MkDocs Search in your AI client of choice. Use the install panel on this page to get one-click setup for Cursor, Claude Desktop, VS Code, and other MCP-compatible clients. This server runs locally on your machine via the stdio transport.

license

MIT

MkDocs Search is released under the MIT license. This is a permissive open-source license, meaning you can freely use, modify, and distribute the software.

readme

MkDocs MCP Search Server

A Model Context Protocol (MCP) server that provides search functionality for any MkDocs powered site. This server relies on the existing MkDocs search implementation using the Lunr.Js search engine.

Claude Desktop Quickstart

Follow the installation instructions please follow the Model Context Protocol Quickstart For Claude Desktop users. You will need to add a section tothe MCP configuration file as follows:

{
  "mcpServers": {
    "my-docs": {
      "command": "npx",
      "args": [
        "-y",
        "@serverless-dna/mkdocs-mcp",
        "https://your-doc-site",
        "Describe what you are enabling search for to help your AI Agent"
      ]
    }
  }
}

Overview

This project implements an MCP server that enables Large Language Models (LLMs) to search through any published mkdocs documentation site. It uses lunr.js for efficient local search capabilities and provides results that can be summarized and presented to users.

Features

  • MCP-compliant server for integration with LLMs
  • Local search using lunr.js indexes
  • Version-specific documentation search capability
  • MkDocs Material HTML to Markdown conversion with structured JSON responses
  • Code example extraction with language detection and context
  • Tab view support for multi-language documentation
  • Mermaid diagram preservation
  • Automatic URL resolution (relative to absolute)
  • Intelligent caching for both search indexes and converted documentation

Installation

# Install dependencies
pnpm install

# Build the project
pnpm build

Usage

The server can be run as an MCP server that communicates over stdio:

npx -y @serverless-dna/mkdocs-mcp https://your-doc-site.com

Available Tools

Search Tool

The server provides a searchMkDoc tool with the following parameters:

  • search: The search query string
  • version: Optional version string (only for versioned sites)

Sample Response:

{
  "query": "logger",
  "version": "latest",
  "total": 3,
  "results": [
    {
      "title": "Logger",
      "url": "https://docs.example.com/latest/core/logger/",
      "score": 1.2,
      "preview": "Logger utility for structured logging...",
      "location": "core/logger/"
    },
    {
      "title": "Configuration",
      "url": "https://docs.example.com/latest/core/logger/#config",
      "score": 0.8,
      "preview": "Configure the logger with custom settings...",
      "location": "core/logger/#config",
      "parentArticle": {
        "title": "Logger",
        "location": "core/logger/",
        "url": "https://docs.example.com/latest/core/logger/"
      }
    }
  ]
}

Features:

  • Confidence-based filtering (configurable threshold)
  • Advanced scoring with title matching and boosting
  • Parent article context for section results
  • Limited to top results (configurable, default: 10)

Fetch Documentation Tool

The server provides a fetchMkDoc tool that retrieves and converts documentation pages:

  • url: The URL of the documentation page to fetch

Sample Response:

{
  "title": "Getting Started",
  "markdown": "# Getting Started

This guide will help you...

## Installation

```bash
npm install example
```",
  "code_examples": [
    {
      "title": "Installation",
      "description": "Install the package using npm",
      "code": "```bash
npm install example
```"
    },
    {
      "title": "Basic Usage",
      "description": "Import and initialize the library",
      "code": "```python
from example import Client
client = Client()
```"
    }
  ],
  "url": "https://docs.example.com/getting-started/"
}

Configuration

The server can be configured using environment variables:

  • SEARCH_CONFIDENCE_THRESHOLD: Minimum confidence score for search results (default: 0.1)
  • SEARCH_MAX_RESULTS: Maximum number of search results to return (default: 10)
  • CACHE_BASE_PATH: Base directory for cache storage (default: <system-tmp>/mkdocs-mcp-cache)

Example:

SEARCH_MAX_RESULTS=20 SEARCH_CONFIDENCE_THRESHOLD=0.2 npx @serverless-dna/mkdocs-mcp https://your-doc-site.com

Cache Location: By default, the server caches search indexes and converted documentation in the system's temporary directory:

  • macOS/Linux: /tmp/mkdocs-mcp-cache (or $TMPDIR)
  • Windows: %TEMP%\mkdocs-mcp-cache

You can override this with the CACHE_BASE_PATH environment variable.

Development

Building

pnpm build

Testing

pnpm test

Claude Desktop MCP Configuration

During development you can run the MCP Server with Claude Desktop using the following configuration.

The configuration below shows running in windows claude desktop while developing using the Windows Subsystem for Linux (WSL). Mac or Linux environments you can run in a similar way.

The output is a bundled file which enables Node installed in windows to run the MCP server since all dependencies are bundled.

{
  "mcpServers": {
    "powertools": {
	"command": "node",
	"args": [
	  "\\wsl$\Ubuntu\home\walmsles\dev\serverless-dna\mkdocs-mcp\dist\index.js",
    "Search online documentation"
	]
    }
  }
}

How It Works

Search Functionality

  1. The server loads pre-built lunr.js indexes for each supported runtime
  2. When a search request is received, it:
    • Loads the appropriate index based on version (currently fixed to latest)
    • Performs the search using lunr.js
    • Returns the search results as JSON
  3. The LLM can then use these results to find relevant documentation pages

Documentation Fetching

  1. When a fetch request is received with a URL:
    • Fetches the HTML content (with caching)
    • Parses the MkDocs Material HTML structure using Cheerio
    • Removes navigation, headers, footers, and other UI elements
    • Processes tab views into sequential sections
    • Extracts code blocks with language detection and context
    • Resolves all relative URLs to absolute URLs
    • Converts the cleaned HTML to markdown
    • Returns a structured JSON response with title, markdown, and code examples
  2. Results are cached to improve performance on subsequent requests

License

MIT

FAQ

What is the MkDocs Search MCP server?
MkDocs Search is a Model Context Protocol (MCP) server profile on explainx.ai. MCP lets AI hosts (e.g. Claude Desktop, Cursor) call tools and resources through a standard interface; this page summarizes categories, install hints, and community ratings.
How do MCP servers relate to agent skills?
Skills are reusable instruction packages (often SKILL.md); MCP servers expose live capabilities. Teams frequently combine both—skills for workflows, MCP for APIs and data. See explainx.ai/skills and explainx.ai/mcp-servers for parallel directories.
How are reviews shown for MkDocs Search?
This profile displays 64 aggregated ratings (sample rows for discoverability plus signed-in user reviews). Average score is about 4.8 out of 5—verify behavior in your own environment before production use.

Use Cases

Web Research & Information Gathering

Fetch and extract information from websites automatically

Example

Research competitor pricing, scrape product reviews, monitor news mentions

Automate 5-10 hours/week of manual web research

Content Monitoring & Alerts

Track website changes, new content, price updates

Example

Monitor competitor blog for new posts, track stock availability, watch for pricing changes

Stay informed without manual checking, never miss important updates

Data Extraction & Aggregation

Extract structured data from multiple websites

Example

Compile product listings from 10 e-commerce sites, aggregate job postings, collect real estate data

Build datasets 100x faster than manual copying

API-less Integration

Interact with services that don't offer APIs

Example

Check form submissions, validate website functionality, test user flows

Automate interactions with any website, even without API

Implementation Guide

Prerequisites

  • Claude Desktop or Cursor with MCP support
  • Understanding of web scraping ethics and robots.txt
  • Rate limiting awareness to avoid overwhelming target sites
  • Knowledge of legal restrictions on data collection

Time Estimate

20-40 minutes including configuration and testing

Installation Steps

  1. 1.Install web automation MCP server via npm or pip
  2. 2.Configure allowed domains and rate limits in MCP config
  3. 3.Test with simple fetch: 'Get content from example.com'
  4. 4.Progress to extraction: 'Extract all product prices from this page'
  5. 5.Set up monitoring: 'Check this URL daily for changes'
  6. 6.Parse structured data: 'Create CSV from this table'
  7. 7.Respect robots.txt and rate limits always

Troubleshooting

  • 403 Forbidden: Website blocks bots—respect their wishes, use official API instead
  • Rate limit errors: Slow down requests, add delays between fetches
  • Stale data: Target site changed HTML structure—update selectors
  • Timeout errors: Site is slow or blocking—increase timeout, try different user agent
  • JavaScript-rendered content: Use headless browser MCP servers for dynamic sites

Best Practices

✓ Do

  • +Check robots.txt and respect crawl rules
  • +Rate limit requests: 1-2 requests/second maximum
  • +Use official APIs when available instead of scraping
  • +Identify your bot with descriptive user agent
  • +Cache results to minimize repeated requests
  • +Handle errors gracefully with retries and fallbacks
  • +Validate extracted data for accuracy

✗ Don't

  • Don't scrape sites that explicitly forbid it (robots.txt, ToS)
  • Don't overwhelm servers with rapid requests—use rate limiting
  • Don't scrape personal data without consent and legal basis
  • Don't ignore copyright on extracted content
  • Don't assume HTML structure is stable—handle changes
  • Don't use scraped data for commercial purposes without permission

💡 Pro Tips

  • Use CSS selectors or XPath for robust data extraction
  • Set up monitoring alerts for extraction failures (structure changed)
  • Implement exponential backoff for retries on failures
  • Store raw HTML for reprocessing if extraction logic changes
  • Combine with data analysis tools for insights from extracted data
  • Consider using official APIs or RSS feeds as more stable alternatives

Technical Details

Architecture

MCP server handles HTTP requests, HTML parsing, JavaScript rendering (if headless browser), and returns structured data to Claude.

Protocols

  • HTTP/HTTPS
  • WebSocket (for real-time sites)
  • Puppeteer/Playwright (for JavaScript sites)

Compatibility

  • Static HTML sites
  • JavaScript-rendered SPAs (with headless browser)
  • REST APIs
  • GraphQL endpoints

When to Use This

✓ Use When

Use for research automation, content monitoring, data aggregation from multiple sources, and when official APIs don't exist. Best for read-only information gathering.

✗ Avoid When

Avoid for sites with APIs (use API instead), sites that explicitly forbid scraping, when data is copyrighted, or for login-required content without proper authorization.

Integration

  • Scheduled monitoring with change detection
  • Multi-source data aggregation pipelines
  • Fallback to web scraping when API rate limits hit
  • Headless browser for JavaScript-heavy sites

Discussion

Product Hunt–style comments (not star reviews)
  • No comments yet — start the thread.

List & Promote Your MCP Server

Share your MCP server with the developer community

GET_STARTED →
MCP server reviews

Ratings

4.864 reviews
  • Amina Abbas· Dec 28, 2024

    Useful MCP listing: MkDocs Search is the kind of server we cite when onboarding engineers to host + tool permissions.

  • Amina Mensah· Dec 24, 2024

    We evaluated MkDocs Search against two servers with overlapping tools; this profile had the clearer scope statement.

  • Min Thompson· Dec 20, 2024

    MkDocs Search is a well-scoped MCP server in the explainx.ai directory — install snippets and categories matched our Claude Code setup.

  • Neel Ramirez· Dec 16, 2024

    Useful MCP listing: MkDocs Search is the kind of server we cite when onboarding engineers to host + tool permissions.

  • Kaira Patel· Dec 12, 2024

    Strong directory entry: MkDocs Search surfaces stars and publisher context so we could sanity-check maintenance before adopting.

  • Neel Gonzalez· Dec 8, 2024

    We wired MkDocs Search into a staging workspace; the listing’s GitHub and npm pointers saved time versus hunting across READMEs.

  • Chaitanya Patil· Dec 4, 2024

    We wired MkDocs Search into a staging workspace; the listing’s GitHub and npm pointers saved time versus hunting across READMEs.

  • Min Robinson· Dec 4, 2024

    According to our notes, MkDocs Search benefits from clear Model Context Protocol framing — fewer ambiguous “AI plugin” claims.

  • Amina Huang· Nov 27, 2024

    Strong directory entry: MkDocs Search surfaces stars and publisher context so we could sanity-check maintenance before adopting.

  • Piyush G· Nov 23, 2024

    Strong directory entry: MkDocs Search surfaces stars and publisher context so we could sanity-check maintenance before adopting.

showing 1-10 of 64

1 / 7